Are algorithms and AI models fundamentally different?
Understanding the differences between algorithms and AI models is critical for us to ask questions about technology that shapes and, increasingly, controls our lives.
Are algorithms and AI models fundamentally different?
Understanding the differences between precisely coded algorithms and models that are not fixed in their programming but learn and can adapt is critical for us to ask questions about the technology that shapes and, increasingly, controls our lives.
Artificial Intelligence (AI) uses models to learn and answer questions or generate new things. Algorithms have long ruled our world and armed armies of bureaucrats with the refrain of, "the computer won't let me do this. I am sorry."
No one would need to know the difference between algorithms and models in an ideal world since both will be cleverly and helpfully hidden behind a humanizing layer. The abstraction breaks through when technology is accused of bias, plagiarism, or misinformation. Policymakers, informed citizens, journalists, and technologists need to understand the fundamental differences between technologies that primarily rely on algorithms and those that leverage machine learning models, or increasingly a hybrid.
The difference between programming and creating models started out as just a semantic difference. Algorithms have been steadily creeping along the spectrum to more into models. In the last decade, the difference has become a kind, not a degree.
Historical Note: Greeks pioneered algorithms to solve geometry problems: The ancient Greeks used algorithms in geometry and mathematics. One famous example is the Euclidean algorithm, a method for finding the greatest common divisor of two numbers. The Euclidean algorithm was described by the Greek mathematician Euclid in his book "Elements" and has been used for thousands of years to perform division and find the greatest common factor of two numbers. The algorithm has also been adapted for use in computer science and is still used today in various forms. Their use of algorithms laid the foundation for the development of mathematics and paved the way for future generations to build upon their work.
Courtesy Visual Capitalist
Chinese abacus as an early computer: In 14th century China, algorithms were used to develop the abacus, an early counting tool. It is one of the earliest examples of using algorithms for practical purposes.
Tell me what to do
Courtesy Visual Capitalist
Algorithms are a set of rules: The traditional way of getting computers to do things is to program them by encoding the algorithm in a language that computers can understand. Algorithms really are rigid sets of instructions to solve a problem. Any apparent flexibility comes from the programmer, who has to think through all the conditions the program can encounter and list those.
Programming is good at capturing business logic: If you were responsible for pricing tickets for an airline, you'd have logic that says, "if the departure date is less than twenty-four hours, on a busy route and on a Monday morning, then raise the prices of tickets by 2% each hour." The airlines' program will include hundreds of such conditions.
Programmers work with subject matter experts (SMEs) to encode this logic. This has led to interdisciplinary subjects like computational biology and the rise in stature of developers on Wall Street for trading.
Failures are severe: This collection of rules will fail if the program encounters a set of conditions it does not recognize. This results in failures like hackers stealing sensitive data from credit card companies because they were able to mimic the conditions of being trusted employees. Another apocryphal example is when NASA lost a Mars orbiter because different parts of its programming used different units of measurement. Programming mistakes can be fixed or debugged by retracing the original logic and finding errors in the algorithm used to code the logic. It is time-consuming, expensive, and sometimes mind-numbing, but it is done by engineers daily.
Correctness and performance: The correctness of an algorithmic system is determined by testing its output against a known list of inputs. Performance is determined by how quickly the program outputs a given set of inputs.
Algorithms are like recipes: A good cake recipe turns out moist and delicious cakes. If you skip any steps, the cake will not rise, be dry, or worse. When it comes to computers, failures are usually the fault of the recipe writer, the programmer, since computers follow instructions faithfully.
Teach a person to fish
Programming with algorithms can only solve some problems: Writing down everything a program is expected to encounter and how it should react is a poor way of modeling reality. This approach hits hard limits with complex problem domains like understanding what people are saying, translating from English to Mandarin, or recognizing objects and people in photos.
Early face detection was an algorithm in the guise of an AI model: The Viola-Jones face detection algorithm was state of the art for teaching a computer to detect faces in a photograph at the turn of the 21st century. It was programmed to look for features found in human faces – eyes, lips, mouth, and nose. It is looking for a bright T-shaped pattern with dark semi-circular patches below it, corresponding to the T-zone of the forehead and nose and darker eyes.
The image above is from an excellent explanation that develops the intuition behind the Viola-Jones algorithm.
The algorithm worked well for its time, but its limitations are painfully obvious. They will only determine faces where people look directly at the camera, among other restrictions.
Teach me, don't tell me: AI gets computers to do things by teaching them. AI is trained, not programmed, to do something. This is similar to saying, "This is a dog" or "This is a cat" to a child when you see either a dog or a cat. The child learns to distinguish cats and dogs and can later spot the difference, even if they are looking at cartoon representations or soft toys.
The model is a closed, black box: The heart of AI, the model, is impenetrable and unknowable. Models learn their domains' words, vocabulary, grammar, and language. They are not acting on a set of rules but ideal outputs. Fixing AI models when they produce the wrong things is different from improving algorithms.
Random but predictable: AI models are inherently random. They might not appear so since the mathematics that drives them forces them to produce predictable outputs (in a narrow range). This forces us to think about correctness in very different ways.
Mathematical note: Random but predictable by a probability is mathematically modeled as a Stochastic process. A system is stochastic if each observation is unpredictable, but over time you can predict, with some likelihood, the range of outputs.
For example, consider flipping a coin. Every time you flip the coin, the outcome is either heads or tails, which is random and uncertain. As you flip the coin multiple times, the sequence of heads or tails forms a stochastic process.
Another example is the random walk. Imagine you're taking random steps, either forward or backward, where each step is a random event. The sequence of forward and backward steps over time is an example of a stochastic process.
In both cases, while the individual events (coin flips or steps) are random and uncertain, the overall pattern or trend of the events can still be studied and understood through probabilities and statistics.
AI adapts to changing conditions: If you programmed a computer to tell a dog and cat apart, it wouldn't be able to adapt when you ask the same question using toys instead of real animals or when the lighting condition is not so good. While the Viola-Jones face detector won't work on photos that don't look directly at the camera, an AI model can quickly learn that. An AI model can give you the wrong answer, like how a young child might get confused between an image of a dog and a cat.
Bias is also learned: AI models can fail in various ways, often by learning to discriminate against some people or using hurtful and toxic language. AI models can also veer off course over time as the input data changes or if the interpretation of that data changes. Fixing these mistakes is about recognizing errors, which are not always obvious, and then either retraining or finding a better model. The advanced models are like black boxes and don't allow an easy understanding of the inner workings. Most car drivers don't have any knowledge of how engines work but are still able to drive to get to work.
AI models are like children: Any parent knows that a child will exhibit the full range of behavior that they think won't get them in trouble - in other words, correct. It is often useless to figure out where the child learned terrible words and behavior. It is better to try to correct that instead of trying to debug it.
There are fundamental differences between algorithms and AI models. These differences are essential for programmers and developers and everyone in society.
We will explore these differences and their implications next week. As the French statesman George Clemenceau said, "War is too important to be left to the generals.".