When regular software produces the wrong answer, something has gone wrong that can, in principle, be fixed. A bug is a bug: find it, patch it, move on. AI doesn't work this way. Its most important failure modes aren't accidents. They're features of how the systems are built.
The clearest example is hallucination. When a language model doesn't know something, it doesn't say so. It produces whatever text fits the statistical patterns it learned during training, whether or not that text is true. As Kristian Hammond, director of Northwestern's Center for Advancing Safety of Machine Intelligence, has put it: "They are designed to write. They are just not designed to tell the truth." That's not a bug to be patched. It's a consequence of what these systems are optimized to do.
Brittleness is a related issue. AI systems trained on one distribution of data can fail badly when the real world presents something slightly outside that distribution. A model that handles thousands of test cases beautifully can stumble on a case that looks, to a person, nearly identical to ones it got right. The failure isn't random; it reflects the edges of what the training data covered.
Then there's bias. Because these systems learn from human-generated data, they absorb the patterns in that data, including the inequitable ones. A hiring model trained on historical decisions tends to reproduce historical preferences. A facial recognition system trained mostly on certain demographics performs worse on others. Better intentions wouldn't fix this; it's a reflection of what the training data contained.
None of this means AI is unreliable in any absolute sense. It means AI is reliable in specific, bounded ways and unreliable in others, and the boundary between the two isn't always obvious from the outside. The systems are confident whether they're right or wrong. They don't signal uncertainty the way a careful person does.
Understanding how AI fails is as important as understanding what it can do. The mechanics of why become a lot clearer once you understand how these systems actually process information — which is exactly where things go next.
