As the technology advances, we might soon cross some threshold beyond which using AI requires a leap of faith. Sure, we humans can’t always truly explain our thought processes either—but we find ways to intuitively trust and gauge people. Will that also be possible with machines that think and make decisions differently from the way a human would? We’ve never before built machines that operate in ways their creators don’t understand. How well can we expect to communicate—and get along with—intelligent machines that could be unpredictable and inscrutable?
The result is that modern machine learning offers a choice among oracles: Would we like to know what will happen with high accuracy, or why something will happen, at the expense of accuracy? The “why” helps us strategize, adapt, and know when our model is about to break. The “what” helps us act appropriately in the immediate future.
It can be a difficult choice to make. But some researchers hope to eliminate the need to choose—to allow us to have our many-layered cake, and understand it, too. Surprisingly, some of the most promising avenues of research treat neural networks as experimental objects—after the fashion of the biological science that inspired them to begin with—rather than analytical, purely mathematical objects.
Yitang “Tom” Zhang spent the seven years following the completion of his Ph.D. in mathematics floating between Kentucky and Queens, working for a chain of Subway restaurants, and doing odd accounting work. Now he is on a lecture tour that includes stops at Harvard, Columbia, Caltech, and Princeton, is fielding multiple professorship offers, and spends two hours a day dealing with the press. That’s because, in April, Zhang proved a theorem that had eluded mathematicians for a century or more. When we called Zhang to see what he thought of being thrust into the spotlight, we found a shy, modest man, genuinely disinterested in all the fuss.
In the language of hydraulic engineering, the process eroding the foundation is known as “solutioning.” If that problem is not addressed, what happens next is “piping”: water begins to travel between the voids, moving horizontally beneath the dam. To illustrate, American engineers have devised a triangular chart. The process begins, at the apex, with solutioning, advances through cavity formation and piping, and ends with core collapse and, finally, dam breach—like a Florida sinkhole opening up, unannounced, beneath a shopping center. Engineers jokingly refer to the chart as the “triangle of death.” Schnittker told me, “Once piping begins, there is no going back. In twelve hours, the dam is gone.”
And yet the rise of machine learning makes it more difficult for us to carve out a special place for us. If you believe, with Searle, that there is something special about human “insight,” you can draw a clear line that separates the human from the automated. If you agree with Searle’s antagonists, you can’t. It is understandable why so many people cling fast to the former view. At a 2015 M.I.T. conference about the roots of artificial intelligence, Noam Chomsky was asked what he thought of machine learning. He pooh-poohed the whole enterprise as mere statistical prediction, a glorified weather forecast. Even if neural translation attained perfect functionality, it would reveal nothing profound about the underlying nature of language. It could never tell you if a pronoun took the dative or the accusative case. This kind of prediction makes for a good tool to accomplish our ends, but it doesn’t succeed by the standards of furthering our understanding of why things happen the way they do. A machine can already detect tumors in medical scans better than human radiologists, but the machine can’t tell you what’s causing the cancer.