In the United States, the ranks of journalists keep shrinking. As I travel around the world for The New York Times, I hear from journalists everywhere about the painful downsizing happening across the industry. This has meant important stories go untold. Costly investigative reporting units pare back their ambition in the face of budget cuts. Expensive trips to conflict zones suddenly seem like a luxury publishers cannot afford, and news organizations everywhere rely more and more on wire services to cover the world. This has reduced the vibrancy and diversity of the journalism we consume, and the world is poorer for it. Above all, local journalism has suffered. Cities that once supported two or more daily newspapers find themselves with one, or none at all.
Category: Economics
Big Data’s Mathematical Mysteries
At a dinner I attended some years ago, the distinguished differential geometer Eugenio Calabi volunteered to me his tongue-in-cheek distinction between pure and applied mathematicians. A pure mathematician, when stuck on the problem under study, often decides to narrow the problem further and so avoid the obstruction. An applied mathematician interprets being stuck as an indication that it is time to learn more mathematics and find better tools.
I have always loved this point of view; it explains how applied mathematicians will always need to make use of the new concepts and structures that are constantly being developed in more foundational mathematics. This is particularly evident today in the ongoing effort to understand “big data” — data sets that are too large or complex to be understood using traditional data-processing techniques.
Our current mathematical understanding of many techniques that are central to the ongoing big-data revolution is inadequate, at best. Consider the simplest case, that of supervised learning, which has been used by companies such as Google, Facebook and Apple to create voice- or image-recognition technologies with a near-human level of accuracy. These systems start with a massive corpus of training samples — millions or billions of images or voice recordings — which are used to train a deep neural network to spot statistical regularities. As in other areas of machine learning, the hope is that computers can churn through enough data to “learn” the task: Instead of being programmed with the detailed steps necessary for the decision process, the computers follow algorithms that gradually lead them to focus on the relevant patterns.
Algorithms Need Managers, Too
In Shakespeare’s Julius Caesar, a soothsayer warns Caesar to “beware the ides of March.” The recommendation was perfectly clear: Caesar had better watch out. Yet at the same time it was completely incomprehensible. Watch out for what? Why? Caesar, frustrated with the mysterious message, dismissed the soothsayer, declaring, “He is a dreamer; let us leave him.” Indeed, the ides of March turned out to be a bad day for the ruler. The problem was that the soothsayer provided incomplete information. And there was no clue to what was missing or how important that information was.
Like Shakespeare’s soothsayer, algorithms often can predict the future with great accuracy but tell you neither what will cause an event nor why. An algorithm can read through every New York Times article and tell you which is most likely to be shared on Twitter without necessarily explaining why people will be moved to tweet about it. An algorithm can tell you which employees are most likely to succeed without identifying which attributes are most important for success.
The Man Who Terrifies Wall Street
Bharara argues that publicizing criminal behavior is a public duty, for the purpose of deterrence. “It’s not my job to put out a ten-point program to fix corruption in New York State,” Bharara told me. “Prosecutors alone are not going to solve the problems. But we do want the problems to be solved. I can say that when you have an overabundance of outside income for legislators, when you have an overconcentration of power in the hands of a few people, and when you have a lack of transparency about how decisions are made and who makes them—that it is our job to point that out. We can give these issues a sense of urgency. A lot of people wake up to the possibility of better government when you start putting people in prison.”
Disgraced Trader’s Struggle for Redemption
“It’s not necessarily about money, it’s about winning,” he told a visiting group of American college students. He told them that to understand trading, they needed to forget everything they learned in economics class and envision the amoral, take-no-prisoners world of “The Hunger Games.”
“The only time when people cooperate is to prolong their own lives,” he said. When rivals are no longer useful, “you stab them in the back.”
He told students he had accepted the fact that he was a rogue trader—but in his telling, it didn’t sound all that sinister.
A rogue trader, he said, “is a risk taker. It’s not a crime. It’s violating the mores established by the institution that you work for. It’s a rebellion against institutional controls that deny individuals opportunities for self-actualization.”
Abnormal Capitalist 