At a dinner I attended some years ago, the distinguished differential geometer Eugenio Calabi volunteered to me his tongue-in-cheek distinction between pure and applied mathematicians. A pure mathematician, when stuck on the problem under study, often decides to narrow the problem further and so avoid the obstruction. An applied mathematician interprets being stuck as an indication that it is time to learn more mathematics and find better tools.
I have always loved this point of view; it explains how applied mathematicians will always need to make use of the new concepts and structures that are constantly being developed in more foundational mathematics. This is particularly evident today in the ongoing effort to understand “big data” — data sets that are too large or complex to be understood using traditional data-processing techniques.
Our current mathematical understanding of many techniques that are central to the ongoing big-data revolution is inadequate, at best. Consider the simplest case, that of supervised learning, which has been used by companies such as Google, Facebook and Apple to create voice- or image-recognition technologies with a near-human level of accuracy. These systems start with a massive corpus of training samples — millions or billions of images or voice recordings — which are used to train a deep neural network to spot statistical regularities. As in other areas of machine learning, the hope is that computers can churn through enough data to “learn” the task: Instead of being programmed with the detailed steps necessary for the decision process, the computers follow algorithms that gradually lead them to focus on the relevant patterns.
→ Quanta Magazine
In Shakespeare’s Julius Caesar, a soothsayer warns Caesar to “beware the ides of March.” The recommendation was perfectly clear: Caesar had better watch out. Yet at the same time it was completely incomprehensible. Watch out for what? Why? Caesar, frustrated with the mysterious message, dismissed the soothsayer, declaring, “He is a dreamer; let us leave him.” Indeed, the ides of March turned out to be a bad day for the ruler. The problem was that the soothsayer provided incomplete information. And there was no clue to what was missing or how important that information was.
Like Shakespeare’s soothsayer, algorithms often can predict the future with great accuracy but tell you neither what will cause an event nor why. An algorithm can read through every New York Times article and tell you which is most likely to be shared on Twitter without necessarily explaining why people will be moved to tweet about it. An algorithm can tell you which employees are most likely to succeed without identifying which attributes are most important for success.
→ Harvard Business Review
The Gaussian copula is not an economic model, but it has been similarly misused and is similarly demonised. In broad terms, the Gaussian copula is a formula to map the approximate correlation between two variables. In the financial world it was used to express the relationship between two assets in a simple form. This was foolish. Even the relationship between debt and equity changes with the market conditions. Often it has a negative correlation, but other times it can be positive.
That does not mean it was useless. The Gaussian copula provided a convienent way to describe a relationship that held under particular conditions. But it was fed data that reflected a period when housing prices were not correlated to the extent that they turned out to be when the housing bubble popped. You can have the most complicated and complete model in the world to explain asset correlation, but if you calibrate it assuming housing prices won’t fall on a national level, the model cannot hedge you against that happening.
→ The Economist
Writers, remember: the more we play the algorithmic game, the more the algorithmic game plays us. (All hail the great Algorithm in the Cloud!)
Algorithm-oriented content is becoming ubiquitous. It doesn’t matter what you read or what topics you search for, a growing percentage of online material is designed ground-up for the acquisition of ‘Likes’ and the courtship of search engines. Food, politics, current affairs, cats, academia — everything. It doesn’t matter what you are interested in, there is an army of people writing about it with strategic intent to leverage the algorithmic landscape for their advantage.
→ James Shelley
The rise of the machine learning :
The quantitative investment world plays down the prospect of machines supplanting human fund managers, pointing out that the prospect of full artificial intelligence is still distant, and arguing that human ingenuity still plays a vital role. But the confident swagger of the money management nerds is unmistakable. Already there are quasi-AI trading strategies working their magic in financial markets, and the future belongs to them, they predict.
→ Financial Times
In the summer of 2014, Puzz had another puzzle to solve. From March to July, the frequency with which an IEX customer could have gotten a better price less than 10 milliseconds after a trade posted rose from about 3 percent to as much as 10 percent. This wasn’t meant to happen. IEX was supposed to protect investors from what’s known as stale quote arbitrage; that’s when a high-frequency trader takes advantage of milliseconds-long delays in how markets update prices to reflect movements on other exchanges. These tiny delays allow high-speed traders to see a price fluctuation on one exchange and then quickly send an order to another market—often a dark pool—that it knows updates its prices more slowly, hoping to pick off the orders resting there at stale prices. It’s a bit like betting on yesterday’s horse race against someone who doesn’t know the result.
IEX prevents stale quote arbitrage with its “magic shoe box,” a metal container in its data center in Weehawken, New Jersey. Crammed into it are 38 miles (61 kilometers) of coiled fiber-optic wire, creating IEX’s speed bump of 350 microseconds (about one one-thousandth of the time it takes to blink). The idea of countering super-fast traders by creating a slower market might seem like a paradox. It’s not. IEX uses the same high-speed data feeds as HFT firms do to monitor other exchanges for price changes. But because IEX didn’t want to be in a technological arms race with the high-frequency traders to process this information faster than they do, it uses the speed bump to slow down all new orders—just enough to ensure IEX has time to update its prices to reflect any movements on public exchanges. This prevents orders on IEX from being traded against at stale prices.
So how, Aisen wondered, could HFT firms be picking off IEX orders despite the magic shoe box? It didn’t take Puzz long to solve the riddle. He discovered that some HFT algorithms could predict price changes—like surfers sitting out past the break, scanning the swell for their next ride—and target orders before the magic shoe box’s speed bump could protect them.
While in Greenwich Ct. one afternoon I will never forget a conversation I had with a leading quantitative portfolio manager. He said to me that despite its obvious attributes “Black Box” trading was very tricky. The algorithms may work for a while [even a very long while] and then, inexplicably, they’ll just completely “BLOW-UP”. To him the most important component to quantitative trading was not the creation of a good model. To him, amazingly, that was a challenge but not especially difficult. The real challenge, for him, was to “sniff out” the degrading model prior to its inevitable “BLOW-UP”. And I quote his humble, resolute observation “because, you know, eventually they ALL blow-up“…as most did in August 2007.
→ Global Slant