AI Mines Hundreds of Thousands of News Articles Per Hour for Stock Tips

Trading and machine learning were made for each other. Both activities are rooted in analyzing noisy data for patterns and using those patterns for making predictions. The relationship is so much so that you can even take graduate-level college courses focused entirely on the problem of applying machine learning to trading. Surely many an engineer has felt the allure of this "dark side."

Generally, in AI stock trading machine learning is used to build models based on financial data, of which there is no shortage of: moving averages of stock highs and lows, moving averages of trade volume, overall market trends, market volatility, etc. This stuff is all right there and easy enough to translate into features useful for training machine learning models, but stock prices themselves react to more than just historical financial data. The market is ultimately subject to the real-world

It seems like a trivial statement, but when you start looking at trading as a math problem it's easy to forget that stocks are tied to companies that do and sell things IRL. And this IRL-ness is much harder for an algorithm to comprehend, let alone use for making future predictions. Nonetheless, an investment management firm called Triumph Asset Management is making a go of it with a new system that parses large volumes of news articles for indicators that can be incorporated into predictive models.

I probably could have predicted that Triumph's presentation at last week's GPU Technology Conference in Silicon Valley would be well-attended, but the over-capacity crowd spilling out into the convention center hallway was a bit surprising. Never underestimate the dark side, I suppose.

Inside was Triumph data scientist Rafael Nicolas Fermin Cota explaining a framework capable of parsing a news article every 3 ms, chewing through hundreds of thousands of articles per hour, a task that not long ago would have taken weeks. Its quarry is sentiment—a common quantitative metric that, in this case, is based on using natural-language processing to assess whether a news article featuring a particular company is saying something good or bad about that company.

Triumph is able to make meaningful sentiment assessments thanks to the GloVe algorithm, a technique recently developed at Stanford. It works by taking text (such as a news article) and analyzing it for the co-occurrence of words, or the frequency that groups of words will appear next to each other. Co-occurrence statistics are then used to carve out substructures from the text data that basically represent meaning at levels beyond single words. (The naive way of doing sentiment analysis is just to go word by word and say whether it's positive or negative and add it all up.)

After chewing through a whole lot news articles expressing a whole lot of sentiments—and then determining whether those sentiments are likely to be "market moving"—a trader should know whether or not it's a good time to sell or buy a stock. And, according to Triumph, this works with 76 percent accuracy. Not bad.

In related gambling news, AI is now confidently beating humans at poker. Farewell, luck.