When Milliseconds Make Millions: Why Wall Street Programmers Earn the Big Bucks

http://goo.gl/of5FJ

HFT was blamed for the huge drop in the stock market in May 2010, called the "flash crash." The Wall Street Journal reportedthat, "At one point, HFTs traded more than 27,000 contracts in just 14 seconds."

A nanosecond is 1 billionth of a second. One company, Fixnetix, recently announced that its  "microchip for ultra-low latency trading is now the world's fastest trading appliance for the financial markets. Customers are now seeing latencies as low as 740 nanoseconds through the stack (wire-to-wire)."

So what kind of language tools and techniques are programmers using to develop such high-performance systems? I suspected it would be something as close to the metal as possible, perhaps even assembly language for certain parts or some esoteric proprietary language. But some surprises emerged in readers' discussions of the previous article, with languages like Haskell and Smalltalk mentioned prominently.

However, it's no coincidence that Aleynikov was a C++ guy. As HFT expert Matt Davey, chief technology officer at Lab49, a financial services industry consulting firm, explained in an interview: "From a HFT platform perspective, C/C++ is the language of choice due to the latency requirements," he said. "The lower the latency, the more C/C++ is important."

Programmer/entrepreneur Jason Roberts told me the languages/tools used by HFT coders "really varies from firm to firm. Some use the Microsoft stack (.NET, C#, etc.), some use Java or C++ and others use more exotic languages like Ocaml."

Roberts has a unique perspective on the HFT industry, having worked as a founding partner for a series of small, proprietary trading companies and for a hedge fund, developing the trading technology. "I built everything from the order managers and feed parsers to the low latency messaging systems, trading UIs and tick capture databases, machine learning libraries and trading engines," Roberts said. "In most cases, I developed everything (or most everything) entirely myself."

Surprisingly, Roberts said he used C++ sparingly. "I never really used anything very esoteric and, in fact, tended to rely on tools that were fairly common and unsophisticated. For example, back between 2001 and 2003, I wrote two entire HFT trading systems (one for option market making and the other for equity pairs trading) using VB6, which even back then would've struck many as silly. The reality was that the applications worked, they were reliable and they were fast (since VB6 is compiled)."

"Moreover," he continued, "I was very productive using VB6, which enabled me to get a lot more done than if I had attempted to build everything using C++. In fact, I would only use C++ for the very rare algorithm that needed an extra boost of speed, and the one I remember best as being slow in VB6 was the square root function, of all things!"

Mike O'Hara, publisher of the High Frequency Trading Review, also listed the "C" languages as being prominent in the industry, along with Java, Matlab and Cuda. The latter, he said in an interview, is used for programming GPUs for an extra boost in performance, which seems to be an increasing trend (more on that later).

A Slashdot.org reader named "TheRaven64" said in the comments about my highest-paid-programmers article that "I periodically get contacted by recruiters for banks, because my CV mentions Haskell, and there's a massive shortage of Haskell programmers." 

The use of Haskell indicates that the new darling of the software development world, functional programming, plays a large part in the HFT industry. TheRaven64 said in his comments that recruiters who contacted him for possible employment in the financial industry "wanted people with any functional programming experience. A lot of these companies use their own in-house languages, but functional programming is popular because it's easy to verify functional programs, and because languages like Haskell facilitate rapid development. Cincom does good business selling Smalltalk to trading houses for a similar reason. Typically, a small improvement in a trading algorithm gives you an advantage for a day -- maybe less. Being able to go from idea to deployment in under an hour is something that Smalltalk and Haskell give you, and that's something that the financial industry values highly."

When one reader asked about the role of testing in such quick deployments, he was answered by yet another reader who said, "I think that was the point. Writing in one of these languages means that you can prove the correctness of the code, presumably automatically, so presumably very quickly."

Of course, there's much more to the nuts and bolts of HFT than the choice of a programming language. "From an OS perspective, the default is Linux, a custom build that has removed all the fat," Davey said. "From a back-testing perspective, there will be some kind of tick database -- Kdb or similar." But other languages come into play also, Davey said when I asked him about needed skills for HFT programming. "On the UI and data side, C# and Java are relevant, coupled with database knowledge."  

High-performance hardware and a tight marriage with software are also important. A reader on Hacker News with the handle "actionbrandon" noted in the comments that, "From the CS side, some of the more cutting-edge stuff I've heard of being worked on involves hacking graphics cards to get a speed edge and building data centers in the middle of the ocean for the same reason." 

That hardware angle was echoed by Davey: "GPU/FPGA [graphics processing units/Field-Programmable Gate Array] usage is not uncommon from a hardware perspective," he said.

That was confirmed by O'Hara, who said "Cuda is increasingly used for programming parallel processing algorithms [pattern matching] on GPUs. Impulse C is used for programming FPGAs."

But the highest-performing systems in the world are worth nothing if the underlying logic is unsound. Or, as Davey worded it, "Having low latency, but a strategy/model that generates no profit, isn't really going to help, and likewise the reverse."

Roberts agreed. Besides a thorough understanding of data structures, algorithms and concurrency issues, programmers "need to understand what it is they're building," he said. "One of the biggest hurdles for HFT firms is that the programmers, quants and traders don't understand one another, which tends to cause a lot of problems. An HFT programmer who really groks trading and some of the theory and math behind finance will be regarded as extremely valuable and can usually write their own ticket."

and : 또, 만약 그리하면, …와, 및, 그리고