Stock Market Manipulation By Millisecond Trading 624
cfa22 writes "Nice piece in the NY Times today on ultra-fast trading on the NYSE and other markets. The 'algos' that make autonomous trading decisions have to be fast, but I wonder: Is network speed ever a bottleneck? Can anyone with inside experience with millisecond trading provide some details for the curious among us regarding hardware architectures and networking used for such trading systems?" According to the article, high-frequency traders generated about $21 billion in profits last year.
Primarily Fiber Channel Networking (Score:2, Informative)
Re:Profits, but for whom? (Score:5, Informative)
In theory it adds liquidity to the market and reduces trading costs for other players.
Also, many of the successful trades will be based on having better information than the market price (over time, it would be very difficult to have much success by being lucky, especially with multiple automatic systems active), the execution of those trades makes the market price more accurate.
A profitable subset of "algorithmic trading" (Score:5, Informative)
I work in the finance industry, and know a few things about this business. It can be very profitable indeed. Since the HF trades are typically finished at the end of each day (or even minute), they are not required to hold much cash (capital) to support their positions. Thus the business is unusual in the finance world for making a profit on, essentially, zero capital. Of course, it costs a lot of money to stay in the arms race.
The article hints at two kinds of HF strategies, and they really are distinct. First, there are the "rebate" strategies that collect those credits for providing markets. Then, there are the "predatory" strategies that try to find the price points of buyers and sellers as described. Other HF strategies include pairs trades (Exxon goes up so RIG will soon), inter-exchange arbitrage where a stock is traded on multiple exchanges, and index arbitrage such as trading the elements of the S&P 500 against the index futures (which has been around almost forever).
Other algorithmic trading includes strategies meant to take on positions slowly (or quickly) and efficiently. A famous old category are the Volume Weighted Average Price (VWAP) strategies that try to trade a little bit at a time throughout the day, so that the average trade price is close to the day's average. Other algos try to take advantage of mean reversion or trends during the day.
There is huge demand for technical people in this industry (I probably get one headhunter call every two weeks), almost all of it in NYC or Chicago. There's demand for network engineers, statisticians, programmers, and traders, and high pay for quality. Surprisingly few programmers these days are really acceptable to the business, because the code has to be so fast and efficient, and almost no one studies that any more.
The short answer (Score:5, Informative)
Most exchanges aim for that kind of speed now, but fail to make it. Some of them, like the London Stock Exchange, http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform [computerworld.com], which made the idiotic mistake of relying on Windows Server and SQL Server, don't even come close to delivering that kind of performance.
For those that come closest, the servers tend to be transaction-optimized RHEL (Red Hat Enterprise Linux) and Solaris. The networks are fiber optic-based. While they may connect to the Internet, the core systems, like those provided by AboveNet, are usually private 10GBe networks. In short, to really take advantage of this kind of high-speed trading you're not going to be doing this from your basement. You need to have a trading station either co-located at the market, or just down the street on a high-speed network no more than a link or two from the exchange's servers.
And, yes, network speed does matter here. So does server, storage and DBMS access speed.
Needless to say, none of the exchanges are exactly forthcoming about what their particular magic technology formula is since being able to deliver high-speed trading consistently has become an important sales point. I know many traders on Wall St. and the City in London who will move from one Exchange to another based purely on their ability to deliver faster trades. For this group, what's being traded is besides the point. It's all about keeping an edge in trading speed over their competitors.
Steven
Re:Human reaction bottleneck (Score:3, Informative)
It seems to me like any potential for exploiting millisecond delays in transaction transmission will be consumed and defeated by the time it takes a human operator to interpret the information and hit the "confirm purchase/sale" button.
That assumes there is still a human in the loop and not a computer doing trades autonomously. Within certain bounds, of course. But still autonomously.
Re:Human reaction bottleneck (Score:5, Informative)
Speed is important (Score:4, Informative)
Bogus artilce by clueless arts graduate (Score:5, Informative)
This is an outstandingly bogus article, what happens when arts graduates attempt to understand anything except celebrity gossip.
>It is the hot new thing on Wall Street,
The first algotrading I encountered was in the early 1990s at Deutsche, and they senior guys there told me of some of the mid80s stuff they'd done.
Not new.
>a way for a handful of traders to master the stock market,
Although algotrading is not exactly mass market, it is about as exclusive an activity as getting drunk.
>peek at investorsâ(TM) orders
That's not algotrading.
>and, critics say, even subtly manipulate share prices.
A major topic in algotrading is actually market impact modelling, ie working out how to make prices move less when they trade.
>Nearly everyone on Wall Street is wondering how hedge funds and large banks like Goldman Sachs are making so much money so soon after the financial system nearly collapsed. High-frequency >trading is one answer.
Actually they're working out how to employ these guys since they've often been shafted by the GS bonus scheme.
>software that a federal prosecutor said could âoemanipulate markets in unfair waysâ â" it only added to the mystery.
He was fed this line by GS as part of a dispute over an algotrader's pay. There is no suggestion that GS was being "unfair", just addressing the issues caused by GS having an inferior tech infrastructure for AT than may competitors.
>"This is where all the money is getting made,â said William H. Donaldson".
I'd love to see the original quote before editing. Bet it was a lot less sexy.
>For most of Wall Streetâ(TM)s history, stock trading was fairly straightforward:
For an arts graduate the writer is terribly ignorant of history, as well as trading.
For instance is he not aware of how the Kennedy family got rich as part of causing the crash of 1929 ?
>Joseph M. Mecane of NYSE Euronext, which operates the New York Stock Exchange. âoeMarkets need liquidity, and high-frequency traders provide opportunities for other investors to buy and sell.â
It's more complex than that. Some ATs are consumers of liquidity, others provide it, and there exist models that imply that heavy AT activity drives liquidity away.
>Average daily volume has soared by 164 percent since 2005, according to data from NYSE. Although precise figures are elusive,
164% is a surprisingly "precise" figure, which PR bunny fed that to him ? Wonder why the PR wouldn't give more ?
>"The result is that the slower-moving investors paid $1.4 million for about 56,000 shares, or $7,800 more than if they had been able to move as quickly as the high-frequency traders."
That's really quite amazingly precise. So precise that I believe not a word of it.
Edit/Delete Message
Re:Performance is a driving issue (Score:3, Informative)
Most of the safety innovations for private cars came from the racing car industry - roll bar cages, seat belts, crumple zones, safety glass. Roll bar cages and crumple zones and safety glass have been into train carriages as well.
Re:Great future (Score:5, Informative)
I don't follow "when traders trade in more than enough for everyone.".
And really, people worked far harder 150 years ago than they work today (farming using animal power is not easy), and I'm pretty sure the common man in the U.S. worked far harder 50 years ago than he does today (working in a factory is 'harder' work than sitting at a computer). People complain that they just can't get ahead, but people drive newer, bigger cars and live in bigger houses and eat better food and buy more crap and on and on, so just looking at the fact that families seemed to have switched from 1 income to 2 is not sufficient.
Re:An abuse of the free market system. (Score:3, Informative)
And it doesn't add value to society. If they generated 21 billion, then 21 billion was necessarily lost by others.
Actually, the stock market is not a zero sum system.
When people and mutual funds lost all their 401K value (billions even) in 2008, no one actually got that money.
Not the hedge funds.
Not the short sellers.
Not the government.
It simply vanished.
Which is why had major deflation.
The issue is that the stockmarket doesn't make money like say the Federal Reserve prints it, but in a sense it basically saying how much it is valued by what people would theoretically pay for it.
So when you have a $1 share that turns into $10 a share, you didn't actually earn that money. It is all theoretical until you sell your shares.
Now if it was at $10 and everyone sold their shares at once, not everyone is going to get $10 because the market has to queue everyone up for sell and buy orders so that theoretical money will not actually be delivered to everyone and someone will get the short end of the stick.
So no... By them creating theoretical money does not one else loose money (except of course shareholders who didn't sell their shares when the stock collapsed)
Re:Some info (Score:3, Informative)
Yeah from some of the stories I have been following on this, it seems some firms even co-locate their machines in the same room with the NYSE trade systems. I imagine that could be quite an advantage over other traders, especially when coupled with some extremely high performance program trade code like Goldman Sachs has been using.
http://www.reuters.com/article/fundsFundsNews/idUSN0518022220090705 [reuters.com]
Re:Profits, but for whom? (Score:5, Informative)
It goes like this. For a given security there are two prices the BID (what you're willing to pay for more of a stock) and the ASK (what you're willing to sell some of the stock you already have for). Most normal folks have only one up at any given time, you're either buying or selling. Then, if a BID equals an ASK you've got a trade.
There are entities in the market known as "market makers" who do both at the same time, however. They're usually not in the market to load up on a stock or unload their stock, so they try to balance the number of orders filled on a given security by raising their ask or dropping their bid in order to maintain equilibrium. Let's look at an example of an inefficient market without adequate liquidity. Let's say there's only one market maker -- he might bid $1.00 but ask $10.00, and clean up on anyone who tries to do business in his market.
So my hypothetical firm sees this huge gap and says, "Sweet. Let's start making market in here as well!" -- and I pad my bid a little bit, say $1.05 and shave a bit off the ask, call it $9.95. All of a sudden the original market maker is cut out - none of their bids end up buying stock, none of their asks end up selling it. So they decide to close the gap even further. 1.10 and 9.90. My firm doesn't like getting cut out of this sweet action so we close the gasp further and you have an arms race towards price discovery where the gap between the bid and the ask are separated by a very narrow margin, constantly being adjusted by what actual demand for the security is like.
Having multiple folks providing liquidity in the market means when you decide you're going to load up on STCK because you think their new product launch is going to do good things for the company your orders are unlikely to drive up the price very much so you can enter the position without paying a big premium to do so. Likewise, a year later you decide you're satisfied with your profits and are ready to exit the position. In a market without this liquidity selling your stock would drive down the price further and cut into your profits on that side.
Efficient price discovery is the purpose of the market. Liquidity is essential to that end. Traders provide that liquidity and, overall, the market benefits because of it.
Microseconds, not Milliseconds (Score:3, Informative)
Re:Great future (Score:5, Informative)
Perhaps from an ever-widening gap between the rich and poor and a destruction of the middle class that is directly related to income taxes not being progressive enough?
As someone who has gone through law school and met a lot of people from wealthy families, I've learned that there are nearly sure-fire moneymaking schemes out there that only wealthy people can buy into. International arbitrage is where it's at. An interesting moneymaking pattern is this: The Indian government apparently currently is incentivizing power plant construction by guaranteeing an X% rate of return every year for Y years plus recoupment of complete investment costs. X is significantly higher than the investment plans available to the hoi polloi because the investor has to be willing to front the initial multimillion dollar investment of building the plant.
So wealthy people sink their $30M or whatever into building one of these and, provided the Indian government doesn't go tits up, the investors get ridiculous returns on their investment with little to no risk.
There are tons of other opportunities like this out there that wealthy families take advantage of on a regular basis. I've heard the stories and I know the involved parties. I just can't afford to participate because of the costs of entry.
Re:Profits, but for whom? (Score:5, Informative)
The housing market is non-liquid. There have been only a half-dozen houses sold in my neighborhood over the last three years, and none of those houses is quite like mine. So nobody knows exactly what my house is worth. "Appraisers" are just professional educated guessers, who try to use judgment and experience to replace the pricing information provided by a liquid market. And as we've seen, a non-liquid market can be very volatile. Also, a house is a non-liquid asset; even a simple house sale to a family member takes weeks and hundreds of dollars to arrange.
But there are some securities (private placements, for one example) that trade very rarely. There actually are appraisers for these types of securities, because with so little trading activity the market price can become stale. These are very non-liquid markets. But as long as you can find a buyer and are willing to accept the buyers price they're no less liquid than IBM stock. You sell them, the trade clears, you get cash the next day. So they're liquid assets (more liquid than a house, in any case). And it's not even meaningful to talk about volatility in these markets because there is so little trade activity.
Re:Profits, but for whom? (Score:3, Informative)
That's the problem with the stock market in general. Stable profit to the company means 0 gain for shareholders. The only way a stock is valuable is for the company to take risks and get bigger and bigger and bigger every year.
Re:Profits, but for whom? (Score:2, Informative)
Ultra fast trading is an interesting idea and done right it can lead to successful short term returns
Ultra fast trading might be an interesting idea, but making a whole lot of profit trading stock because you have access to some information before it is made public (even if just 30 milliseconds before it is made public), isn't that a form of insider trading?
The only reason those ultra-fast traders make that much money is because they have access to information before every one else, before it is made public. I don't see how that can be legal.
Re:Profits, but for whom? (Score:5, Informative)
What you are describing is the market before these high frequency trading systems were developed. Everyone has access to the same information, they process it to make a decision on the current and projected value of a stock, and they place orders accordingly. Moving from the ancient floor and telephones to an electronic system with internet brokers increased the speed of the system and the number of players. And these high frequency trading systems do the same thing, but they do something else as well.
1) They use speed and volume to muscle in on traders who have performed their analysis and placed their orders and take a portion of the profits the slower traders may have acquired through their analysis and order. They didn't add liquidity, efficiency or even set the market price, they are simply stepping in between the buyers and sellers and creating multiple unnecessary trades to ensure a portion of the profits from those trades end up in their pockets.
2) And the systems are used to manipulate the pricing of stocks by placing fake orders that are removed before the orders can be acted upon because there was no intent to buy or sell in the first place, just to put some numbers on a board to force a reaction.
What it comes down to is you have a small group that have created a shadow market of orders that most traders never see and the use of systems to place and remove bogus orders on the real market to manipulate prices. I have watched NASDAQ level 2 quotes over the years and it has reached the point where today they are meaningless as the orders you think you see are not a reflection of the true orders hiding in the shadows and you can watch massive blocks of orders appear and vanish for no obvious reason as the stock price changes. In fact, I swear you can sometimes see orders flying back and forth as the high frequency trade systems of competitors wage a cyber trading battle on the open market. NASDAQ is actually paid to provide access to that information and it sure looks like its just a scam at this point.
Re:Profits, but for whom? (Score:3, Informative)
No. They make money by taking advantage of public information and trading BEFORE other people act on that public information.
There is a short time window between the time of disclosure of public information and when people start to act on that disclosed information.
No, it actually all happens before the information is public. Here's some info from the article you didn't read :
While markets are supposed to ensure transparency by showing orders to everyone simultaneously, a loophole in regulations
allows marketplaces like Nasdaq to show traders some orders ahead of everyone else in exchange for a fee.
(Emphasis mine). So they act upon the information they got before every one else.
With every winner, there is a loser. (Score:5, Informative)
"high-frequency traders generated about $21 billion in profits last year"
Many of those trades took money from the guy who comes home at night and trades for his 401K. With every winner in the stock market, there is a loser, and it is big banks like Goldman Sachs [rollingstone.com] that are usually winners.
I suspect that talking about "market liquidity" is just avoiding the issue: The U.S. financial system is corrupt.
Re:Profits, but for whom? (Score:3, Informative)
Don't wait for "we", its up to you.
The people who are actually the means of production generating the wealth need to stop over paying the 2% for the products the 98% produced and the 98% need to stop working for wages that are well below what their production is worth to pay the 2%'s wages that are well above what they actually produce.
Easier said than done but it can be done with a lot of pain, paying off debts, cutting expenses, building savings, quitting jobs, refusing employment offers, starting personal businesses, etc.
The only way to stop the 2% is to cut off their supply of revenue and production.
Re:Profits, but for whom? (Score:2, Informative)
And the market is ALWAYS hungry for more suckers. It's the only legal Ponzi scheme out there.
I think you are sadly mistaken. Incase you were wondering, FDR created the largest, legal Ponzi scheme ever - Social Security. :)
Re:Welcome to 1992 (Score:3, Informative)
Correction: I'm told I'm behind the times. The current advantage is in microseconds, not milliseconds.
Re:Profits, but for whom? (Score:4, Informative)
Explain to me how not doing the bailout would have destroyed mainstreet? Oh, a bank fails? FDIC is there for that. Loans get harder to get? Happened anyways. Unemployment goes up? Happened too. The only thing the bailouts did was keep the rich guys who gambled and lost....still rich.
This also ignores the hundreds of small local and regional banks that are absolutely fine because they properly managed their risks.
Re:Profits, but for whom? (Score:3, Informative)
Treasuries have low volatility because the prices are based on interest rates, inflation, and supply/demand. Those numbers change slowly and have a very predictable impact on prices.
Tech stocks have high volatility because the prices are based on things like market acceptance (will a new product be the next iPod or the next CueCat?), effectiveness (will the promising new drug work in people?), legal environment (will this new file-sharing service be made illegal to protect the RIAA?), and investor sentiment (will a lot of people want to jump on this bandwagon in a few months?). Since it's all based on guesses the prices can move as fast as people's opinions change, and the "plausible" range of prices becomes bigger, allowing for bigger swings.
Using housing prices as an example, it's easy to calculate mortgage interest, rent, taxes, insurance, etc. prior to buying a house. The market got crazy when we increased the demand for houses with lax lending policies, which pushed up prices, which attracted speculators, who started buying houses based on what they thought other people would pay for them rather than for what they were worth. The volatility (both the rise and the fall) was caused by lots of investors guessing everything would go up, followed by lots of investors guessing they would go down. The fundamental value of a house as a place to live or as rental property never changed much at all. (Which is why so many of the non-speculative fundamental home buyers just walked away from the market until the speculators went away, and are coming back into the market now.)
Only Affects Technical Trading (Score:3, Informative)
Whenever a stock is deep on either side of the price elasticity curve, the influence of fundamental traders will present too much of a "noise" signal for technical manipulation to be effective. I'm writing up a on this subject here. [blogspot.com]
Re:Profits, but for whom? (Score:1, Informative)
Perhaps it is a good thing the creditors and investors suffering repercussions from a collapse. It will make them reconsider investing in such a busines model in the future. High returns are not without high risk, and any investor should understand this.
In terms of wealth distribution, say hello to the rest of the West. I am from Australia, and we have sufficiently socialist government policies that all citizens can get basic healthcare, and finnancial assitance in times of hardship (unemployment been one of those times). There is still the reward of earning more for working harder (as there should always be), but there is a much smaller disparity between rich and poor than in the USA. This allows people to not be as limited by circumstance of birth.
Re:Explanation must agree with the facts. (Score:3, Informative)
Not really. There's a *very* large pool of regular investors and smaller houses that are trying the day-trading thing, and that's the group that's losing out to automated sub-millisecond trading. Day trading is *dangerous*, and better than half of the people who try lose a lot of money to it.
This is why there are rules about how much money you have to keep in reserve if you get classified as a day trader, to try to keep people from seeing their entire life savings vanish. Currently, day-trade flagged accounts have to keep at least 25k in the account.
For most long-term investors, the rapid trading really has very little effect, because they're not really playing the same game, so they're not the ones paying such a price.