Market Data Firm Spots the Tracks of Bizarre Robot Trading 483
jamie spotted a fascinating story at The Atlantic about "mysterious and possibly nefarious trading algorithms [that] are operating every minute of every day in" the stock market:
"Unknown entities for unknown reasons are sending thousands of orders a second through the electronic stock exchanges with no intent to actually trade. Often, the buy or sell prices that they are offering are so far from the market price that there's no way they'd ever be part of a trade. The bots sketch out odd patterns with their orders, leaving patterns in the data that are largely invisible to market participants."
Spotting the behavior of these bots was possible by looking at much finer time slices than casual traders ever see — cool detective work, but as the story points out, discovering it is just the beginning: "[W]e're witnessing a market phenomenon that is not easily explained. And it's really bizarre."
Re:Here's an explanation for you: (Score:5, Informative)
Karl Denninger [market-ticker.org] has been reporting this problem [market-ticker.org] for a few years now.
Re:Here's an explanation for you: (Score:5, Informative)
Re:A Solution to this and the eBay 'sniping' probl (Score:2, Informative)
The reasons are actually well known (Score:5, Informative)
Re:Nope, it's right on (Score:5, Informative)
Why should Americans have all the fun? Could be Chinese bots... I hear they like money, also...
Lower ping times helps these bots a lot.
Re:Is there a chance (Score:1, Informative)
Re:Nope, it's right on (Score:5, Informative)
I don't see what is the mystery here. If two people are negotiating a price, and both of them have a hidden high/low price for which they are ready to settle, then the dominating strategy in a game theory sense is to move your price by the smallest step possible. That way, you always hit your opponents price that is best for you and worst for him.
Of course, in face to face markets, this is insulting:
http://www.youtube.com/watch?v=3n3LL338aGA [youtube.com]
but, we are talking bots with a really low ping here. And that's what those patterns are.
At least those with increasing prices by one cent. Those where the bids are going down don't fit this explanation.
Re:I mostly agree! But let's soften it a little. (Score:5, Informative)
Once you abandon the idea that the market gives a damn about the solidity of retirement accounts or the portfolios of the masses,
Easy to "abandon," since that was never the purpose. The stock market exists to marry investors' capital with business opportunities and to provide an easy means for selling and buying ownership shares of corporations. Corporations use the stock market to raise capital. Individuals or organizations use it to buy/trade ownership of corporations. That's it.
The stock market is not designed to be a retirement savings device.
Re:Nope, it's right on (Score:3, Informative)
Re:A Solution to this and the eBay 'sniping' probl (Score:3, Informative)
I've never really understood the complaints about eBay sniping.
I suggest you spend more time considering the issue.
Set your maximum bid at the actual maximum that you want to pay. Whether someone snipes or not, if your bid is the highest you will win. If it's not, you won't.
But this a suboptimal strategy that will result in you paying more for the item than you could otherwise get away with. There is a psychological and competitive aspect to bidding, that induces people to up their bids. By bidding your maximum and then leaving the following will often occur: (Say you bid $100.00)
Here's typical scenario... ... yeah, what's another $5, and bid again. ooops outbid by you to $56. Again... what's another couple bucks... oops outbid again at $57. They give up and wander away. You win the auction, at $57.
Another Regular Person X bids against you, $50, and sees that you've outbid them at $51. They think to themselves, $52
But if you had sniped, Person X would have bid $50, saw they were top bidder and walked away. You come in and snipe $100 at the last second and you walk away with a winning bid of $51. Not sniping cost you an additional 12%. That basically amounts to a stupid tax on your proposed bidding strategy.
Meanwhile from the sellers perspective, they hate sniping because they "lose" money. The auctions end before the true 'maximum' bid is allowed to be discovered. That 12% you would have saved by sniping is 12% the seller would have gotten.
So regular buyers and regular sellers both are irked by sniping, while the only people who benefit are snipers. The entire point of an auction system is to place goods into the hands of the person willing to pay the highest amount. In economic theory an auction is a 'perfect market' where demand and supply meet exactly. Sniping distorts it by enabling auctions to end before the true price is properly set.
I'd think that the simplest solution would just be to extend the auction slightly every time there is a new high bid. Add 5 or 10 minutes every time the bid increases, and sniping would be totally ineffective.
I also suggested this to ebay 10 years ago, as a simple fix. Technically, I'd say 5 or 10 minutes isn't enough. In practice the auction should probably be extended an extra day so that all interested parties have time to check and revise their bids. (If an auction ends at 3am, having a window of opportunity to revise my bid until 3:10am isn't really enough. You need enough time for participating parties to receive their email notifications that they've been outbid, and to come back and update if they wish.)
Some people have argued that this would extend an auction indefinitely, but I disagree. I would however, bump up the bid increments to help prevent auctions from being drawn out. If a Pez dispenser is going to sell for $1.10, dragging it out another day so someone else can bid $1.20 is just stupid.
Now some sellers value having a fixed closing for auctions for whatever reason and for them... implement a silent auction where all bids are held in secret until the end.
Re:Correct the market (Score:2, Informative)
Those are features of an automatic trading system, not of high frequency trading. As I wrote, high frequency trading is an abuse of that system. You can have one without the other.
Re:Free Market = good; Capitalism = Usury (Score:5, Informative)
Re:High Frequency Trading Should Be Banned (Score:5, Informative)
In the absence of sensible regulation there are many abuses of the "free market" that effectively destroy it and turn it into a rigged game to benefit the already rich and powerful. Monopolies. Cartels. Price fixing. Trading on one's own account ahead of a customer.
Or we could do nothing and not fix a non-problem. After all, the market currently is far from "destroyed". "Monopolies, cartels, price fixing, trading on one's account ahead of a customer"? If any of those exist (for example, there aren't any monopolies resulting from high frequency trade), then all you have to do is develop your own high speed market program and profit from the opportunity. Or only trade with brokers that have passed some sort of fairness audit (if you desire fairness over profit).
These special access high-speed connections to the stock market exchange are market fixing tools, pure and simple. They allow the trading firms to skim the market for their own profit, thus defrauding every market participant in the world who lacks these powerful and privileged tools.
Once you strip the needlessly negative connotation from the above statement, it reads a bit differently:
These special access high-speed connections to the stock market exchange are market making tools, pure and simple. They allow the trading firms to provide, for a profit, extremely short term liquidity and price information, thus aiding every market participant in the world who is trying to sell large orders and who lacks these powerful and costly tools.
Requiring all buys to be held for a "long" time (a minute?, an hour?) would kill a lot of these shenanigans. Also requiring the link to go through a regulated buffer that introduces a random delay of a second or so would also take the wind out of their sales (pun intended). Or maybe we just impose a fee on each transaction so that they aren't free. Sub-millisecond trading loses a lot of luster if you automatically incur a charge equal to 0.1% (or something) of the stock's value.
Why would we want to kill these "shenanigans"? And why do you think a delay would stop the shenanigans (rather than introduce bizarre oscillations and such into the stock market).
Re:Here's an explanation for you: (Score:4, Informative)
A smart lobsterman will not sit idly by but will sell futures on his haul (before he leaves0 at $3.75, guaranteeing him that price (up to some quantity) instead of trying to sell on the spot market. Or he'll refuse to sell to the trader at $3 and hold on to the lobsters (they don't go bad overnight ya know) and leave the trader on the hook for his put option with no supplier. In fact, he can probably gouge the trader out of $4.50 because the trader absolutely has to make good on his contract or else face a fairly stiff penalty (unless the buyer is a rube).
Any way you look at it, the trader is screwed. He has no leverage and no arbitrage. The only he has is an obligation to sell something that he may not be able to deliver.
Re:I mostly agree! But let's soften it a little. (Score:4, Informative)
I didn't mention: the stock market also gives the true investor an opportunity to depress the amount he invests in a company relative to its potential value in the stock market. So he can rip off both ends.
This is astonishingly lucrative. That is why you will never be allowed into that club. Anything that has real ROI is reserved for people who actually compete at it. By the time the shares reach the secondary market (the exchanges) there is no profit margin to be had in known conditions, and any trading is speculation on imagined future events.
By entering the stock market, you are entering a casino, with rules on the conduct of the game but no rules on the setting of odds, and no information on the actual odds.
Speculation is just as rampant in the real estate market as in the stock market, but that doesn't make owning a home a gamble.
Two things about that: 1. at least you have a house, even if you overpaid for it. try getting a bank loan on a pile of stock shares. bankers know how the stock market works, and will not even give you a loan against the "par" value. 2. it was rampant speculation that led to the 2006-2007 bubble in real estate. a third of all sales were "investment" sales; i.e., to people who never planned to occupy or rent or significantly improve on the property. hundreds of thousands of people who honestly were intending to live in their new homes found out that the houses they bought were indeed caught up in a massive gamble, and they are making mortgage payments on a house that's worth as little as half what they paid for it. it's the biggest gamble they ever took, and many of them didn't even know it.
Re:Nope, it's right on (Score:5, Informative)
Hell, even Jersey bots are out of luck.
NYSE (Arca) is already in Weehawken, NJ, and everything (including NYSE proper) is moving to Mahwah, NJ, beginning Monday, 2010-08-09.
Re:Nope, it's right on (Score:5, Informative)
At least those with increasing prices by one cent. Those where the bids are going down don't fit this explanation.
And that is what this junk is, completely bogus bids with no intent other than to cost your competitors clock cycles.
I worked for a couple of years at one of the big trading exchanges in Chicago. Our offices were on a lower floor, and whenever our traders got off the elevator, coming back from lunch, they would hit all the floor buttons to delay the traders returning to the higher floors, and anyone else unlucky enough to be on the same elevator. But that was one of the minor reasons that I quit that business sector. The piles of spilled cocaine on the bathroom floors, and my boss asking me "Do you love money? I love money. In order to be in this business you have to love money!" were two others.
Re:finally adjudicated? (Score:4, Informative)
Stocks must be liquid for markets to work at all efficiently.
Liquidity is a result of an efficient market. Liquidity is not a driver of efficiency.
-Rick
Re:Nope, it's right on (Score:3, Informative)
you can colo in the exchange (for a price). moving the profits does not need to be fast...
Re:Here's an explanation for you: (Score:2, Informative)
What you fail to include is the price that the trader paid for the $4 put option. Likely a good sum since the current price is $4. Additionally, all the trades are executed at $4 and are not completed on an exchange (they are contracts). How does that drive the price down to $3??? Sellers of puts and calls are typically speculators and not consumers of the underlying product. If the trader had advanced information of significant supply changes (in the lobster case, a supply increase), he would likely sell calls at the current market price and safely pocket the option premium since the market price would likely decrease. Even this advantage is fleeting as significant option volumes will affect the exchange prices.
Re:Nope, it's right on (Score:3, Informative)
Re:The reasons are actually well known (Score:5, Informative)
It actually doesn't need to be that nefarious (of course, it could be).
You see a lot of this sort of thing in the derivatives markets (I work in the industry - I was the lead architect of the CBOT's Order Routing System) and it's caused by auto-spreaders. A spread trade in derivatives involves finding a pattern between two or more products and trading the differential in prices (e.g., the March Corn contract and the June corn contract tend to move largely in sync, but the spread between them can grow and shrink, so you combine a buy and a sell when the spread narrows and a sell/buy when it grows again).
Most of the easier kinds of spreads are handled natively by the exchange trading engines - they imply prices into and out of the underlying contracts and trade the package of contracts as an atomic unit. But someone who wanted to trade non-standard spreads (like those across exchanges, for instance NYMEX energies vs. ICE energies) has to do it differently - you have to create a synthetic spread by watching the prices of the underlying products and "legging in" the different products you want to buy or sell when your price target is reached.
The easiest and least sophisticated way to do this is to wait until your prices all line up (say you want to buy the NYMEX Oil contract for 10 cents less than you sell the ICE Oil contract for) and then throw in market orders. Then you wait for the spread to move and throw in market orders when you're in the money (you sell the NYMEX Oil contract for 12 cents more than you buy the ICE Oil contract for). Bingo - you make money and you don't really care what either contract was really "worth" - you just care about the differential.
Problem is - market orders suck. The price can move away from you (screwing your differential) and you end up behind all the limit orders that were in before you (increasing your chances of a price movement). So, the smarter way to do it is to place part of your order into the market as a limit order that tracks against the price of the other market. As that market moves, you cancel/replace the leg or legs that are "in the book" so that you stay in sync with your overall strategy. If your "in the book" order(s) starts to fill, you know you've hit your target and you can drop the final part of your spread into its market, giving you a much better chance of getting your differential.
Now, imagine that you are doing a pretty complicated spread (four or five different underliers that all relate in some model you have) - depending on which ones you put into book and who else is spreading slightly different contract combinations, you get a lot of weird orders being inserted, canceled and replaced at prices all over the map. It can appear semi-random, but for each algorithm, it actually is highly deterministic.
I don't know if that's what's going on here, but I wouldn't necessarily rule it out. A number of exchanges (including the CME) are trying to stop this sort of thing, because the transaction volume going into and out of the exchange (and the associated price changes that need to get pushed out) is hugely expensive. So, these days you have to maintain a certain ratio of orders to fills (i.e., don't cancel or replace a lot) or you start to get fined.
Re:Intent (Score:4, Informative)