Slashdot Log In
Stock-Picking Computers
Posted by
Zonk
on Fri Nov 24, 2006 10:32 AM
from the finally-useful-math dept.
from the finally-useful-math dept.
eldavojohn writes "A while ago, Slashdot ran an article on Algorithms used to augment or replace analysts. Today, the NY Times is running an article on stock-picking computers with quotes from the lovable Ray Kurzweil." From the article: "'Investment firms fall over themselves advertising their latest, most esoteric systems,' said Mr. Lo of M.I.T., who was asked by a $20 billion pension fund to design a neural network. He declined after discovering the investors had no real idea how such networks work. 'There are some pretty substantial misconceptions about what these things can and cannot do,' he said. 'As with any black box, if you don't know why it works, you won't realize when it's stopped working. Even a broken watch is right twice a day.'"
Related Stories
[+]
Technology: Algorithmic Investors on Wallstreet 249 comments
eldavojohn writes "Recently, setting up prediction markets that people play was the big thing to guess the future. But is there a chance that computers will replace investors? From the article: 'Quantitative investment managers use a model to identify sets of characteristics for their investments. Computing power is now relatively cheap. Obviously, computing power can access data almost instantaneously and simultaneously. Asset classes and financial instruments within those asset classes can then be screened and investments are selected. They reflect the manager's views.'"
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Efficient markets (Score:5, Insightful)
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
Re:Efficient markets (Score:4, Insightful)
Anyway, it's already being done, ergo it's possible!
Parent
Re:Efficient markets (Score:5, Insightful)
And this fact is EXACTLY what stabilizes the market!
As soon as there's a discernable pattern, somebody's going to exploit that pattern in order to make more money, and as soon as that happens, the original pattern gets interrupted, thus stabilizing the marketplace. Perfect? No. But damned good. Some regulation is needed to keep these market forces from being overwhelmed - but the cost of this regulation is a pittance compared to the benefits gained!
Money is an awfully effective invention for distributing wealth, which is why the Star-Trek "utopia" where nobody needs money is not going to happen anytime soon. So long as there is differentiation between different people (and thus resource distribution potential) there will be money.
Parent
Re: (Score:3, Interesting)
The communist ideal of Star Trek, as I see it, is possible only because of the replicator.
Consider what happens when it is possible, given an example of a given product, to duplicate it en mass at near-zero cost. Suddenly nearly everything's free. Manufacturing costs nothing, so it's all in the pattern that tells the replicator what to build. T
Re: (Score:3, Insightful)
Yes, but it is even more effective at concentrating wealth.
While you probably didn't intend it, that is one of the benefits of money and banking. Back two thousand years ago, almost no one could plan ahead 20 years except the wealthy. Any savings you might have accumulated could easily be stolen. IMHO, that was one of the reasons land was so valuable. Someone couldn't break in and take it and it still had use even if everything on it was destroyed. Now, anyone can concentrate wealth, not just the extrem
Re: (Score:3, Insightful)
While this thread has passed all bounds of decency or on-topicness, I do happen to think that money still is far more effective at distributing wealth than concentration. First, money forms a small part of the actual wealth out there which also consists of real estate, labor and education, equities and other securities, savings, etc. Sure one can often trade real estate or stock directly for someething else, but these things aren't in themselves money (unless you're about to claim everything with value is m
Re:Efficient markets (Score:4, Insightful)
Sure you can create programs that handle arbitrage opportunities, or detect shortterm effects (market movements lasting less than 1 hour), and these make lots of money for those lucky people who have realtime prices and no brokerage costs (I.e. investment banks, etc).
Stock prices for a company will move on news. Prices may drift around on speculation, but eventually a company will post its trading figures and you will know exactly how much that company is worth at that point in time. Unless these technical analysis programs know which comanies are moving product, who is about to sue who, which companies are in secret negotiations, what the future price of oil will be, etc, then they are going to miss price movements caused by events external to the markets.
Parent
$1 says it won't work. (Score:2)
Trader 1 (on the phone): Buy May belly contracts at...
STOCKBOT:- That's a big mistake, Sir.
Trader 2: Why shouldn't we buy now, STOCKBOT?
STOCKBOT:- The price is going to keep going down.
Trader 1: Randolph, this isn't Monopoly money we're playing with.
Trader 2 (on the phone): This is Randolph. Hold that belly order a moment.
Trader 2: Tell me why you think the price of pork bellies is going down.
STOCKBOT:- It's Christmas time. Everybody
Re: (Score:2)
Re: (Score:2)
While he cannot say the nature of the programmed trading algorithms, he does not that all their best years are when the market goes DOWN, because their algorithms are better picking
Re: (Score:3, Informative)
Um, that IS beating the market: all investments are a combination of risk and expected reward (e.g. treasuries are low-risk and low-return, junk bonds are higher risk and higher return.) If you can reduce risk without reducing return, you can make buckets of money (people will line up outside your door wanting to give you capital.)
That said, don't trust any academic studies on this topic. There are
Re: (Score:3, Informative)
If the Efficient Market Hypothesis were true, stock pickers like Cramer should have been driven out of the market by now. Some investors do, on average, beat the market. See Warren Buffet. Now. The hard part is figuring out if your analyst is the next WB, or just some MBA who isn't too stupid and had good luck on top of not being too stupid... for the last 5 years until he regresses to the mean for the next 10 years. So. If you could write software that picked *analysts* then maybe you'd have somethin
Well it makes a change (Score:3)
Quite futile (Score:2, Interesting)
Anyone who really knows anything about this subject wil not post. Too much going on in trading land...
Hence AC post...
Re: (Score:2, Interesting)
Re:Quite futile (Score:4, Funny)
So in this light, how am I suppose to interpret the rest of your post?
Parent
Seems only reasonable... (Score:4, Interesting)
However, it still can't predict things that a human can (yet). I doubt that a computer can incorporate thigns like global news, company announcements, and other such real world variables into how it makes judgments. That was the one thing that the article didn't really talk about.
So I doubt we will see these "black boxes" replacing brokers, simply suplimenting them.
Re: (Score:3, Informative)
Here's [psu.edu] a guy who incorporated yahoo message boards in his stock market prediction software a few years ago.
Actually I was surprised how few references I could find to this sort of thing. Still I don't believe this is an indication that it's not happening; rather, I think market prediction is a black art because investors don't want anybody else to know
Re: (Score:2)
[Note: the following example is based on my understanding of the stock market, which is most likely wrong]. For example, say you had 1000 shares of Google, cu
Re: (Score:3, Insightful)
Completely, but it's good that you know what you don't know--you'd lose a lot less money trading that way. It's impossible to parse stock news and figure out if it's good or bad. Using your example, if Google announces that it will exceed its previously projected earnings, it's just as likely the stock will crash as skyrocket. This is because market participants are working with earning estimation at
Spam! (Score:2)
It also can't take into account stock-pumping spam, which has graced so many of our inboxes lately.
Humans *really* bad at predictions. (Score:2)
However, it still can't predict things that a human can (yet).
People are really bad at predictions. Managed funds rarely even keep up with simple index linked funds.
I doubt that a computer can incorporate thigns like global news, company announcements, and other such real world variables into how it makes judgments.
No this really should be pretty simple. We already have an algorithm that can classify a message as spam or not spam with 99.5% + accuracy, basically good or bad. The same algorithm could be used to classify any news article as good or bad for a particular stock by looking at the direction the stock takes after the news appears. You could even train the classifier automatically by looking a couple of days
Technical analysis (Score:2)
If it was really possible to predict price behaviour on trading patterns these opportunities would be fully exploited until such a prediction would not work.
Why not simply buy a good profitable company?
Or a company that will be good and profitable?
This would be my request... (Score:2)
Most companies provide free trading stations but the tools/software to predict what might happen in the market cost thousands of dollars, yet these tools are never 100% accurate. So, does anyone know where one can grab Open Source versions of these? Thanx.
Re: (Score:3, Informative)
However, while it's fun to play around with a system like this, I must warn you that the realities of trading make it very hard to profit even if it looks good on paper. You probably know this, since you "got burned" before. Make sure you consult a professional before investing, or I can pretty much guarantee that you'll get burned again.
Re: (Score:2)
Re: (Score:3, Interesting)
Care to post a link? Thanks.
sure.
http://www.npr.org/templates/story/story.php?story Id=6203264 [npr.org]
My own portfolio is a bit simpler. UK index linked mutual fund, developing country mutual fund, government bonds, commodities (gold silver), housing stocks. Basically about 20% in each sector. Try to spread your portfolio over several sectors which don't all go in the same direction at the same time.
The strategy is simple but it's the important bit because it stops you buying at the top of the market. It's called rebalancing.
Every month add t
It Works (Score:2, Informative)
Re: (Score:2)
If you have level 2 access, you can pretty accurately predict where a stock is going. Sure, you don't know from day to day, but there are reasons for market introduced delays and market makers.
Re:It Works (Score:4, Insightful)
Automated trading systems 'generally' are used to take a position in a stock that has already been picked.
So, trader A in Goldman Suchs wants to take a long position (buy) 100K shares of IBM, so he assigns that trade to the algorithmic trading engine, which might offer him various algorithms to help fulfill his position at the best possible price, ranging from %vol, VWAP, 'iceberg' or other type of algorithm.
notice, though, that the trader already had the stock to trade chosen, he didn't let the algorithmic engine choose it for him.
Parent
It's a prime directive issue (Score:4, Interesting)
First, no matter how well you can predict based on patterns, when you are picking individual stocks, there is such a huge influence from the chaos of human nature that, from day to day, no matter how appropriate your predictions are (based on history), they may have nothing at all to do with reality.
Additionally, if you get enough of these stock picking systems in operation, they can actually change the dynamics of the market, keeping them from being accurate for years as they all try to account for the activities associated with each others' predictions.
The problem with stocks is that in order to know how they are going to perform, you have to know not only what the company is going to do and how their customers are going to respond, but also how the investing public is going to take that news. It's an odd mix of fundementals and faith, in my experience.
The problem is markets are choice based (Score:3, Informative)
You wanna use your computer? (Score:2, Informative)
Or you could try genetic algorithms, download the info on the whole market plus historical info, give the algorithms access to the lot, plus downloaded financial news, classify the financial news as good or bad for a stock using a bayesian classifier, add that to the pot and then use evolution to see which algorithms survive best in the market to date.
You may need to build a supercomputer to run enough algorithms to perfo
Mr. Lo is not smart enough to teach at MIT (Score:5, Insightful)
[ Yes, I am joking. I'm quite sure Mr. Lo is brilliant -- just maybe a touch too honest. :-) ]
Re:Mr. Lo is not smart enough to teach at MIT (Score:4, Insightful)
You're assuming that he cares about earning lots of money. More likely, he has enough money, and he wants to do work which he finds interesting -- in other words, he's not turning down the offer because he's honest; he's turning down the offer because he doesn't want to waste 20 years of his life.
Parent
It's harder than it looks (Score:2)
Re:It's harder than it looks (Score:4, Informative)
Parent
The problem is too big (Score:2)
About Markets and mathematics.. (Score:3, Interesting)
The last work of Mandelbort (the 'fractals' father) 'The (Mis)behaviour of markets' http://www.amazon.com/Misbehavior-Markets-Benoit-M andelbrot/dp/0465043550/ [amazon.com] is quite interesting.
Sigh, markets are chaotic, much more chaotic than current market analisis states.
Program trading is the new day trading (Score:5, Interesting)
In the industry it's called "program trading" and refers to automated, algorithmic trading of instruments such as stocks, futures, forex. This is regularly done by many banks and large funds, and also small investors. In fact there is a discount brokerage which I'll just call IB here, that has an API which lets anyone program their own computerized trading. It's a bit "too easy" to do.
That doesn't mean it's always profitable in the long term, but without a doubt people are profiting at least in the short term. The software has multiple strategies, well documented approaches and algorithms. Generally the trading robot is trying to ride trends.
As someone who follows these things, here are a few criticisms I'm aware of:
1. These short term trading activities require high leverage, because trades have to be for large amounts of money to make them worthwhile. You need large amounts of money to make this work, because things like trading costs eat into profits tremendously. Again, like day trading.
2. High leverage is risky because one big mistake or unpleasant event could wipe out tons of past small gains. Risk management becomes a key issue. Some would argue that perceived risk in markets these days is unreasonably low. Does this unbalance the risk/reward equation?
3. Market-wide, we know program trading has increased dramatically on US exchanges. Add to this the undocumented program trading (smaller traders who don't have to report it to anyone) and basically there are a ton of computer algorithms out there today trading stocks. Everybody can't make money at the same time, so to profit the participants have to use even greater leverage = more risk.
4. Programming flaws, bugs, or improper risk management could have tremendous market-wide implications. Take for example the huge market moves in 1987; the drop was a "20-sigma" event and not anywhere within the realm of possibility back then. Obviously the models failed to handle it. Similarly, the next time we have a "big event" in markets, today's algorithms might fail. If a large number of computers choke while trading, could bad things happen?
5. So under unstable market conditions, the program trading could lead to increased volatility (like daytrading caused volatile markets during the crash). But under stable market conditions, like we have today, program trading seems to smooth out daily movements. Notice that the US markets hardly move as much as 1% in a day; trends are smooth and volatility is extremely low. The VIX, a volatility measure, has hit historic lows.
Humans can't do it... (Score:3, Insightful)
Of course, the entire planet's GDP is only 60 trillion, so even a little mistake means complete global meltdown.
.
Re:Why? (Score:5, Funny)
No, no, no, there's a better way. Get me a case of beer and a copy of the WSJ and a dartboard. I'm at least as random as a monkey after 24 beers.
Parent
Re:Why? (Score:4, Funny)
Remember: sticks and stones may break your bones but feces just splatters.
Parent
Re:Why? (Score:5, Interesting)
I had a friend who worked in the AI department of Lockheed (about 20 years ago) and they developed the software that was used for the Robert Prector's Elliot Wave newsletter. Every two years they would give the program to a couple of people to try for 6 months. These people would invest $10,000, use the program and follow the guidelines, then evaluate the results. I was privy to the outcomes of three of these tests in the mid-nineties, and the lowest was earned $15,000 and the highest was earned $36,000. These are pretty good results. (However, the stock market was steadily climbing during those years and I wasn't able to compare results with EWT competition. Still, if I was able to consistently get 30%/year on my investments...)
Back in the 70's, Dean Witter had a program called PACE. I know two people who had a system for using it that earned them over $100,000/year, and they never deviated from the program while I knew them.
Then I have a friend who is a very conservative money manager (manages a couple hundred million of other people's money), and over the period that stocks crashed (remember Enron and Worldcomm?) he only had two clients lose any money, and the biggest loss was less than %15. He claims that these programs are mostly bunk. (This guy is a perfectionist, and I bet a computer is no more disciplined than he is.)
These programs are not investment management programs. The principles of investment management are pretty simple. The best book I know on the subject is still Benjamin Graham and David Dodd's book, "Security Analysis". However, the problem is finding opportunities that comply with the principles. Systematic data analysis by computer could have a profound effect, and that's what most of these programs do.
BTW, the article mentions that profits are slowing down: In Robert Prector's book, "The Elliott Wave Theory" and in his newsletter, he sort of predicts that as information becomes more available for analysis trading will be done more rapidly on spreads that may show profits as low as 1.5%
Parent
It could happen (Score:4, Interesting)
I support the investor's right to be as stupid as they wish. A buy and hold regulation won't do as much good as something like the Fair Tax Plan, increased personal debt reduction, a balanced budget, and reduced spending. Presumably, investments are property that an investor should be able to divest anytime. Leave the government out of it.
Parent
Re: (Score:3, Interesting)
The dollar is due for a complete collapse. China has trillions of dollars and they are seriously considering insisting on other currencies for payment, same with oil producing companies all of whom hate our guts and would love to see the dollar collapse.
If you have money to invest do it on commodities. Food, minerals, oils, water etc. The smart corporations have already
Re: (Score:3, Informative)
In fact, in the Wall St. Journal's long-running contest, the experts have out-performed the darts 29 to 21 times ( ahref=http://www.webtrading.com/issue18.htmrel=ur l2html-21923 [slashdot.org]http://www.webtrading.com/issue18.htm> ). This does not mean the experts are all that great - the score is 26 to 25 againsT the Dow Jones Industrial Average.
However, in the less-long-running contest of Wall St. Journal readers ve
Re: (Score:3, Informative)