The Rise of Machine-Written Journalism 134
Hugh Pickens writes "Peter Kirwan has an interesting article in Wired UK on the emergence of software that automates the collection, evaluation, and even reporting of news events. Thomson Reuters, the world's largest news agency, has started moving down this path, courtesy of an intriguing product with the nondescript name NewsScope, a machine-readable news service designed for financial institutions that make their money from automated, event-driven trading. The latest iteration of NewsScope 'scans and automatically extracts critical pieces of information' from US corporate press releases, eliminating the 'manual processes' that have traditionally kept so many financial journalists in gainful employment. At Northwestern University, a group of computer science and journalism students have developed a program called Stats Monkey that uses statistical data to generate news reports on baseball games. Stats Monkey identifies the players who change the course of games, alongside specific turning points in the action. The rest of the process involves on-the-fly assembly of templated 'narrative arcs' to describe the action in a format recognizable as a news story. 'No doubt Kurt Cagle, editor of XMLToday.org, was engaging in a bit of provocation when he recently suggested that an intelligent agent might win a Pulitzer Prize by 2030,' writes Kirwin. 'Of course, it won't be the software that takes home the prize: it'll be the programmers who wrote the code in the first place, something that Joseph Pultizer could never have anticipated.'"
The reporter is now a touch more obsolete (Score:5, Interesting)
News agencies have already been turned into commodities, they just don't realize it yet. Now the reporter is being sent down that same drain. With original reporting set to become a 'premium' by the news agencies, their market is only shrinking.
Where were the reporters when millions of jobs were outsourced by H1B's or sent overseas? At best most stories were brief, with no follow up, and no outrage at the loss of middle class America. The same thing has happened in Europe and elsewhere as well.
Now the reporter faces the inevitable market forces that they previously ignored, and they expect anyone left to care? The programs will only get better, the markets and stories it applies to will only improve, and for the vast majority of stories the quality will be imperceivable to the average person.
Re:nonsense (Score:5, Interesting)
This strikes me as the sort of thing that, without excessive amounts of variation, would get filtered out quickly by the general public. Sure, a machine can write *one* story on a baseball game that is interesting to read. But after the hundredth version of the same story that you've read, the public would stop reading the text entirely and just filter for the important bits. At that point, you might as well just have a table with the interesting stats.
The challenge isn't to write one story. It's to create a machine that can write N stories that remain interesting and fresh, and with less effort and cost than it would take journalists to just write N stories the traditional way.
Re:The reporter is now a touch more obsolete (Score:2, Interesting)
There was a time when the collective consciousness of this country embraced technology as a way to free humans from the mundane activities of life. No more work to do, all the menial tasks performed by machines. A utopia of leisure and enlightenment.
This changed over the last hundred years into the dystopia view of the terminator and 1984. Technology advanced but economic and political theory has not. Machines became the enemy because we made them competitors. There will soon be a time when there simply are not enough jobs to be done by humans. People will become obsolete in the work force.
The kick in the ass is that we already have the resources to give every person on the planet a decent life. That surplus of labor and resources will only increase as technology continues to advance.
So why not a utopia? If everyone was equal and happy and content then there is no reason to be rich. The best thing about being rich it getting to shove it in the face of the poor (and the point is for everyone to be poor).
This story is really about the friction between the natural evolution of technology and the need for our masters to make sure we never see the benefits of it..
steakthskynet (Score:3, Interesting)
Re:The reporter is now a touch more obsolete (Score:3, Interesting)
Where were they? They were cherry-picking the easy stories that they knew would get viewers. Stories about H1Bs are hard because those happen 1 at a time and only the collective tale of ALL of them means anything. Who do you interview? What do you actually report?
After the fact, we can point and say 'Wow! That's a lot of jobs lost!' and complain about it. While it's happening, it's nearly invisible.
I agree with your point, though... Reporters are not covering the -real- news. That's why sites like Slashdot exist now. (Not saying ./ is perfect, mind... Just that it is responding to the gap left by 'real' reporters.)
Re:The reporter is now a touch more obsolete (Score:3, Interesting)
There are really 2 good ways to handle this.
1. To place artificial barriers to entry. For example, you could say that any piece of 'news' must be presented by a certified journalist. Just like any prescription must be done by a doctor. Or lawyers have their own provisions. For programmers you could have certain requirements like any piece of software in use must have a certain number of maintainers... Basically turning such jobs into professions with the same level of protection as doctors, lawyers... Oh I'm sure we can come up with some excuse like 'quality' to enforce this :)
2. We embrace the lowering of costs and focus on reducing our cost of living. As good private sector jobs (auto-workers, engineers...) go away, that high-end tax base drops as well. So the payment to the public sector should drop as well. There is no intrinsic reason a teacher should earn more than a waitress. I know I'd rather be a teacher than a fast food server. I've been both :) We can then focus on low property taxes... having a lost cost of living. And we would all be able to have more 'stuff' as things would cost very little. We could simplify the legal and tax system as well as remove the medical monopoly from doctors...
The alternative is the current unstable system. Whereby those in the private unregulated sector keep pushing themselves to efficiency and 0 cost. While those in regulated professions and the public sector keep pushing for more. Basically a form on indentured servitude by those in the public sector and regulated profession upon those in the private sector.
Re:Mod parent up. (Score:3, Interesting)
Inevitable. Most of the Financial world is overstated swings in outlooks, leading to crazy stock price gains and losses.
I have a very simple solution to daily manipulation of financial manipulation. A sliding scale of capital gains taxes, based solely upon how long one owns the financial instrument they are trading.
10 years capital gains tax free.
Or something like that. The problems with our current market are due either directly or indirectly to short term outlooks to income generation, ie next quarters profit/loss statements.
Holding a stock or bond long term is almost insane these days. YET that is the purpose of stocks / bonds, long term financing.
That is my solution.
Re:Mod parent up. (Score:3, Interesting)
I saw one of the stocks I owned go up when Company A released a press release that Company A signed a deal with company B.
The stock of Company A spiked again 3 days later when Company B released a press release that it had just signed a deal with Company A.
If there are quant systems out there listing to the wire and trading on info like this, the system will surely be gamed. What is worse is that if a human were watching the blips come over the wire would he necessarily catch the problem?
They've been doing crap like this in their accounting for years, Enron charges X to company Y, and Y charges X back to Enron, both of them had 2X extra sales in the quarter, but no money or goods actually change hands, now it extends to journalism.
Re:nonsense (Score:3, Interesting)
Feeling defensive, eh? Thanks for informing everyone you're "literate" and therefore better than those filthy ordinary people. Pick up a newspaper, and tell me how much writing would actually be *improved* with a machine writer, eh? Writing isn't sacred, it's just another occupation like woodchopping or running the cash register at the 7-11.
What you describe is "everyday" writing. This is like the vending machine that reads "insert dollar bills face up" or "team A scored 10 points while the opposing Team B scored 15." It's purely practical, get-the-job-done sort of writing where only the technical correctness is important.
Writing can also be beautiful, powerful, and artistic. A well-written editorial, penned by someone who has a deep understanding of the issues, can and has moved entire communities to change their minds on important issues. A beautifully written book tells the perceptive reader as much about the mind and spirit of the author as it does about the story that is being told. A horror story requires some sense of what is horrible.
The ability to both produce and appreciate good writing is on the decline. Measured scholastically, the grade-level at which the average American reads and writes is significantly lower now than it was say, 50 years ago. A good look at many online forums will also tell you that this skill is not highly valued. You can say that's because only some cultural elite are capable of enjoying it, though the GP made no such claim. You can also say that there is simply less interest in such things.
You can invent and try to substantiate any number of unique explanations for it. The truth of the matter is that in most situations, challenging your readers is now considered highly undesirable. They'll read a competitor's paper written in simpler langauge before they'll grab a dictionary. There was a time when this would have been viewed as laziness and an unwillingness to meet a worthy challenge. It would have been viewed like a wasted opportunity to better yourself that should not have been passed up, just like most people today would not like to pass up a higher-paying job. I'm not saying that previous generations were one homogeneous block who all felt this way, but many more people once did.
Over the last few generations, some kind of cultural shift occurred. People now care much more about avoiding the small-but-significant effort of learning something new than they care about improving important skills. They generally won't do it at all unless it's required by an authority like a boss or a professor. Even then it's in a rote, mechanical sort of way that deprives them of true appreciation. Even then it's reluctant, with a "gun to their head" in the form of losing their jobs or failing school.
Can you comprehend how sad that is?
Re:Mod parent up. (Score:5, Interesting)
The purpose of stocks is for a company to raise capital by selling shares to the public, in return for some promise of potential future dividends.
The fact that people trade, hold, or speculate on these stocks in the secondary market (all the busy noise that is the "stock market") is nearly irrelevent. A company should have no reason to care about the price of its stock. Sadly, due to double-taxation of dividends, this has gone completely to shit. People who aren't speculating buy stock not for dividends, but to trade it to the next guy at a profit, because this is tax-favorable over dividends.
Companies do all sorts of crazy BS because of the expectation of stockholders to be rewarded not with dividends (a system that you simply can't game for long) but with rising stock prices (a system that is almost entirely gamesmanship).
Individual investors have the absolute right to seek short term gains or long term gains a their preference. The government has no business meddling in that preference. The "problem with the current market" is that it is no longer grounded in the reality of being able to pay dividends, because some previous generation's ideas about social engineering through taxation punishes dividends as the means of earning a profit on one's investment.