ProPublica's Guide To News App Tech

dstates writes "ProPublica, the award winning public interest journalism group and frequently cited Slashdot source, has published an interesting guide to app technology for journalism and a set of data and style guides. Journalism presents unique challenges with potentially enormous but highly variable site traffic, the need to serve a wide variety of information, and most importantly, the need to quickly develop and vet interesting content, and ProPublica serves lots of data sets in addition to the news. They are also doing some cool stuff like using AI to generate specific narratives from tens of thousands of database entries illustrating how school districts and states often don't distribute educational opportunities to rich and poor kids equally. The ProPublica team focuses on some basic practical issues for building a team, rapidly and flexibly deploying technology and insuring that what they serve is correct. A great news app developer needs three key skills: the ability to do journalism, design acumen and the ability to write code quickly — and the last is the easiest to teach. To build a team they look to their own staff rather than competing with Google for CS grads. Most news organizations use either Ruby on Rails or Python/Django, but more important than which specific technology you choose is to just pick a server-side programming language and stick to it. Cloud hosting provides news organizations with incredible flexibility (like increasing your capacity ten-fold for a few days around the election and then scaling back the day after), but they're not as fast as real servers, and cloud costs can scale quickly relative to real servers. Maybe a news app is not the most massive 'big data' application out there, but where else can you find the challenge of millions of users checking in several times a day for the latest news, and all you need to do is sort out which of your many and conflicting sources are providing you with straight information? Oh, and if you screw up, it will be very public."