A British Supercomputer Can Predict Winter Weather a Year In Advance (thestack.com) 177
The national weather service of the U.K. claims it can now predict the weather up to a year in advance.
An anonymous reader quotes The Stack: The development has been made possible thanks to supercomputer technology granted by the UK Government in 2014. The £97 million high-performance computing facility has allowed researchers to increase the resolution of climate models and to test the retrospective skill of forecasts over a 35-year period starting from 1980... The forecasters claim that new supercomputer-powered techniques have helped them develop a system to accurately predict North Atlantic Oscillation -- the climatic phenomenon which heavily impacts winters in the U.K.
The researchers apparently tested their supercomputer on 36 years worth of data, and reported proudly that they could predict winter weather a year in advance -- with 62% accuracy.
The researchers apparently tested their supercomputer on 36 years worth of data, and reported proudly that they could predict winter weather a year in advance -- with 62% accuracy.
No it can't (Score:5, Insightful)
Re: (Score:3)
They have made similar claims in the past. Back in the 90s I seem to recall that their fancy new computer would make predictions super accurate. In practice the Met Office seems to be one of the worst, far inferior to AccuWeather and the like.
In fact, the BBC recently ditched them, although I think it was mostly due to the cost rather than them being inaccurate.
Re:No it can't (Score:5, Funny)
Hey, but 62% accuracy... that's 12% better than me!
Re:No it can't (Score:4, Insightful)
Re: (Score:3)
62% of the time... it works EVERY time.
Re: (Score:2)
I too am above 50% (Score:2)
Michel, if you are disappointed then try my own algorithm : "weather tomorrow will be the same as today" -I guarantee the accuracy is higher than 50%.
We are running just behind them indeed -maybe they have a similar predictor!
Re: (Score:2)
Should be well worth it in terms of things like planning for agricultural products, natural gas supplies, etc.
The real issue however is that they've validated it with hindcasting. Which is certainly something, but isn't as ideal as you'd want. It's trivially easy to fit any arbitrary past dataset to a statistical model if you have enough parameters that can be tweaked, but that doesn't mean that you're actually capturing the underlying dynamics. That said, from the sound of it it's built around a physica
Re: (Score:2)
>The real issue however is that they've validated it with hindcasting.
I suppose you do it right and compare against the future.
Could you send me the S&P max and min valuation for 2017, 2018 and 2019 please?
Re: No it can't (Score:2, Interesting)
It's NOT a statistical model, at least not in the sense you're likely thinking. Although the actual scientific article is ridiculously paywalled, it's definitely a dynamical model that numerically integrates partial differential equations (like the Navier-Stokes equations) forward to produce a solution. They are using an ensemble of the dynamical model solutions to create their forecast. Similar models already exist, such as NOAA's CFS, which are ensembles that make predictions several months in advance. No
Re: (Score:2)
"If a significant majority of the solutions show one phase instead of the other, it suggests that there is some predictability of the feature (in this case, the NAO) despite the uncertainty and errors from the initial conditions."
There is, however, some chance that the models share some common incorrect assumption. They don't agree because they are correct, but because they are similar to each other. The Literary Digest effect -- sort of.
Or they may simply agree by chance.
Re: (Score:2)
No, you can't predict the weather on a given day months in advance (the limit is believed to be around 21 days) because the atmosphere is a chaotic system and small errors grow too large to make predictions useful.
Less than that if there's a volcanic or solar event.
Re: (Score:2)
Re: (Score:2)
Weather != Climate (Score:3)
And you would know, being so obviously well-informed about weather simulations.
If the person who wrote the summary knew anything about "weather simulations" they would be aware that climate is not weather!!!
Re:No it can't (Score:5, Funny)
I call BS on the headline. Let the damn thing prove it can do it before we claim it can. And doing regression model tweaking doesn't prove anything.
Why? Predicting the winter weather in Britain is pretty simple. This little program will get it right about 90% of the time:
#include <stdio.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
int main()
{
char date[32];
time_t rawtime;
time (&rawtime);
struct tm *timeinfo = localtime (&rawtime);
strftime(date, sizeof(date)-1, "%d.%m.%y_%H:%M:%S", timeinfo);
printf("[%s] Weather prediction: Precipitation\n", date);
sleep(86400);
}
Re: (Score:1)
Why? Predicting the winter weather in Britain is pretty simple
Uhhmm, wrong, predicting British weather is a PITA,( some might say a ROYAL PITA), the meandering jet stream and retrograde systems that come off the continent are a bitch to figure, Its not at all like the USA where you can watch storms march across 3000 miles and rightly predict rain on Tuesday.
Re: (Score:1)
Yes you can. It *will* rain on Tuesday in UK. You just didn't specifically ask how much and where...
Re: (Score:2)
If nothing else, a good demonstration of the popularity of Python when C is overkill
#!/bin/env python
print "Weather prediction: Precipitation"
Or bash
echo "Weather prediction: Precipitation"
Re: (Score:3)
I don't see any stylish manipulation of timestamps into a correct imperial format in your code.
Re: (Score:3)
Miyagi on Latin (Score:2)
What have bishops got to do with it?
Re: (Score:3)
As William Shakespeare might have said "Computers can predict weather a year ahead. So can I. So can any other man. But will the weather come when I summon it?" See Henry IV Part 1: Act 3, Scene 1, Page 3 nfs.sparknotes.com/henry4pt1/page_133.html for details.
Re: (Score:2)
It is definitely BS.
NOAA can't even predict weather one day in advance in many cases.
I've seen them issue heavy storm alerts when it doesn't even rain
Re: (Score:2)
Re: (Score:2)
They can predict climate that far ahead
They can only predict weather up to 7 days ahead ....
The title is bogus, the artical is correct ...
Re: (Score:2)
I call BS on the headline. Let the damn thing prove it can do it before we claim it can. And doing regression model tweaking doesn't prove anything.
The farmer's almanac does as as good a job of prediction as will any mega or super computer.
Re: (Score:2)
If you look into it in a bit more detail, the actual claims made are much less ridiculous than the headline makes them sound.
They're basically claiming to be able to predict (with lots of uncertainty) whether or not next year's winter will be particularly severe. This is useful, but not nearly as precise as the headline makes it seem.
Re: (Score:3)
I can do it with 100% accuracy, for the american midwest. My winter forecast calls for snow and cold, with periods of total darkness between sunset and sunrise. Crystallized hydrous precipitation will at times accumulate and form drifts, and will under certain conditions be made into snowballs and snowmen.
Re: (Score:2)
That plus a constant wind chill in the plains... I live in Kansas.
Re: (Score:2)
Oh I have much more specific forecast for Kansas. This winter's forecast also calls for flat, flatter than a pancake [improbable.com]
Re: (Score:2)
I always find the Hollywood portrayal of Kansas funny because I live in the flint hills on the east side of the state which are not flat and also where the majority of the states population lives. It's only the western two thirds of the state that is actually flat with fields as far as you can see and no people or towns.
Re: (Score:2)
On i70 it's really flat from Hayes all the way past the Colorado boarder and then you find yourself asking where are the mountains... That stretch always makes me extra sleepy and if I don't have the cruise control set I'll be doing 90-100mph with out realizing it.
After 40 splits off from i70 it's only about 70-80 miles to the border but yeah there are more things than the flint hills and rolling hills that people don't expect like a 33,000 acres of lake and damn on the republican river.
Re: (Score:2)
If I could be right about a coin toss 62% of the time, I could get very rich, very easily.
Since the weather has a huge impact on the economy (not least because heating/cooling costs) if this model is as accurate as claimed, that's pretty useful.
fallacy (Score:2)
using historical data, that's just as silly as "cooking the books". There is one and only one way to test the validity of such a system, and that will take over a year....
Re:fallacy (Score:5, Interesting)
No, what they mean is they test it by feeding it the data from 1995, then comparing its predictions to what the weather was actually like in 1996. They are doing exactly what you say is the only way to test the validity of the data - they just started collecting data long ago.
THAT SAID, 62% correct doesn't seem all that awesome unless they use very tight margins. Does the computer say it'll be -10C and then count it as a fail if it's actually -11C? -15C? Does it say 'Good enough' if it says "Rain and 5C" and instead we get "Snow and -2C"?
Re: (Score:2)
Did they access to the 1996 data when they developed the model?
Re: (Score:3)
Did they access to the 1996 data when they developed the model?
Well of course they did. How else could they test the 1996 predictions the model made from the 1995 (and earlier) data?
You build models by using prior data to adjust the model's parameters to "predict" new data, until the accuracy of the prediction is optimal.
You seem to imply that they cheated somehow. Generally, scientists are honest, with the exception of a small minority who are discovered by their peers and vilified.
Re: (Score:3)
Because the human mind is incapable of bias and groups of minds are incapable of systematic bias?! There's a reason we say a real test of a prediction requires waiting for the real future. And this should be obvious to everyone. And before anyone tells "troll", smart intelligent honest people are as subject to bias as anyone, except because they know they are smart and honest, they are also subject to what's called "expert bias". It's just one more thing to be aware of as we pursue greater knowledge and ins
Re: (Score:2)
A test of predictions doesn't necessarily mean waiting. If you use the data available at the nominal time of prediction, and haven't used this particular prediction in your training data, it's perfectly valid.
Re: (Score:2)
You can do the same thing with the stock market, and many people do. It works until it stops working.
Then a new system is announced, to great fanfare. Lather, rinse, repeat endlessly.
Re: (Score:1)
You can do the same thing with the stock market, and many people do. It works until it stops working.
Then a new system is announced, to great fanfare. Lather, rinse, repeat endlessly.
To be fair, the stock market is responsive to predictions - all market movements are based on the actions of humans and of programmed systems, both of which will change if you provide people with a way to predict the outcome (since acting differently to how they would otherwise have done is precisely how they will profit from the predictions) - whereas the weather mostly isn't affected by those sort of factors.
Re: (Score:2)
Using 1996 data to construct a model that conforms to 1996 data is cheating.
Re: (Score:2)
If the model only uses constants found in this list [infogalactic.com], the data from the past is already cooked into the algorithms it uses. It really needs to do better than 62% verifying (not predicting) itself using the past.
Or, possibly, past data was funny in some way and the model is superior, in which case call me in 2 years to tell me that it is off to a good start.
Re:fallacy (Score:4, Insightful)
The problem with that approach is that you will tweak the algorithm until it works in 1996.
In other words, you will incorporate 1996 into the test set.
This is the big problem with almost all climate studies, and the reason why people that understand statistics really hate the current climate "science" as it is done. You really do need to make a prediction, and then test the prediction. If you get it wrong, you cannot re-try against the same data set until it works.
Re: (Score:2)
But if they manage to tweak the model so it fits the data set of 1980, 1981, 1982 etc. all the way up to now, isn't that essentially the same as starting today and predicting 2017, then tweaking the model a bit and using that to predict 2018, then tweaking it a bit because you missed something and so on?
Re:fallacy (Score:5, Insightful)
They kept adjusting the algorithm over and over until they got the right answer from 1980 onwards. The huge risk with that method is overfitting [wikipedia.org], and if you develop an algorithm this way, it's important to also show that you've managed to avoid overfitting.
You can do the same thing with stock market data: adjust it until you get nearly 90% correct returns on a test interval, then you will find that the next year, the model is completely wrong because of overfit. Even if you incorporate the next years data, you will still get incorrect results because the nature of the stock market is chaotic and also random.
Re: fallacy (Score:2, Insightful)
Your claims of overfitting would mean something if they used a purely statistical model, but it's not - it's a physical simulation, constrained by laws of thermodynamics.
It amazes me that people say how trivial it is to fit statistical models perfectly to any random data (the stock market always gets mentioned here), yet don't think to wonder why they "only" got 62% accuracy. You'd think that this would be a huge red flag that your assumptions are wrong, but instead it's waved away as them all being dumber
Re: (Score:2)
I need to introduce you to my broker. He always tells me to get into the stocks and indexes that just did well. Of course he also always says past performance is no guarantee of future results. All these models (climate, stock, economic etc) are great until they don't work.
Re: (Score:2)
Gee, if only there were statisticians involved in climate research. Too bad nobody ever thought of that one.
Re:fallacy (Score:4, Insightful)
There are an infinite number of functions that can go through the data points of the past. I could make you 1,000 perfect stock predictors for past data.
Ask yourself, how did they refine and improve this model over time? It's nothing but a pile of cooked books
Re: fallacy (Score:5, Insightful)
And more ignorant nonsense gets modded Informative. The anti-science here is getting worse. Posters like you not only drastically overestimate your own knowledge of unfamiliar fields, you then insist to others it must all be a scam.
Weather and climate models aren't some arbitrary curve-fitting; they're physically based using ridiculously detailed physical simulations of air movements and ocean currents, starting from an observed state and running the simulation forward. Read up [arstechnica.com] a little [arstechnica.com], and maybe you'll learn how to learn again.
Re: (Score:2)
Weather and climate models aren't some arbitrary curve-fitting; they're physically based using ridiculously detailed physical simulations of air movements and ocean currents, starting from an observed state and running the simulation forward.
Just because a set of predictions is based on a physical model does not necessarily make it a better set of predictions. Physical models are still hypotheses, in that the basic premises behind the model and even the construction of the model itself have not been demonstrated as an accurate representation of reality. It is not until the model turns out accurate predictions that are significantly better than random that the hypothesis stands a chance of being correct.
I'm a reservoir engineer for a large oil
Re: (Score:2)
You are the ignorant one, thousands of models are made and then they are culled. They most certainly are a type of "arbitrary curve fitting"
Re: (Score:1)
So in short they are feeding it the Farmers Almanac.
Re: (Score:2)
IIRC, the Farmer's Almanac claims to have a method for making predictions. Could be. AFAIK, their method is proprietary.
I've long suspected their editorial staff assembles in a Pub a few weeks before the printer's deadline and throws darts while the soberist attendee takes notes. But they could have more rigor. Or maybe throwing darts works.
Re: (Score:2)
No, what they mean is they test it by feeding it the data from 1995, then comparing its predictions to what the weather was actually like in 1996.
Sure, and when one algorithm doesn't work, you try another, and another, and another. Then after 19 failures, you find an algorithm that works on the data from 1995 to predict 1996 weather with a 95% confidence level.
You can do the same thing with jelly beans [xkcd.com].
bullshit (Score:3, Interesting)
It is predicting climate, not weather.
Re: (Score:2)
Re: (Score:3)
So the best modeling offered can predict next years climate with 62% accuracy. That says a lot about climate modeling over the next century.
Keep in mind that while short term predictions can be chaotic, it's sometimes easier to see long-term patterns emerge, and to extrapolate data from those trends, like trending lines through a scatter plot. I agree that anything looking a century out is guesswork at best, but I'm not sure I'd say the same looking a decade out.
Historically, many climate-related doomsday predictions have been laughably innacurate. [aei.org] It's for this reason that I continue to be somewhat skeptical about current doomsday or long te
So can I (Score:5, Insightful)
Re: (Score:3)
So, you think that supercomputer will only figure out whether it rains on the day?
Or do you think it might forecast a little more information about the day as well? (air pressure, wind, temperature, ...)
I laud the attempt to improve the models behind the forecasts - though, I don't think I'd buy options on winter fuel for 2017/2018 yet, simply because of that computer.
On the other hand - if you know the average chance of precipitation for a given day in the American midwest - I don't think it would help y
I'mt still sticking with ... (Score:3)
Or just ask an indian. When asked how he could tell how cold the winters would be, one old chief just said, "I watch how much firewood the white man splits."
Re: (Score:2, Interesting)
When asked how he could tell how cold the winters would be, one old chief just said, "I watch how much firewood the white man splits."
That only sounds silly to people who don't consider humans to be animals.
If the chief would have answered that he looked at how much food beavers stockpiled then no-one would have found it funny.
White mans gut feeling works just as well as any other animals gut feeling and is a hell of a lot better than flipping a coin.
I live in the Uk and call BS (Score:1)
I've been living in the UK for the last three years an as I use a telescope I look at the weather daily. The national weather service barely gets a prediction correct FOR THE SAME NIGHT never mind days or years in advance! I am not kidding, there is something like an 80% or a bit more chance for clouds and a lower chance for precipitation, but the forecast only gives you slightly more information than that. Eg instead of 80/20 cloud odds, if the MET office thinks it will be cloudless, consider the odds to b
Re: (Score:2)
I can somewhat attest to that - having lived in the UK from 2004 to 2009... In 2008 I tried taking some photos from our offices in Canary Wharf. I looked at the whether forecast every day before deciding whether to take camera+lenses+tripod to work that day. ...and for many of those days, the "visibility" forecast of the MET was pretty much the opposite of what happened -- hardly any visibility on days when it said "good" visibility -- and clear views on many days where the forecast was for poor visibility.
With 62% accuracy (Score:1)
it's not much more than a coin-toss, though.
No historical data (Score:2)
This is only impressive if they didn't use any historical data at all to create the new super computer. If they did use historical data then the answer would be correct by definition. The way to test this is to use historical data make a prediction and then wait a year then compare to the real data only then you have any valid comparison.
Re: (Score:2)
This is only impressive if they didn't use any historical data at all to create the new super computer. If they did use historical data then the answer would be correct by definition. The way to test this is to use historical data make a prediction and then wait a year then compare to the real data only then you have any valid comparison.
That's really dumb. Why would you wait a year, when you have 35 years of data? You test the model on 1980's data, and see how accurate it was by checking with 1981's data, and so on. They can do that 35 times, if they're looking a year ahead. If they're only looking a month ahead, they can do it 12 x 35 times.
Re: (Score:2)
Yes that is ok if you don't alter or fix the model based on the result of the historical data. But if your modal is based on or altered to fit the historical data, then what you got is just a model that can predict historical data very well. You have no way to know if that model can actually predict the future in any way unless you actually test it.
Yes but can it... (Score:2)
Predict the weather next week?
Weather (Score:2)
Little better than random chance, then.
Pisses me off that the biggest IT investments and supercomputers exist for meteorogical purposes that perform little better than chance.
Though important, for shipping, air travel, etc. it's not THAT important to get a tiny little percentage over just looking around and thinking it's going to piss down in a moment, or sticking a box in the North that lets you guess how long until the same weather hits the South.
Just seems one enormous waste of money to me. And who exac
Re: (Score:2, Insightful)
Little better than random chance, then.
Chance only gives you 50/50 for a coin flip. The weather has many more states, so chance will give you something much worse than 50/50.
Re: (Score:2)
Weather in the UK is pretty simple to predict: foggy or rain
Is this computer by any chance (Score:2)
62% is fail (Score:1)
Is this part of the "everyone wins" generation come to life in practical science now? lel
Re: (Score:2)
So, 12% better than a coin flip.
Great, really, just great.
Re:62% is fail (Score:4, Insightful)
Re: (Score:2)
It's not very difficult (Score:2)
Really ? (Score:4, Informative)
Re: (Score:2)
In his textbooks, does he end each chapter with the words "I'll be back!" by any chance?
Re:Really ? (Score:4, Informative)
If you want to estimate the error, if n is the number of months of the forecast and eps is the measurement precision, the error is given by:
10^(2.5n) times epsilon. As you can see the error rapidly increases, although the formula I transcribed from Arnold's textbook is quite rough (toroidal Earth, steady flux and negligible viscosity). Not a bad approximation for estimating trade winds [wikipedia.org] flux, however.
People at MET probably took care of the propagation of numerical errors in the calculation, by increasing the grid density and maybe setting up a system capable of working with quadruple precision. However the problem again is the needed precision of input data, that increases exponentially with the time forecasted.
I can probably do better than 62% accuracy (Score:2)
North Atlantic Oscillation (Score:3)
... and also impacts winters in the northeast USA.
So Can I (Score:2)
A British Supercomputer Can Predict Winter Weather a Year In Advance
Yeah, so can I: cold, with occasional snow and sleet.
My 100 year calendar can do it as well (Score:2)
It says: The weather will get better or worse or it will stay as it is.
So what? (Score:2)
I live in MN.
I can predict "winter weather" - whatever the hell that means, precisely - to the same degree of accuracy 10 YEARS in advance.
"In 2026, we will see 'winter weather' in Nov, Dec, Jan, Feb and well into March".
If at least 2/3 of the years follow the normal weather patterns, I've just beaten their supercomputer.
predicting weather is one thing (Score:2)
... but does it run Linux?
CRAP Reporting (Score:1)
Wow. The reporting in this is just CRAP. I didn't read the scientific paper but the abstract (summary) and even the anouncement from the MET (both of which u can only find by first browsing to the link in the summary) makes it clear they are talking about CLIMATE not weather. And it's only about 1 major phenomenon that has a heavy influence on UK climate.
So this isn't about predicting the daily weather but rather general predictions about whether or not there will be general periods of "wet, stormy and warm
Operating system and hardware? (Score:2)
No, it can't. (Score:2)
I call Bullshit. (Score:2)
5 years wrong in a row.... (Score:2)
62% accuracy is only a bit better than a coinflip. you will relatively often see it predict incorrectly 5 years in a row.
bollocks (Score:2)
Bollocks on their predication rate. Real forecasters report skill [wikipedia.org]. By contrast, actual progress on predicting the North Atlantic Oscillation, perhaps an achievable goal, would be huge.
Both of these issues are covered in Judith Curry on Climate Change [econtalk.org], a podcast from 2013 which, as it happens, I consumed yesterday.
Concerning the rush to embarrass themselves by reporting their weather prediction rate, it's because of the taxonomic land grab.
Headline is bullshit (Score:3)
Re: (Score:2)
It's good for the NAO. When you're pushing the boundaries, anything over 50% is good.
For long-term climate models, things like the NAO average out across many years. For short-term weather forecasting, you have a week or more before the system diverges enough to cease to be useful. But it's tougher working on those in-between scales.
Re: (Score:3)
When you're pushing the boundaries, anything over 50% is good.
Is it? It depends on the data, the model, the thresholds for "correct forecast," etc. There are lots of places in the world where a "persistence" forecast (i.e., today will be the same as yesterday) will net you a greater than 50% accuracy within a reasonable margin of error. And one should also always consider forecasting models against general predicted climate averages. Again, taking those into account, a forecast system just using climate averages might do pretty well too.
It really depends on what
Re: (Score:2)
Re: (Score:3)
You're talking about overfitting.
The thing is they aren't doing that regression-and- frigging-the-coefficients thing. It's a physics based, bottom up, method.
Nice armchair.