AMD Forces a LibreOffice Speed Boost With GPU Acceleration 144
New submitter samtuke writes: AMD processors get rated and reviewed based on performance. It is in our self-interest to make things work really, really fast on AMD hardware. AMD engineers contribute to LibreOffice, for good reason. Think about what happens behind a spreadsheet calculation. There can be a huge amount of math. Writing software to take advantage of a Graphics Processing Unit (GPU) for general purpose computing is non-trivial. We know how to do it. AMD engineers wrote OpenCL kernels, and contributed them to the open source code base. Turning on the OpenCL option to enable GPU Compute resulted in a 500X+ speedup, about ¼ second vs. 2minutes, 21 seconds. Those measurements specifically come from the ground-water use sample from this set of Libre Office spreadsheets.
Not the right tool (Score:3, Interesting)
Re:Not the right tool (Score:5, Insightful)
It can be. Don't generalize to use cases you don't know, especially when people with no real programming skills are concerned. I honestly don't know any other software that is both as flexible and accessible as spreadsheets when it comes to doing computations on heaps of (mostly irregular) data.
Re:Not the right tool (Score:4, Funny)
When the only tool you have is a spreadsheet, there is no place to sit because...
Re: (Score:3)
Even for people WITH programming skill, a spreadsheet is often faster when you need stuff done.
When working with hardware, there are often pesky register settings that need to be configured just right - a spreadsheet
Re:Not the right tool (Score:5, Insightful)
Re:Not the right tool (Score:4, Insightful)
I wonder if a spreadsheet is really the right tool for computations that take several dozens of seconds on modern hardware, even without GPU acceleration. I am inclined to think it is not.
It depends on the nature of the data and the calculation itself. If it is a relatively small data set and does not require relational tables, but a very complicated set of equations, then a spreadsheet might be a reasonable choice vs a database.
Re: (Score:2)
I wonder if a spreadsheet is really the right tool for computations that take several dozens of seconds on modern hardware, even without GPU acceleration. I am inclined to think it is not.
No, it isn't but many accountants and business controllers are using spreadsheets, some of them also learn SQL, but that isn't really all that much better for financial calculations (though better as a database which they also use spreadsheets for).
Re: (Score:2)
Exactly my thoughts also. Intensive computation and using a spreadsheet? You're doing it wrong.
Re: (Score:1)
OK, so what do you want to do? Say that a large part of the workforce should sit idle or go back to school just because they didn't learn the optimal language for this specific problem?
Spreadsheets are typically used by non-programmers to perform calculations that are cumbersome or impractical to do manually.
Or should they just offload the job to a programmer? They can probably put together a spreadsheet in half the time it takes them to write a half-assed and partially incorrect specification for the progr
Re: Not the right tool (Score:2)
Re: (Score:1)
A spreadsheet is easier to conceptualize, organize and manipulate than a database for most non-IT professionals and users, at the cost of higher risk on the data.
Re:Not the right tool (Score:5, Interesting)
really depends on use case. Our spreadsheets (finance, derivatives) can get damn big, but there are 3 reasons they persist: ease of modification, speed of the interface, and easy integration with powerful analytics libraries we use.
Now I have functioned in a python based environment before, and that had some huge benefits (especially when working on tick level data, or data that was just a pain to manage in VBA until I got output down to a reasonably visualizable size) , and I regularly push for trade level data and details to be put off into a SQL database as it is pretty easy to write flexible queries to get what I want out. But visualizing data, interacting with historic data (user forms for display), generally integrating with many other financial libraries (bloomberg and reuters for realtime, internal quant libraries for complex calculations), and having a fast interface out of the box is amazing.
I've been at places that have tried to replace excel as the interaction layer. The problem is, for all its problems, most coders cannot hack together, on their own, a better GUI that is as performant or easily interacted with. Sometimes it isn't the data analysis layer (which if at all possible, we like to farm off somewhere else for perofrmance), but everything else that makes the spreadsheet far superior. And of course, I can modify and adapt someone else's work far faster than anyone using code. On a regular basis I can build up a complete tool in excel 10-20x faster than any coder can write me something outside of it. And most of the time a 95% correct answer in 1 hour is far more useful than a 100% correct answer in 3 days.
Now saying that, once the office ribbon started, that was the beginning of the end. Slowly the interface is getting too clunky to waste my time with when it was the simplest things I required. Now I try to do a lot of my work in a proper coding language and write out files I can parse quickly in vba and display in excel.
Re:Not the right tool (Score:5, Interesting)
It's really amazing what you can do in a spreadsheet.
Several years ago I was involved with management of optical wavelength switching gear (DWDM) in conjunction with a large, national telcom. They had some very well designed tools with very nice GUIs to allow things like building an optical path. Things that require managing complex database and doing a lot of checking on availability of resources and validity of the circuit.
It was all written in Excel!
I was amazed at it all. Nothing looked at all like a spreadsheet. and it actually worked and seemed pretty maintainable. I'm sure that they would have been delighted to see this sort of things as the one issue was the time it took to update the screen when certain changes were made (re-calculation).
Re: (Score:2)
The idea of a spreadsheet that needs GPU acceleration is alarming.
Re: (Score:1)
Wrong tool (Score:5, Informative)
This is good of course, however, whenever I see a spreadsheet program used for any serious computation, I cringe. There are far better tools out there if you require real number crunching. Think Python + Panda for instance, or R, or Matlab if you are really into commercial programs, otherwise a nice interactive web page will usually do the trick. For accounting use a real accounting program, there are plenty out there. Spreadsheet programs are the lowest common denominator that allow the sharing of table-like information, but almost universally they are the wrong tool for the job. Just in the last week, I have seen spreadsheets used for a program logic workflow, a timetable, a university course schedule, to compute an FFT, to exchange student marks, to discuss a budget (with lots of deletions and remarks), and even for a presentation. In each and every case a more suitable, open-source, freely available, multi-platform application exists.
Of course this is software that people know, so usually we have to deal with it. As a rule I accept to work with other people's spreadsheets, but I usually refuse to create one ex-nihilo, unless there is a compelling reason to. For instance I teach a course on optimisation, and I do show how the solver in Excel / {Libre,Open}Office works. I have also on occasion shown people how to use a pivot table (never use those if you can help it).
The most severe problem I see with spreadsheet is that they have their use but they are fragile. It is too easy to load an extensive table into them and inadvertently modify just one cell, potentially undoing a lot of work. This is easy to detect if your spreadsheet is small, but if it span multiple tabs and an ungodly number of rows, you will not detect your error. Of course the format of these spreadsheet is obscure, and version control is typically not supported.
Personally the worst I have seen was one spreadsheet used for the accounting of 90+ separate research projects, spanning 30,000 cells. The accountant in charge of it was the person most attentive to detail I have ever seen. She was careful and the only person using it, which made her indispensable. We put in place a year-long plan for her retirement, involving scrapping her spreadsheet, entirely replacing it with a direct interface with SAP via a php-based web page. It was many months in the making, of course this was not a trivial project but we've pulled it off. In the process we discovered a huge number of accounting errors thanks to it, typically invoices that were never billed, to the tunes of nearly one million dollars. It took us several months to correct them.
The morals of this is never, ever use spreadsheets program for non-trivial work.
Re:Wrong tool (Score:5, Insightful)
...Just in the last week, I have seen spreadsheets used for a program logic workflow, a timetable, a university course schedule, to compute an FFT, to exchange student marks, to discuss a budget (with lots of deletions and remarks), and even for a presentation. In each and every case a more suitable, open-source, freely available, multi-platform application exists.
You've just stated the reason people use spreadsheets right there. One multi-purpose program vs umpteen specialist programs, each with their own UI quirks that have to be learned to make the most use of the programs. Why bother when a good old spreadsheet will work.
Re: (Score:2)
The morals of this is never, ever use spreadsheets program for non-trivial work.
This is only really a valid conclusion if you compare it against the lost billing rate of other solutions though. I have seen organisations that would happily allow billing stuff to fall through the cracks with specialised software simply because it was easier than trying to fight the software for certain situations.
I can imagine that someone with Excel might end up saving more money for that reason, although I certainly agree the average complicated spreadsheet probably has a lot of errors.
Re: (Score:2)
About 20 years ago I did some contract work for a hospital whose accounting group did all of their day-to-day work with Quattro Pro, then a nightly process batched all of the spreadsheet work into the actual accounting system that ran on a System/390. The spreadsheet was useful for the bookkeepers an
not just AMD (Score:3, Insightful)
AMD engineers have contributed OpenCL code which is an open standard that can run on many different accelerators (some not even GPUs [parallella.org]). this is distinctly different from CUDA which only works with Nvidia stuff.
Re: (Score:2, Insightful)
Nvidia is really starting to piss me off with their proprietary shit like this, Gsync and their 'nVidia hair' bollocks for games.
Re: (Score:1)
I did not like them being allowed to merge with Intel. Nor did I approve of the AMD/ATI merger. It seems the stack has lost a bit of potential because of those changes. I can not say where things would have been, nobody can be certain, but I think things *could have* been better. I may have rose tinted glasses but I seem to recall that nVidia was making some very good headway in the open source area and that ATi was following suit. Now? It seems to be less so, less frequent, and with less dedication. Again,
Where's the... (Score:2)
Re: (Score:2)
Re: (Score:2)
Get OneNote for free on all your devices: Windows Windows Store Windows Phone Mac iPad iPhone Android Amazon Web Chromebook Clipper
I don't see linux on that list.
Re: (Score:2)
What's going on with Base? (Score:4, Interesting)
I may be completely wrong but no features have been added in ages, no code updates etc. etc.
Does anyone know what's going on?
Also, would Base benefit from this GPU enhancement?
A win for FOSS and AMD (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Evilly forced. They're going against the natural order, as defined by Intel's compiler.
Intel has gone to great lengths to keep AMD as slow and uncompetitive as possible. Obviously there must be some greater good reason, like demonic invasion or zombie apocalypse.
Re: (Score:3)
Modern GPUs can be hundreds and even thousands of times faster than CPUs at highly parallelizible simple math. That's why things like bitcoin mining, which are exactly that were done on GPUs before they moved to even more specialized hardware.
Re: (Score:1)
You are greatly misinformed. I was surprised to wake up and see so many bad answers, and my answered down-modded to oblivion.
An good multi-core CPU is 500 GFlops, and it's reasonably easy to attain it. A good GPU is 1.5 Teraflops, and it's nearly impossible to reach that in practice.
And that's for single-precision floating-point, if you want double-precision, like all of science does, it's not really that good.
Sure there are different models, exact numbers might vary, but it's in that ballpark, especially i
Re: (Score:3)
Interesting hypothesis.
I'll make it simple. please show me a bitcoin miner that can produce numbers you suggest on CPU in comparison to numbers produced on GPU while staying on approximately same cost levels.
Re: (Score:2, Insightful)
You say you "worked" in HPC. Are you sure your knowledge is still even remotely current? Are you sure your focus on CPU optimisation didn't leave you blind to what everyone else was doing?
In my experience, a good multi-core CPU is good for more like 125 GFLOP/s, and that varies highly from program to program thanks to the highly variable quality of code & compilers. You may be able to eke out better performance than that with skilful optimisation, but the vast majority of commercially available CPU-only
Re: Title condradicts summary (Score:5, Informative)
"A GPU is not 500 times faster than a CPU, more like 2 or 3 times"
Just on FLOPs alone you're dead wrong. The latest and greatest E-series Xeons (V3) have barely enough power to match the 9800GTX+ - about 800 GigaFLOPs. The 980 GTX Ti is roughly 5.6 TeraFLOPs.
Re: (Score:1)
Well, I put in the TOP of the line server hardware versus top of the line GPU hardware.
If we go to the desktop region, then the performance disparity increases!
Example, the latest Core i7 5960X is only 384 GigaFLOPs. It can't even get to half the power of a 9800GTX+.
But that doesn't even begin taking into account the efficiency of algorithms and code utilized.
Re: (Score:3)
GPUs are optimized for massive parallelism and can be much faster than CPUs that aren't for tasks fitting them. They can also be much slower if the task doesn't fit them. 500x faster than a normal processor is well within a reasonable speedup range.
But I also agree partially with you - the reason we see such a huge speedup in this case is probably because the CPU comparison point is running badly optimized code. With a few exceptions reports of huge speedups for GPU computing is because the CPU is feed with
Re: (Score:3)
...the reason we see such a huge speedup in this case is probably because the CPU comparison point is running badly optimized code...
Actually, it's because spreadsheet problems tend to be "embarrassingly parallellizable".
Re: (Score:2)
Really? While some uses of spreadsheets could be considered that most should be faster to run on a CPU - if the CPU uses optimized code. Parallelizing the calculations have inherent overheads.
Re: (Score:2)
500x faster than a normal processor is well within a reasonable speedup range.
Not really. The improvement is usually around 10x.
Re: (Score:2)
Yes but I didn't claim otherwise. The fact is that a GPU running code fitting it can get over 500x the performance of a CPU. However most real world code isn't as parallelizable as e.g. 3D rendering so overheads will strongly reduce the GPU performance.
As I wrote "With a few exceptions reports of huge speedups for GPU computing is because the CPU is feed with severely suboptimal code". Because a CPU running really shitty code is a "good" comparison point if one wants to promote GPGPU though not realistic.
Re: (Score:2)
Re: (Score:2, Informative)
I honestly thought that we'd got away from this 500x nonsense a few years ago. I would suggest that AMD is one source for the information that 2-3 is more reasonable. AMD, Qualcomm, Khronos, any of the members of the OpenCL committee you talk to as well as NVIDIA insiders if you catch them at a conference. I gave multiple public talks countering any factors over about 10 when I worked at AMD, which were approved by management.
Just think the raw numbers through. The GPU has, say, 32 cores. The CPU ALSO has m
Re: (Score:2)
Let's look at the actual setup used in this benchmark: AMD A10 7800B
4 Steamroller CPU cores (2 modules):
2x128 bit FMAC per module = 2x4 Single precision FMAC = 8 FMAC per module
16 FMAC/clock
8 GCN compute units:
4x16 single precision FMAC per compute unit = 64 FMAC per CU
512 FMAC/clock
Compute throughput:
CPU: 3500MHz x 16 = 35GFlops
GPU: 750MHz x 512 = 384GFlops
So we get more that x10 the (single precision) throughput using the GPU.
But that ignores the fact that GPUs are designed to tolerate long average memory
Re: (Score:3)
No it really doesn't. Just because NVIDIA's marketing department calls something a core does not make it comparable to a CPU core - notice that even AMD's marketing is being more up front about this by talking about an APU having some number of "Compute units" which is the sum of the CPU cores and the GPU CUs.
Even if the GPU did have thousands of cores, though, that does not affect the numbers I discussed. Instead of having 32 cores with 64 ops per cycle you just get 32*64 cores with 1 op per cycle. You sti
Re: (Score:1)
What you're missing is that each compute unit is made up of several shader units, which are vector processing units each with their own vector/scalar/general-purpose registers. This means that for data-parallel tasks, there really are thousands of threads executing in parallel, and that's why GPUs really are hundreds of times faster than CPUs at data-parallel tasks.
You can bluster & nit-pick all you like, but it doesn't change the fact that most developers who've successfully ported code from CPU to GPU
Re: Title condradicts summary (Score:5, Insightful)
Depends on what you mean by "faster." If you mean clock frequency, then perhaps. Also perhaps if you mean an individual core of a CPU vs a core of a GPU.
In this sense, it's the time to perform massively parallel instructions. GPUs are generally hundreds of times faster than CPUs for such calculations. Part of this is because a CPU can have a few cores, but a GPU generally has thousands of floating point units. The other part is that CPUs are general purpose central processors while GPUs are very specialized to optimize them for specific kinds of tasks.
Think of it like a CPU is 4 guys with Swiss Army Knives while a GPU is a team of 1,600 guys each with a battery powered, professional screwdriver. Guess which one's faster at screwing 1,600 wood screws into 400 posts for a building. Now guess which is faster at cutting a traced outline on a single piece of paper.
http://www.nvidia.com/object/w... [nvidia.com]
Re: (Score:3)
I'm not sure where you are getting this. For repetitive math a CPU will be doing something like 4 instructions per clock cycle while a GPU will be in the low thousands. Also AMD tends to have more parallelism running slower than NVidia so for the right kinds of computations it is even better.
Re: (Score:3)
There are entire branches of computing and computing hardware dedicated to this idea. Before Linux Journal gave up on the idea of dead tree publication, you could always see the ads for multi-GPU compute machines (usually on the back cover).
3D rendering is all about heavy duty math.
Then there's PureVideo/VDPAU and VAAPI. The idea that dedicated silicon can do things better than a general purpose CPU is a pretty old (and obvious) one.
Re: (Score:1)
Compared to MS office? Ms office is atleast as slow. I use libre office as much as i can. All work documents and spreadsheets, that i start i make with libre office. I like it, it works for me. Sure they are not the most complex stuff and i don't do that on daily basis. Still the speed is quite fine for me.
MS office takes 500 megs on my machine, and that's just excel, powerpoint and word installed. Libreoffice is 450 megs and that's base, calc, writer, draw, impress and math installed.
Do you need to install
Re: (Score:3, Informative)
I just ran the test on both MS Office and LibreOffice. MS Office was 5.6x faster than LibreOffice with OpenCL turned off. MSOffice score 9705.51206654159, LibreOffice score 54296.0680468936. LibreOffice scored 218.087068492849 with OpenCL turned on. Hardware Core i3 2100, 8GB RAM, nVidia GTX 770 .
Re: (Score:1, Flamebait)
Or possibly stop using a Mac and switch to the platform it's optimized for, ie PC. Apple may own the phone, but they're still 2nd bastard cousin on the desktop.
Besides, are you authorized by Cupertino to use ghetto opensource software on their sleek Williams Sonoma OS? Aren't you supposed to be using iLife or whatever it is the reanimated corpse of Steve masturbated out of his dessicated gonads? Wouldn't surprise me if there's code in the OS to slow down unapproved applications just to keep you in the A
Re: (Score:1)
That was a factually correct statement. Mac OS is the bastard cousin when it comes to desktop software. The flaming didn't start until 4th sentence.
Re: What about the rest of it? And Firefox? (Score:5, Insightful)
My opinion of Apple and Apple products has changed. Not that I'd ever buy an Apple product, but I went with a non-technical friend to an Apple store; she wanted me to go along when she bought a new Macbook. I was amazed at the high level of service and the extent of the support structure. She paid twice as much total for her Macbook as I paid for my Asus Zenbook (she bought the training and extended support which added $400 to the total), but she has a level of comfort that is rather high for a non-technical user.
I was pretty impressed, and I can see why people might want to buy the Apple brand. They don't care about the closed design and complete control by Apple. They want to have a place to go to get help and support, of a type and level that they can understand and feel good about. This particular Apple store was packed with people and always is. There's a reason, and I learned what it is that day.
Re: (Score:1)
"closed design"? really? I think you are thinking about Apple from 15 years ago. OS X has a lot of open source in it and its far easier to use open source software in it than on Windows. OS X also uses a lot of open standards, unlike MS "extend and embrace" system. As for the hardware, you can run Linux or Windows on it if you want, so I wouldn't exactly call that "closed design" or "complete control" either. OS X also has free developer tools unlike some other OS.
Re: (Score:2)
"closed design"? really? I think you are thinking about Apple from 15 years ago. OS X has a lot of open source in it and its far easier to use open source software in it than on Windows. OS X also uses a lot of open standards, unlike MS "extend and embrace" system. As for the hardware, you can run Linux or Windows on it if you want, so I wouldn't exactly call that "closed design" or "complete control" either. OS X also has free developer tools unlike some other OS.
I think by "closed design" they're talking about Apple's hardware. Over the years, it's become less tinkerable, particularly the Macbooks as they've exchanged accessible battery, RAM and hard drives for ultra thin and lightness and battery-life. It's comforting to some that a few years down the road you can upgrade some parts and extend the life of the machine . But your typical consumer might not care so much, and the newer Macbooks are so freakin' thin and still feel solid like they're not gonna crack i
Re: (Score:1)
I am a fan of OSS but, really? Easier to use open source software? In what way is it easier? You install the damned thing and use it. There is much more OSS for Windows than there is for OS X. The end result is the same. How is it any easier, regardless of the platform you use? Opera is as easy on Linux as it is on OS X as it is on Windows. Install the damned thing and use it. The platform does not, nor should it, change that.
Re: (Score:2)
OS X is a unix-alike. Most of the software that runs on Linux will run on a Mac without a problem.
Your idea of open source seems to be of the double-click variety. There's a LOT of open source software that requires a ./configure; make to install. That works on Linux and Mac. If one of the developers is a masochist there might be a way to build it using Visual Studio on Windows, or more likely a way to build it under Cygwin.
Re: (Score:2, Interesting)
Re: What about the rest of it? And Firefox? (Score:4, Interesting)
You make a good and valid point, but the people who are buying Apple for comfort will buy a new one every few years because that maintains their comfort level.
I learned, with my visit to the Apple store, that buying Apple for most people is not about
1) Being able to tweak and customize (in general; yes, you can do that if you know how)
2) Worrying about cost[1]
It is much more about
1) Comfort and assurance
2) Style
3) Relative ease of use, and lots of hand-holding.
As technical people we tend to forget how hard things can be for non-technical people. (And no, this is not because we're in any way "better" than they are, we just are good at different things.)
[1] It's true that many Apple buyers have plenty of cash. But I'm active in the local writers' community and it's amazing how many really poor writers (poverty is chronic for most writers) will scrape together every last cent to buy a Macbook.
Re: (Score:2)
Hm... I took my six year old macbook pro in to an Apple Store the other day. They were happy to work on it. They ran a 24 hour diagnostic, free, after which I took it home and fixed the problem (they would have happily done so). My friend took in his 5 year old one, and they replaced the mainboard in it for free because it was one of the first lead free batches that had known faults.
Re: (Score:2)
Re: (Score:2)
She paid twice as much total for her Macbook as I paid for my Asus Zenbook
I'm curious about that. Last time I (well, my SO actually) bought a Zenbook (UX21 as it happens), it was very similar to the equivalent 11 inch macbook air. It was I think it was slightly cheaper but had a larger drive. The outer dimensions, and critically the weight were pretty much the same.
The main driving factor in spec/price seemed to be driven by who has most recently released a new model, the new models always being slightly be
Re: (Score:3)
If you knew anything about computer history you'd know that the GUI was inevitable. Steve & Bill may have popularized GUI interfaces through their positions but they weren't the only people thinking about computer graphics. In the same year the Macintosh was released 1984, Digital Research announced GEM & MIT announced X, Xerox was already developing their own graphics system.
In 1985 we got GEOS, AmigaOS, and the first shaky Windows.
The world could have gotten along fine without Apple or Microsof
Re: (Score:1)
I seem to recall the GUI came from Xerox and the mouse was around in the late 1960s. The mouse even had three buttons on it. I forget the name of the talk but you can still find it online if you know what to look for. I re-watched it not too many months ago and then was surprised to see someone link it here in the past week. They had file headers, cut and paste, and function keys - back before 1970. The GUI was a Xerox thing - SPARC as I recall - that Jobs borrowed. This is not a slight, or not intended to
Re: (Score:1)
Nice. Thank you. My memory is old, it does not do error correction. Upgrades are expensive and ineffective as their is a bottleneck due to processor speed.
I did not fat finger it. I added the extra letter thinking it was correct. (I am not scared to be mistaken.) I knew that GUI had pre-dated Apple with Xerox being commercially available before Apple did so. I know it was also in labs before that. I seem to recall a Bell Labs solution that was GUI as well though I do not know if it, directly, ended up in a
Re: (Score:1)
There was this one time, at a truck stop in Pennsylvania, but they liked it so it does not count. It's not gay if your bags don't touch.
Re: (Score:2, Informative)
Ok, it's fantastic that LibreOffice spreadsheet calculations are faster now. But what good is that when the rest of it is so goddamn bloated?
Because speeding up spreadsheet calculations matters a lot to some users?
"So goddamn bloated" is a subjective term. I use LibreOffice regularly and it works for me. Pretty damn impressed in fact. Sure, faster is already better but your exaggeration is wild.
Re: (Score:2)
Ok, it's fantastic that LibreOffice spreadsheet calculations are faster now. But what good is that when the rest of it is so goddamn bloated?
Because speeding up spreadsheet calculations matters a lot to some users?
"So goddamn bloated" is a subjective term. I use LibreOffice regularly and it works for me. Pretty damn impressed in fact. Sure, faster is already better but your exaggeration is wild.
Huh. Troll? Somebody has an agenda.
Re: (Score:2)
It's a nice alternative to XML-stored, CSV-formatted data.
Re: (Score:2)
Oh nice, you've used IBM Cognos too?
Re: (Score:2)
Wait, what? O.o
I was trying to do a joke here, but you make it sound like something as stupid as that really did exist?
Re: (Score:1)
CSV has way too much industry penetration. It is not even suitable for an address book but it gets used for far more than you might want to imagine. I guess we could call CSV the original noSQL... It was wrong then and it is wrong now. It is simple and fairly easy to manipulate and transfer to other formats, I suppose. Anything large should not be kept in CSVs but it is not unusual to see it. Once upon a time many shared hosting companies did not offer things like MySQL or PHP - just PERL was a blessing. Du
Re:Spreadsheets (Score:4, Informative)
As someone who has never found a use for spreadsheets anyway, I wonder whether there is anyone who uses LibreOffice spreadsheets? For what is this used, accounting?
Just wondering...
Spreadsheets are tremendously useful for a number of things. Accounting is a perfect & common use. Doing cost estimates or comparisons, doing sensitivity analysis (what happens to the answer if I change one parameter?), anything where you have a table of data and you want to analyze it, sum it, average it, etc.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
They're difficult to debug (there was a story on Slashdot about a study on that a year or two ago).
Re: (Score:2)
For simple accounting, and for really complex accounting. Supposing you have accounts for 50 divisions around the world in various different currencies and local accounting standards, and you need to put them together as a set of group accounts. There isn't really any alternative to doing it on a spreadsheet.
Re:Spreadsheets (Score:5, Informative)
That's funny because spreadsheets actually came from accounting. The first spreadsheet wasn't created by a geek or a programmer, it was created by an accountant. It's simply the electronic form of something that was already being done on paper.
Spreadsheets not suitable for finance or accounting?
That's where they came from.
Re: (Score:2)
Re: (Score:2)
What is a specialised tool? Something that compares the sums of rows and columns? Something that makes simple if-then decisions and applies tax formulations depending the final or intermediate answers? Something that stores large tables of expenses, and revenues? Something that can do simple look-ups like how many hours has a certain person worked within these weeks?
Quite often the specialised tool is a fancy GUI that sits on top of a spreadsheet.
Re: (Score:2)
The big one is data transformation. Most companies don't have some kind of seamless integration from department to department and system to system. Each department has some sort of input, and most output multiple subsets of that data to different places/departments/people in different formats like as reports, as data loads into other systems, or a smaller data set for further work. Excel allows non-IT folk the ability to handle this work without having to ask a techie every time they want to make a chang
Re:Spreadsheets (Score:4, Interesting)
I like to play with orbital mechanics - "hard science fiction" scenarios such as orbital catapults and the like, and spreadsheets are a decent way to quickly run the numbers for a large range of parameters. For example, a few hundred mile tumbling-cable space elevator around the moon could grab payloads directly off the surface and launch them on Hoffman transfer orbits to Mars or Venus, without ever exceeding a fraction of a g acceleration.
Re: (Score:3)
They also told me how much I should be offering for junk components in Eve if I was still to turn a profit on reprocessing. I've not played the game for years, but spreadsheets are a pretty important tool. They also tell you which ore is most profitable to mine.
Re: (Score:2)
Re:Spreadsheets (Score:4, Interesting)
Re: (Score:2)
I use them as a mapping tool for old-school grid-based CRPGs like Wizardry and Might & Magic.
Re: (Score:3)
It's almost a no-brainer, but for people who are on the fence on putting data in spreadsheets or a database: Start with spreadsheets, but organize them in a database table-like form (as opposed to an interface-like form with little mini-tables scattered around one subsheet, which is what most people tend to start out with).
1. Use only one header row per subsheet
2. Make as many rows as possible just straight data rows, with every cell conforming to what is dictated by the header for that colu
Re: (Score:2)
Back in the day, I had a teacher using spreadsheet software for a grade book that could automatically sum the scores and apply a curve instead of having to do it all by hand.
I've done that too. The system was at a uni for a 3rd year module. The point with that is that everyone did the first two years, so the curve for those years was easy. For the 3rd year there were not enough people to grade on a curve, acurately, so the curve was derived from that cohort's second year results.
I did a spreadsheet, realise
Re: (Score:2)
As someone that works with data sets all the time, (Score:5, Informative)
here are my answers. Spreadsheets are used in several cases:
1) When you have a small-to-medium-sized dataset (100m data points) and want to do a particular set of calculations or draw a particular set of conclusions from it just once or twice—so that the time invested in writing code in R or something similar is less than the time needed just to bung a few formulas into a spreadsheet and get your results. Once you get into analyses or processes that will be repeated many times, it makes more sense to write code.
2) Similar case, when you need to work with essentially tabular database data, but the operations you're performing (basic filtering, extracting records based on one or two criteria, just handing data from one person to the next) are either so simple or will be repeated so rarely that a MySQL database is overkill and just emailing a file back and forth is easier.
3) When you are working with data as a part of a team, and certain members of the team that are specialists in some areas related to the data, or (for example) members of the team that are doing your data collections, aren't particularly computationally expert. Spreadsheets are hard for laymen, but it's doable—a dozen or two hours of training and people can get a general, flexible grasp of spreadsheets and formulae. It takes a lot longer for someone to become basically proficient with R, MATLAB, MySQL, Python, etc., and you really want those specialists to just be able to do what they do to or with the data, rather than focusing their time and effort on learning computational tools. Spreadsheets are general purpose and have a relatively shallow learning curve relative to lots of other technologies, but they enable fairly sophisticated computation to take place—if inefficiently at times. They're like a lowest-common-denominator of data science.
We use Spreadsheets all the time in what we do, mostly as a transient form. The "heavy hitting" and "production" data takes place largely in MySQL and R, but there are constant temporary/intermediate moments in which data is dumped out as a CSV, touches a bunch of hands that are really not MySQL or R capable, and then is returned in updated form to where in normally lives.
Should have read "or less" (Score:2)
100m data points *or less*
Re: Should have read "or less" (Score:3, Informative)
"Or fewer"
Re: (Score:3)
Re: (Score:2)
I bought a harddisk the other day.
I copied the table of harddisks and prices into a spreadsheet from a computer store's website. I then set a column to extract the number immediately before "TB" in the string from the site, and another number to extract the number immediately after "$". Then I created a column that divided the two. Finally I applied a colouring to the cells based on value from lowest (green) to highest (red) and finally sorted the result by TB and then $/TB.
The end result was a list of ever
Re: (Score:1)
See... I just can not wrap my head around spreadsheets or databases. No, I know how they work. No, I can not use them for anything. I can set information in one and extract information from one. I can do work on those results. I am slow... In your example, I could have done all that in my head faster than I could have done it in a spreadsheet. My degree is in Applied Mathematics but I simply can not conceptualize the use of spreadsheets or databases. I can use them in limited cases where I am already famili
Re: (Score:2)
I really can do the math in my head faster than I could do it on the computer
You're assuming a fixed data size if you think you can beat a spreadsheet, and that is precisely where the benefit of a spreadsheet lies.
Were we talking about a handful of harddisks I would agree, but I was talking about every harddrive available from every manufacturer from a given online retailer. By typing in 2 very simple division equations, 2 very simple string lookups, and hitting autocolour followed by sort on 2 columns I had completely sorted and sifted through the cheapest $/TB drive for a reasonab
Re: (Score:1)
Good guesses. I have my PhD in Applied Mathematics and a MA in Electrical Engineering. And I *might* be able to do all 138 drives faster than you can do it in a spreadsheet. It is strange, I look at numbers and just *get* them. I can not really explain it. Early on, in basic mathematics, I had a hard time doing proofs. Showing my work was a problem. Eventually I had a teacher who explained the concepts in an abstract form and that is when things just sort of clicked.