Slashdot Log In
Afterlife Will Be Costly For Digital Films
Posted by
kdawson
on Sun Dec 23, 2007 03:34 PM
from the same-time-next-year dept.
from the same-time-next-year dept.
Andy Updegrove writes "For a few years now we've been reading about the urgency of adopting open document formats to preserve written records. Now, a 74-page report from the Academy of Motion Picture Arts and Sciences warns that digital films are as vulnerable to loss as digitized documents, but vastly more expensive to preserve — as much as $208,569 per year. The reasons are the same for video as for documents: magnetic media degrade quickly, and formats continue to be created and abandoned. If this sounds familiar and worrisome, it should. We are rushing pell-mell into a future where we only focus on the exciting benefits of new technologies without considering the qualities of older technologies that are equally important — such as ease of preservation — that may be lost or fatally compromised when we migrate to a new whiz-bang technology." Here's a registration-free link for the NYTimes article cited in Andy's post.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Linus has already solved this problem (Score:5, Funny)
- L. Torvalds
Re:Linus has already solved this problem (Score:5, Insightful)
- L. Torvalds
Parent
Re: (Score:3, Informative)
http://en.wikipedia.org/wiki/Doctor_Who#Missing_episodes [wikipedia.org]
I'm not sure I'd call it "open sourcing" but... (Score:5, Insightful)
I wouldn't call it "open sourcing" exactly, but let's just say that films won't soon go extinct [thepiratebay.com], at least as long as there are people willing to copy them.
Actually, that's how books survived. The only ancient books we have now are the ones people thought were important enough to copy regularly, plus a few random things that survived for a ridiculously long time.
Parent
Important Enough to Copy (Score:3, Interesting)
Huge amounts of fundamental culture simply disappears because it is so transparent or ordinary to those it affects. The next generation comes along and they forget about it because of that apparent mediocracy. For example, breast feeding was normal, ordinary, and public in America up through the 1950's. Movie and later Television rule-makers didn't allow showing it unless it was part of some National Geographic type presentation
Re: (Score:3, Interesting)
Re: (Score:3, Insightful)
The problem with this type of storage and distribution, is that it strongly favors only what is popular.
This comment really hits the nail on the head. Even worse is that it only favors what is popular at a given moment. What is popular today might not be as popular tomorrow, and what is popular after that could be different still. If we relied on the interest of individuals to preserve content, then all it takes is one uninterested generation for valuable content to be lost forever. It doesn't matter if people for the next thousand years would love to have that content, since once it is gone it is gone
Why? (Score:3, Insightful)
Re: (Score:3, Interesting)
yes and whocares - now for the cost (Score:5, Insightful)
Yes - You don't need to have 5.25" drive now to read back data that you stored onto an 'old' IDE drive 2 years ago. And that's a bad example because you can still get 5.25" drives. 200 years from now when we're working with crystalline storage methods, we won't have to read back from HDD platters.. just from the holographic storage drives that things were transferred to with the last generation of storage devices.
Will we still have film projectors 200 years from now? Possibly not.
Whocares - because the formats used to store digital film aren't exactly H.264 or whatever fancyschmancy codec the copyright-infringent care about; google 'digital intermediate'. And yes, those formats do tend to change, but they all remain lossless and, again, things can be transferred with each generation.
Will we still know what to do with film 200 years from now? Ahhh.. there's the kicker.. probably, yes.
This is also where the cost comes in - you have to keep upgrading to the latest formats and the latest storage devices to ensure that there will be no 'digital divide', so to speak.
With film, you don't incur this cost. It's lossy in an analog sense, but if somebody looks at a film reel 2,000 years from now - and we assume to still have the same visual system in our watersacks - it will be trivial for them to see, literally, that it is a series of pictures which, in succession, appear to animate. Even if there's no device to play them back then, it would be trivial to build one from scratch using very rudimentary knowledge.
With digital, even if you have the latest format and the latest hardware to read the device it's stored on, it is non-trivial for the layman to read this file and be able to put it back into a picture; in fact, it tends to take people with intricate knowledge of the device and the storage format.
Personally I'm all for doing both, costs be damned, if the material is important enough. That said, do we really need to hold on to all material forevermore? Like a history book, it should be enough to retain the highlights (be they positive or negative), and not cling onto minutiae, as a society. Similarly, like family archives, those who believe something to be well worth the preservation for future generations (either within the family or civilization as a whole), will - or at least should - do so on their own and have history prove them right, or wrong.
Parent
Re: (Score:3, Informative)
Re:Why? (Score:4, Informative)
Actually, the damage to Metropolis is due in large part to editing rather than damage of the film stock. Metropolis was edited early and often; the only time the whole, original film was viewed was during its original (and brief) German first-run. Subsequent German, US, and other world-wide releases contained major deletions, reordering of scenes, and other changes which significantly changed the storyline of the film. The only reason that we now know the original order the scenes were meant to go in, and just how much has been lost, is due to the discovery of the original score and title cards.
Parent
Well mosty of it is crap anyway (Score:4, Insightful)
Preservation was a lot easier when the media lasted longer but by far the largest problem is the increase in the amount of data.
What is interesting is that old analog film & tape also degrades, but does so more gracefully. They also get degraded by reading, not just by storage. Archives of old footage etc have largely been converted to digital to allow older signals to be accessed without damaging the originals.
Re:Well mosty of it is crap anyway (Score:5, Informative)
Also, the line in the article regarding digital editing is incorrect. Films are edited in digital form on the computer, but the edit decision list is given to a negative cutter who cuts the negative. There is no loss of quality editing digitally.
Parent
Re: (Score:3, Insightful)
Quantity vs quality (Score:3, Interesting)
I'm not convinced we need to keep 90+% of youtube or Friends and similar crap for people to watch 100 years from now.
Why not just... (Score:5, Insightful)
Re:Why not just... (Score:4, Funny)
Parent
Re:Why not just... (Score:4, Insightful)
If just a DVD-quality copy of the final cut, then it's certainly not a problem.
If you are aiming at preserving the final cut in its glourious uber-HD, lightly compressed form, things get a bit trickier.
If you want it all - all the shots, the various data (textures, models, etc) used in digital production in their raw, original form, well, in that case we are speaking of storage space well beyond what you found even in a heavy torrent user's computer.
Parent
how much? (Score:4, Informative)
I cant help but relate some personal experience here. I know its not production quality, or lots of information, but I recently pulled out my Apple IIe from storage. It included the original 5 1/4 floppy disks and drives.
There was also a cardboard box with ~150 floppy disks, some as old as 20+ years. NOT A SINGLE ONE WAS BAD. Yes, "Zork" still works!
Could it possibly be that the quality of media just isn't up to the demands of a longer life of storage anymore? We all know how Cadillac runs that racket, as in sell the crappy car, and make the money off replacement parts. Has media storage gone the same way? As in 'sell the media, but just good enough to work for x years' before being replaced. And with the demands to increase revenue year over year for public companies, perhaps that time-frame has become shorter and shorter over the years to keep the money flowing in.
Or am I just being too cynical? But you know, a world where such works as "Zork" can survive and "Legally Blonde" can not, on their respective media, might not be that bad.
Re: (Score:3, Insightful)
So, higher density = shorter shelf life. I've tried to read in some 10 year old DAT tapes, and no luck at all (not that I needed the data, just to see if it would work).
Those who forget history... (Score:5, Insightful)
> Those who do not study history are doomed to repeat it
Yes, and those who do study history are doomed to watch in frustration
as it is unwittingly repeated by those who do not
Hard drives don't "degrade" (Score:5, Informative)
The reasons are the same for video as for documents: magnetic media degrade quickly,
The myth of bit rot on hard drives is just that- a myth. It's been perpetuated for two decades by the idiot Steve Gibson, selling his own snake oil (Spinrite), and unfortunately, not enough people are calling him on it [radsoft.net]. I thought it actually did something too, until I read that post from someone who actually knows how modern drives work. As the author points out, there's a track that can only be written at the factory, and if what Gibson claimed were true, ALL drives would be dying left and right after a few years. Funny how I've found drives made almost a decade ago working just fine now...
The problem hasn't changed; it's mostly obsolescence in drive interfaces, and the drives themselves (for tapes.) PATA is common these days, but everything is going towards SATA, for example.
Both DAT and 8mm were in common use as little as 6-7 years ago...but you'd be fairly hard pressed to find a place to but either now save eBay. And...do YOU want to entrust a backup to an ebay drive?
Is it really that hard to solve? (Score:5, Interesting)
Re:Is it really that hard to solve? (Score:5, Informative)
Beyond that, single-bit errors in encoded data streams (e.g. MPEG2, AVC, MP3, AC3) can lead to large distortions in the decoded data. You really have to store everything raw in order to reduce the chances of severe corruption and increase the chances of recovery.
Parent
Re: (Score:3, Interesting)
The problem isn't necessarily the medium of storage itself, its the whole of how the information is encoded. After awhile, the machinery and knowledge of the format will be lost.
With normal film, hold it up to a light, the image is there. Suppose that in 200 years someone wants to play back the film - even if such a machine did not exist, it would be easy to construct.
I recall reading a similar problem nasa ran into... they wanted to resurrect s
Re: (Score:3, Insightful)
Type "dvd glass master" in Google's search box and you'll find out.
My favorite part: (Score:5, Insightful)
Leaving out the humongous math error, why can't you just store the digital fucking media in the same salt mine? The things that damage analog film are the same things that damage digital media.
Is it any wonder we have the expression "lies, damned lies, and statistics"? This article is all three, with some incompetency thrown in.
And yet (Score:3, Interesting)
Back in 90/91, I worked for a company that did burning of CDs and Laserdisc (compressed data for the DOD). The CDs cost something like 5 or 10 each, and the laserdiscs were a couple of hundred each. IIRC, These were based on gold, and would last something like 50 or 100 years without losing a single pixel. I would guess that hollywood could easily afford these.
nonsense (Score:3, Informative)
The consumer format wars between Microsoft, Apple, Sony, and other companies have no influence on this.
Stupid article and stupider people (Score:5, Insightful)
Who gives a rats ass if a given copy of a film will degrade in 10 years. I can make a 100% perfect copy of the thing in minutes. Copy the data every year. Hell copy it 100 times. Copying also makes the obsolescence of formats meaningless.
I still have emails and RTF documents written in 1994. These are 100% perfect copies of the original data. Is that somehow to be interpreted by brain-dead fear-mongers that any day now my data will be "obsolete" since the obviously 15-year old media is almost degraded beyond recognition? Or are people a bit more intelligent and realize I have already copied this from hard drive to disc and back about 30 different times?
Not a new problem (Score:4, Insightful)
Second, I just heard that the studio that produced Aerosmith's first album has lost the masters, so they're going to re-record it.
This kind of problem isn't new, and blaming it on electronic media is silly.
Yes, you do have to take steps to ensure the availability of it in the future - but the same is true of analog versions too. If you don't have a good filing system, or your 'vault' is the backseat of a car in southern California, the reels are going to get damaged/destroyed/lost, too.
I was on a railroad photographers' list for a while, and I remember the digital/analog debate came up one time. Someone said, "I'll be laughing when you lose all your files because your hard drive crashed and don't have pictures any more!" Obviously he never considered he could easily lose his negatives/slides, or have them damaged in a flood or fire. Analog media has different risks and storage requirements, but they BOTH require proper storage. (And, frankly, digital has the additional advantage that it can be easily backed up at multiple sites with no loss in quality.)
The problem is older and more extensive (Score:5, Interesting)
Technicolor dye transfer (imbibition) prints were much less fugitive. Color separations onto black and white film stock (often termed YCM for yellow, cyan, magenta) are much more robust. Production of these separations (and imbitition relief "matrix" films) was intrinsic to the Technicolor printing process (even if the film was shot in conventional tripack negative, then transferred to Technicolor for printing), and films where these intermediates were saved (or where someone presciently thought to have a set of YCMs made), are much safer for the future than anything kept only on color stock.
In the 70s there were some photo places (especially in Los Angeles) that marketed Eastman Color Negative 5247 movie film (short-end remnants from the movie industry) as a cheaper alternative for 35mm color negative still photography, and printed this onto 5283 color print film (same as movie prints) for 35mm slides.
I recently found a few boxes of these that I had shot back then (and stored under entirely careless, or Arrhenius/Murphy if you prefer, conditions). I am not good at evaluating color negatives by eye, but the positives were faded either to mutated colors or to almost nothing.
Even simple technologies can have amazingly short shelf lives under conditions of disuse. I recently turned on my stereo system after close to 3 years of not being used. The amplifier, CD player, and LP turntable all failed to operate. Part of this might have been due to de-formed electrolytic capacitors; these appear to have more-or-less repaired themselves after a couple of hours with the power turned on. Both the CD player and the turntable suffered additional electromechanical problems that required a combination of manual exercise and cleaning to rectify.
None of these devices have anywhere near the scary sophistication of a modern hard disk drive.
Seeing as I cannot remember what I last set my external firewall password to, imagine the additional challenge of future Hollywood being bitten deeply in the butt by present Hollywood's favored time-bombed destined-to-be-lost-art proprietary DRM technologies, with the keys long since dissipated in Hollywood's perennial miasma of mergers, acquisitions, lawsuits, cocaine, and personal vendettas.
Analog still not dead (Score:3, Insightful)
On the other hand, digital archival of data, which can offer incredible clarity and potentially 1:1 accuracy in restoration often becomes an all-or-nothing proposition if even a tiny bit of the data is lost or altered. Even with file formats/codecs that offer some form of error correction or redundancy, the final result we may end up seeing could be little more than randomized shifts between a blank screen and a perfect image... all of which are swapped in and out so quickly, we may not see the recoverable parts long enough to identify any usable pattern.
For example, try comparing something like the "scrambled" channels (mostly the porn channels) on cable television back in the early to mid 90s to something like DirecTV during a heavy rain storm. Even though the cable stuff was typically visible warped and uncomfortable to look at, you at least had a good idea of exactly what was going on behind the scrambling, even without the audio channels. But, try watching a DirecTV signal under less than ideal weather conditions, and the best you get is a bounce between a random mosiac and pitch black, combined with severely degraded audio pops here and there. You're luck if you can even get a useful picture of anything on the screen, let alone being able to comprehend what is going on in the show itself.
That said, how difficult would it be to create a micro-film drive (photosensitive analog scanner/burner) that could not only store any document on a computer in an analog form, but do so in a format that could be interpreted entirely by the human eye using a proper magnifying device. For that matter, why not create a hybrid device that would store both an easily visible analog form of a document as a high-resolution thumbnail, along with a digital version using pattern of dots similar to how data would be stored on an optical disc. This way, no matter what device you use to extract the information, you'd always have the means to access the data you need.
Re:Just imagine. (Score:5, Insightful)
Seriously, IF the older films are an authentic art that deserves preservation, the why is most of it scrapped on the cutting room floor? why are all the really old films sitting still on their Nitrate Stock in archives in hollywood slowly turing from film to dust?
AS others point out, released to the Net a movie is saved in various codecs, on various media (hard drive, tape CDR DVDR laserdisc even film FOR FREE just like music and most other data is. Horrible thought that; information in the hands of the people.... unsupervised, heck UN TAXED!
In the 15th century the Church tried desperately to put an end to this new Printing Press because it was putting their scribes out of work. They even excommunicated printers. Now we do the same only we use Lawyers.
I await the next turn of the wheel to see what damn foolishness humans are yet capable of..
Parent
Re: (Score:3, Funny)
Re:$208,569 (Score:4, Insightful)
Parent
Re: (Score:3, Interesting)
Re:$208,569 (Score:5, Funny)
And some, god help them, migrated to Apple's Almost Better Than The Competition So You Can Feel Better About Using A Proprietary Format For Only Three Dollars a Pop Codec (.aabttcsycfbauapffotdapc). Those Apple Engineers cost bocoup bucks.
Parent
Re:$208,569 (Score:5, Interesting)
My calculator says a 2 hour movie at 24 frames/sec will have about 175,000 frames.
A few more button presses tell me that's a bit north of 6 terabytes of data.
Let's quadruple that to include all the cut scenes and unused footage, to 25 terabytes.
TB drives are available now for $400 or so each. They use under 10 watts idle.
Building a 30 drive RAID would thus cost $12,000, and require perhaps 500 watts if run constantly, including cooling. Let's bump that to $15,000 to pay for controllers and chassis.
Three such arrays (in case of earthquakes, etc... keep 'em at opposite ends of the continent) would cost an initial $45,000, take up perhaps 7u of rack space, and need 50 kWh per day for all three. At 30 cents per kWh, that's 15 bucks a day, or $5500 per year. Let's double that, assuming those 7u cost you $5500 a year.
So... my numbers, triply redundant, come to an initial investment of $60,000 (profit, hey!), and a yearly cost of $20,000 (more profit!).
How the hell they came up with $208k is beyond me. I'm thinking I should start a company that does this for the studios, it's looking quite lucrative.
Parent
Re: (Score:3, Funny)
Yes, a punch-card system is perfect...until somebody drops the deck...
Re:So pretty much ... (Score:5, Insightful)
The answer is simple, copy it over frequently.
Yeah, from the article, are several silly things are going on here:
Parent
Re:So pretty much ... (Score:5, Insightful)
In all seriousness, the biggest obstacle to preserving a history of our culture is copyright. If the owner of the copyright doesn't care to preserve the piece of our history that they have their monopoly on, the information will simply deteriorate and there is nothing legally that can be done about it. We can only hope that the evil dirty thieving pirates save our history for future generations.
Parent
Re: (Score:3, Interesting)
It is a pretty simple problem to solve. You set up a smallish data centre on three continents. You install some LTO4 tape libraries and start replicating the data to each over the internet. With LTO4 you are looking at ~600TB per 19" rack, and when you are not accessing the data (most of the time) you are not consuming power. Add in some checksumming and
Re: (Score:3, Informative)
Re: (Score:3, Interesting)
How about printing a few copies of a binary bar-code record in big books of archival quality paper for terms of a few centuries? Or how about blowing the bit pattern into any other format with some longevity on some nice passive substrate like a non-flowing glass if you'd like to keep them for a few millennia? Two hundred plus grand a year per film to maintain, my aching ass. Give me two million bucks - the supposed cost to archive just ten films - and I *guarantee or your money back* that I can design (an
Re: (Score:3, Interesting)
No. The trick here is only half archival; the other half - and it's not complex, just apparently not obvious - is that it should take any half-competent tech no more than a day or so to rig up a reader using discrete components of current technology, the task having intentionally made simple. An optical diode, resistors, a transistor, maybe a lens system and an XY table. Not "drives" and metaconstructs like them. This way, the components can be emulated if required (doubtful, but possible) by higher techno
Re: (Score:3, Interesting)
Re: (Score:3, Informative)
Putting the movies on film for archiving isn't really an option, since too much of the quality would be lost.
Incorrect:
For any given resolution, there is a size of film that beats that resolution. Figure on about 100 line-pairs per millimeter of film for still frames, less for movie frames, and you'll be in good shape. If you want a nice margin of error, quadruple the size or resolution of whatever you are copying to in each direction. If your film hits the silver screen at 4,096 pixels in the vertical dimension, this is roughly 2000 line pairs or 20 millimeters of film, well within the bounds of standard movi
Re: (Score:3, Insightful)