Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
News

Saving Digital History 133

Posted by timothy from the problem-that-keeps-getting-discovered dept.
Gavinsblog writes "The Washington Post is reporting that the Library of Congress in the U.S. plans to initiate the $100 million National Digital Information Infrastructure and Preservation Program (NDIIPP). It is hoped that the project will lead to the preservation of data that is constantly changing on the Internet. But I wonder who will choose what is worth saving?" This may remind you of the LOC's effort to preserve and digitize the audio collection in the National Recording Registry.
This discussion has been archived. No new comments can be posted.

Saving Digital History More

Saving Digital History

Comments Filter:

  • one persons trash... (Score:5, Insightful)

    by trefoil ( 153310 ) <brentsNO@SPAMeasystreet.com> on Sunday February 16, 2003 @06:24PM (#5315931)
    is another persons treasure.. I'd say just save it all and allow others to sift through and decide what is worthwhile and what is worthless.. just like the library..
    • Save it all? I don't think there would be an article if that was an option. You see, there are time and space limitations imposed by the problem that you simply have not considered. Constantly "saving it all", even those parts that have not changed since the last instance, is a poor idea.

    • From the article:

      On top of the $5 million the library received for planning the initiative in 2000, the plan approved yesterday releases another $20 million of funding to develop a system for evaluating and storing digital information. Just as the library receives more than 20,000 printed pieces each day but keeps less than half, it now faces the herculean task of deciding what digital information should be saved for future generations.

      --
      The library doesn't keep all of the printed information it receives, keeping all of the information online is an enormous, if not possible task. The archive.org has terrabytes upon terrabytes of data, and they don't even come close to having everything that was on the web at any one time. With the budget they're talking about, keeping all of this information would most definitely not be possible.

    • Re:one persons trash... (Score:4, Insightful)

      by whereiswaldo ( 459052 ) on Sunday February 16, 2003 @10:59PM (#5316947) Journal
      Repeat after me:

      Disk space is cheap.
      Disk space is cheap.
      Disk space is cheap. ....
      Save everything. ;)

      "The Navy has both a tradition and a future--and we look with pride and confidence in both directions."


      Admiral George Anderson, CNO, 1 August 1961.
  • ...DMCA?

    It's nice that the government can ignore it at will, at least till someone in Hollywood notices...

  • skip slashdot. (Score:5, Funny)

    by Anonymous Coward on Sunday February 16, 2003 @06:28PM (#5315942)
    No need to add slashdot as one of the website. They keep reposting stories here as an initiative to preserve their own history.
    • hey, I thought that was funny... Why mod it down as flambait???
    • What the....?

      $RecursionLimit::"reclim": "Recursion depth of 256 exceeded."
    • I know that was meant as a joke, but skipping sites like Slashdot or Kuro5hin, which tend to maintain their own archives quite well, wouldn't necessarily be such a bad idea. Perhaps the sites most worth saving wouldn't be the Slashdot stories themselves, but the sites linked to by them. The slashdot effect tends to make people want to take their content off-line, which means that the most interesting content lasts the shortest. Archiving the contents of those sites ought to be the top priority.

  • New Media Doesn't Last (Score:5, Insightful)

    by spun ( 1352 ) <loverevolutionary&yahoo,com> on Sunday February 16, 2003 @06:33PM (#5315963) Journal
    It all degrades faster than plain old ink on paper. There are plenty of books that last hundreds of years if kept in appropriate conditions. Film decays pretty rapidly. Tapes don't last, even CDs and DVDs wear out pretty quickly. Gopher is all but gone. Web pages disappear daily.

    The irony is that, while digital files could be preserved indefinitely in absolute perfection, many are being completely lost in much less time than it would take a book to turn to dust.

    Kudos to the folks at the Library of Congress, and other projects like the Wayback Machine [archive.org] who are working to preserve a surprisingly ephemeral media.

    • Re:New Media Doesn't Last (Score:5, Interesting)

      by Xzzy ( 111297 ) <`gro.h7urt' `ta' `rehtes'> on Sunday February 16, 2003 @06:44PM (#5316020) Homepage
      > There are plenty of books that last hundreds of
      > years if kept in appropriate conditions.

      My suspicion is that punch cards will make a return at some point. ;) No, really. Not only does it resolve the longevity issue, but it could also solve the issue of obsolete reading hardware (seems to me it'd be easier for a distant generation to rig up a punch card reader than a cd-rom drive). Punch cards are in a rather obvious format as well, if worst came to worst and humanity nuked itself back to the stone age.. in ten thousand years a disc that looks like a mirror is probably harder to translate than a piece of paper with regularily spaced holes.

      I think the only difference will end up being the material used; how many centuries could a stainless steel plate with pin sized holes last in a library's basement?

      • Re:New Media Doesn't Last (Score:5, Insightful)

        by Mostly a lurker ( 634878 ) on Sunday February 16, 2003 @08:50PM (#5316483)
        Sorry, but this idea will not fly on a number of grounds. Consider how many punch cards would be needed to save even 4.7GB of data (contents of one DVD). It would take over 50,000,000 cards (even if they did not contain sequence numbers). The creation and storage costs would be astronomical and reading them back in to find any data you wanted would take weeks -- just for a single DVDs worth of data. Further, much of the most useful data (images and sound recordings) are more difficult to store on punch cards than almost any other alternative medium.

        • Re:New Media Doesn't Last (Score:3, Interesting)

          by Xzzy ( 111297 )
          I didn't say it was practical, or suggest that the density would be anything to write home about.. but the fact of it is, we haven't yet been able to develop a digital storage format that is longer lived than punch cards. ;)

          i'm sure something better that's got a life of a thousand years or more will come along eventually, but speaking in the here and now the only way to get that is with holes in a piece of paper.

          • Re:New Media Doesn't Last (Score:4, Insightful)

            by JustaGiga ( 193029 ) <{moc.liamg} {ta} {amlussum}> on Sunday February 16, 2003 @11:23PM (#5317026)
            It's not only a concern that physical media may become obsolete, but also the algorithms in which data is encoded on the media. We have lots of old backup media (reel to reel tape, 8mm tapes) at work that are probably still readable, but no one knows how the data was encoded on that media (or more importantly,) what information is on which tape.

            Most commercial tape backup solutions have proprietary encoding solutions, and who knows if that company is going to be in business/supported in 50 years. In fact, for true(r) long-term storage, it's recommended to copy the data from the commercial tape backup solution copy to plain old tar.

            Keeping an archive on media that will be around in 50 years seems like a minor point compared to finding the exact tape with the right data you need in a format you can still decode.

            -JG
      • How about punched tape? It would be more compact, I think. Probably best to use some kind of long-lived plastic rather than paper (mylar maybe?). The holes could be really close together and maybe 32 or 64 bits across the width of the tape.
    • The difference being that this archiving is _digital_, though...

      Didn't you pay attention in that IT class when they were explaining the difference between Digital and Analogue? Digital's main advantage is its reproductability. So if, say, the CIC Lib^H^H^H^H^H^H^HLibrary of Congress were to refresh the information once every five years or something like that, then you've got an indefinate storage period. The problem with it is that it needs constant maintenance. The reason this is better than analogue archives is pretty simple... when analogue decays, it's pretty much never going to achieve its original quality. You can do things to try and make it similar, but you're never going to get it as pure as the original.

      With digital archives, you can avoid the decay simply by transferring. This isn't an option really with analogue because once you transfer, you tend to lose quality. But bits are simply 1s or 0s, and digital transfer can be perfect. Throw some md5 checksums in there to make sure that you don't corrupt the data, and boom... you've got perfect digital copy.

      • Re:New Media Doesn't Last (Score:2, Insightful)

        by spun ( 1352 )
        Yes, I understand the difference between digital and analogue. I didn't learn that in IT class, I learned it when I was 10, on my own, building a robot from scratch using a Z-80 microprocessor.

        That is why I said, "The irony is that, while digital files could be preserved indefinitely in absolute perfection, many are being completely lost in much less time than it would take a book to turn to dust."

        Did you even read my comment before firing off a snide reply?

      • An indefinite storage period is only part of the problem. Even if you keep the 1s and 0s by copying them every five years, file formats go out of scope, and even if you keep the software the file was saved in, the OS that ran it may well be dead (most are, after all) and even if you save a copy of the data _and_ the application that can read it _and_ the OS, what hardware are you going to run it on?

        So its a nested set of problems, with no one solution -- copying, conversion and emulation will all be required.

        There are two major advantages of analog over digital: the first is that inaction over a period of years does not destroy analog material. If you put a stack of paper in a box in the early 90s, it's probably fine. That degree of inaction, however, can be the death knell for digital material. If you put a stack of CD-ROMs or disks away in the early 90s, chances are at least some of that material is gone.

        The second is that while analog degrades slowly, bit-sensitive digital data (encrypted, compressed or executable files) degrades extremely quickly. If you make a mistake handling a book, say, you may end up with one torn page, but if you lose even a small piece of a bit-sensitive file, the entire thing vanishes forever.

        -clay
    • It all degrades faster than plain old ink on paper. There are plenty of books that last hundreds of years if kept in appropriate conditions.

      What scares me too is a lot of the stuff today is not only on very ephemeral media, its also encrypted so that it is readable only under very special circumstances.

      It seems that content is doomed once the technology used to decrypt it is gone.

    • Its so true take teh doomsday book as an example the origonal text survives (albiet in a dead language latin) where as the BBC project from the 1980s need to be resqueued only a two decades after it was compiled
    • Well I don't know this for a fact, but they're NOT magnetic or even electronic. A laser beam is shone on the surface and it is diffracted and reflected back in patterns corresponding to the data on the CD. But the mirrored part isn't on the surface; clear plastic protects it, so I can't imagine any way this can degrade.

      DVDs are like CDs but the data is layered. The layers reflect different wavelengths of light so that allows the format to take advantage of depth. DVDs are thus slightly thicker than CDs but not larger.

      • The mirrored part IS on the surface, or about one layer of paint away from it (it's right underneath the label). Which means the data is vulnerable.

        Also, the materials decay. There have already been reports of early CDs becoming unreadable because the aluminium started corroding. Who knows what will happen in 50 years?

        Yes, CDs are relatively stable, but even the manufacturers aren't promising a CD will be readable in 100 years.

  • WaybackMachine (Score:5, Informative)

    by ChunKing ( 513714 ) on Sunday February 16, 2003 @06:33PM (#5315964)
    Isn't this already being done by the WaybackMachine (http://www.waybackmachine.org)?
  • I am sure this would be violate the DMCA somehow. Just thought I would point out that the government has no clue what they passed.

    Mike
  • It's called Wayback ^_^. But agreed, we need a more comprehensive method of archiving for posterity...but how do we go about doing that? Hard Drives don't last forever, nonvolatile memory is frikkin expensive, and optical media dies after 10-15 years...
  • It is hoped that the project will lead to the preservation of data that is constantly changing on the Internet.

    In related news, the Library of Congress has also purchased a subscription to Playboy.

  • That's good. (Score:4, Funny)

    by Black Parrot ( 19622 ) on Sunday February 16, 2003 @06:34PM (#5315975)


    I deleted all my porn, and I was afraid I wouldn't be able to get it again when I need it.

    • <stonerchick>i was on slashdot and all of a sudden beepbeepbeepbeep and all my porn was gone....it was really good porn</stonerchick>

  • Something Old, Something New (Score:5, Funny)

    by SparklesMalone ( 623241 ) on Sunday February 16, 2003 @06:36PM (#5315980)
    How much energy should humanity spend remembering its past? I love history, but frankly I'd rather they fund more discoveries (i.e. NASA) than archive drivel like my slashdot musings.

  • Quality not quantity (Score:4, Insightful)

    by wiggys ( 621350 ) on Sunday February 16, 2003 @06:42PM (#5316009)
    We already suffer from information overload as it is. Why bother to save the hundred million Geoshities webpages anyway? What's the point of keeping all the data when it's boring and irrelevant?

    Plus not all the data can be saved anyway... sites such the Internet Movie Database, Amazon.com, and even Multimap are database-driven. Even assuming you get access to the underlying database you still need to preserve the code which gets used to generate the pages. And for what purpose?

    Add to that the problem of accessibility. If the data isn't laid out in an easy-to-browse fashion then it's as good as dead anyway. I prefer to browse a library by topic, not searching for keywords and hoping a nice book pops out.

    • Could this be just the back-door excuse John Poindexter needs to get his infomration awareness office to mirror databases?

      One Noid; not a pair.

    • As a society. . . (Score:2, Interesting)

      by kfg ( 145172 )
      we have an incredible fascination with spending today looking at where we were yesterday instead of where we are or where we're going.

      I'm not talking about history. I love history. My shelves are well stocked with various dead trees delineating history.

      I'm talking about our own lives. When we go on vacation we tend to spend most of our time *documenting* our trip rather than living it. Then we live it "in absentia" as a kind of recreational post mortem.

      It's a fascinating to thing to observe, but I admit it puzzles the hell out of me.

      This point was driven home to me a while ago when someone pointed out how odd it was that I only have one photograph of my SO of 10 years. I only have it because my mother took it. In my mind why would I want a photograph when I could just look at *her*?

      KFG

    • Geocities (Score:3, Insightful)

      by Detritus ( 11846 )
      Geocities web pages may be exactly what a future historian is interested in. They tell you something about the common culture and people. Why do you think archaeologists are so fond of ancient trash dumps?

  • the big red dot !?!? (Score:3, Interesting)

    by Brigadier ( 12956 ) on Sunday February 16, 2003 @06:43PM (#5316013)


    This may sound like a joke but I really hope they save the big red dot. I dont know if the website is still in existence but a while back there was a website that had a big red button. When you clicked it, it said you have clicked the big red dot. The counter had some ridiculous number. This was back when it was envogue to show off your hit count.

    • TRBBTDDA (Score:3, Informative)

      by trmj ( 579410 )

      I believe you are talking about The Really Big Button That Doesn't Do Anything.

      A novel concept in its time, it was a strangely addictive big red button on a website. Established in 1994, and linking back to itsef, it was more repetitive than Taco's story postings.

      As interest in it waned, though, they added a message board-ish thing that let people comment on the button. As it was quickly misused, the best comments were left and the worst deleted.

      There, the very first MS bashing in large amounts began with comments like, "Huh? A button that does nothing? Must be a new Microsoft product..."

      Although dead at the age of 5, its final resting place [pixelscapes.com] is in its original home, Spatula City [pixelscapes.com].
  • ... Of Being a Freelance Hacker and Concert Promoter has come true... All I need is a couple of swords and a Pizza Delivery Job and I can make tons just gathering Intel...

    Stephenson's a Genius... We're basically looking at the first instance of the CIC Database....

    Now we can start looking at the Metaverse and nanodrugs.... I seriously can't wait...
  • But I wonder who will choose what is worth saving?

    Well, for start they may as well mirror Google cache and go from there. Panel of recognised authorities should not have too much trouble deciding the standards for the worthiness of existing material. They will need high level of independence, perhaps total autonomy, to be able to do fair job.

  • sloshdat and Mod Point for history (Score:4, Funny)

    by QEDog ( 610238 ) on Sunday February 16, 2003 @06:48PM (#5316040)
    "But I wonder who will choose what is worth saving?"

    Well, maybe they can come up with a system where people post what they think it is important in history and then some of the same people moderate that using a unit called Mod Points up or down to see if they are or not worth saving... maybe call it sloshdat.

    A mechanism would be deviced to protect the figures that make history against the people reading the history, and effect that could be called Sloshdatted.

    I'm sure that with a system like this, historic figures such as many of the presidents would be Modded Down, while anyone who trashes an established monopolistic corporation would appear in the history books.

    A system like this, would, without any doubt, save and Mod Up a comment like the present one for future generations.

    • The only problem is that what people think is historically significant, usually isnt significant to historians. The most information about a people is found in garbage dumps of ancient civilizations. Who knows what future historians will want to look at
  • That is ALOT of pr0n.
  • that the goatse man will NOT be preserved in this way......

    *shudders*
    • Please tell me that the goatse man will NOT be preserved in this way......

      Oh, but he MUST be preserved! How else can future historicans understand just how much we fear that site today?

  • National Security (Score:3, Interesting)

    by handy_vandal ( 606174 ) on Sunday February 16, 2003 @07:00PM (#5316090) Homepage Journal
    It is hoped that the project will lead to the preservation of data that is constantly changing on the Internet.

    One possible reason: because the OIA and Company [slashdot.org] might need the data to track down terrorists, etc. (Much the same way that the FBI keeps a collection of outdated phones books.)

    After all, when the events of Iran-Contra [chadwyck.com] blew over, Congress quietly passed a bill authorizing the CIA to use any Federal agency for cover. Why not the Library of Congress? Indeed, where else? Makes perfect sense.

    • Re:National Security (Score:2, Insightful)

      by tjic ( 530860 )
      Do you think that the intelligence agencies are only
      now realizing that this is a useful idea? This article isn't about the black archives - you can assume that they've existed for years and have no such funding constraints.

  • Choosing what should stay.... (Score:3, Insightful)

    by MosesJones ( 55544 ) on Sunday February 16, 2003 @07:03PM (#5316094) Homepage
    The answer is simple... what represents the goverment mindset of the day will be chosen to represent that mindset in the future. Cynical ? Of course not, why would they be even handed ? Will they store what Al Jazeera (sp?) says rather than what the Washington Post says, why would the views of Palestine be represented over the views of Israel.

    Or of course they will stear clear of politics and pick only science and absolute news, thus making it pointless for future historians.

    Saving what is said OVER what is already saved is an interesting idea, but will this be targeted beyond those people who already retain everything (like CNN and the BBC) or will it include them ? The BBC store everything, "Just in case", will this money record that information yet again, or will it concentrate on other fields after ensuring that the BBC information is already available?

    Historians of the future will have more information than historians of any other generation. Their problem will be that the miriad of views reflected via this information doesn't mean an increase in the spectrum of political opinion, but the ability of everyone to be opinionated.

    Their worst problem is that the leaders of the day (Bush, Blair et al ) don't stand out like the leaders of previous years. Will anyone rate the speach of Powell or Bush against, Churchill or Kennedy ? Nope. So how to judge politics of today, how to judge what should be stored, we have no leaders of merit, we have only retoric. So choose what to store, and realise that history will judge as much what you choose to save, as what you saved. This is a different problem to that which has faced historians up till now.

  • Our ancestors won't be able to read the data if you put it on a floppy -- Dell and Apple are trying to rid the world of them.

  • Open plea (Score:4, Funny)

    by grub ( 11606 ) <slashdot@grub.net> on Sunday February 16, 2003 @07:09PM (#5316111) Homepage Journal

    Dear U.S. Library of Congress,

    Although not a U.S. citizen, I implore you to retain redundant backups of the website goatse.cx [goatse.cx]. Losing this website to a disaster would be tantamount to losing the collective works of Shakespeare, DaVinci and Picasso. The goatse.cx guy [goatse.cx] is an artist in the truest sense of the word.

    Yours very truly,

    grubby
  • It will be interesting to see how this pans out. Currently I suppose many of items we have from preserved from the past exist because they had some inherent monetary value or were of sufficient quality that time and effort was taken to preserve them. This is not the case with digital media as basically its all 1's and 0's to store the content, which tends to have highly subjective value. What will the future judge to be historically important I wonder, the first recorded blog ?, the first mail promising increased penis size? web services? (insert anything you can think of here)?

    I was wondering also about how they actually plan to physically store this information for extended periods of time. I was going to post a question about it until something occurred to me. In 500+ years time I cant really imagine many people will give a crap about much of the digital material that is being churned out today. It will most likey be a case of viewing sonething like AOTC, falling on their asses laughing at the "special effects" but reaching male consensus that Natalie Portman was a babe.

    • ..the first recorded blog ?, the first mail promising increased penis size?

      When google groups went up, the did specifically mention the first major 'spam' (C Greencard) in their press release.

      It all went to shit after that.

  • We need to take extra precautions to preserve some "movies", because, ahhh, they contain certain "positions" unlikely to be witnessed before or since outside of their "industry." I will therefore generously donate 500 burnt CD's of such movies to the people compiling this digital library.
  • begin sarcasm string:>:/ They abandoned the TIA program where the government saved every record made ever because people cried "big brother". Now they want to "preserve" every record and all data ever made, and people cry "hell yea" end sarcasm string

  • Actually.. (Score:4, Informative)

    by NotAnotherReboot ( 262125 ) on Sunday February 16, 2003 @07:21PM (#5316152)
    From the article:

    On top of the $5 million the library received for planning the initiative in 2000, the plan approved yesterday releases another $20 million of funding to develop a system for evaluating and storing digital information. Just as the library receives more than 20,000 printed pieces each day but keeps less than half, it now faces the herculean task of deciding what digital information should be saved for future generations.

    --
    The library doesn't keep all of the printed information it receives, keeping all of the information online is an enormous, if not possible task. The archive.org has terrabytes upon terrabytes of data, and they don't even come close to having everything that was on the web at any one time. With the budget they're talking about, keeping all of this information would most definitely not be possible.

  • Wonder what Disney will think (Score:3, Funny)

    by UTPinky ( 472296 ) on Sunday February 16, 2003 @07:24PM (#5316164) Homepage
    So what I want to know, is if one of Disney's movies get archived, will they sue the Library of Congress?

  • How do we know its real (Score:3, Interesting)

    by Timesprout ( 579035 ) on Sunday February 16, 2003 @07:26PM (#5316172)
    I noticed in the article that one of the topics on which information was being preserved about was 9/11 and that got me thinking.

    On a broader scale news media love the internet because they can make outlandish claims when a story first breaks and then modify it as the facts become available. How do we know whats being preserved is accurate ?

    Secondly, do we trust the people controlling all this nice, easily modified information not to change it to suit some political whim ?

    They say the victor writes the history book. Digital storage will allow the victors to run a few drafts by their spin doctors first.

  • From the viewpoint of meme theory... (Score:4, Interesting)

    by asparagus ( 29121 ) <koonce&gmail,com> on Sunday February 16, 2003 @07:35PM (#5316198) Homepage Journal
    The important information will save itself without outside help.

    For example if talkorigins.org was wiped out of existance tomorrow, the theories it has created will live on in the minds of those who have read them. These essays can be easily recreated by re-reading the various creationist works. On the other hand, if the various creationist works were destroyed, they would probabally not be recreated because they have already been refuted.

    The history of information is the history of massive portions of it being eliminated, but then either re-printed, re-discovered, or re-invented centuries later.

    The Catholic church 'knew' the earth was the center of the universe.

    Along came Copernicus with his helio-centric theory, and the popes tried to lock him in his house for his entire life.

    Now, if the modern versions of these men were to make the same claim, they would be soundly laughed at.

    So, while this is a noble effort, it is merely a collection of data. Time itself the bayesian filter that will determine which parts of the internet are important.

    -Brett
    • I agree with your point, but one thing to note...

      People have been saying Christianity is 'dying' or 'going to die' for thousands of years.

      It hasn't happened yet.

      Just some food for thought.
      • Christianity is a meme, just as science is.

        It has managed to survive by constantly evolving itself through appropriation of new theories. A hundred years from now, I believe that fundamentalist preachers will be espousing DNA from the pulpit and damning those who believe in quantum mechanics.

        The more things change, the more they stay the same.

        -Brett

    • Re:From the viewpoint of meme theory... (Score:4, Insightful)

      by cshirky ( 9913 ) on Sunday February 16, 2003 @09:30PM (#5316643) Homepage
      "The important information will save itself without outside help."

      That's whistling past a pretty big graveyard.

      The problem is that time changes the definition of interesting. Would you be interested in the ads from a copy of the NYTimes.com from 1998? Probably not, unless you wanted to chuckle at the 667Mhz Pentia selling for $2500.

      Would you be interested in the ads from a copy of the New York Times in _1898?_ Those ads are a view into a world you never inhabited, and expose the preoccupations of the era in a way that the articles don't.

      We can look at the 1898 ads, not because the important information saved itself, but because archivists did. Someday the ads from 1998 will have the same interests for historians and anthropologists. Who will do the archiving there?

      If we leave it to the present to sort the good from the bad, the future will never know what we considered unimportant. If you'd asked anybody in 1960 what that era's biggest technological revolutions of the time were, they'd have all said atomic energy and space travel. The real answers turned out to be the transistor and the birth control pill.

      We are just about the worst possible people to ask what's important now, because we're too close, and it would be hubris to pretend otherwise.

      -clay

  • great (Score:1, Interesting)

    by Anonymous Coward
    Sounds great, why is it going to cost 100 million dollars? Can we say pork?
  • Seems like they should hire google to create a CVS type cache of sites. Can you imagine the amount of storage would be necessary to back up "the internet?!"

    • I dunno, I'll donate that box of floppies I have around here somewhere. Maybe if we all looked behind our collective couches we'd find enough floppies to take over the ... ahem ... back up the internet.

      The internet can't be that big that it can't fit on a couple of floppy disks, surely?

  • Nineteen Eighty Four (Score:3, Insightful)

    by Anonymous Coward on Sunday February 16, 2003 @07:52PM (#5316255)
    <rant mode="tinfoil hat">

    In Nineteen Eighty Four, The Party embraced the digital revolution because they could easily control what the news said about them. (Who controls the past controls the future...)

    Anyway, the point is the government may not be the best to be in charge of this.

    </rant>
  • things that NDIIP catalogues from the distant past will one day be reported as 'news' on slashdot.

  • Wtf ? (Score:1, Redundant)

    by IanBevan ( 213109 )

    the $100 million National Digital Information Infrastructure and Preservation Program (NDIIPP). It is hoped that the project will lead to the preservation of data that is constantly changing on the Internet...

    1. Who is to be the judge of what is worth saving ? I mean, let's be honest, there's a *truckload* of 'internet' out there !!

    2. Wouldn't $100 million be better spent on a new hospital or two ? Just a thought...

  • Preservation vs DRM (Score:5, Interesting)

    by dpilot ( 134227 ) on Sunday February 16, 2003 @08:42PM (#5316444) Homepage Journal
    Since the public domain died back in the 1920's, and since this is about digital content, it stands to reason that pretty much all of the content that LOC is talking of preserving will be covered by some sort of copyright, and an increasing portion will be protected by some sort of DRM. What will the LOC stand be on this?

    Since the LOC seems to hold some of the strings over implementation of the DMCA, they can obviously craft a loophole for themselves. But it will be interesting to see what that loophole is, and how it will work. Will they simply leave the stuff under DRM, and have their own copy of keys, or will they manage to have an unprotected copy?

    Enquiring minds want to know.

  • Actually cheaper to save everything (Score:3, Insightful)

    by Mostly a lurker ( 634878 ) on Sunday February 16, 2003 @09:00PM (#5316514)
    I think the practical solution with online data will be to save everything and worry about indexing and selection decades hence when we have much better technologies to carry out these tasks.

    The actual cost of storage is not that high. The highest costs are involved when human intervention enters into the equation.

  • Can you imagine the computational power required for such a task? Now that's what I want on my desktop! (Where's the link to TheOnion's PlayStation 5 story when you need it?)
  • I this it is the responsibility of the creators of the content to deem whether it is important enough to keep on the Internet or not, or else to archive it. If somebody else tries to archive the Net then I believe we'll end up with 95% fluff, and 5% good stuff. The Internet is now so large at a single point in time, it's sometimes hard to find something current let alone wading through years of archives. I say to forget spending the money on archiving the Internet which is already being done to some degree by TheWayBack machine [archive.org], and leave the responsiblity in the hands of the content creators/publishers. The good content will continue to survive.

  • Google already does this (Score:3, Informative)

    by mrm677 ( 456727 ) on Sunday February 16, 2003 @09:54PM (#5316733)
    Google does not evict anything out of their cache. They just keep adding capacity. Hence Google can already see changes to websites. Granted I'm sure that this data isn't durable though.
  • I wonder if google could do some sort of archive using it's cache system? That is, a snapshot of a page's cache is permanantly recorded at regular intervals.
  • Oh no, metal gear solid sons of liberty comes true!

  • Archive.org, and its limitations (Score:4, Interesting)

    by Animats ( 122034 ) on Monday February 17, 2003 @12:27AM (#5317352) Homepage
    There is, of course, archive.org. [archive.org] That's a surprisingly small operation for what it does. A few volunteers work on the server farm (less than a thousand commodity PCs), and there's a little office at the Presidio of San Francisco. The web crawl is done at Alexa, and the Archive is filled from Alexa's backup tapes, which is why it runs so far behind.

    There's a live backup of the Internet Archive at the Library of Alexandria [bibalex.gov.eg] in Egypt. Thus, no single government can censor the archive. More duplicates may be established in other countries.

    Perhaps unfortunately, it's easy to remove material from the archive. Just put a "robots.txt" file on your site, and not only will it not be captured again, the archive will immediately refuse to display copies of the blocked site. This seems to be enough to keep the militant copyright holders happy.

    Most text is saved, but not all pictures, and very little video. This is good enough for most historical purposes.

  • Yes it is already here.
    click here [archive.org] if you want to see how slashdot.org has changed over the years.
  • It's always a good idea to save a piece of history. Traditionally, it's been done by writing a book. As we've seen, a book can be read thousands of years later. But what about digital information? The media types changes rapidly and todays storage is obselete tomorrow. So, how will the historians read a "Seedee" 100 years from now? Ok, assuming they actually managed to build a device that can read the data of a CD, the data will most likely be corrupted, since CD's has limited lifespan.

    Now, the only way to accomplish this is to make it a dynamic storage. That is, go with the flow and when a new sooper dooper storage device is invented, copy the data to that, thusly ensuring two things. 1) The data is "refreshed" 2) The data can be read by the contemporary hardware.

  • I think that a lot of these posts are missing a major point. There is a big difference between preservation of digital media (what NDIIP and LOC are doing) vs storage of digital media (what GOOGLE and P2P systems do). When you preserve digital media you have to try to make it so that in the future (100s of years or more) people will still be able to access and view/hear, etc. this data. This might mean continually updating the file format ("format transformation") or it might mean trying to create hardware/software systems that can play back this media ("emulation"). This is a MUCH harder task than just storing the media and many schools and research centers are trying to figure out the best way to do this.

Slashdot Top Deals

How many QA engineers does it take to screw in a lightbulb? 3: 1 to screw it in and 2 to say "I told you so" when it doesn't work.

Close