Wikimedia Trying P2P Video Distribution 85
bigmammoth writes "One potential problem with campaigns and programs to increase video on Wikimedia sites is that video is many times more costly to distribute than text and images. The P2P-Next consortium has created an HTML5 streaming BitTorrent browser add-on to try and help experiment with ways to reduce the costs of video distribution. As described in a Wikimedia tech blog post, once the SwarmPlayer add-on is installed, and when using the multimedia beta, video on the site will be streamed via the hybrid HTTP / BitTorrent SwarmPlayer. For smooth playback the Swarmplayer downloads high priority pieces over HTTP while getting low priority bits from the BitTorrent swarm. The same technology is available for experimentation with any site via the standalone version of the Kaltura HTML5 Media library."
Great (Score:5, Interesting)
This is good news. It'll:
a) make it a lot easier to compete with the likes of youtube.
b) be very easy to take advantage of, once integrated into CMS's.
c) make it a lot harder to argue that P2P is only something that pirates use, rather than simply modern technology.
Re: (Score:2, Informative)
a) Wikimedia is (allegedly [wikipedia.org]) encyclopaedic media, whilst youtube is cats. Not direct competition.
b) My outbound connection is extremely limited.
c) The people who need to be convinced fight against "websites" and "torrents", but they would have a real "Oh my God - it's full of data!" moment if they understood what they were talking about. It's all just bits. Encrypted bits even more so.
Re: (Score:3, Insightful)
I'm pretty sure this player can be used for more than encyclopaedic media, even for cats. As long as it's free to anyone to put on their sites, it's a competition on the Youtube business.
Also, even if we think it's absurd, people see Colour in data and that won't change soon.
Re: (Score:3, Informative)
people see Colour in data
I don't even see the colours anymore. All I see SYN, SYN, ACK, ACK, FIN, ACK
Re: (Score:3, Interesting)
Are you thinking multicast? Because this is the real need here.
Can anyone fill us in on where is multicast on the internet right now? It seems pretty far away - even further away than IPv6.
Re: (Score:3, Informative)
IPv6 includes multicast IIRC.
IPv4 multicast is basically broken by NAT, so is unlikely to ever get used on the internet itself.
Re: (Score:2)
The problem with multicast is that everyone is supposed to need the same packets at the same time, like in a real time TV feed. AFAIK it doesn't work so well when each person might want to start watching a video at any given time.
Re: (Score:2)
The problem with multicast is that everyone is supposed to need the same packets at the same time, like in a real time TV feed. AFAIK it doesn't work so well when each person might want to start watching a video at any given time.
True, multicast isn't long tail friendly. But if a video gets linked from a Slashdot story, there will likely be enough viewers to justify starting a multicast swarm every so often. Send the first minute as unicast and the rest as a rolling multicast swarm. Then while a player is receiving the first minute over unicast, it can receive the next minute over multicast. Players on sufficiently fast connections might join multiple swarms at once to pull in the whole video more quickly.
Re: (Score:2)
But that requires an immense quantity of swarms (multiple per video), which means the routers will have to store huge in-memory databases to keep track of the downstream routers who need each packet stream, because unlike with unicast, the destination IPs are not contained in the packets themselves.
Re: (Score:3, Informative)
the routers will have to store huge in-memory databases to keep track of the downstream routers who need each packet stream
Congratulations. You've just explained why multicast was never deployed over the public IPv4 Internet even before NAT had become widespread.
Re: (Score:2)
You could also have worded it:
"Hey, instead of just sending you a packet addressed to every host, I'll send you a single meta-packet and you can figure out who needs it from there."
Multicast has advantages (less bandwidth usage), but at a cost of much beefier routers. IIRC, setting up routing of multicast traffic can be a pain.
Re: (Score:2)
a) Wikimedia is (allegedly [wikipedia.org]) encyclopaedic media, whilst youtube is cats. Not direct competition.
No, you're confusing Wikimedia with Wikipedia. Wikipedia is an encyclopaedia. Wikimedia Commons [wikimedia.org] is a collection of media that complements Wikipedia. Wikimedia is a non-profit organisation that runs a large number of open content wikis, and was founded by the same people as (and used to share some infrastructure with) Wikia, which is a for-profit wiki-hosting company.
An open content archive of cats doing funny things would be within the scope of Wikimedia's goals.
Re: (Score:1)
Only if you can explain how cats doing funny things are useful for an educational purpose. [wikimedia.org]
Re: (Score:2)
d) What if I want to grab a video that nobody is seeding? This happens to me a lot on torrents I grab from isohunt. For example I tried to get "The Odd Couple" but it got stuck at 5% and never went any further. I imagine the problem would be even worse for unpopular encyclopedic videos.
>>>"video is many times more costly to distribute than text and images."
It doesn't have to be "many" times costlier. I routinely share 25-minute episodes of Penn&Teller with friends via email, and they are
Re: (Score:2)
HTTP will still be an option - if it can find seeds, great, if not, it'll work like today. No loss for the user.
Re: (Score:2)
>>>I wouldn't want to watch that.
53 kb/s. No worse than when we watched videos back in the days of dialup (either 28k or 56k). The point is that videos don't have to take up a lot of space if you make them SD quality.
Re: (Score:2)
Google Gears (Score:2)
Google Gears also started as a plugin before being swallowed by HTML5.
Video where handled by plugins before being considered by the VIDEO tag of HTML5.
3D Web also started as a plugin before WebGL emerged as a standard.
see any tendency ?
Well, if things keep that way, bittorrent P2P server offloading could be integrated into HTML5.1 /. effect, if links on the main page where modified to leverage such swarming, in addition to direct HTTP access to the /.ed server.
could even help mitigating the
If enough propone
Re: (Score:1)
just like some are injecting bogus packets into eDonkey networks, as MD4 is not secure anymore
Do you have reference on that? I mean, the injection part, no the MD4 is broken part.
Since preimage attacks on MD4 are far from practical, you cannot inject bogus packet to infect a given file. You need to create a special file that exists in two versions: a legitimate one and a bogus one. Then you would have to get people to download you file, and now you can inject the bogus version. But I would be really surprised if someone was actually doing this: it's much easier and just as efficient to infect so
Re: (Score:2)
Aha, so that's why nobody is using Flash for video.
Re: (Score:1)
"Installing this plugin will give you great new features" is a much better argument to install it than "installing this plugin will help the owners of the web sites you're visiting to keep their costs down."
Re: (Score:2)
Also, it's often hard to keep a YouTube video, and I'm not always online. There's a nature film called "The Bear", for instance, that's a keeper, but it's hard to save the stream.
Legitimate P2P use ?~ OMG!~ (Score:2)
c) make it a lot harder to argue that P2P is only something that pirates use, rather than simply modern technology.
But... but... I thought that only evil pedo-terrorist pirates are using Torrent, to siphon the hard gained profits out of the pocket of the poor record- / motion picture- industry !~
More seriously : I'm actually surprised that it took so much time until someone decided to implemented it. Leveraging P2P to offload server load for user-made and -uploaded videos (just like it's already used to offload bandwidth requirement for distributed TV - like torrentocracy - and for upgrades - like in WoW. ) just make pe
Why not just use Youtube? (Score:3, Interesting)
Why not just use Youtube to host the videos, after archiving them in a Wikimedia store?
Or is this more about control than openness? What value does hosting them at Wikimedia have over Youtube?
Youtube isn't that restrictive as long as you aren't infringing copyright..
Re:Why not just use Youtube? (Score:5, Insightful)
1) less intrusive ads
2) you would be relying on a commercial third-party, which is bad. What if Youtube suddenly decides to go pay-per-view? What if it closes?
3) you, and not Google, should get to decide what is "fair use"
Re: (Score:3, Interesting)
Doubly important in the case of Wikipedia - whose "fair use" justification is frequently "we couldn't find an image usable under the normal interpretations of fair use, so we used this one anyhow".
If WP is abusing fair use, let's go file an IFD (Score:2)
Wikipedia - whose "fair use" justification is frequently "we couldn't find an image usable under the normal interpretations of fair use, so we used this one anyhow"
As far as I can tell, all non-free media on English Wikipedia, other than WMF logos, is supposed to be an excerpt (factors 3 and 4) used in context of commentary on the image's subject (factor 1), and the subject has to be of a nature that free images cannot be produced (factor 2). Most of these are of A. a notable non-free work of authorship or B. a notable person who is dead or extraordinarily reclusive. Can you cite specific abuses of fair use on Wikipedia so that I can file an IFD?
Re: (Score:2)
You mean FfD [wikipedia.org].
Re: (Score:2)
Why, Google are the ones who will get sued, not you.
Re:Why not just use Youtube? (Score:5, Insightful)
Not being at the whims of a private company that can do what they please with the videos, including censoring?
It's not about Wikimedia having control, it's about not giving control to some company. Bittorrent puts us all in control.
Re: (Score:2, Insightful)
That's quite naive. First of all youtube would ban a lot of media that wikimedia would not (a lot of biology media regarding human anatomy for instance...) secondly youtube is of the stance that "there's no such thing as fair use".
Re: (Score:2)
Youtube isn't that restrictive as long as you aren't infringing copyright
In the case of one of my videos that criticized the practices of The Tetris Company, it took YouTube substantially longer than the standard 10 to 14 business days specified by the DMCA to handle my counter-notice.
The bad new is (Score:2)
Re: (Score:2)
Someone will put a transcript on the wiki (Score:2)
Re: (Score:1)
I hope they won't go the route of a lot of news sites and make stuff that should be text video.
Wikipedia is all about collaborative editing. As long as making a collaborative video is out of reach, they will stick to text.
Re: (Score:2)
It will make it much harder to use Wikipedia as a reference. You will want to look up something quickly and be presented with four our five possibly relevant 10-minute videocasts on the subject.
Which is especially awkward if you're part of the population that only has access to dialup. Slashdot already takes nearly a minute to load on my connection and it takes nearly half an hour to download a YouTube video, I can just imagine how well WikiYouTubePedia would work.
Re: (Score:2)
Which is especially awkward if you're part of the population that only has access to dialup.
AT&T covers 97 percent of the United States. What part of what country doesn't have satellite or EDGE by now?
With Chrome... (Score:3, Informative)
What about unpopular videos (Score:2)
P2P is good for content which everybody wants right now but what about the situation where you have an encyclopedia full of videos and few of them are accessed by different browsers in any given day? Client side caches can't hold on to this stuff for ever. I wonder if there is any benefit from using P2P in this case.
Re: (Score:1, Interesting)
This is pretty much the right idea. Say CNN mentions some hot topic, a bunch of wikinerds go update that topic, and the people who don't want to read can watch a video on it. If only one or two people are watching it, no advantage, but if thousands of people are... then the bandwidth isn't hosed in a few seconds.
I wish more software actually worked this way, the "outbound bandwidth" being consumed is not an issue because once you have it, you're not sharing it forever, just for the length of time it takes t
Re:What about unpopular videos (Score:4, Interesting)
Didn't you read that this is a hybrid system? If there are no seeders, everything will come over HTTP.
Jeez, people really aren't even bothering to read even the summaries now.
Re: (Score:2)
No I understand that, I just question the benefit of P2P at all for the wiki. I doubt that it will have many videos popular enough to benefit from this type of distribution, and I doubt P2P would save wikimedia much bandwidth.
Re: (Score:3, Insightful)
The worst-case performance probably isn't appreciably worse than it is now, and the best-case performance is much better, so the average is probably at least a little bit better. Furthermore, getting code like this into more people's browsers can increase the accessibility of the technology for other sites. So even if it's a dead-end for Wikimedia, it's a potential boon to video sites.
Really, if it doesn't bother Wikimedia, the only ones bothered by it should be ISPs and content creators who want to be take
Re: (Score:2)
the average is probably at least a little bit better.
The average case can't just be "a little bit better". It has to be better enough to pay the bills of programmers and server administrators.
if it doesn't bother Wikimedia, the only ones bothered by it should be ISPs and content creators who want to be takedown-happy.
Wikipedia already has processes for deleting non-free media with dodgy fair use rationales [wikipedia.org].
Re: (Score:2)
Why would they send it over HTTP, instead of having their server participate in the swarm as a seed? That seems the natural way to do it.
Network neutrality (Score:2)
Re: (Score:2)
Net neutrality often concerns me - why shouldn't I be able to play less for what I call low priority traffic and have someone else's prioritized above mine.
Re: (Score:3, Informative)
--Google CEO Eric E. Schmidt
Re: (Score:3, Insightful)
Network neutrality is about the ISPs treating data as equal, not about the clients or servers.
Re: (Score:2)
PLEASE SEED! (Score:5, Funny)
Seeding problem (Score:2, Interesting)
Re: (Score:2)
They really need to build P2P into the HTTP protocol. It is the best of both worlds. When nobody is accessing a site, then a single webserver can easily cope with the load, and it serves the purpose of being the "seeder" of last resort, and also providing hashes/etc to authenticate the content. When a lot of people are hitting a site the webserver just becomes one seed of many and the swarm takes over.
Amazon S3 has an offering like this, but it is either http or torrent, not a seamless switch from one to
Re: (Score:2)
It's almost ironic, seeing as how Opera is the only mainstream* browser with a built-in Bittorrent client.
*For rather small values of mainstream
Now then... (Score:2)
Re: (Score:2)
All a server needs is a government surveillance option to get the IP of all who showed interest in loading eg. protest images/vids of undercover cops ect.
P2P is not really the issue, trying to track and find the origin of a file seems to be.
backups and rotation (Score:2)
a obvious use for Bittorrent would be serve big backups of wikipedia as bittorrent.
but the backups of Wikipedia are not served that way, because make no sense, since at the speed that change, you will have people seeding a old version no one wants anymore.
bittorrent has not appeal for files that can change often.
just saying...
Re: (Score:2)
Not true; Wikipedia dumps are often updated, but most of the content stays the same. You could perfectly serve a complete dump every six months, for example, and then serve via HTTP only the diffs to the last dump.
In fact, in the download section of Wikipedia they show* the usage of rsync to download only the changes between the current version and what you have on the disk, which means you could perfectly use a well seeded torrent of the dump [torrentz.com] and then sync it to the current revision using rsync.
*it seems t
Re: (Score:1)
Interesting (Score:1)
I'm definitely interested in this and will be checking it out once I get home from school. What other possibilities does this technology hint at?
Hello NAT (Score:1)
How would it help if 90% of web users are behind NAT/proxy?
Who does have real IP on the desktop at all? Do I have to open inbound port directly to my browser?
I'd like to keep all P2P traffic on my router, so it doesn't get anywhere inside my LAN
Re: (Score:2)
Re: (Score:1)
I highly doubt that it would be 60%. Do you know single person having non-NATed computer for web browsing?
In order to create P2P distribution you need at least one side able to accept inbound connections, but NAT/proxy would forbid it.
Re: (Score:1)
Do you know single person having non-NATed computer for web browsing?
Most of the machines I use for browsing are non-NATed. My machine at the university has a public IP. My home machine also has a public IP, because I want to be able to use it as a server, and it's easier to have a single machine than two. Even my laptop gets a public IP when I use the hotspots of my ISP [wikipedia.org].
Re: (Score:2)
Back in 2006 (Score:2)
Just great (Score:1)
There goes another one of my ideas. Of course #bittorrent isn't logged. I should really start a darned blog :-(
why not use metalink? (Score:1)
it would be cool if they could use metalink [metalinker.org], an internet standard for describing files offered in hybrid ftp/http/p2p content distribution systems, already used by a lot of open source projects.