QuijiboIsAWord writes "Zap2It Labs, which provides free TV listing data for personal use, has long been the main source of program guide information for users in the US and beyond. They've announced via their webpage that,
due to abuse of the service, data will no longer be available after September 1st. There is no other direct source, and no option to pay for the service even if the users wanted to. Without a data feed of this type, users will be reduced to scraping websites at best. Is this going to be a killing blow for MythTV?"
I don't use MythTV, and so I was surprised to see that it relies on a private third-party source for TV listings. Isn't there any way to obtain this information in an "open-source" manner?
I don't use MythTV, and so I was surprised to see that it relies on a private third-party source for TV listings. Isn't there any way to obtain this information in an "open-source" manner?
I can't imagine how that would work. Ultimately, you need information from the studios, and that's going to require a business relationship. TV Guide has such a relationship, as did these folks it seems (or perhaps TV Guide and these folks have a common feed).
I'm a little shocked that these guys didn't just go commercial, though, and build a MythTV add-on that allows you to subscribe to their product.
They rely on advertising to make money and aren't going to freely help people with a device that most people are going to use to skip advertising.
They seem to forget that a substantial chunk of those viewers wouldn't be watching at ALL, if it weren't for DVRs. I like a show that's on when I'm usually playing with my kids. If I didn't have a DVR, I wouldn't watch that show period. Yes, I skip through many of the commericials during playback, but not all of them, and not if the commercial catches my eye, or is for a product I'm interested in. I even rewind to watch a commercial from the start (like if I skip into the middle of a Mac/PC ad I haven't already seen) etc.
Before I had a DVR I hit mute and/or pipped the commercials while I browsed the channel guide, or checked on the hockey game, or something. Its not like I was sitting there 'attentively watching' all the ads before.
I expect advertisers are probably losing eyeballs overall as people adopt DVRs, but its probably not nearly the issue they think it is.
Not unless someone volunteers to compile the voluminous amount of information themselves, then set up a distribution method.
This was always MythTV's achilles heel, more than even HDTV. For all the talk about "Unlike Tivo, MythTV can NEVER be shut down or crippled," MythTV always had this dependency on a third party, for profit service. It's possible someone could replace them, but they're going to want SOME form of revenue (and since no one is going to tolerate ads on their MythTV, or pay for the service, this is unlikely).
Isn't there any way to obtain this information in an "open-source" manner?
The best way to get the information is from the stations and cable operators.
Unfortunately, MythTV and other PVR users are in the game of cutting out ads; TV programming is purely to sell ad space, and always has been, save when programs were entirely paid for by one company and the show was branded in their name. What motivation do TV stations have to assist people who are purposefully going out of their way to cut out the ads?
Without a DVR, there is zero chance that I'll see any show that doesn't air between 7-10PM on weekdays. For that matter, the odds of me seeing shows that ARE between 7-10PM is slim, since I'm often doing other stuff too. From their POV, they'd probably be better off with me seeing bits of pieces of commercials at FF speed than with me seeing nothing at all.
I think the "abuse" they're talking about is programs that are pre-configured to hit their service so that everyone on the Net who tries them out hammers their servers.
It seems fair to start charging, but odd that they're just shutting it off. They say they're willing to license to other companies, so presumably they're hoping someone will come along and offer a package to the MythTV folks by licensing the data and re-selling subscriptions.
Based on their previous complaints and this message, I think the problem was people were using the free data set, then redistributing it, probably for profit, possibly indirectly (say, selling devices that used Zap2It's free service). Zap2It makes money selling their data set and they were very generously offering it for free to individuals. But you weren't allow to redistribute it.
I think you're trying to suggest that Slashdot readers are, generally, hypocrites because they don't care if new technology or business models drive old ones out of business. Of course, that's utter nonsense. This isn't about Zap2It's business model being harmed by new technology or new business models. This is about Zap2It offering something to individuals for free out of simple generosity. This generosity has been abused and does threaten their business model, so they going to stop being generous. Zap2It doesn't suffer in the slightest, only those who benefited from their generosity do.
Now, if a third party started giving away television schedules without relying on Zap2It's data, harming Zap2It's business model, then we'd be on the same ground. Of course, if that happened and Zap2It whined about it, I don't think you'd see much sympathy here.
No. For us, all you had to do was register with labs.zap2it.com and then you can get the listings for private non-commercial use. So we (mythtv users) weren't abusing the system. I imagine that the abuse they are talking about is people signing up for the service and then reselling their data (that they themselves aggregate from other sources).
Some reasons and other comments given for the scheduled discontinuence, copied from the forum, copied from the mailing list:
Continued use of the service to support commercial products, in violation of the agreement.
Commercial products continued to grow despite previous appeals that this activity stop.
There are significant changes to the supporting data structure forthcoming and we could not devote resources to the continued upkeep and enhancements of the service.
Maintenance of the service is impacting our resource pool for other projects.
We sought alternative options but were unable to find a solution.
We recognize the hardship this creates for the user community. We are open to alternative solutions and would consider proposals that met the needs of the user community and our company.
We looked into options to turn this into a paid service however we do not have the infrastructure at this time.
While no longer linked from the front page, this link [tribune.com] still works.
Basically, the content was provided free provided it remained for non-commercial use. After all, commercial ventures have to pay for those listings and if they could get it for free, nobody would pay.
I hope they at least tried to weed out the abusers before just cutting the cable.
... Without a data feed of this type, users will be reduced to scraping websites at best. Is this going to be a killing blow for MythTV?"
I fail to see how, unless the screenscrapers specifically mentioned don't work. I wrote a scraper for my MythTV box to pull PPV information from somewhere, since it wasn't coming from Zap2it. It's not that hard, really, and took me all of an hour to do.
I first became aware of MythTV some years ago from a developer that was excitedly working on the project. With all the effort that was going on at the time, nobody seemed to have a clear-cut idea of a long-term, stable way of getting TV listings. "Scraping web pages" was the only plan.
Looks like five years later, it's still the only plan.
Tribune put new management in Zap2it. They have been unresponsiveand treated the Data Direct service like a redheaded stepchild for a year now. The data has been bad, with long outages on it from time to time for a while now. Many of us that have used myth and other xmltv systems have tried to pay for a subscription for a couple of years now and they refuse.
it's the new management, they hate that OSS people are getting access to the data and want to stop it.
That's a good thought, but I think you miss the mark. Data Direct was created as a response to the various screen scrapers that existed before it - scrapers which had to pull down and parse entire HTML documents instead of simply using compact, per-user xml feeds. This service was intended to reduce total bandwidth use, and as far as I'm aware it succeeded at doing so.
It's possible that there has been a shift in management and that this history lesson was forgotten, but if their intention is to save ban
How, exactly, does one go about "misusing" TV schedule listings? Is this really because Zap2It was making all other forms of TV listings obsolete while not making any money at it? The announcement is quite vague. Does anybody have details on what's going on?
I use a Windows based PVR that works with the Web site titantv.com Clicking on the 'record' button on the Web site sends a small file that a helper app converts to a Windows scheduler entry. I can't believe it would be that hard to adapt to Linux.
The Titan TV web site includes advertising and also does tracking. While personally, I don't care if anyone knows I watch StarGate and Myth Busters, privacy issues may be a concern for some.
It doesn't appear that this will affect their main listings on their website, just the programs that tap into their database. I've always just setup an easy shortcut to their web page to get listings, so I really don't think this will affect my usage in the least.
This will be the killing blow for MythTV and other open source DVRs? I think not. MythTV predates Zap2It and managed to do okay. Yes, it relied on screen scraping, but it worked. Furthermore, I know I'm perfectly prepared to pay a small monthly fee for a good data source. Maybe $5 a month? Since a company offering such a service doesn't need to recoup costs for selling hardware below cost (as Tivo does), such a price should be feasible. Since Zap2It was free, there wasn't much incentive for someone to offer the service, but now there is. I'm hoping the free market will see the opportunity and we can work something else.
What's wrong with just "scraping" web pages (I assume that means writing code which automatically downloads the webpages and captures the data of interest, rather than requiring a human to do it. I do this all the time with Perl code.)?
There's multiple sites out there with TV listings: Yahoo TV, Zap2It, MeeTV (the one I use), etc. Just write perl scripts to capture the listing information from these sites, and modify MythTV to allow the user to choose any service he wishes. Of course, some of these sites may (stupidly) screw with their HTML in order to throw off these scripts, but that's easily worked around with regular updates. So we just need to have a "myth-scripts" distribution site where your Myth box automatically checks for updates to the perl scripts every day and downloads them if necessary, just like we already do with many other things.
No, it's not quite as reliable and efficient as a static interface to this data, but if these companies are stupid enough to remove static interfaces, thinking we're just going to go back to doing everything manually and looking at all the ads, this seems like a reasonable solution. There's no way of preventing automated scripts from downloading webpages.
What's wrong with just "scraping" web pages (I assume that means writing code which automatically downloads the webpages and captures the data of interest, rather than requiring a human to do it. I do this all the time with Perl code.)?
There's multiple sites out there with TV listings: Yahoo TV, Zap2It, MeeTV (the one I use), etc. Just write perl scripts to capture the listing information from these sites, and modify MythTV to allow the user to choose any service he wishes. Of course, some of these sites may (stupidly) screw with their HTML in order to throw off these scripts, but that's easily worked around with regular updates. So we just need to have a "myth-scripts" distribution site where your Myth box automatically checks for updates to the perl scripts every day and downloads them if necessary, just like we already do with many other things.
No, it's not quite as reliable and efficient as a static interface to this data, but if these companies are stupid enough to remove static interfaces, thinking we're just going to go back to doing everything manually and looking at all the ads, this seems like a reasonable solution. There's no way of preventing automated scripts from downloading webpages.
They can't prevent it - but if they decide they don't like you doing it, they can come up with all kinds of ways to make it hard.
For instance, they could replace their nice little HTML table with some flash code - that's not going to be impossible for you to read with scripts, but it'll be a lot harder. Another option would be to use images instead of text - possibly even breaking up the images into smaller images, to make it harder for automated tools to pull it and OCR it. They could load their page with b
Me thinks that someone doesn't quite grasp the strength of OSS: That strength not just being free. Ultimately, it's the dissemination of data. Which is precisely why this problem is uniquely suited to being solved by OSS.
This will be a minor set back at worse. But, like any set back, it will make the overall product stronger.
That said..."NNOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO OO!!!!!!!!!!!!!". Zap2it was damn handy. Thank you Zap2it, for a great service.
It sucks that it' shutting down (I use it for GB-PVR)- but they did do it for free for years. Let's not demonize them, no matter the reason, and start looking for alternatives.
by Anonymous Coward
on Wednesday June 20 2007, @03:18PM (#19585681)
Shouldn't Google or Yahoo or Microsoft or any of the other big media / tech companies of the 21st century provide this information as a free WebService? Seems like something Google would jump on top of since their mission statement is to organize the worlds information. Well, TV listings is information.... get on that Google!
What if I was to write a web service that exposed the data garnered from website-scraping? You could just write a standard XML request, wrap it in SOAP tags and send it to the web service, and you'd be returned whatever information you requested- by channel, time, or show name...
Rather than a million screen scrapers each hitting various listing sites, better one screen scraper sharing the resulting data with a million video rebels. This solution would probably leave everyone happier.
Now there's just the question of who? Who is expert at spidering the web? Who likes to provide new cheap-to-free services in their quest to take over new markets. Who would love to put yet another spike into Microsoft's side by removing yet another possible revenue source for them? Who doesn't have to worry about financing such a small, cheap service alongside their already multitudes of underutilized servers and bandwidth?
Of course it would be the best ever... With version numbers for each day and diffs available that only contain the changes. The whole thing would be available as an rss feed and would be free, of course.
Like many others, I'm a little surprised that they aren't moving to a subscription model. Clearly they know better what their available resources are, and what they are and are not capable of handling, but it seems like a missed opportunity to walk away from a situation where their servers are getting hammered, and start charging a small fee. Many, many of the MythTV users would happily pay a few dollars a month to have a steady stream of information. Sounds like it could be a million dollar annual income right there, and that's got to be very hard to walk away from for any company.
As for commercial abuse, if they know it's happening, they presumably are taking steps to quash it as well, without much luck. Probably like playing whack-a-mole.
I was happy to find your clear, concise, comment down here all by itself. It makes it easy for a clean response.
Let's say that MythTV implemented your paid service plan and began charging the princely sum of $2 per month for the data.
I would give it all of 7 days before that paid for data became available for free. Someone, somewhere, would buy the data for $2 per month and load it up for others to have free of charge. It would be a daily torrent that you could pull, or a streamed RSS feed, a static layout site with a downloadable screen scraper, or any one of a dozen other ways I can think.
So now instead of a million dollar revenue stream you'd get a thousand dollar revenue stream coming from the 500 users who would actually be wiling to pay when a free source is available.
If you can answer the question of how to prevent the above scenario from happening I can put you in touch with some content providers who will pay REAL money for your idea. The kind of money that allows people to retire for life...at the age of twenty.
A year ago it was announced that an alternative paid service through TechnoVera [slashdot.org] was available to replace Zap2It with part of the funds going back to OSS and mythtv -- and no periodic surveys. Couldn't users switch to this service? -- or is it no longer available? (I've never used this service myself.. any users care to respond?)
MythTV can read the broadcast schedules on the airwaves - see EIT [wikipedia.org].
At least that's what I use in the UK. I can also still scrape the Radio Times website in XMLtv as well.
Excellent link. However, using EIT has at least one drawback:
Unlike DVB systems, an EIT in ATSC covers a limited time span. [...] It's possible to transmit up to 128 EIT tables for any virtual channel, but it's very rare that so many are transmitted - terrestrial systems have to transmit EITs covering at least the current time slot and the next three time slots.
So at least in the U.S., the EIT would be of limited utility to MythTV users, since it is unlikely that one would be able to use this data to schedule multiple days in advance. Also, the article you linked only indicates that EIT is a standard part of ATSC (the terrestrial digital TV broadcast standard), and wouldn't necessarily be present in analog broadcasts (which are going to be around for a little while longer) or in digital cable transmissions (which use QAM here in the U.S.). It's entirely possible that a broadcaster might supply EIT data, and a cable operator might strip it out to save some bandwidth.
I've seen some of the newer LCD HDTVs sold here in the U.S. which have the ability to display program guide data, no doubt culled from available EIT data, but this is really intended to allow viewers to see what's on right now or in the immediate future without resorting to channel flipping. I bought a Sharp Aquos one generation before this feature became standard, so I missed out, but I really don't feel like I'm missing too much because of the described limitations.
Not having to pay a monthly fee to automatically record shows you watched previously
Not having to deal with commercials inserted when you fast forward 30 seconds.
Now, if they could only find a way to strip thos FREAKIN' RUDE graphics that certain vile channels overlay on top of the movie/show. I can deal with a tiny little station identification, but those rude obnoxious ads obscure vital parts of the show about one out of every 3 times.
I swear my Blood pressure goes through the roof.
I don't suppose you expect to get your TiVo service for free, do you?
You might want to read the summary, or even the title. From what our fine/. editors have implied, you won't be getting your MythTV feeds free either.
TCO indeed...when you figure in TCO for MythTV, you are taking into account the risk you take in your reliance on a free service that you have absolutely no control over, right? Because that's kind of necessary.
Average joe doesn't look at it like that. He says a service. One service "just works" the other requires work. For the majority of people, they'd rather pay for a finished product then build it themselves. The apples / oranges would be each of these products are designed for different audiences.
You forgot being able to watch a recording on your laptop while on the toilet. I'm puzzled as to why MythTV doesn't advertise this feature a bit more, since it's one of my favorites.
That being said, you can't honestly suggest that MythTV is always a better choice than Tivo, unless you've conveniently forgotten about the teeming masses of people who couldn't install a capture card if their life depended on it. Ease is the killer feature for Tivo and bundled PVRs.
MythTV doesn't fast forward, it does commercial skip. Automatically. It hits a commercial break and just jumps past.
Tivo used to allow a really nice FF feature to skip commercials. Now they overlay advertisements on top of the advertisements you are fast forwarding through. Not to mention the advertising in the rest of the UI. If you own one, you should know what I'm talking about. If not, Google found me someone's blog with pictures [blogspot.com]
Unlike printed guides, which have to limit show descriptions to one or two lines at best, digital guides can be as detailed as you wish. By providing your own RSS feed of your schedule, you could increase the information given, which should increase the number of people willing to watch your channel. Even though I have a DVR I still watch (interesting) commercials, so don't completely write off my viewership.
And here's how you can directly make money: By providing your own guide information, you can insert sponsorship lines and charge for them.
"Tonight on Lost, sponsored by Coca-Cola, Jack and Kate have more awkward sexual tension, while Hurley tries to cheer everyone up, Sawyer acts rudely, and Sayid kills someone with his feet."
Even us DVR users will very often look at the guide information. Voila, you've just sold an ad!
I think you are on to something. I've been keeping tabs on the NZ Mythtv forums & mailing list (am about to move there) and EPG seems an ongoing issue for them.. I've often thought what you've suggested would be their answer.
People further down in the thread suggested that TV channel EPG is right up Google's alley, and I agree... just how do you persuade them???? Hopefully there are a few Mythtv users in the Googleplex that have the sway to pull this off. Google have the funds to purchase the EPG data, AND the customer base to insert ads in the stream (however ads would not be click-able).
For those folks with DirecTV, that's not an option. In fact, at this point, we have no options at all other than writing a guide scraper for TVGuide.com. Fortunately, their website data is fetched in Javascript using xmlHttpRequest, so it is probably straight XML in some dialect that could be converted into the same format as Zap2It uses with very little effort. If done carefully (request the entire schedule exactly once, then only refetch the current day each day and fetch any new days added to the schedule), it might not add enough server hits for them to even care.
This is troubling (Score:4, Interesting)
Re: (Score:3, Interesting)
I don't use MythTV, and so I was surprised to see that it relies on a private third-party source for TV listings. Isn't there any way to obtain this information in an "open-source" manner?
I can't imagine how that would work. Ultimately, you need information from the studios, and that's going to require a business relationship. TV Guide has such a relationship, as did these folks it seems (or perhaps TV Guide and these folks have a common feed).
I'm a little shocked that these guys didn't just go commercial, though, and build a MythTV add-on that allows you to subscribe to their product.
Re:This is troubling (Score:5, Insightful)
They seem to forget that a substantial chunk of those viewers wouldn't be watching at ALL, if it weren't for DVRs. I like a show that's on when I'm usually playing with my kids. If I didn't have a DVR, I wouldn't watch that show period. Yes, I skip through many of the commericials during playback, but not all of them, and not if the commercial catches my eye, or is for a product I'm interested in. I even rewind to watch a commercial from the start (like if I skip into the middle of a Mac/PC ad I haven't already seen) etc.
Before I had a DVR I hit mute and/or pipped the commercials while I browsed the channel guide, or checked on the hockey game, or something. Its not like I was sitting there 'attentively watching' all the ads before.
I expect advertisers are probably losing eyeballs overall as people adopt DVRs, but its probably not nearly the issue they think it is.
Parent
Re:This is troubling (Score:5, Insightful)
This was always MythTV's achilles heel, more than even HDTV. For all the talk about "Unlike Tivo, MythTV can NEVER be shut down or crippled," MythTV always had this dependency on a third party, for profit service. It's possible someone could replace them, but they're going to want SOME form of revenue (and since no one is going to tolerate ads on their MythTV, or pay for the service, this is unlikely).
Parent
Re:This is troubling (Score:5, Insightful)
Given the choice between:
1. paying Comcast's fees, DVR service, etc
2. paying Tivo or
3. paying for a subscription to an XML TV Listing service, and keeping my MythTV box
I'll take #3.
People will pay for it as long as no free alternatives are out there.
Parent
TV stations are hardly interested in helping... (Score:4, Insightful)
Isn't there any way to obtain this information in an "open-source" manner?
The best way to get the information is from the stations and cable operators.
Unfortunately, MythTV and other PVR users are in the game of cutting out ads; TV programming is purely to sell ad space, and always has been, save when programs were entirely paid for by one company and the show was branded in their name. What motivation do TV stations have to assist people who are purposefully going out of their way to cut out the ads?
Parent
Re:TV stations are hardly interested in helping... (Score:4, Interesting)
Parent
Re:This is troubling (Score:5, Interesting)
Parent
Anybody know what the "abuse" was? (Score:4, Interesting)
Re: (Score:3, Interesting)
It seems fair to start charging, but odd that they're just shutting it off. They say they're willing to license to other companies, so presumably they're hoping someone will come along and offer a package to the MythTV folks by licensing the data and re-selling subscriptions.
Re:Anybody know what the "abuse" was? (Score:5, Informative)
Parent
Re:Anybody know what the "abuse" was? (Score:4, Insightful)
I think you're trying to suggest that Slashdot readers are, generally, hypocrites because they don't care if new technology or business models drive old ones out of business. Of course, that's utter nonsense. This isn't about Zap2It's business model being harmed by new technology or new business models. This is about Zap2It offering something to individuals for free out of simple generosity. This generosity has been abused and does threaten their business model, so they going to stop being generous. Zap2It doesn't suffer in the slightest, only those who benefited from their generosity do.
Now, if a third party started giving away television schedules without relying on Zap2It's data, harming Zap2It's business model, then we'd be on the same ground. Of course, if that happened and Zap2It whined about it, I don't think you'd see much sympathy here.
Parent
Re: (Score:3, Insightful)
--Ajay
Re:Anybody know what the "abuse" was? (Score:5, Informative)
According to a posting on mythtv.org... [mythtv.org]
Parent
Re:Anybody know what the "abuse" was? (Score:5, Informative)
Basically, the content was provided free provided it remained for non-commercial use. After all, commercial ventures have to pay for those listings and if they could get it for free, nobody would pay.
I hope they at least tried to weed out the abusers before just cutting the cable.
Parent
How is MythTV dead? (Score:4, Informative)
Achilles Heel (Score:4, Interesting)
Looks like five years later, it's still the only plan.
This is quite bad :( (Score:3, Insightful)
Re:This is quite bad :( (Score:5, Interesting)
it's the new management, they hate that OSS people are getting access to the data and want to stop it.
Parent
Re: (Score:3, Insightful)
That's a good thought, but I think you miss the mark. Data Direct was created as a response to the various screen scrapers that existed before it - scrapers which had to pull down and parse entire HTML documents instead of simply using compact, per-user xml feeds. This service was intended to reduce total bandwidth use, and as far as I'm aware it succeeded at doing so.
It's possible that there has been a shift in management and that this history lesson was forgotten, but if their intention is to save ban
Misuse? (Score:3, Insightful)
TitanTV (Score:4, Informative)
The Titan TV web site includes advertising and also does tracking. While personally, I don't care if anyone knows I watch StarGate and Myth Busters, privacy issues may be a concern for some.
Going, going, gone? (Score:5, Interesting)
At first, they made me fill out a big online survey as "payment" for the service. The first time it was about 30 questions.
The third time (this is like every 3-6 months) they only asked one question.
For the last year, the survey has been "click here to renew."
What's with that? I'm willing to give up some personal time and info to pay for this service, and they can't even think of a way to leverage that?
The main listings (Score:4, Informative)
I've always just setup an easy shortcut to their web page to get listings, so I really don't think this will affect my usage in the least.
The killing blow? I think not. (Score:4, Insightful)
I don't see the problem (Score:4, Interesting)
There's multiple sites out there with TV listings: Yahoo TV, Zap2It, MeeTV (the one I use), etc. Just write perl scripts to capture the listing information from these sites, and modify MythTV to allow the user to choose any service he wishes. Of course, some of these sites may (stupidly) screw with their HTML in order to throw off these scripts, but that's easily worked around with regular updates. So we just need to have a "myth-scripts" distribution site where your Myth box automatically checks for updates to the perl scripts every day and downloads them if necessary, just like we already do with many other things.
No, it's not quite as reliable and efficient as a static interface to this data, but if these companies are stupid enough to remove static interfaces, thinking we're just going to go back to doing everything manually and looking at all the ads, this seems like a reasonable solution. There's no way of preventing automated scripts from downloading webpages.
Re: (Score:3, Interesting)
What's wrong with just "scraping" web pages (I assume that means writing code which automatically downloads the webpages and captures the data of interest, rather than requiring a human to do it. I do this all the time with Perl code.)?
There's multiple sites out there with TV listings: Yahoo TV, Zap2It, MeeTV (the one I use), etc. Just write perl scripts to capture the listing information from these sites, and modify MythTV to allow the user to choose any service he wishes. Of course, some of these sites may (stupidly) screw with their HTML in order to throw off these scripts, but that's easily worked around with regular updates. So we just need to have a "myth-scripts" distribution site where your Myth box automatically checks for updates to the perl scripts every day and downloads them if necessary, just like we already do with many other things.
No, it's not quite as reliable and efficient as a static interface to this data, but if these companies are stupid enough to remove static interfaces, thinking we're just going to go back to doing everything manually and looking at all the ads, this seems like a reasonable solution. There's no way of preventing automated scripts from downloading webpages.
They can't prevent it - but if they decide they don't like you doing it, they can come up with all kinds of ways to make it hard.
For instance, they could replace their nice little HTML table with some flash code - that's not going to be impossible for you to read with scripts, but it'll be a lot harder.
Another option would be to use images instead of text - possibly even breaking up the images into smaller images, to make it harder for automated tools to pull it and OCR it.
They could load their page with b
Killing blow? (Score:3)
This will be a minor set back at worse. But, like any set back, it will make the overall product stronger.
That said..."NNOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
Thanks (Score:5, Insightful)
It sucks that it' shutting down (I use it for GB-PVR)- but they did do it for free for years. Let's not demonize them, no matter the reason, and start looking for alternatives.
Parent
MythTV devs are working on this... (Score:5, Informative)
And it isn't just MythTV that uses the guide data provided...
Google should provide a WebService (Score:5, Insightful)
What if I was to write a web service? (Score:4, Interesting)
Any takers?
Rather than a Million Screen Scrapers... (Score:5, Insightful)
Now there's just the question of who? Who is expert at spidering the web? Who likes to provide new cheap-to-free services in their quest to take over new markets. Who would love to put yet another spike into Microsoft's side by removing yet another possible revenue source for them? Who doesn't have to worry about financing such a small, cheap service alongside their already multitudes of underutilized servers and bandwidth?
Google?
We need GoogleTVListings (tm) (Score:3, Insightful)
Of course it would be the best ever... With version numbers for each day and diffs available that only contain the changes. The whole thing would be available as an rss feed and would be free, of course.
Why not use Guide+? (Score:3, Interesting)
Could a computer not do the same thing?
Make it a paid service (Score:3, Insightful)
As for commercial abuse, if they know it's happening, they presumably are taking steps to quash it as well, without much luck. Probably like playing whack-a-mole.
Let's all hope Google comes to the rescue.
Re:Make it a paid service (Score:5, Interesting)
Let's say that MythTV implemented your paid service plan and began charging the princely sum of $2 per month for the data.
I would give it all of 7 days before that paid for data became available for free. Someone, somewhere, would buy the data for $2 per month and load it up for others to have free of charge. It would be a daily torrent that you could pull, or a streamed RSS feed, a static layout site with a downloadable screen scraper, or any one of a dozen other ways I can think.
So now instead of a million dollar revenue stream you'd get a thousand dollar revenue stream coming from the 500 users who would actually be wiling to pay when a free source is available.
If you can answer the question of how to prevent the above scenario from happening I can put you in touch with some content providers who will pay REAL money for your idea. The kind of money that allows people to retire for life...at the age of twenty.
Parent
What happened to Mythtv's paid service, TechnoVera (Score:3, Interesting)
It has another source (Score:5, Informative)
Parent
But EIT has limitations (Score:5, Informative)
I've seen some of the newer LCD HDTVs sold here in the U.S. which have the ability to display program guide data, no doubt culled from available EIT data, but this is really intended to allow viewers to see what's on right now or in the immediate future without resorting to channel flipping. I bought a Sharp Aquos one generation before this feature became standard, so I missed out, but I really don't feel like I'm missing too much because of the described limitations.
Parent
Re:Myth will survive (Score:5, Informative)
Not having to pay a monthly fee to automatically record shows you watched previously
Not having to deal with commercials inserted when you fast forward 30 seconds.
Parent
No it doesn't (Score:3, Insightful)
With a TiVo, you can plug in three cables and press power, at which point you are done but for the watching. You're claiming MYTH can do this too?
So no, it does NOT offer everything TiVo does.
Re:No it doesn't (Score:5, Insightful)
Parent
Re:No it doesn't (Score:4, Informative)
You might want to read the summary, or even the title. From what our fine
TCO indeed...when you figure in TCO for MythTV, you are taking into account the risk you take in your reliance on a free service that you have absolutely no control over, right? Because that's kind of necessary.
Parent
Re:No it doesn't (Score:4, Insightful)
Parent
Re:Myth will survive (Score:5, Insightful)
That being said, you can't honestly suggest that MythTV is always a better choice than Tivo, unless you've conveniently forgotten about the teeming masses of people who couldn't install a capture card if their life depended on it. Ease is the killer feature for Tivo and bundled PVRs.
Parent
Re:Myth will survive (Score:5, Interesting)
Tivo used to allow a really nice FF feature to skip commercials. Now they overlay advertisements on top of the advertisements you are fast forwarding through. Not to mention the advertising in the rest of the UI. If you own one, you should know what I'm talking about. If not, Google found me someone's blog with pictures [blogspot.com]
Parent
Re:Myth will survive (Score:5, Insightful)
And here's how you can directly make money:
By providing your own guide information, you can insert sponsorship lines and charge for them.
"Tonight on Lost, sponsored by Coca-Cola, Jack and Kate have more awkward sexual tension, while Hurley tries to cheer everyone up, Sawyer acts rudely, and Sayid kills someone with his feet."
Even us DVR users will very often look at the guide information. Voila, you've just sold an ad!
Parent
Re:Myth will survive (Score:4, Insightful)
People further down in the thread suggested that TV channel EPG is right up Google's alley, and I agree... just how do you persuade them???? Hopefully there are a few Mythtv users in the Googleplex that have the sway to pull this off. Google have the funds to purchase the EPG data, AND the customer base to insert ads in the stream (however ads would not be click-able).
Google... please buy ZAP2IT LABS!!! please!!!
Parent
Re:Oh no! (Score:5, Insightful)
For those folks with DirecTV, that's not an option. In fact, at this point, we have no options at all other than writing a guide scraper for TVGuide.com. Fortunately, their website data is fetched in Javascript using xmlHttpRequest, so it is probably straight XML in some dialect that could be converted into the same format as Zap2It uses with very little effort. If done carefully (request the entire schedule exactly once, then only refetch the current day each day and fetch any new days added to the schedule), it might not add enough server hits for them to even care.
Parent