Robo-chattel? New Legal Challenge to 'Bots 109
milomilo writes "Extending on the eBay vs. Bidder's Edge case, the NY Times reports (free registration required) that a Manhattan judge has granted a preliminary injunction against Verio from using 'bots to harvest up-for-renewal prospects from Register.com's WHOIS. The theory's that bots use up a piece of the target system's resources, denying its use to the owner. (Question: would search engines be different, presumably because they also confer a benefit on the target by making it findable?)"
I think I speak for everyone here when I say (Score:1)
Stupid! (Score:1)
How about not placing a machine on a free, publicly accessible network if you`re not up for letting people use it? Its not like its being DOS`d or anything, is it?
Does this include Slashdot? (Score:1)
this is just another step in a long process.. (Score:4)
I.E. you nonmaliciously (meaning, it isn't a DOS, you're actually getting information) ask for large gobs of information off of some site, the way these bots did.. or the way a spambot might.. they call this "denying services", but still, it's a simple the questioner requests, the answerer replies. If it's "unauthorized use".. well, how can you talk about unauthorized use on a public server? How can these things, authorization and to who, be implied on a public internet? Should it be the job of the requester to not go where they clearly shouldn't be, or the job of the requestee to keep them out?
Or look at it in terms of a port scan. I request things from each of these ports, thus figuring out which are open (and thus vulnerable to attack). I've seen people try to procecute this based on "unauthorized usage of machine".. well hold up, who said you had to authorize something? This person is just sending pings to ports, on a machine that by its presence on the internet you have implied responds to traffic. Why on earth would you need "permission" prior to using a system? If so, how would that permission be obtained? .. but of course none of this changes the fact that the port scan is almost always part of a malicious cracking attack.
Or, let's say-- hypothetically-- there was a single-line javascript that, if accessed from a windows NT machine, would cause the kernel to be overwritten by 0s. If you put that up on a web page, would that be "hacking"? You didn't break the machine yourself; you politely ask the machine to break itself, and it complies. Is that your fault?
But then, when you get down to it, all forms of "cracking" could be seen as requests. I request you process this block of information that just happens to cause a buffer overflow... you didn't have to process it, now did you? That last bit doesnt' really sound reasonable.. you have to draw a line somewhere, you have to note somewhere that it's no longer a request but an attack. Somewhere, for the sake of sanity, you have to draw the line, and how do you do that? Intent? How do you prove intent in court? What's the difference between the slashdot effect and a DDOS, at an abstract level?
But still how the hell can you say it's illegal to ask for something because the questioned might give you an answer even though they don't want to...? That's where the law is heading, where it's been heading for awhile, and that's completely absurd.
There is no right answer here, is there?
Necessary (Score:1)
Robots and search engines (Score:3)
Question: would search engines be different, presumably because they also confer a benefit on the target by making it findable?)"
The standard search engines, such as google, altavista, etc, know and obey robots.txt, which is the same as Register.com's policy of not allowing spambots search through their site. If, after a robots.txt file is in place and the search engine continues to index it, I would say there's a good legal case there.Now, more interestingly is tools that 'mirror' web sites; they still are using a resource that you've made publicably available, except doing it over a timeframe that is much shorter than a human can do it, which usually means more resources used up at the server end. These bots tend not to follow robots.txt rules, and are only defeatable by User-Agent blocking. If the above ruling stands, does it apply here?
Take it a step further: Ebay has taken action to stop meta-Ebay sites that index their site and make it easier than ebay's search engines to find things or to search multiple auction websites. Even though the information that is up is publically available from ebay, and IIRC, they still won, mostly because the information is still ebay's property and they didn't like it on other sites.
Which all leads to an interesting question: when you click on a link, does that start a clock in which you have temporary copyright ability to download the information to your local computer, and after some time, that ability 'expires'? If so, sites that index or mirror without further authorization could find themselves in trouble...
Re:Public Spaces (Score:1)
Moreover, the issue isn't you walking into a store just to look around with no intention of buying, it's as if you had 30 thousand you's go into the store, clogging every square foot, just looking with no intention of buying.
Still not correct (Score:1)
Re:this is just another step in a long process.. (Score:1)
But I seem to remember that the link you have to click to do this says something like "Do not click this link" so maybe it's OK.
I've been fighting the temptation to put one like that on my website (jsoftco.8m.com [8m.com] - shameless plug!).
Re:I have just one question... (Score:2)
AMSTERDAM - TEASERS
TEASERS
(EX-CHooters)
Damrak 36
1012LK Amsterdam
T. 0031-20-4287508
FIRE WARM UP PARTY in the Teaser's to 22.04.00 of 13.00 - 16,00 o'clock
The Teasers of sport bar, typical American bar, was always the meeting place of the NFL Europe fan (particularly with the Scots) for their Party's.
Special flag are the waitresses, who are inferior to our Pyro's hardly. And if one has then times birthday, then the waitresses let themselves also which "nice" be broken in (not truely, Living putting)! Who would not loving have exchanged with you gladly!
Freely after the slogan: Man, those are thick, man!
By the way: The Teasers was called in former times Hooters , like the American branches. The name did not change, but in the concept to anything. It continues as in the Hooters. It was probably probable more a license problem. The Disco is inferior also during the day no Discothek. A D.J. with a violent sound system and Lightshow brought still each Partymuffel in tendency. The meal is very good in addition, not cheap. The Damrak connects the Dam Square with the main station. The bar can be attained by both workstations within 5 minutes. Beside the Teasers an excellent typically Dutch Frittenbude with megaportions of Pommes and thousands saucen is direct.
cracking/port scanning/malicious behavior (Score:1)
Re:Nice link, Hemos (Score:5)
What really ticks me off is that "The Old Media", through which many people still get their news, has latched on to Slashdot as "The New Media", meaning that Slashdot will be reflecting on my own efforts, and the efforts of anybody else trying to run a 'new media' style website. This is why I post this; Slashdot's flub-ups are personal and affect us all. The flub-ups affect people running new media sites (by tarnishing the reputation in the eyes of the Old Media press who doesn't care to dig past their original generalizations), they tarnish the reputation of Open Source (as they have been labelled the spokesperson of the Open Source movement by the same collection of media entities), and they tarnish the reputation of VA Linux. (Hey, anybody at VA listening? This is not good return on your investment!)
Slashdot editors, wake up! You are not invincible. You can be replaced, and in Internet time, too. Please get some ethics, before you convince thousands or millions that the New Media doesn't have any!
The Answer is in the Code (Score:1)
I have a column [webtechniques.com] on this very issue (linking as "trespass to chattels") in the current issue of Web Techniques [webtechniques.com]. One of things I discuss there, in the context of streamlinking, is that a better solution to these issues generally is for developers to use links generated on the fly or session-specific permissions if they want to block linking.
This may not be a solution for Register.com, as it has to follow standard protocols for the implementation of Whois, but we shouldn't let this Register.com precedent carry over to other scenarios. For standard linking problems, the better solution is to use code to prevent linking, not court orders.
-- BretRe:Nice link, Hemos, or how /. almost got me fired (Score:1)
I'd also suggest that, proof-reading links aside, it's your own fault for reading Slashdot at a client's site. No wonder your boss isn't giving you any billable hours if you spent them reading and posting to a news site.
I loaf around at work by reading Slashdot though, so I'm hardly holier-than-though.
Re:This is somewhat understandable... (Score:1)
Even better analogy: you give your client list to your secretary, your competitor asks your secretary for it, and s/he hands it over willingly. There's no legal issue here; you just need to set up your web server to not return certain results if you don't want them to be public, or at least don't return them to certain people.
Re:Nice link, Hemos (Score:1)
For example my last one was yesterday, about the first genetically modified monkey borned in the US. 2 links where given, one to the BBC and one to CNN.
If this is how Slashdot works today, it's sad
Public Spaces (Score:1)
Oh yeah this will help......... (Score:2)
There are a million ways around the injunction. (see example above) I think from a moral stand point the judge is correct. Unfortunately being morally correct doesn't mean a damn thing.
SPAM uses extra CPU cycles, in some cases spam causes users to go over quota (is that a DOS attack against my users?) Is SPAM outlawed, is anyone REALLY doing anything about it?
NO
Do i think that this ruling will change the unsrupulous? No. Hopefuly Verio will show us that it is a "good guy" and go about it's business the right way.
Re:Public Spaces (Score:3)
Just because the machine is connected to the public internet does not mean that the machine is open for anyone to use however they please. This is enshrined in UK law these days (Note the gradual disappearance of "Welcome to hostname" for login prompts, it can be argued it's an explicit invitation for hackers to enter your machine
I mean, your telephone is connected to the public network but would it be OK for me to set up a bot to constantly dial your home to see if you'd dropped the price on the car you were selling?
Rich
Re:This is somewhat understandable... (Score:1)
---
seumas.com
Newsflash: Hemos drops Hooters girls down its pant (Score:2)
Btw, think about the 'hooters' admin that have its site slashdotted right now...
A new slashdot first (Score:1)
Re:Nice link, Hemos (Score:1)
If there were more serious stories, the ratio between serious and dumb replies would be higher. I've often seen that some stories only get 10-20 posts, not more. And they don't even appear on my Slashdot homepage. I've to use "older stuff" to see them.
Excuse me (Score:1)
Re:search engines (Score:1)
Re:I'll be damned. (Score:2)
ObOnTopic post:
Question: would search engines be different, presumably because they also confer a benefit on the target by making it findable?
The difference between what Verio is doing and search engines is one of implied permission.
Web sites either grant or deny permission to search engines based on the
the AC
Maybe this was just a simple hack of
Re:Oh yeah this will help......... (Score:2)
When Canter and Siegel first pulled their little stunt, it was a bunch of geeks spread around the world that got upset. Who cares about that? Since then, spam moved to e-mail and more and more people are starting to use the internet: Judges, politicians etc. As the volume of spam increases and these people get more affected, we're starting to see rumblings of legislation against it. I am sure that one day, spam will finally be outlawed in one country then other countries will begin to follow suit.
Rich
Re:search engines (Score:2)
free space with the likes of AOL/Freeserve/Geocities you have no control
over the indexing of your site. Additionally, some (albeit poor) ISPs
don't offer configuration of this file."
If you can't control your ISP's robots.txt, in the header of each page
put:
<META NAME="ROBOTS" CONTENT="NONE">
I have 70MB of pages on my ISP and robots were costing me a bundle
till I put this in. Now all major robots ignore it and only a few
oddball wiseguys occasionally download the entire site (even though I
offer them a compressed version of the entire site at my ftp site)
Good and Bad ... (Score:2)
This is good in the sense that another company should not peruse a site to gather contact information for marketing purposes. I've always thought these kinds of practices were dishonest to say the least. I think this kind of behavior should be curtailed.
On the other side of the coin. This is bad, because of this:
"If I don't like your linking to my site, or searching my site, even though it is open to the public, and I say, 'Stop,' you have to stop . . . whether you are actually hurting me or not."
Crawlers and Links shouldn't be penalized. It's a way of finding useful information (as opposed to finding new business... ...A way of getting contact information of people who probably don't want to be contacted, anyway.)
I also understand the reasoning behind the robots.txt file. If the information being gathered by a crawler will be outdated (in the case of eBay auctions) then it's a good way to selectively remove portions of your site for searching, because it's not appropriate.
---
Re:So... who actually clicked the link? (Score:1)
Does anyone else want to... (Score:1)
Stopping by (Score:1)
And if your someone whom writes down the prices, it's ok, as long as you do it politely and do not get in the way of my customers.
Just think about it. If I had a site that get
Best case I think is that we set up a robot tos page . this page will charge the bot indexing company a certain rate for reading the web site. That robot tos page will be mentioned in robots.txt as a non-indexable page. Therefor if a rouge robot get's into the system I have an acountable party to bill for abuse of my system.
any further ideas ?
I also think i could catch alot of spammers this way
michael
Re:Nice link, Hemos (Score:3)
1984? (Score:2)
I see they corrected the link, but made no comment (Score:1)
At least admit you fucked up, instead of pulling the old switchero and pretending you actually read this stuff first...
They're dodging the point? (Score:1)
The point of the issue as I see it (contradictions and argumentation more than welcome!) is that the information regarding which domain names are up for grabs should be public knowledge, otherwise it becomes restricted data that can only be used by Register.Com for marketing purposes, creating a form of "lock in" effect which enables them to aggressively market their own customers whilst preventing others from doing so in as targeted a way. Is this anti-competitive?
The argument about the cost to the company of the use of their server, and these numbers hovering around 2%, are irrelevant - if they want to remove the load, all they have to do is provide a table with the expiry dates of the registrations, which is the information people like verio want.
Should this information be private? Are these registration companies entitled to use their status to hoard this information and therefore erode the competitiveness of the market they inhabit.
I don't know the answer to these questions, but it pisses me off that the judge in question has dodged the issue.
Re:a binding Robots Exclusion Standard? (Score:1)
For instance, I don't let image-surfing software suck up my bandwidth looking for copyright violations or whatever. The images I have are either original to me, or part of a snapshot archive of newsgroup regulars and events, and there's positively no benefit to me in return for bandwidth hogging.
Now, the bulk of the graphics are in particular directories that are blocked in robots.txt, but you ought also be able to say "no *.jpg" or "no *.cgi" or whatever. Or "no image indexing bots," or whatever.
search engines (Score:3)
- Russ
Wrong link in the article (Score:1)
"Requests" can be blocked (Score:1)
Nice link, Hemos (Score:3)
-B
Re:Public Spaces (Score:1)
Off-the cuff, didn't read the article comment (Score:1)
Clickable correct link (Score:4)
Technical solution - fair queueing (Score:2)
Load-sharing boxes for server farms ought to have this feature. And it should go into Apache.
Correct link (Score:4)
I'll be damned. (Score:3)
I did notice, however, that the required registration at the "New York Times" [rheinfire.de] was not free...
information wants to be expensive...nothing is so valuable as the right information at the right time.
Attitude (Score:1)
The link in the story has obviously been corrected (it pointed to http://www.rheinfire.de/pamster.htm), so you might find this an interesting read:
From an interview by german computer and technology magazine c't with Rob Malda (aka CmdrTaco) (translation of translation follows):
c't: Haben Sie ein Problem damit, etwas zu korrigieren?
Malda: Das hängt von der Geschichte ab. Wenn ich einen Absatz geschrieben habe, der einen Fehler enthält und es Hunderte von Kommentaren über diesen Fehler gibt, würde eine Fehlerbereinigung diese Kommentare sinnlos machen. Die Nutzer-Kommentare sind wirklich wichtig. Wenn man sie nicht liest, ist es so, als würde man sich nur den Anfang eines New-York-Times-Artikels anschauen, aber nicht auf Seite 8 umblättern, um ihn weiterzulesen.
Translation:
c't: Do you think that correcting things is problematic?
Malda: That depends on the story. When I have written a paragraph with an error and there are hundreds of comments about this error, correcting it would make all these comments inane. User comments are really important. Not reading them is like glancing at the beginning of a New York Times article and then not turning the pages over to page 8 to read the rest of it.
Does this mean... (Score:1)
'Cause a Bot doesn't use up any more bandwidth or system time than I do when I keep hitting 'refresh' to see something update, or bids change.
They can't accuse the bot of doing anything wrong without accusing every user of the page. Unless they have a terms of use page first.
-Steve
If they dont want ease of use (Score:1)
You can turn off the search crawlers (Score:1)
ummm...this confers the same benefit (Score:1)
This is somewhat understandable... (Score:2)
If this was not Register.com's WHOIS service that was being used, then I would consider it a little more like a company that makes photocopiers looking in a public phonebook for big businesses, calling them up and saying "Hey, we'd like to do business with you and we'll beat whatever your current photocopier service is charging you".
---
seumas.com
'Bots at Hooters? (Score:3)
Oh my god! The Hooters girls were bots all along? I feel so dirty!
(And yes, the Hooters girls do use up resources on the target system, if you get my drift)
Re:I'll be damned. (Score:1)
I have just one question... (Score:3)
--
People or Robots (Score:1)
What is the difference? Intent; having enough lawyers.
The (costly) answer might be to write software that determines the load a user places on a server and block that IP for X time if the load is deemed excessive. Eventually, you could get to the point where you could make the bot look like the load created by a user, then what do you do if that replicate that bot over 1000 IP addresses. The (real) answer: if you offer anything for free, you have to depend on the user to not take advantage of that free service. After all if people acted like businesses there would never be any change in the leave a penny/take a penny cups.
Corporations are inherently evil since their only goal is to make money. The only salvation is to hope that good people run them.
Re:Public Spaces (Score:1)
Well, there's one difference between the situation described in the summary and your analogy. Verio is not a customer. Phrase the question this way: just because a retail store is publically open to customers, does that give a competing store the right to come in and take copies of all the name/address/telephone info from the sales slips so they can advertise to the customers? I don't think so. That's what Verio's doing, though, and IMHO Register.com's entirely within their rights to kick Verio out.
Although the article linked to in the summary doesn't seem to have anything to do with the situation described. Someone trolling, maybe?
Re:search engines (Score:3)
Secondly, robot.txt is often a server level setup file. If you get some free space with the likes of AOL/Freeserve/Geocities you have no control over the indexing of your site. Additionally, some (albeit poor) ISPs don't offer configuration of this file. Whether it is the fault of the ISP, the search engine, or you, for the crawling of your site would be a matter for further debate.
Onion
Re:Nice link, Hemos (Score:5)
I'm really, really glad that the submitter didn't slip a really objectionable link in there. I'm also really, really pissed off at Slashdot for this kind of crap. This is total incompetence. (I'm not even taking into account the duplicate stories [slashdot.org] on the Chinese rocket lanunch [slashdot.org] in the Science section...)
This kind of fsck-up at virtually any other major online content provider would be grounds for immediate dismissal for the employee in question, for crying out loud. READ YOUR DAMNED FRONT PAGE SUBMISSIONS!
information wants to be expensive...nothing is so valuable as the right information at the right time.
Don't forget the implied inverse (Score:3)
"Everybody must be allowed to access web resources" is a statement from the POV of the accessors. Consider that statement from the point of view of the server managers: "We must allow everybody to access our resources in any way they choose."
Do you really want to make that statement? If you put up a public resource, must you allow people to abuse it if they wish? Or can you take actions to stop such abuse, esp. as it nearly always does real, if not always a lot of, damage. In the case of Bidder's Edge vs. eBay, eBay was suffering real slow-down of service, which affects its bottom line. Must eBay allow it?
Perhaps the real danger is not so much the rulings per se, but the legal doctrines being used to make them: "Under the reasoning in the Register.com case, "you don't have to prove harm or show any evidence of harm," he said. "Harm will be presumed." He said that he fears the Register.com case will "spread like Kudzu" through the court system."
At any rate, just recognize that things are somewhat more complicated then they may seem at first. It's tempting to oversimplify in either direction, but the truth is probably complicated.
Bots at Hooters? (Score:2)
(Translation: Click the link, Hemos. Or even just hold your mouse over it to see where it goes.)
--
Verio called 1 day after registering! This week !! (Score:1)
So Mr. Verio calls me up, asking me about my hosting and design needs, and is actually not so bad for a salesman. He sends me some info from their website and agrees to follow up on Monday with me if I'm interested about using their services.
Now that I've read this article, I don't know what to think/do yet. It looks like Verio provides a level of service which would be pretty affordable for my domain needs, but I also don't want to perpetuate bad business practices by giving them my business. I'll have to prepare some thoughts on how he got my name and information and grill the poor guy next time he calls.
Another Judge who doesn't understand technology (Score:2)
This is 1.11% of the "total load", meaning 1.11% of the CPU usage, not 1.11% of the CPU. Nothing here argues that the machines were fully loaded. Had they been, then yes, eBay was prevented from using that load. But it's doubtful that eBay had a 100% load all of the time, so it's doubtful they were using those resources.
Looks like a short-sighted ignorant ruling (Score:2)
The simple fact is that every single interaction between any two computer systems requires resources in the way of memory, processor time, network bandwidth and sometimes disk access. Now the judge seems to indicate that if you go over my service as it was designed, and retrieve information, and then use that information in a way I don't like, I can forbid you from using my service.
That's completely unrealistic. Unless the searching routine is basically stomping the server by requesting as fast as possible, there is no real damage being done that isn't done by a regular cyber-squatter wannabe trolling the database in this case. If there is an issue with the DoS sort of effect, ask the other party to back off, or alter the server software to restrict the rates at which requests are accepted from certain IP addresses or blocks. Better yet, negotiate a new service where the database query is run locally by the whois provider, bundled up, and distributed for fee to anyone interested.
The thing is, it is public information. This sort of legal adjustment of the reality is foolhardy in the extreme. If I take a quote from an article on a news site, citing the reference properly, and use it as a portion of a work that results in something the originating news site doesn't like, can they forbid me from using the site now?
The ramifications are a lot further reaching than just bots. It's all a matter of degree.
Re:search engines (Score:1)
Someone else mentioned a per-page META tag alternative to robot.txt, and I would also point out that the server-level configuration problem is between you and your ISP, as are many other access and content issues. The outside user only really cares (legally at least) about how their browsing/searching behavior interacts with the ISP, and then the ISP has a relationship with the site that is completely seperate.
- Russ
Hemos Quality Seal (Score:1)
This raises some very interesting issues... (Score:1)
Namely, if this cases wins and is upheld, then look next for a lawsuit seeking to shut down the SEC 'FraudCrawler' [usatoday.com] and any other government owned or contracted law enforcement crawlers that come along after it.
Here's why.
If Register.com wins, then the courts will have recognized bandwidth as an asset on an equal legal footing to money or physical property. Once bandwidth has gained this legal stature, then suddenly the fourth admendment of the constitution kicks in:
"The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures , shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized."
Because unauthorized consumtion of bandwidth would then be legally equivalent to a seizure, the government would have to obtain and serve a search warrant on the owner of any server that they wish to examine the contents of!
"A microprocessor... is a terrible thing to waste." --
Re:Technical solution - fair queueing (Score:1)
Re:I'll be damned. (Score:2)
Web sites either grant or deny permission to search engines based on the /robots.txt file. If a web site wants to be indexed, they put permissive rules in robots.txt. Verio is spidering for their own commercial gain, and ignoring a number of posted policies against it. That is apparently what the judge has ruled on, violating an explicit request not to harvest. What is funny is that register.com doesn't have a robots.txt file, so does that give people permission to spider the site?
robots.txt...it's not just a good idea, it's the law.
-jon
Re:Technical solution - fair queueing (Score:2)
One might view that as a feature.
Maintaining a list of proxies you like might be necessary. (Or not, depending on whether you want visits from AOLers). This is why network routers have weighted fair queuing.
Re:Correct link (Score:1)
Re:search engines (Score:1)
-B
Maybe, maybe, maybe (Score:1)
Correct (no reg) Link (Score:1)
You Mean http://partners.nytimes.com/2001/01/12/technology/ 12CYBERLAW.html? [nytimes.com]
(No reg version)
You do know that Verio *is* Network Solutions (Score:1)
Re:1984? (Score:1)
We only acknowledge small faults in order to make it appear that we are free from great ones. -- La Rouchefoucauld
Link to the story (Score:1)
Although, the other page was a Hoot!
---
Interested in the Colorado Lottery?
I'm actually glad (Score:1)
My $.02
--
Re:Nice link, Hemos (Score:1)
Here I go, trespassing again. (Score:1)
Did this bug the living shit out of anyone else? Is there one company, or two? None of my friends who work for Verio have mentioned a sister company called Vario. And it would have been funnier if they had spelled it Varyo.
Actually, I think the Rhein Fire had a better editor.
Sufficient Harm (Score:2)
There's a very important difference here (Score:2)
US Law (Score:1)
Not that I'm suggesting anything, but if this restriction is a bad thing (TM) for for freedom, how about us non-US citizens start taking an interest in WHOIS spidering.
Re:Nice link, Hemos (Score:1)
It's getting longer between the good posts, you have to surf at at least +2.
--------
Re:This is somewhat understandable... (Score:2)
WHOIS and Junk mail (not hooters flyers) (Score:1)
I know its from that source because my name is spelled strangly and they send it to me at the semi-fictious enity that is the "company" my web site is registered to.
a binding Robots Exclusion Standard? (Score:2)
Sure, one might argue that people might be unaware of the standard, but that is seldom an excuse. I may be unaware of fire/electrical codes, but I'll still get in trouble if I don't adhere to them, because I'm putting others at risk and thereby imposing a cost upon society (fire trucks and insurance don't come free). Web crawlers that index data in violation of the Robots Exclusion Standard impose a cost on companies and society just as well, in the end requiring people to by bigger pipes, faster servers, and so on (thereby using more power, dumping more old computer components into landfills and more chipmaking chemicals into the environment).
My point is that web crawler operators live in a society, just like everyone else, and they too must be held accountable for the consequences of their actions, particularly when they willfully disregard the requests of web site operators as expressed in
So... who actually clicked the link? (Score:4)
-Todd
---
Re:Off-the cuff, didn't read the article comment (Score:1)
And that's how we ended up in court.
Actually, I live in Texas, so I just shot the fucker.
(In Texas, if someone is harming your property, or someone else's property, and you feel that the only way to stop them is to respond with deadly force, it is not felony homicide.)
Re:Stupid! (Score:1)
Uh... You have just described the planet earth.
Re:Does this mean... (Score:3)
Outside of the issue of bandwidth, there is the issue of profitability. Part of many websites income is derived from banner ads. If a bot scours a website to harvest the content, it prevents the end users from seeing some of the advertisements they would have normally have seen by exploring it on their own. One example of where this hurts is a Meta Search engine that trolls several search engines and produces a compiled list of search results. Search engines have millions of dollars invested in hardware, software, bandwidth and staff to make it all work. Every single query has a real monetary cost associated with it. Every free service has its cost. A Meta Search engine bot like that only does about %5 of the total work involved producing the results or content that it displays. Now, a site like that makes its OWN profit from users with minimal money out of its own pocket (of which none goes to the companies doing most of the real hard work).
I completely understand why companies would get cranky about someone repeatedly grabbing computationally intense data from their site and profiting from it as they suck money and resources away from the provider of said data.
One way websites are about to track when people visit a site is "tracking gifs". Usually very small 1x1 pixel images that give them a general idea of how many visitors a site is receiving. Reports are generated and they get PAID by advertisers based upon this info. Bots RARELY grab anything but content. If you go to a webpage with images embedded in it, your web browser individually requests (most of the time barring cached data) each image. Since bots don't tend to request this graphical "fluff" intended for hyoomans, owners of the site notice an increase in site traffic and resource drains and a decrease in "ad impressions" when a bot "isn't polite". Yes, I do realize that some make revenue in other ways, but bots can use up resources faster than humans can regardless. With people your traffic usually scales slowly up or down. You can add or remove hardware to deal with the demand. Sometimes when a bot hits your webserver it is such a huge spike in requests for data it kills the server. Then all the legitimate users get cranky. Or, a more minor form is that it makes the s e r v e r sllluuuugggggiiish.
Anywhoo... Thats my $1.25 on the matter. Cheers. - Sartian
Re:Clickable correct link (Score:1)
Re:Wrong link - MY APOLOGIES. (Score:2)
This seems similar to other laws (Score:4)
Similarly, if you worked at one store and went to your competitor's, pen and paper in hand, and strolled the aisles noting their prices (so your store can meet/beat them), you might be asked to leave. Despite the fact that you are just writing down prices that are clearly there to be read.
Finally, various retailers, esp. car dealers, place "No wholesaler or retailer" restrictions on their best sales, even though their products are meant to be bought and other retailers may want to do just that.
It seems to me that analogous laws already exist. Just because something is available in the public realm it doesn't follow that anyone can avail themselves of it to any extent; at least not under current U.S. laws.
-----
D. Fischer
What the Judge found (Score:2)
http://channel.nytimes.com/2001/01/12/technology/1 2CYBERLAW.html [nytimes.com]
Although here is this interesting bit from the middle of the article:
Finding web pages (Score:2)
This ruling is obviously rediculous; who hosts a website that can't handle 1 extra connection?
But just to play devil's advocate, search engines are slightly different, since you can always specify in Robots.txt which robots, (none even) can access your site, and what they can access (effectively controlling their time on your site).
I would like to see a law against robots which do not adhere to robots.txt though...
Oh wait, this is the internet and it shouldn't be online if it's not meant to be accessed... sorry, I forget sometimes...
It somes down to mass for profit motives (Score:2)
Re:Technical solution - fair queueing (Score:2)
In what way is that fair? In fact, it would be fairer to give regular/frequent/recent users higher priority because they're the ones doing whatever it is you want them to do (like, viewing banner ads, if you're /., and completing transactions if you're a brokerage).
Load-sharing boxes for server farms ought to have this feature. And it should go into Apache.
There are better solutions [cisco.com] already in the marketplace for allocating resources for network services.
Re:Nice link, Hemos (Score:4)
This only proves that Slashdot really doesn't care to check a story before posting it. I don't see what's so hard about clicking on a link to see if it works? Or to see if it goes somewhere sensible? I mean, we're not even talking about checking facts here or anything. Why is it that something so basic as checking URLs seems no longer relevent to the Slashdot editors?! What are they doing now???
Sorry for this rant. I hate the downward trend of Slashdot recently. This wrong link almost made me give up Slashdot forever... there are better, less crowded, less trolled places around that I think I'll move to.