Forgot your password?
typodupeerror
Databases The Military News

How a Team of Geeks Cracked the Spy Trade 187

Posted by kdawson
from the connecting-the-dots dept.
drunken_boxer777 sends us to The Wall Street Journal for a lengthy article on a small tech company, Palantir Technologies, that is making the CIA, Pentagon, and FBI take notice. The submitter adds, "And yes, their company name is a reference to what you think it is." "One of the latest entrants into the government spy-services marketplace, Palantir Technologies has designed what many intelligence analysts say is the most effective tool to date to investigate terrorist networks. The software's main advance is a user-friendly search tool that can scan multiple data sources at once, something previous search tools couldn't do. That means an analyst who is following a tip about a planned terror attack, for example, can more quickly and easily unearth connections among suspects, money transfers, phone calls and previous attacks around the globe. ... With Palantir's software 'you can actually point to examples where it was pretty clear that lives were saved.'"
This discussion has been archived. No new comments can be posted.

How a Team of Geeks Cracked the Spy Trade

Comments Filter:
  • Call me dense... (Score:4, Interesting)

    by Anonymous Coward on Friday September 04, 2009 @12:29PM (#29312557)

    But what is the reference?

  • Name? (Score:5, Funny)

    by Anonymous Coward on Friday September 04, 2009 @12:30PM (#29312571)

    > ..a small tech company, Palentir Technologies..

    > ..Palantir Technologies has..

    > The submitter adds, "And yes, their company name is a reference to what you think it is."

    A spellcheck company?

  • Reference to LotR (Score:5, Informative)

    by Anonymous Coward on Friday September 04, 2009 @12:33PM (#29312615)

    It was the seeing stone that Sauron used in Lord of the Rings.

    That is the tool the evil guy used to control the world. Sounds appropriate.

    • by Hadlock (143607) on Friday September 04, 2009 @12:55PM (#29312893) Homepage Journal

      Thanks, my internet is down, I was unable to google that myself.

    • No, they were a gift from the Valar to the NÃmenÃreans to see other times and places.
      Sauron just happened to get a hold of one of them, and used it as an avenue to warp Saruman and Denethor's minds. There is no evidence he used the PalantÃr for their intended purpose.

      • by Abreu (173023)

        And, in other news in the "that's obvious" department, Slashdot still does not have proper Unicode support!

      • They were gifts from the Eldar to the Numenoreans. If I remember right, there's a line in The Silmarillion which implies that Feanor might have made them.

        There's a good story in Unfinished Tales about the use of the palantiri.

    • Re: (Score:2, Informative)

      by hooeezit (665120)
      That's the FUD propaganda. The more balanced perspective is it's the seeing stone anyone could use to see their future. But it was put to evil purposes instead by Sauron.
      Technology by itself is not good or evil. It's how one uses it that makes it so. Remember that the Internet came out of a doom-and-gloom project to create a nuclear-war resistant communication network.
    • by denzacar (181829) on Friday September 04, 2009 @01:12PM (#29313091) Journal

      Good guys used it too. To defeat Sauron AND to "keep the world safe".

      In fact... Good guys made all 7 Palantir mentioned in LotR.
      Sauron got his hands on one of those and used it to corrupt Saruman and Denethor.

      So... No. It is not "the tool the evil guy used to control the world."
      The message would be that "power corrupts". In this case - power in the form of knowledge or information.

      What Palantir really lacked was a decent firewall. No protection whatsoever.
      Very intuitive user interface though. And they were practically indestructible.

      • by AP31R0N (723649)

        The dwarves drilled three holes in one of the 7 and used it to bowl the first perfect game in Middle Earth history.

    • Re:Reference to LotR (Score:4, Informative)

      by jollyreaper (513215) on Friday September 04, 2009 @01:16PM (#29313149)

      It was the seeing stone that Sauron used in Lord of the Rings.

      That is the tool the evil guy used to control the world. Sounds appropriate.

      The Palantir themselves were not evil, it was simply put to an evil purpose. The last surviving one was so corrupted by Sauron's influence it could never be used peacefully again but you can no more blame the Palantir for that than you could blame a wrench for becoming radioactive when left sitting next to a leaky reactor. Really, the only bit of truly evil magic in the entire book was the Ring itself and, seeing as it bore a measure of Sauron's own power, I think of it less as an object than as a character with a will and mind of its own.

      There is no evil in science, technology, or magic; evil lies not in the tool but the hand that wields it.

      • by Fnord666 (889225)

        no more than you could blame a wrench for becoming radioactive when left sitting next to a leaky reactor.

        Damn, I hate when that happens. They do bring good money at a garage sale though.

    • by alta (1263)

      Mod -1, Unnecessary definition ;)

    • by Alphanos (596595)

      The larger role played by a Palantir was the one used by Saruman. He was chief among the wizards sent to oppose Sauron and the forces of evil, but the knowledge granted by the stone corrupted him such that he turned against his order and sought power for himself.

      Even more appropriate.

  • by Sporkinum (655143) on Friday September 04, 2009 @12:34PM (#29312631)

    With a name like Palentir, it sounds like trojan spy program, not a Google like search tool.

  • Great! (Score:4, Funny)

    by Saija (1114681) on Friday September 04, 2009 @12:35PM (#29312635) Journal

    Palantir Technologies has designed what many intelligence analysts say is the most effective tool to date to investigate terrorist networks

    What? a crystal ball to fight the terrorist:

    A palantír (sometimes translated as Seeing Stone but actually meaning "Farsighted" or "One that Sees from Afar") is a stone that functions somewhat like a crystal ball. [wikipedia.org]

  • Bad summary (Score:5, Informative)

    by gurps_npc (621217) on Friday September 04, 2009 @12:36PM (#29312653) Homepage
    The summary seems to be a description of a meta-search engine, which is rather common. (Dogpile).

    The actual product seems MUCH more interesting than the silly summary. It compartamentalizes secret info, so if you are classified for level 5, you can still search and find info that is level 6, even if the file also has level 4 information. It can also tag information so that if your level 5 clearance is not enough to tell you how person A is connected to person B, you can still know that the connection exists.

    • Re:Bad summary (Score:5, Insightful)

      by fuzzyfuzzyfungus (1223518) on Friday September 04, 2009 @12:55PM (#29312899) Journal
      The tech sounds quite interesting; but I'm not sure I love the idea of having intelligence agents operate on a "Yes, person A is linked to person B. You aren't allowed to know why; but the omniscient computer assures you that it is so." basis.
      • Re: (Score:3, Funny)

        by sukotto (122876)
        Trust the computer. The computer is your friend. (There is no ultraviolet classification)
      • Re:Bad summary (Score:4, Insightful)

        by steelfood (895457) on Friday September 04, 2009 @01:33PM (#29313445)

        It's all about need to know. If you knowing why is necessary to draw a conclusion, you'll eventually be granted this access.

        Under the old system, you outright wouldn't even know that a connection exists, nevermind whether you need to know whether that connection is important or not.

      • by pnuema (523776)
        This is a tool for intelligence analysts. They just write reports and hand them off to people who do have the clearance. I'd be willing to bet good money that the people who make operational decisions are not the ones using this tool.
      • by rm999 (775449)

        Why not, if it helps them do their job?

        I feel like there is an implicit assumption on Slashdot that government agents don't know how to do their jobs properly, even though we know very little about what they do.

        And then we complain when lay people assume tech/IT people don't know how to do their jobs correctly.

    • Re: (Score:2, Insightful)

      by NoYob (1630681)

      The summary seems to be a description of a meta-search engine, which is rather common. (Dogpile).

      The actual product seems MUCH more interesting than the silly summary. It compartamentalizes secret info, so if you are classified for level 5, you can still search and find info that is level 6, even if the file also has level 4 information. It can also tag information so that if your level 5 clearance is not enough to tell you how person A is connected to person B, you can still know that the connection exists.

      Yeah, but if you are classified for level 5 and look at level 6, which presumably is above your classification, then you are in fact looking at classified work even if it has level 4 work - which means the levels of classification are being broken and the security is compromised. And if person A is a 5 and looking at classification 6 which is connected to person B it in effect blows any security clearances out the door. Of course, person C who is a 4 looking at person B who is a ....I've gone cross-eyed, d

      • by PitaBred (632671)
        I think in his example, he thought that Level 5 was higher than Level 6. Level 1 was the most secret. At least, that's what seems to make sense to me.
    • CIA and FBI computer systems are infamously way, way behind. They only got wikis in 2006. Now they can finally google something.

    • by JLavezzo (161308)

      I think you're describing PL4 security. That's not the trick here. When they say different sources, they mean different TYPES of sources, geospacial, dates, ip addresses, telephone logs, video metadata, random XML, SQL. Dogpile searches multiple sources of unstructured text.

    • Fact A can be Secret
      Fact B can be Top Secret and
      Fact C can be confidential
      Sometimes taking C and then correlating to something with A+B (with the B removed) will then result in TS (same as B). So, I'd think it's a touchy area. In the 90s a similar "classification by association," was commonly referred to as Elements of Essential Friendly Information (EEFIs), such as a recall roster and leave schedules. If the enemy has the recall roster and suddenly one particular part of a unit gets a 3-ring recall
  • by reporter (666905) on Friday September 04, 2009 @12:36PM (#29312659) Homepage
    When you aim the Palantir tool at terrorists, you can easily identify useful patterns in disparate data. These patterns reveal information about the names and the plans of the culprits.

    What happens when you aim the same tool at ordinary people like Slashdotters? You will discover sexual orientation, adultery, etc.

    In other words, the same tool saving us from the terrorists can also defeat the last barriers protecting our privacy. If an intelligence officer in the government hated a particular SlashDotter (due to her articles in this forum), that officer could easily identify her address, her friends, her bank accounts, her adulterous lover, etc. Can you say, "blackmail"?

    • by Dancindan84 (1056246) on Friday September 04, 2009 @12:57PM (#29312939)

      If an intelligence officer in the government hated a particular SlashDotter... her friends... her adulterous lover

      A female SlashDotter with friends and a lover... it would take a top tier spy tool to find that unicorn.

    • Re: (Score:3, Insightful)

      by MobyDisk (75490)

      No software cannot determine your sexual orientation, nor your hidden adultery. It does not read minds, or hearts. It does not magically know events from the past or the future.

      Now, if you went online and posted about your homosexual adulterous relationship on a board that publicly reveals your IP address, then yes, a tool could indeed find it. In that case, who defeated the last barrier of your privacy? Did the tool? Or is it your own darned fault?

      • Re: (Score:2, Interesting)

        by Anonymous Coward

        No software cannot determine your sexual orientation, nor your hidden adultery. It does not read minds, or hearts. It does not magically know events from the past or the future.

        I do not know that you can definitively say that. What the tool can ferret out is going to depend on what data it has, how much data it has and the quality of that data.

        Does the tool have access to credit card data? Does the tool have access to hotel reservation data? Car rental data? Flight reservation data? Phone records? Movie rental data? Other data types too numerous to list that do exist and that the government might get access to?

        Given enough data I don't think anyone can say with absolute cer

    • by steelfood (895457)

      Which is why it's a good idea to use separate identities everywhere and keep them separate and distinct.

      It's rather difficult and annoying as hell, but that's if you want to be safe. The paranoid will go as far as to access these separate identities from entirely separate systems. Certainly, their access patterns may be similar or the same, but that's only if they can make that connection.

      The other thing to do is to change your speech patterns for each identity. The idea is to try to mimic the general speec

    • by Alpha830RulZ (939527) on Friday September 04, 2009 @01:57PM (#29313947)

      Adultery? You mean like using your other hand?

    • by afxgrin (208686)

      Who will babysit the babysitters....?

  • Hayden Panettiere ?

    Ok, well thats the first thing that came to my mind...
  • "Hi, I'm Alex Karp," Mr. Karp said, offering his hand. No response. "I didn't know you really don't ask their names," he says now.

    Real spies have fake names and ids. There's no reason not to give the guy a name, as long as everyone in the room isn't named "Bob".

    • by AndrewNeo (979708)

      But that would run the risk of blowing that fake name's cover. Of course, he should have been able to just make something up on the spot..

  • There has been this notion that somehow if you can shove a bunch of data through algorithms that somehow you can catch terrorist networks.

    More likely you're just wasting time and here's why: terrorists don't act or usually exhibit predictable and trackable behavior like normal people. Typically they deal with disposable cell phones, cash and other "untrackables".

    These guys have managed to come up with Yet Another Terrorist Tracking Tool®
    • Re: (Score:2, Interesting)

      by PainKilleR-CE (597083)

      but as TFA points out, the people they're looking for often do things that should get them caught, like using the same address and phone number when buying the plane tickets in the case of the 9/11 hijackers. The basic idea is to find a better way to process the data they already have, and to give people the ability to process data that will help them, even when they don't necessarily have access to it (ie the use of data classified at a level higher than the searcher has access to).

      The problem generally ha

    • Re: (Score:3, Informative)

      by brainboyz (114458)

      Except, despite all of this, they still exist in a trackable world. They live and have stuff delivered to addresses, they access information that leaves a data trail, and use identifiers which do the same. If they share anything, or a field observer notices a meeting then it gets tagged as a meeting and connection; then any activity at all is tracked back to a single node (bank account, address, person, phone number, etc) then you can link ALL connected nodes to that activity. Cash, disposables, and other "

    • by alta (1263)

      If you can even USE the word 'Typically' about something, your are implying that they do something often enough to make it traceable. Then you further your anti-argument by giving examples such as disposable cell phones.

      Very interesting. You must be a terrorist posting disinformation.

      Or maybe I am for pointing it out ;)

    • Your post is not only ill-informed, but totally illogical. Which is it, are terrorists unpredictable (which itself is a pattern) or do they exhibit certain typical behaviors?!

      Never too late to read TFA, you know...

    • by dkleinsc (563838)

      Shouldn't it be "Yet Another Terrorist Tracking Apparatus", or "YATTA!" for short? Or is that the Japanese version?

  • by Daniel Dvorkin (106857) * on Friday September 04, 2009 @12:48PM (#29312811) Homepage Journal

    ... we hatesss it, Preciousss, yesss we doesss.

  • After all, all the Seeing Stones are not yet accounted for. Who knows who might be watching?
  • by alta (1263)

    Cool, so they just invented Splunk! Cool. Is it any cheaper than splunk, because if it is, I'll use it.

  • > The software's main advance is a user-friendly search tool that can scan multiple data sources at once, something previous search tools couldn't do

    OMG! Did someone finally discover the hidden "UNION" conjunction in SQL?

  • This sounds similar to Starlight [wikipedia.org], which the NSA uses for all kinds of "connect the dots" type intelligence activities.

  • Actually their software works like this:

    1: Announce software that will bust terrorist networks.
    2: Only terrorists buy software to test out their own network security.
    3: Software phones home.
    4: PROFIT!
  • Just be careful not to have your name mentioned in the same document with a bad guy's name in it. This technology really is that simple and that dangerous.

  • There is a particularly entertaining game mentioned in the video called "Turbo Hearts", which rules.

    I found this good explanation of how to play:
    http://ericanderson.us/2009/09/04/how-to-play-turbo-hearts/ [ericanderson.us]

An Ada exception is when a routine gets in trouble and says 'Beam me up, Scotty'.

Working...