US Marines Outsmart AI Security Cameras by Hiding in a Cardboard Box (petapixel.com) 86
United States Marines outsmarted artificially intelligent (AI) security cameras by hiding in a cardboard box and standing behind trees. From a report: Former Pentagon policy analyst Paul Scharre has recalled the story in his upcoming book Four Battlegrounds: Power in the Age of Artificial Intelligence. In the book, Scharre recounts how the U.S. Army was testing AI monitoring systems and decided to use the Marines to help build the algorithms that the security cameras would use. They then attempted to put the AI system to the test and see if the squad of Marines could find new ways to avoid detection and evade the cameras.
To train the AI, the security cameras, which were developed by Defense Advanced Research Projects Agency's (DARPA) Squad X program, required data in the form of a squad of Marines spending six days walking around in front of them. After six days spent training the algorithm, the Marines decided to put the AI security cameras to the test. "If any Marines could get all the way in and touch this robot without being detected, they would win. I wanted to see, game on, what would happen," DARPA deputy director Phil Root tells Scharre in the book. Within a single day, the Marines had worked out the best way to sneak around an AI monitoring system and avoid detection by the cameras. Root says: "Eight Marines -- not a single one got detected." According to Scharre's book, a pair of marines "somersaulted for 300 meters" to approach the sensor and "never got detected" by the camera.
To train the AI, the security cameras, which were developed by Defense Advanced Research Projects Agency's (DARPA) Squad X program, required data in the form of a squad of Marines spending six days walking around in front of them. After six days spent training the algorithm, the Marines decided to put the AI security cameras to the test. "If any Marines could get all the way in and touch this robot without being detected, they would win. I wanted to see, game on, what would happen," DARPA deputy director Phil Root tells Scharre in the book. Within a single day, the Marines had worked out the best way to sneak around an AI monitoring system and avoid detection by the cameras. Root says: "Eight Marines -- not a single one got detected." According to Scharre's book, a pair of marines "somersaulted for 300 meters" to approach the sensor and "never got detected" by the camera.
AI tactics (Score:5, Funny)
Re: (Score:1)
Towels around heads (Score:3)
Ravenous AI beast of Trall, so mind bogglingly stupid it thinks if you cant see it, it can't see you.
Re: (Score:2)
So the tactics needed to fool AI surveillance systems basically come down those used by cartoon characters, Tom and Jerry and Roadrunner and Wiley Coyote. Good to know.
Well then, the problem is easy to solve - they have simply need to include all those episodes in its training set.
Re:AI tactics (Score:5, Funny)
Well then, the problem is easy to solve - they have simply need to include all those episodes in its training set.
Well, they tried that. Problem is, they also shared the training data with Tesla - then afterward there was a bad crash after a Model S tried to drive through a tunnel painted on the side of a cliff.
Re: (Score:2)
That's because the steering wheel came off [insideevs.com].
Re: (Score:2)
Re: (Score:2)
Wait, it came off without using the official Lego piece separator tool?
Re: (Score:2)
Re: (Score:2)
The best safety feature is when the Tesla self driving goes off the side of a cliff, but then hangs there for a few seconds giving passengers a chance to climb out and jump to safety.
Re: (Score:2)
As long as they don't look at the camera and wave "bye bye", they'll never fall! But no one ever thinks of that...
Re: (Score:3)
The best safety feature is when the Tesla self driving goes off the side of a cliff, but then hangs there for a few seconds giving passengers a chance to climb out and jump to safety.
Sensing the weight of all the passengers and cargo so that it just teeters perfectly was very tricky to get right, being an edge case and all.
Re: AI tactics (Score:2)
As dumb as this is⦠yeh⦠I mean, what did they expect when they trained it to detect people walking around? That it would suddenly understand every possible way you could sneak up on it?
Re: (Score:2)
All you need to avoid detection systems is a cardboard box and a pack of smokes.
Re:AI tactics (Score:4, Interesting)
I always thought the "walking very slowly" scene in Sneakers was absurd, and the security system would definitely flag them. But maybe not?
Re:AI tactics (Score:5, Informative)
Mythbusters did a whole thing on that stuff. Walking slowly DOES work, but even better was just holding a bed sheet in front of you. Clip of the sheet working:
https://www.youtube.com/watch?... [youtube.com]
The sheet blocks the heat detection, cause it just sees a room temp sheet. Boxes would be similar, but probably not quite as good since a person in a box will eventually heat up the box.
Re: (Score:2)
I've seen this done with just a sheet of insulating foam board with a couple of handles glued to one side to grab on. $25 at a home supply store.
The interesting question would be is the AI so narrow that the Marines could have dressed in Navy Whites and just walked up and touched it? Maybe wear a sign that says "pine tree" and just walk up?
Re: (Score:2)
Re: (Score:2)
However, in the movie, there was also the added issue of moving too quickly. I believe it was more than three inches per second. So you would also have to walk that slowly with the sheet or else the sensor would notice the change.
Re: (Score:3)
The narrator is correct. The sensor they're testing in the clip is an ultrasonic motion sensor. However, it's well known that these types of sensors make for poor security, so they're usually used for non-security applications like turning on the lights.
The thing they held up to "trick" an IR sensor was a sheet of glass. However, this wouldn't work against real IR sensors because it would also block the background, so they'd notice the change in temperature.
In Sneakers, they're trying to trick two sensors a
Re: (Score:2)
I used to walk past a motion detection sensor in the computer lab by walking slowly (since the security team sometimes forgot to turn it off early in the morning).
Re: (Score:3)
So the tactics needed to fool AI surveillance systems basically come down those used by cartoon characters, Tom and Jerry and Roadrunner and Wiley Coyote. Good to know.
Well Wile E. Coyote was a genius.
Re: (Score:3)
Well Wile E. Coyote was a genius.
Card-carrying, and that's a fact.
https://www.skewsme.com/img/wi... [skewsme.com]
Re: (Score:3)
Well Wile E. Coyote was a genius.
I believe you'll find that's a _Super_ genius.
Re: AI tactics (Score:2)
You can also kill facial recognition with some face paint. AI is fucking retarded.
Re: (Score:2)
That's because all the things people are being lead to believe about AI are silly nonsense. It drives me nuts. Remember how frustrating it was watching creationists during the Dover trial? It's like that, but with people that really should know better.
Re: (Score:2)
I believe the cardboard box is a key disguise in Metal Gear Solid. Something that seems intended more for a laugh than being serious. And yet...
This also reminds me of how to defeat the motion sensor done by Mythbusters.
Re: (Score:2)
The article doesn't give a date or what type of AI system they were using, so that probably helps figure out when they did these tests.
Re:Important note (Score:5, Interesting)
Next they'll train it on boxes and people will use bed sheets with eye holes to get past it.
(skip five years)
AI now shoots anything that moves.
Re: Important note (Score:2)
shoots anything that moves
Not shoot. But alert the security desk where they monitor the cameras. For this, you don't need AI. A simple motion detector will do. My home cams work this way. It isn't unusual to get up in the morning and see clips of the local racoon family walking by. For a manned security system, this would aleviate most of the monotony of watching a bunch of screens with no activity.
Re: (Score:2)
Re: (Score:2)
[Shows images of somersaulting marines and marines in boxes]
AI: None of these fit your classification.
Human: HA! Fooled you!
Re: (Score:2)
I think that even an mail cop will be able to spot that.
Well (Score:5, Insightful)
Adapt, improvise, overcome!
Someone played metal gear (Score:4, Funny)
So they did the ol' solid snake box move,
Hiding in boxes since 1987! (Score:2)
https://www.youtube.com/watch?... [youtube.com]
Stare (Score:2)
And then there was Sir Rodney (Score:3)
This story's about a week old. And you all missed the one gyrene who did a Sir Rodney (from the Wizard of Id): he cut a small bush or tree and walked up using it.
Oh, and then there were two who somersaulted up to it.
Snake!!! (Score:4, Funny)
I knew all my videogame training would become useful :)
Re: (Score:1)
300 meters of somersaulting? (Score:4, Funny)
Wouldn't it have been easier to be captured by the enemy and waterboarded?
Re: (Score:2)
Re: (Score:2)
Maybe moonwalk up to it, then the AI mistakenly thinks that you're walking away from it.
AI detection schemes are lacking (Score:1)
I have a security system with human / motion detection. It picks up snow, leaves etc. But sometimes people walk across and no alert. Cars drive through, nothing. I also have some PIR motion sensors and they're only good for 5-20 feet and in a FOV. Maybe the systems I have are garbage but I can imagine even the best systems its tough to not have false positives yet still not let things slip past.
This is important because its tough for humans to sit and watch security cameras all day. People get bored, attent
When thinking inside the box ... (Score:1)
.... pays off
Cardboard ghillie (Score:1)
Is the best ghille!
And we now know how to fool The Terminator...
Re: Cardboard ghillie (Score:2)
I think something like this has already been developed for Seattle sidewalks [blogspot.com].
According to Scharre's book, a pair of marines (Score:2)
Re: (Score:2)
Could they shoot straight before?
Re: (Score:2)
Re: (Score:2)
Ah, but after summersaulting though. You're dizzy, the middle ear going crazy, etc. That's why biathlon is a real thing at the olympics, because being able to shoot accurately immediately after physical exertion is a whole new level of difficulty and yet an important skill in the military.
Also, can they summersault while being hidden in cardboard boxes?
Re: (Score:2)
Modern biathlon derived from hunting, not military (Score:2)
That's why biathlon is a real thing at the olympics, because being able to shoot accurately immediately after physical exertion is a whole new level of difficulty and yet an important skill in the military.
The US military incorporates physical exertion (running, PT, etc) to get breathing and heart rates elevated during some shooting exercises. The military shooters also have a more challenging cognitive environment as they have to estimate distances to the target, wind speed and direction, etc. The modern biathlon shooters are are firing at targets a small fraction of the distance of the military shooters. The modern biathlon is really derived from hunting while skiing. The true military style training with t
is it better then some min wage rent a cop looking (Score:2)
is it better then some min wage rent a cop looking at video all night?
Better training for the AI (Score:1)
All this says is that the Marines performed significantly different behavior from their normal "walking around in front of" the security cameras, which is what the AI was trained on. A more appropriate training set will now include all the various evasive techniques they came up with and anything else anyone can think of.
Re: (Score:3)
What it says is that people deploying AI systems need to think very careful about how they did the training.
The problem we have right now is AI is a magic box that will solve all my problems, is how users are starting to think. ML is in this way exactly like every expert and automation system that has come before it. Its limited by YOUR imagination. We have gone through this with computer security from day-0 this new wizbang thing will completely solve this - no it won't not if someone else has an interes
Re: (Score:2)
I've been beating that drum for years, but it looks like the creationists are winning this one. We need more stories like this to inject a little reality.
Re: (Score:2)
Yes, exactly. I think this is the Achilles heel of AI systems for now. Fundamentally they only seem to pattern match and parrot back what we tell them. There's no real insight, thinking, or generalizations. AI systems really don't seem to have any concept of "that's a human, even though they're behaving very differently from any human I've seen before. Any six month old infant has more insight than that. So does our family pet rabbit.
It really seems we're missing very fundamental aspects of what makes somet
Re: (Score:2)
Well the claim intelligence isn't ONLY pastern matching is likely true given ONLY is pretty limiting. I am not sure as confident we can conclude that its not MOSTLY about pattern matching.
Another component of this is that our digital analogues (see what did there) for things like vision really aren't. Computer vision isn't a copy of what your brain and eyes do. It might achieve some of the same ends but the data it produces. I am not a neural scientist but I bet my brain isnt getting a serialized set of has
Re: (Score:3)
Better system: train it on what is "normal," and flag deviations for human review. You might have to explicitly flag frozen video, frame-to-frame discontinuity, or looped video as aberrant. But other than that, this seems better for using machines for what they're good for and humans for the interesting parts.
Anybody else feeling like playing Metal Gear ? (Score:1)
ChatGPT, meh (Score:2)
People are freaking about ChatGpt? Put some jarheads in front of it and watch the magic.
OK, so ChatGPT is now able to use "fuck" as noun, verb, adjective, adverb, and exclamative in the same short sentence. Meh. The real magic will happen when those art generating AI programs receive training from Marines. :-)
Re: (Score:2)
OK, so ChatGPT is now able to use "fuck" as noun, verb, adjective, adverb, and exclamative in the same short sentence. Meh. T
There's merely first grade level English around here...
Re: (Score:2)
Gundam Style (Score:2)
We laughed at him, but he's on to something. [knowyourmeme.com]
The Manual: Monty Python's How Not to Be Seen (Score:4, Insightful)
It's like a manual at this point...
https://www.youtube.com/watch?... [youtube.com]
Re: (Score:2)
I was looking to see if anyone referenced Monty :-)
Re: (Score:2)
All these young'ins and their Metal Gear...
I will admit I'm rarely seen, but I do buy lots of refrigerators which have... large packaging.
Fun Times Ahead (Score:2)
What a waste of taxpayer dollars (Score:2)
How insane, that we waste billions of dollars developing AI to detect humans walking, and the product gets engineered without infrared camera input built into its AI algorithm? While I'm not sure it would have worked for the somersaulter, the IR input would have easily studied and identified the movements for the marines hidden in the cardboard box and behind the fir tree.
On the other hand, at least now we know how to defeat Skynet.
Re: (Score:2)
Cardboard like in most boxes is basically a wavy sheet of kraft paper glued between two flat pieces of kraft paper. With air gaps in the middle.
It's surprisingly insulating. As such, an IR sensor wouldn't be able to see through it.
Re: (Score:2)
It's surprisingly insulating.
It's a super material whose kryptonite is rain.
Tactics learned in VR (Score:2)
We know that the special ops guys practice tactics using VR similators.
And I've done this myself in Second Life! Just rez a cube, momentarily set it Transparent, and step into it!
Re: (Score:2)
We know that the special ops guys practice tactics using VR similators.
I've been practicing for years. When we finally get invaded by dragons, I'll be ready!
(dohvahkiin, dovahkiin, not a single sardine)
That guy running in a box (Score:2)
Looks like something out of Minecraft.
Re: (Score:2)
Snake??! (Score:2)
SNAKE!?
SNAAAAAKE!!!
Re: (Score:2)
Damn it, beaten!
Thinking outside the box.. (Score:1)
Marines obviously play TF2 (Score:2)
https://www.youtube.com/watch?... [youtube.com]
Semper Fi! (Score:2)
As a former sailor, allow me to offer the following compliment:
NEVER underestimate the determination and skill of a squad of US Marines offered a challenge!
It never ceases to amaze me that the Marine Corps can take a quasi-random pool of 18 year olds from all across the nation and turn them into skilled, mature, adults who can work together in any situation. Well done.
Oh, and the AI guys might want to consider hiring some retired marines...