Security – Scenario 2015

Catching a thief in a crowded shopping mall can be devilishly difficult. But smart cameras that translate image content into words will make it possible to search vast video databases for a particular person. Here's one possible scenario.

Never a dull moment. That's what I like about this job. None of that old-fashioned security stuff. You know—with the banks of monitors and the sleepy-looking guys peering at pictures of delivery bays or rear entrances over cups of steaming black tea. Things have changed. Now we're on the move. But there's more. Here in the SkyMall in Singapore—a giant commercial complex that opened a couple of years ago—they've got security systems like you wouldn't believe. Sure, everyone's got cameras. But these guys are different. They're smart. They watch for certain kinds of situations: a package left in an aisle, someone walking from car to car in the underground parking lots, faces, license plates, even certain tones of voice. Any of these things can trigger what we call "an event." Most events are insignificant, but they're routinely routed to the nearest security person. After all, how's a camera supposed to know that a slap on the back is harmless?

The thing is that just about everyone these days has biometric templates—coded but incomplete versions of their voice, fingertip and face scans—in government databases and on their PDAs and smart cards. It's convenient and private since using the system is just a question of verification—not identification. You can access and update your government benefits from home or from kiosks in places like SkyMall. When customers enter a parking lot or leave a store with some merchandise, all they have to do is touch a reader with a finger. The reader creates a digital template of the fingerprint's geometry and compares it wirelessly to the information on the customer's PDA. If the templates match, the transaction takes place.

Getting personalized information is also a breeze. Just take a look at that lady over there at the kiosk. She's probably checking to see if any of her favorite products are on sale. When she places her hand on the scanner, the information is downloaded to her PDA—complete with aisle and shelf numbers.

Still, crime is hard to stamp out. A couple of days ago, for instance, a Japanese tourist's shopping bag disappeared from the SkyBar and the only clue was that the man could vaguely remember that a middle-aged woman with a beige jacket had been nearby. Not much to go on. Most people around here wear beige all the time. But he also remembered that she had worn a pair of pink sneakers. Bingo.

Only a few minutes after the theft had been reported, I was drawing a triangle and a square—visual shorthand for a person with a bag or briefcase—in my PDA's data entry window. I told the system to search the databases of the cameras around the SkyBar for the last hour. But if I had left it at that, the system would have come up with hundreds of images of people with briefcases or shopping bags. Adding "beige top" to the search list wouldn't have narrowed the results by much. But "pink lower section" was a winner. A few seconds after I hit the search button, images started popping up on the PDA. "Do any of these look like the person you remember?" I asked. The man—Mr. Tanagucci—looked carefully. There had been several women and even a couple of men who had worn pink shoes. But there was one woman who looked familiar to him.

I asked the system to search for all images matching her description for the last hour. Bingo again. An image taken only 40 minutes earlier showed her in a shoe store without a shopping bag. But the latest images showed her with two shopping bags, and Tanagucci said that it appeared that one of the bags was the right size and color. She was definitely our girl—pink sneakers and all.

Scrolling forwards in her video archive, I could see that the woman had gone for the elevators and was already in the parking lots, which, by the way, are outfitted with plenty of cameras. Now the pictures coming in were real-time. I saw her in a car. The vehicle's electronic license plate ID—automatically transmitted to the nearest camera each time a car's ignition is started—appeared on my screen along with exactly the notice I had expected to see: "Driver and vehicle do not match," it said. Now things were getting serious. But clearly, SkyMall's investment in high-resolution cameras with face recognition capability was paying off. Using the fingerprint reader on my PDA to confirm my command, I put one of our newest security systems into effect: I requested cameras at all exit gates to search for the suspect's car and to deny release of exit barriers if it tried to leave. Five minutes later Tanagucci and I found the stolen vehicle. Two security men were already on the scene. "Where is she"? I asked, looking around. The two looked annoyed. I checked my PDA. No more images. Then I looked in the car and knew why the trail had suddenly gone cold. There they were on the front seat: a pair of bright pink sneakers.

Arthur F. Pease

Biometrics: A Question of Identity

Biometric Technology: Body Language

Smart Cameras; Getting the Picture

Netwoks: Viruses, Worms & Hackers

Interview with Marc Rotenberg