University Cooperations – Siemens Corporate Research

Publications        Pictures of the Future Fall 2003

To keep an eye on what's happening throughout this high-stakes area, SCR maintains close ties with top universities in the U.S. and Europe. And, like the smart vision systems the department is developing, research projects have become increasingly focused. "In the mid-'90s, the research we conducted in collaboration with universities was purely academic," explains Dr. Visvanathan Ramesh, who heads the Real-Time Vision Department. "Today, the model is very different. We work closely with Siemens' operating companies to find out what their customers would like to have. Then we target top universities and develop highly focused research projects to bolster our own work in that direction. We are definitely getting more bang for out bucks than ever before."

Face Zooming. One of the many combined SCR-university research projects that have paid valuable dividends involves video monitoring with multiple cameras to detect people, localize them and zoom in on their faces. Part of the project was conducted by Dr. Michael Greiffenhagen, a Ph.D. student from the University of Erlangen, who was with SCR for four years. "My work was a perfect model of communication," recalls Greiffenhagen, who now works for Siemens Information and Communication Networks Group. "My professor in Germany knew what I was doing at SCR and felt that it was high quality work; but Dr. Ramesh was my local supervisor. For me, it was the right mix of basic science and practical applications."

The work focused on analyzing the stability of algorithms used in detecting people and acquiring images of their faces by means of intelligent zooming under varying lighting conditions. Greiffenhagen performed a unique, systematic design and analysis of the performance limits of the system. In addition to estimating where people are in an area at any given time, his system is also able to tell how accurate its person location function is so that a second camera can adaptively zoom in on the person's face. "The project gave us several insights into design of practical video monitoring systems, thus influencing our commercial products in the related field of traffic monitoring," says Ramesh.

Seeing the Big Picture. Yanghai Tsin is a graduating Ph.D. student at Carnegie Mellon University's (CMU) School of Robotics in Pittsburgh, Pennsylvania. Since 1999, he has worked on projects with Dr. Ramesh. But all the projects have one thing in common: they have to do with building statistical models that take into account the physics of the image formation process to build physics-based video surveillance systems.

As anyone who has taken pictures of the same scene with different apertures knows, the aperture determines the level of detail. Shoot "wide open" and you'll get washed out bright areas, but rich detail in unlit areas of the image. Shut the exposure down, and you'll get no detail in dark areas, but plenty where the scene is bright. But if a camera could take a vast range of exposures for every picture and then seamlessly piece the best exposures together into a single image, it would be technically possible to have unlimited corner-to-corner detail. That's the general idea behind Tsin's concept of a "high dynamic range image." Guided by Prof. Takeo Kanade at Carnegie Mellon and Ramesh, who is on Tsin's Ph.D. committee, the project combines the best of academia with far-signed industrial goals.

While the concept of "high dynamic range images" is not new, Tsin's work has concentrated on developing a complete statistical model of the camera and using that model to accurately estimate the high-dynamic range image along with its uncertainty. Tsin uses this technique to monitor a parking lot and determine the differences between two pictures due to physical or illumination changes. As part of his work, Tsin also developed a system that stitches together several images taken with a pan-tilt camera to give a high-resolution overview of a large area. The work holds the potential of developing surveillance technology to monitor wide open spaces that would never be blinded by reflections, and would never miss events, regardless of the level of contrast and illumination variations in a scene.

"The work that is being conducted by students such as Greiffenhagen and Tsin gives us an understanding of what the state of the art is," says Ramesh. "And because it is a joint effort, each party – the student, the university, SCR and the operating company – comes out of the process with a better handle on the problem's potential solutions."

Ramesh points out that SCR now earmarks approximately three to four percent of its funds for university cooperation. Projects typically involve one or two professors at a target university and one or two students per project. To date, projects have been successfully executed at the University of Rochester in Rochester, New York, where a student worked on 3D image reconstruction from video; Lehigh University in Bethlehem, Pennsylvania, where two students analyzed how to quantify the performance of video analysis systems; and Princeton University in Princeton, New Jersey, where the emphasis was on pattern representations.

Meanwhile, other projects are underway at Columbia University in Manhattan, Brown University in Providence, Rhode Island, the University of Maryland, and Michigan State University. "Is our method good for Siemens?" asks Ramesh. "Absolutely, because it puts us in the mainstream for a very modest price. Is it good for the students? Absolutely, because it gives them the big picture and allows them to solve real world problems. The poof of the pudding is that several students we have funded have decided to join SCR."