Pictures of the Future    Fall 2005

Sensors See the Light

Advanced sensing systems enhance safety. A night-vision system can detect pedestrians (large image) better and sooner than full-beam headlights. Sensors for blind spots (bottom right), for automatic parking assistance (center), and for trafficsign recognition—and combinations of these (left)—provide drivers with crucial safety-related information

It’s a nightmare scenario for any driver. There you are, peacefully driving along a dark country road, and all of a sudden a pedestrian or a deer appears in the headlights, right in the middle of your path. With adrenaline pumping, you brake and swerve to avoid contact.

In today’s vehicles, systems such as ABS and ESP already help drivers deal with such situations. Moreover, with the advent of new and more powerful assistance systems, such situations might be prevented altogether. A night-vision system, for instance, can recognize an obstacle on the road long before it appears in the headlights, and warn the driver accordingly. Recently, in the context of the 2005 International Motor Show in Frankfurt, Siemens VDO Automotive presented a car featuring a total of ten ground-breaking assistance systems.

"Even in ten or twenty years, assistance systems will still be there to support drivers—but not to replace them. The driver will always have the final say," says Dirk Zittlau, head of Driving Assistance Systems at Siemens VDO Automotive in Regensburg. Zittlau, an electrical engineer, also rejects the fear that these technologies could somehow take control of vehicles. "The driver will always be able to overrule such systems. Besides, it will probably be possible to deactivate some of them as well, if drivers feel distracted."

Cutting Fatalities by 50 %. Behind the push for advanced driver assistance systems is the stated goal of the European Union to halve the number of road fatalities by 2010, based on the 2001 figure of around 40,000. Experts agree that this is barely feasible without the use of electronic assistance systems. Indeed, experience with the Electronic Stability Program (ESP) already confirms this. According to a study by the U.S. National Highway Traffic Safety Administration (NHTSA) covering the period between 1997 and 2002, the number of single-vehicle accidents—i.e. accidents that don’t involve another road user—and fatalities involving large SUVs equipped with ESP have both fallen by two thirds. Similarly, the number of single-vehicle accidents involving passenger cars with ESP has fallen on average by 35 % in the U.S.

What Consumers Want. Demand for driver assistance systems is also coming from consumers themselves, primarily because the average age continues to edge upward (see Pictures of the Future, Spring 2005,  The Graying Society: When I´m 64). "In the future, more and more older people will want to keep driving, despite increasing physical impairments and slowing reactions. Here, our lane-change assistant—to name just one example—can help to compensate for a driver’s limited mobility and restricted vision," says Zittlau. Using radar sensors to monitor traffic to the right and left of a car, this system warns the driver of any objects that pose a danger should he or she wish to change lanes or execute a right or left turn. In addition, before the doors are opened to leave the vehicle, the same sensors also warn of approaching cyclists, skaters or cars concealed in the mirror’s blind spot.

It’s stop-and-go time on the highway again, and all that braking and accelerating is really stressing drivers out. To help prevent rear-end collisions in congested traffic, Siemens is developing a system that can maintain a constant distance from the vehicle in front, automatically braking and accelerating the car as required. What’s more, the system can even do this when driving around corners, as demonstrated by a test vehicle at the end of April 2005 in the "Invent" project sponsored by the German Ministry of Education and Research to promote user-friendly technology for intelligent transportation. The car is fitted with a video camera and an infrared laser (lidar), mounted on the windshield, plus radar sensors fitted behind the front fender. "The various sensor systems and the areas they scan all complement one another," explains Dr. Georg von Wichert from the Competence Center for Autonomous Systems at Siemens Corporate Technology. As Wichert explains, this mix of different sensors increases the power and reliability of the system as a whole: "After all, there’ll always be times when individual sensors are impaired by ambient conditions." The image-processing system hooked up to the video camera is able to calculate—on the basis of their distance ahead—whether vehicles in front are accelerating or decelerating. In addition, the system also registers road markings—which serves to determine, among other things, lane width and curvature. Using this information, a motor fitted to the steering column of the test vehicle is able to hold it automatically in the middle of the lane and even follow the car in front as it drives around a corner. In combination with a lidar sensor, a radar system measures the distance to, and the speed of, vehicles in front. Unlike today’s systems such as automatic cruise control (ACC), which work best at high speeds and are unable to steer the vehicle, this new system can control the car fully automatically at low speeds of between zero and 50 km/h. This is made possible by special software that checks the data from all three sensors—video, radar and lidar—millisecond intervals in order to verify that their readings do not contradict a model of the vehicle environment. The latter features descriptions of probable vehicle movements, as based on the laws of nature. If, for example, the sensors have registered a vehicle in front that is driving at a certain speed, the model tells the system that the vehicle will probably continue to travel at that speed. Because the system knows its own speed, it can calculate where the vehicle in front should be by the time of the next reading, which in turn is compared with the new sensor data. In this way, the system can determine the precise position of the vehicle in front and therefore ensure that it remains at a safe distance. Of course, the driver can also overrule the system at any time and independently accelerate or brake the vehicle.
                                                                                                               Rolf Sterbak

The Lane Departure Warning (LDW) system functions according to the same principle. Here, a video camera monitors lane markings. If the software detects that the vehicle is in danger of crossing the white line, it produces a clearly discernible vibration in the automatisteering wheel so as to warn the driver. In the future, the LDW could also be transformed into a camera-controlled lane-guidance system, which would automatically correct the steering whenever the vehicle threatens to stray off course.

Virtual road signs. In the future, important traffic information could be projected on the windshield

In another highly useful development, Siemens VDO has come up with software that can read speed limits on traffic signs. If the driver violates a speed limit, the system displays a visual warning signal. "But it won’t automatically reduce your speed," Lutz emphasizes. "Furthermore, the driver can always deactivate the system," he adds. In addition, there are now plans to extend the system to cover stop and yield signs. Meanwhile, another application from Siemens VDO—adjustable cruise control (ACC) featuring variable speeds—utilizes a new light-based detection technology to measure distance. This so-called lidar sensor is comparable to conventional radar sensors, but is substantially cheaper. Moreover, unlike radar sensors, which have to be fitted behind the radiator grille or the fender in order to protect them from the elements, lidars can simply be installed behind the windshield.

Researchers at Siemens Corporate Technology in Munich are busy looking at how machines can develop a sense of direction in unfamiliar surroundings. For some time now, Michael Fiegert and his team from the Competence Center for Intelligent Autonomous Systems have been working on self-propelled robots for applications such as transport systems in hospitals and cleaning equipment in supermarkets and warehouses. "The same principles can be applied to other moving objects that utilize visual-recognition systems. In the future, this might include cars equipped with cameras," says Fiegert. At present, Fiegert’s team is using laser scanners to generate 3D images of a robot’s immediate surroundings. In the future, however, these will be replaced by cheaper video cameras. The principle is the same. Images of certain places are taken from different perspectives and at varying resolutions. On this basis, characteristic features are determined and then saved in the form of a code. The same characteristic features are then coded from another angle in order to calculate their 3D positions in the space under consideration. Using this method, a mobile machine is able to work out its position in relation to its environment. Whenever it returns to a given location, the machine recognizes it on the basis of the characteristic features saved in its memory. Siemens has recently installed this technology in a driverless forklift designed to independently transport pallets around a warehouse. The project is already so advanced that Fiegert is confident that the system will be used in practice in the near future.
                                                                                                                     Rolf Sterbak

Synergistic Networking. Combining various systems—a major Siemens VDO strength—brings extra benefits. If a car’s navigation system recommends leaving the highway, LDW will then monitor whether this maneuver can be carried out safely. And should the driver be unable to change lanes in time, the navigation system will have already calculated an alternative route before the turn-off point, so as to spare the driver needless stress. In another example, a so-called electronic horizon prevents a similar scenario. As your car turns off the highway on to the exit road, it’s conceivable that the adjustable cruise-control system could lose the vehicle ahead and then accelerate to the preset speed. However, using digital maps and a GPS device, the system recognizes that your car is on the exit road and adjusts the cruise control accordingly. Networking also brings benefits in other areas, not least when different applications utilize the same sensors. For example, the small CMOS camera is utilized by the system to read road signs, the LDW system, the stop-and-go assistant and—albeit with a different lens—the night-vision systems.

Navigation systems featuring augmented reality could soon be providing drivers with enhanced assistance. With this system, the driver’s actual view of the road ahead is supplemented by virtual, computer-generated information. "We’re currently working on a prototype that may well develop into a product in a few years," says Dieter Kolb, a software engineer at Siemens Corporate Technology. Today’s systems present route information as a pixilated graphic or a map representation on the display unit, accompanied by voice output. In the future, however, the route could be displayed as a colored strip superimposed on images of the road ahead, as captured by an onboard camera. The driver would then see the route as if it were literally painted on the road. And because the display would show the same view of the road ahead that the driver sees through the windshield, motorists would have no trouble orienting themselves. Moreover, the "pathfinder" provides clear routing information at night or when the driver’s view of the road ahead is obscured by other vehicles. For example, the system shows the driver the course of the road after a hill or a tunnel and therefore gives advance warning of any sharp bends or other potential trouble spots. In the next project stage, route information will be projected directly onto the windshield—although only when required, so as to guarantee a free view of the road at all other times. If navigation assistance is needed, the driver simply presses a button on the steering wheel in order to project routing information onto the head-up display on the windshield. A major advantage of augmented reality is that the driver always keeps an eye on the road ahead even when consulting navigation data. In other words, this solution will not only improve the ergonomic characteristics of today’s navigation systems but also increase road safety.
                                                                                                               Rolf Sterbak

Siemens VDO has therefore produced a comprehensive concept to ensure maximum user-friendliness and minimum strain on drivers. This includes siting display devices in locations where drivers intuitively expect to find them—a TFT display in the instrument cluster, for example, or a larger screen in the center console and warning lights where they can best be seen. The centerpiece here is the fully configurable head-up display, which projects driving-related information straight onto the windshield, directly in the driver’s field of vision. Siemens VDO is currently the only company to supply such a system, which has featured in the BMW 5 series since 2003.