Software - Quality

German comedy star Dieter Nuhr is having a hard time. With women. With his mother. But most of all with computers. Just the other day, he says, he experienced an especially interesting computer crash. "Unexpected fault," was the message on the display. "It’s reassuring to know," sighs Nuhr, "that all the other faults met expectations."

The audience loves it. It’s nice to know that you’re not the only one having problems. Everyone knows about the tribulations common in office software, and sometimes a sense of humor makes them seem more bearable. It would be a lot less funny, however, if the software running cars, trains, factories, medical equipment and power plants experienced similar faults to those all of us are familiar with when on our PCs.

As software assumes ever more functions that used to be performed by hardware, fears of such failures are increasing. In cell phones, for instance, software accounts for 70 % of the devices’ value. Lines of code have quintupled between the launches of Siemens’ S25 and S65 cell phones. And that adds up to more code than controlled NASA’s rockets of the early 1960s. "Software is becoming ever more complex," concedes Dr. Frances Paulisch who directs the Software Initiative, a program established at Siemens in 1996. But complexity doesn’t necessarily tell you anything about error rates. Indeed, eliminating errors from program code is not the principal purpose of the Inititive. Instead, the Initiative is designed to support the 30,000 software developers at Siemens by establishing optimized processes.

Scrutinizing Software Quality. With this in mind, the Software & Engineering/Processes department at Siemens Corporate Technology (CT)—an internationally renowned assessment center for software processes—uses a catalog of 250 standardized questions to interview key project development people. The questions are designed to sort out the strengths and weaknesses of an organization’s software development processes. Questions cover topics such as the competencies project manager must have, and the value of quality assurance measures. Based on the responses, a maturity profile is formulated with a maturity level ranging from 1 to 5. The international Capability Maturity Model Integration (CMMI) is used as a yardstick. In the CMMI, a maturity level of 1 is assigned to disorganized processes, while a 5 is awarded to software whose development is continually improved, based on specified metrics. By 2005, the Software Initiative aims to achieve a maturity level of 3 for R&D projects at Siemens, which corresponds to international targets. In several groups, this goal has already been reached or even surpassed.

Following evaluation, project participants are provided with feedback and, if necessary, with training. Such training includes review techniques in which a software engineer must present programming results to team members. While there may be some grumbling at the beginning, participants invariably come to appreciate their colleagues’ advice. "It’s 20 % technology and 80 % psychology," confides department head Ludger Meyer. His 36 employees conduct worldwide assessments for all software units throughout Siemens.

A high CMMI level, however, is no guarantee that software will be flawless. Still, there’s a measurable correlation between CMMI levels on the one hand and software quality and costs on the other. "When the processes are OK, it’s easier to estimate the costs and the time required," notes Paulisch. To reach level 4 or higher, which is advisable in safety-critical applications, one must increase the use of metrics. These are measuring methods that define the quality of the program code—a parameter of error density. Another level 4 requirement is the modularity and reusability capability of the software, which reduces both cost and complexity.

Modules Assure Quality. One Group that has embraced software modularity is Siemens Medical Solutions. Instead of developing new software for each product, developers at Med now rely on the syngo software platform for CT, MRI and other medical imaging systems. This "platform strategy" reduces the chances of errors. Engineers at Med rely on a proven development methodology that moves from defining specifications, to design, implementation and testing in successive but overlapping stages designed to minimize development time.

Still more advanced, though not yet widely used, are incremental methods, in which the entire development process is subdivided into a succession of mini-waterfalls with predefined interfaces. Gustav Pomberger, Professor of Software Engineering at the University of Linz, Austria, is exploring such processes in collaboration with Siemens. Pomberger is an advocate of prototyping. In this strategy, developers create testable prototypes of their software at an early stage. The prototypes are designed to give the client a foretaste of a program’s ultimate suitability for an application. If changes are needed the prototype is modified and perfected in feedback loops. Pomberger has refined the concept of prototyping for certain decision processes in a spiral process model (see diagram above). In a pattern resembling a snail shell, software engineers work in successive loops from the center (awarding of the project) outward through prototypes, evaluations, concepts and tests.

Yet another approach to software development can be found at Regensburg, Germany-based automotive supplier Siemens VDO. There, Stefan Hohrein, who heads VDO’s System and Software Initiative, is committed to close coordination of software development with systems engineering, hardware and mechanical development. This approach can serve as a model throughout Siemens, since most of the company’s software is created for embedded systems, in which software is very closely linked with hardware, as is the case in cell phones and automotive infotainment systems. In automobiles, hardware is increasingly being replaced by software, because it is more flexible. According to a study by the Mercer Group, software will account for 13 % of the value of a typical car by 2010, compared to 4 % in 2000.

Systems and software development at Siemens VDO follow well-defined processes, but testing is determined by customer requirements. When a program is complete, individual program sections are first tested separately. The next phase is an integration test, which ensures that individual functions interact smoothly. The final hurdle is a system test on the complete device under realistic environmental conditions. To keep complexity to a minimum, automakers are developing a type of operating system with uniform interfaces that allows proven program modules to be ‘docked’ and reused (see  Standardization). The intent is to reduce the number of control devices in premium cars from the present 70 to about 20.

Programs that Test Programs. Automated tests during an early phase of development are gaining in importance. Basic syntax errors in code are detected upfront by compilers, which translate a program into machine language. New test methods are designed to ensure smooth interaction between all program sections. To meet this objective, the Fraunhofer Institute for Computer Architecture and Software Engineering in Berlin and its sister institute for Experimental Software Engineering in Kaiserslautern, have developed Quasar, a software tool that tests functions in automobiles before components are produced.

Quasar’s first test object was a car door with integrated pushbuttons for seat adjustments. To begin with, Quasar was used to represent the manufacturer’s—DaimlerChrysler—requirements in lucid diagrams. For example, seat adjustments had to be limited to very low driving speeds, which required a link with the speedometer. As the next step, several hundred combinations of functions were then simulated and tested for consistency. Only then was the seat adjustment software engineered. Quasar also comes in handy in later stages. It simulates sensors and automatically activates all microcontroller functions. Safety-critical functions such as the electronic steering systems of the future can’t even be developed without such test tools. "Manufacturers will need to prove that steer-by-wire is just as reliable as mechanical steering," says Prof. Holger Schlingloff, Quasar project manager.

Code Inspector. Across the Atlantic, at Siemens Corporate Research in Princeton, New Jersey software engineers are developing "code inspector," an analytical tool that sniffs out bugs in the source code of the most important programming languages, including C (20 % share at Siemens), C⁺⁺ (30 %) and Java (12 %). This sniffer has already proven its value in several sectors and saved time and money by providing early fault indications.

Code inspector also provides specific key figures that customers request. The German Federal Railroad Administration, for example, uses a catalog of quality criteria for software written in C⁺⁺. Code inspector enabled Siemens Transportation Systems to generate the documentation at half the expected cost.

The next logical development would be to endow code inspector with the ability to automatically correct errors in code. "That’s already feasible for some quality criteria," asserts Jean Hartmann, who helped create code inspector in Princeton. "But as a former software developer I wouldn’t appreciate having a machine messing around with my code."

Bernd Müller