The use of Silk products to test the Agency’s applications is regarded as a landmark among projects within the Bundesagentur. Since its roll-out in 2004, COLIBRI (the unemployment benefit application) has had no significant errors. The key to this success, according to Dr. Brandes, is the ability to have fully automated regression testing.
As the labor market’s biggest service provider, the IT department of the Federal Employment Agency must ensure that the wide range of vital applications and software products used to support Germany’s workforce and unemployed people meet stringent quality standards. To help face this major responsibility, Doctor Christian Brandes, Software Test Architect within the Bundesagentur für Arbeit, has relied on the capability of Silk for many years. Dr. Brandes has worked with Silk since 2001 to help in his role of defining and establishing testing standards for all IT projects across the agency, which at present number more than 100.
The IT systems of Germany’s Federal Employment Agency face several substantial challenges. There are a number of large, very important IT applications, and a complex infrastructure of 170,000 connected PCs and over 11,500 servers. The systems are used to support highly visible activities, such as the correct calculation and payment of over 14 million benefit payments per month. The financial risks of getting calculations wrong would be counted in the millions of Euros for even a €1 miscalculation per payment, and the risks to public confidence and the reputation of the agency would also be catastrophic.
For these reasons avoiding calculation errors is mission critical for the Agency. “We have a large number of very important projects that affect the lives of millions of people,” Dr. Brandes explains. “There are several high-profile applications which have been tested using Silk Test™ and Silk Performer™. For example, COLIBRI, which looks after the calculation and payment of unemployment benefits; KIWI, which manages ‘Kindergeld’ or child benefit payments; and we also have to ensure the quality of the central customer database, zPDV, as well as many more.”
With such a large number of highly visible applications that handle vast sums of money and affect the lives of thousands of German people, the Agency needed a reliable and robust testing tool to ensure that its applications met the high standards demanded. The Silk story started back in 2001 for Dr. Brandes and has continued to evolve and develop over the years, with Silk Test, Silk Performer and – later – Silk Central™ Test Manager making up the Agency’s comprehensive testing portfolio.
The use of Silk products to test the Agency’s applications is regarded as a landmark among projects within the Bundesagentur. Since its roll-out in 2004, COLIBRI (the unemployment benefit application) has had no significant errors. The key to this success, according to Dr. Brandes, is the ability to have fully automated regression testing. He says, “If any changes have been made to the system, we are able to run 30-40,000 tests overnight to test that the changes have worked, and that none of the functionality that was previously working has been affected.” Maintaining performance standards demands that tests are carried out before each release, including hot-fixes, and automated testing was a priority for the Agency. Using Silk it is possible to run distributed tests across ten parallel machines each needing different test environments and performing tests independently of each other. Each machine runs around 3-4,000 tests. It is, as Dr. Brandes says, “a complex configuration, but our investment has paid off.”
Despite the large scale of testing required, and the critical importance of the applications, today’s performance testing team of six people is able to address the significant demands placed on it. It executes in the region of 30-40 tests per year and each test takes around 30 days in total – from preparation through execution to reporting.
Silk Central Test Manager was brought on board when the Agency realized that it lacked a professional integrated tool to provide a framework for improving productivity, traceability and visibility across all types of software testing. Together Silk Central Test Manager, Silk Test and Silk Performer were key elements of the ‘test factory’ that the Agency established in 2006 – a dedicated professional testing team for IT projects.
While Silk Central was identified as a key success factor for coordinating functional and business tests, it also added an extra dimension by enabling keyword-driven testing. As some test cases were proving difficult to automate, Dr. Brandes made the decision to use keyword testing for both manual and automated testing. As he explains, “Our test specifications are now very tightly written using a formal description of test steps. We developed a keyword editing tool and process to insert keywords from a dropdown menu rather than allowing ‘normal’ German.” Test cases are defined using this notation, which is a sub-set of the German that testers would ordinarily use. Combining Silk Central with the keyword method (both editor and processes) has automated the generation of Silk Test scripts while still being able, at the same time, to perform these tests manually. Another advantage is that the quality of test specifications has increased significantly compared to the specifications previously written in ‘normal’ German. The solution has been widely accepted and the keyword automation framework and notation developed by Dr. Brandes and the test factory has started to roll out across other IT projects.
Using Silk products, such as Silk Test and Silk Performer, has ensured that the Agency’s systems perform accurately and predictably and deliver high levels of customer satisfaction. As Dr. Brandes says, “The COLIBRI application has 50,000 users across Germany, and customer surveys universally return high levels of customer satisfaction in terms of functionality, reliability and dependability. Our use of Silk Test and Silk Performer makes a strong contribution to this result and I am very happy with our decision to use such mature and powerful tools.
Naturally, there will be new challenges, especially as Germany moves to e-government, giving citizens access to government services from home PCs and as migration to SOA becomes a focus for the Agency’s work. Silk products from Micro Focus will continue to play an important role in helping the Bundesagentur für Arbeit achieve its very high levels of software quality now and into the future.
