Improve the flexibility and speed of software testing by eliminating dependence on in-house infrastructure
This European government agency is responsible for running robust IT systems that manage the efficient collection of more than EUR 100 billion in tax each year.
To help taxpayers understand their obligations, this government agency creates online interactive tools for the public, as well as internal case-handling systems.
As part of a major modernization program spanning both public-facing and internal tools, the agency needs to test systems on an ongoing basis for performance, stability and scalability, both during development and subsequent maintenance.
An agency spokesperson says: “We wanted to perform continuous large-scale testing for simple use cases with large numbers of users as well as highly complex use cases. To avoid managing internal infrastructure, we looked for a cloud-based solution offering short lead times, known costs, significant automation, repeatability, and easy scalability for high-volume testing.”
For the public-facing application, the agency aims to scale to 50,000 concurrent users and response times of less than 3 seconds for at least 99% of users. For the internal application, the target for concurrent users is much smaller – in the hundreds – but here the functionality is significantly more complex, and data loads are in the gigabits per second range. Underpinning these applications are dozens of subsystems and multiple interfaces, all of which must be capable of meeting the targeted performance, stability and scalability.
Miracle42 – SPOKESPERSON
To enable high-speed delivery within this complex software landscape, the agency is adopting a “shift-left” approach, whereby testing conducted earlier in the cycle and on a more frequent basis. To facilitate this transformation, the agency engaged miracle42, an IT services company, to deploy Micro Focus LoadRunner Cloud, a 100% cloud-based load testing solution that can support scaling to more than 5 million virtual-user tests.
A miracle42 spokesperson comments: “We were already working with the agency and had built a strong reputation for being able to manage multiple system environments. We recommended Micro Focus LoadRunner Cloud as it provides the capability to script complex scenarios relatively easily, then programmatically parametrize them to cover other scenarios. LoadRunner cloud also makes it easy to scale up performance testing.”
The parametrization enables testing against known SLAs and against acceptable limits for failures in transaction processing, adding significant sophistication to automated testing. Choosing the cloud rather than an on-premises deployment improved cost-efficiency and gave the agency greater flexibility to grow the solution, since it eliminated any need to deploy and maintain a large server farm for testing.
The agency spokesperson says: “To emulate user behavior, we employ VuGen (Virtual User Generator) in LoadRunner to create test scripts using the DevWeb protocol. Working with miracle42, we recorded various testing scenarios, abstracted them, and added parametrized data so we that can run multiple permutations. In this way, we can quite accurately simulate the actual workload in all its complexity.”
The spokesperson adds: “We execute the scripts on LoadRunner Cloud, which is a tremendously useful tool. It’s a huge benefit not having to orchestrate in-house infrastructure for performance testing.”
Agency – SPOKESPERSON
Within the continuous integration/continuous deployment (CI/CD) chain the agency is building, the CI server triggers the execution of automated test scenarios in LoadRunner Cloud. Broadly, these scenarios have three purposes. First, to validate integrations between subsystems, so that developers can be confident that the interfaces will emit and accept/reject parametrized data in the expected way. Second, to confirm that the interfaces can handle the expected volume and velocity of incoming data – potentially including garbage data that needs to be rejected – without being overloaded. And third, to validate elaborate test cases that put stress on multiple subsystems to simulate the distributed and complex load of multiple case workers working in parallel.
“To ensure that we can meet our SLAs, it’s vital that the interfaces between application components are working correctly,” says the agency spokesperson. “This means that we need to test all of the workflows that are supported by the case-handling domain, enriched with data from the actual system. That’s the capability we’re building in LoadRunner Cloud.”
Miracle42 is helping the agency to create a performance baseline for all the individual components of all subsystems. This makes it easier to identify bottlenecks in performance and find probable root causes once the components are working together to support a complex workflow.
“Automation is a significant factor in the ease of performance tests in LoadRunner Cloud,” says the miracle42 spokesperson. “Within a couple of minutes of receiving a new request, we can have a test scenario running. We can script scenarios and generate data, then schedule the tests to run on demand when triggered by a CI server such as Azure DevOps or Jenkins. This automation helps dramatically reduce the cycle time between development and feedback, which will help us achieve a higher rate of releases with better quality.”
Once this multi-year digital transformation project goes fully into production, the agency will move into an operational mode for continuous performance testing. miracle42 is working to transfer skills to the agency team so that they can be fully autonomous in adapting the testing regime to changes in the system. Testing will also be an important ongoing element in monitoring performance against SLAs, in particular as the agency will gradually increase the number of concurrent users from being in the hundreds to being in the thousands.
“The system we are building represents a great leap forward in our capabilities, and will support a significant improvement in the quality of case handling,” says the agency spokesperson. “These enhanced capabilities bring with them a major increase in complexity, which in turn puts greater pressure on testing. With LoadRunner Cloud, we are confident that we can meet the challenge with an efficient, highly automated approach.”
