Long-Term Testing Excellence

TMNAS aims to streamline processes in order to enhance services and deliver cost savings for the companies it serves. Developing new applications to serve customers and employees more efficiently is a key part of this strategy, and the company’s development teams strive to deliver high quality software within short timeframes.

TMNAS had built up a powerful set of on-premises software testing systems to ensure software quality, but the company found that ever-increasing demand for performance testing was putting these systems under pressure. Expanding from its original deployment of Micro Focus LoadRunner at its Pennsylvania headquarters, the company first added instances in other US locations, and then upgraded to LoadRunner Enterprise to meet growing internal demand.

Naveen Karakavalasa, IT Manager, Automation and Performance Engineering at Tokio Marine North America Services, comments: “As a service provider, it is vital that we provide rapid feedback to software development teams to improve business satisfaction levels. LoadRunner had supported us extremely well over the years but maintaining the growing on-premises infrastructure was becoming a significant overhead.”

Migrating to the Cloud

To eliminate its maintenance challenges, TMNAS chose to migrate to LoadRunner Cloud and convert all its existing test cases to run on the new cloud architecture.

“Migrating to LoadRunner Cloud immediately gave us the flexibility we needed as a team to run performance tests whenever we want,” says Karakavalasa. “We are no longer constrained by having a fixed number of LoadRunner Controllers, so we can run more tests concurrently.”

Moving to the cloud also made it easier for TMNAS to advance its plans for adopting continuous integration and continuous development (CI/CD). The organization wanted to integrate its load and performance scripts and test cases into its cloud delivery pipelines, so that developers could trigger test cases right from the build.

“Using LoadRunner Cloud Extension for Azure DevOps, we have successfully integrated our tests into our cloud delivery pipelines,” says Karakavalasa. “We also built an automatic feedback mechanism to push performance test results seamlessly back to developers, facilitating continuous performance testing.”

Whenever a developer pushes code and triggers a build in Azure Build Pipeline (CI), this then triggers a release in Azure Release Pipeline (CD) and automatically pushes the release to LoadRunner Cloud for testing. TMNAS has created a series of quality gates and reports that provide feedback to developers and test owners. Only once a release has passed the relevant tests is it released to production.

“The LoadRunner Cloud Extension for Azure DevOps automatically allocates the required testing infrastructure, moves the relevant scripts to it, and starts the tests,” says Karakavalasa. “LoadRunner Cloud lets us define an SLA for each feature, so once the tests are complete, we can compare the results against those predefined SLAs and back to the developer on which tests passed, and which failed.”

For any test failures, the relevant quality gate waits for the developer to respond with comments for review by the test owner, who can then monitor the status of the defect. For lower-priority defects, the test owner can provide conditional approval so that code can be moved to the next stage without delay.

Optimizing Quality and Customer Satisfaction

The continuous performance testing capability enabled by LoadRunner Cloud Extension for Azure DevOps provides almost immediate feedback to developers after each commit, with detailed information on failures that helps them understand what root-cause analysis they need to execute. Smaller tests are continuously repeated every 15 minutes, helping developers identify and resolve issues earlier, which ultimately reduces the cost of fixing defects. With LoadRunner Cloud, TMNAS also benefits from the elimination of the manual effort previously associated with preparing and managing on-premises test infrastructure, planning tests, and publishing test results.

“Our LoadRunner Cloud environment helps developers understand failures faster, from the perspective of their application’s performance requirements,” says Karakavalasa. “That rapid feedback all ties back to the delivery pipeline, where we can determine whether each build is good enough to move to the next stage. Ultimately, the Micro Focus solution helps us improve quality, driving higher levels of customer satisfaction.”