IT within TMNAS is delivered as a central service to all TMNAS group companies. When this was first introduced, Todd Soisson, Vice President of Business Experience for TMNA Services, had a very small Quality Assurance (QA) team that soon ran into capacity issues:“At the time we were using a freeware software testing tool for automated regression testing, and it just wasn’t cutting it. We were getting a new build every other day, and many new fixes were breaking older functions that had otherwise been working. Problem was, the freeware tool wasn’t catching everything and the scripts were difficult to maintain.”
Not long afterward, they also began receiving complaints from agents in the field regarding the slow performance of another core system. Further complicating the matter was the fact that test cases and associated metrics were being manually managed in spreadsheets. Soisson: “This testing model was obviously not scalable and we saw our workload piling up, while releases were in danger of running late.” TMNAS realized test automation could be the answer and started investigating the market for solutions.
Micro Focus UFT One is engineered to facilitate distributed test execution of multiple tests simultaneously. A proof-of-concept demonstrated that this could substantially speed up TMNAS’s testing cycle. Initially just one license was purchased, and Soisson was delighted with the results: “We went from three to four days of manual testing to an eight-hour automated overnight testing session. When we showed the CIO the ROI we were able to achieve with one license, he got really excited and challenged us to put a roadmap together to purchase more licenses, and start automating the entire regression test process. The early success with UFT One developed confidence in my team and quickly paved the way for additional Micro Focus solutions like ALM/QC and LoadRunner Enterprise.”
Micro Focus ALM/QC is an integrated IT software quality management solution that enables consistent QA procedures across the application lifecycle by enabling test case management and defect fixing in a centralized location. Micro Focus LoadRunner Enterprise is an automated performance and test automation solution that provides an accurate picture of end-to end system performance under varying loads, so that issues can be identified and resolved before an application goes live.
TMNAS identified core test cases and built a framework to automate these. Over the next couple of years more and more test automation was added and a Quality Center of Excellence was established that included the newly formed automation team. Performance, functional, and end-to-end regression testing are all included and can be remotely executed and monitored through a Virtual Automation Lab, hosting 12 dedicated test automation VMs. Proactive dashboards and metrics provide a real-time view of test automation within TMNAS. Soisson: “We started with just one automation engineer and have grown the team to nine resources. We run the automation team as a development team rather than a test team, and we feel the value added to the business is what gives us our credibility.”
With 780,000 test checkpoints and 100 percent of regression tests being automated, all tier 1 applications undergo a monthly performance validation before being released into general production. This includes 150 hours of automated regression testing to cover all core applications, running high risk business workflows worth $4.5 billion to TMNAS.
The response from the business has been overwhelmingly positive: “If we have to make an emergency change to a key application, our internal clients feel safe in the knowledge that our test automation processes will run overnight and nothing catastrophic will have happened to their application when they come in in the morning. They trust our team and the processes we’ve introduced,” says Soisson.
Hemant Anugonda, Senior Manager of Quality Services at TMNAS, adds: “We’ve introduced parallel threading, leveraging TestNG, UFT Developer, IntelliJ, and Eclipse, running up to three testing threads on the same machine. This lowers test execution times significantly and enables us to release our versions much faster. The use of headless browsers (i.e. browsers without a graphical user interface) helps us run our test automation scripts without even rendering them on a screen. This allows us to reduce our regression test time by 70 percent, time we can use to execute deeper and broader application tests.”
TMNAS has ambitious plans for the future and the Quality Center of Excellence will play a central role in supporting them. It is moving towards a SaaS model to create more efficiencies and enable faster upgrades. In preparation for this, the foundations are being laid for a Test Data Management team to service test data requirements for TMNAS. They will enhance the automated testing framework as a self-service solution.
In response to increased use of mobile applications, TMNAS is expanding its mobile test capabilities. This triggers additional security requirements and the Quality Center of Excellence is expanding its remit to include security testing as well. Integration of automated smoke tests with Octopus Deploy supports continuous integration and deployment, shortening the time to release new versions. A large infrastructure transformation program is underway and the Quality Center of Excellence aims for all associated application testing to be automated and in a self-service model, through the use of Micro Focus solutions.