“Silk has been a key tool in AAA Life becoming more agile. Ongoing regression testing will happen with minimal additional effort thanks to effective test case reuse through Silk Test and Silk Central.”
Software Test Analyst
AAA Life offers a variety of competitive life insurance, annuity, and travel accident products. With over 1,000,000 policies in force at the start of 2014, many customers have relied on AAA Life to help provide protection for their loved ones.
The QA Team of 10 at AAA Life manages a very diverse and inter-connected technology environment with complex business processes spanning multiple applications. Some key business processes rely on legacy applications dating back many years, often with custom code, and this makes testing a complex challenge. Geoff Ellis, Director, IT Application Development, explains: “We previously documented all our test scripts in Excel and stored them in a hierarchy of folders on a shared network drive. This made it very difficult to reuse existing test plans. We put considerable focus in the last 12 months updating and formalizing many of our development processes and testing procedures, and having our team work on the launch of a major new business initiative, ExpressTerm Life Insurance, gave us the opportunity to really put these new tools and processes to good use.”
ExpressTerm will allow users to visit AAALife.com, receive a quote for life insurance, complete an application fully online, and receive an immediate response and confirmation of an active policy, all within an approximately 15-minute single session. This ambitious project relies on many diverse and complex automated integration points with multiple distributed systems, and extensive quality assurance across the entire system was an absolute necessity.
Ellis had another objective in mind: “We are committed to an Agile software development cycle and this gave us the opportunity to prove the benefits of test-driven development, continuous integration, and test automation to make the team more effective.
TEST PROCEDURES INTEGRATED IN SILK
Following a short market evaluation where Open Source testing tools were compared with commercial solutions, the suite of Silk tools was chosen for its superior functionality and integration potential. The process started by converting all the test plans into Silk Central® on a project-by-project basis. Previously, the QA team would be working one or two sprints behind development and requiring multiple weeks of regression testing for each release, creating significant overhead.
Tom Barlow, Software Test Analyst, explains how the Silk tools are used within the new process: “Using Silk Central as our test repository, the Business Analysts and QA team work together to define the acceptance criteria before development starts. As soon as development checks code into the central build server, the QA team runs fully automated regression testing using Silk Test®, including new functionality testing. At the end of each day, a new code build is deployed to the test environment and the same set of automated tests is rerun. Integration testing and regression testing of other products to ensure that existing functionality has not been affected is also performed at this point. We have found it to be so much more efficient to test alongside the developers in the DEV environment. Basic coding errors or mistakes resulting from misinterpretation of requirements are caught immediately and can be resolved within minutes. At the end of each two-week development sprint, the team demonstrates all new code and functionality, all of which has already been tested by the QA team.”
Silk Performer® is used to performance test the transactional sites that support Express Term. Previously, load-testing was conducted but not as effectively as possible. Ellis comments on the process when Silk Performer was implemented: “Using Silk Performer allowed us to run load testing much more regularly, and with less effort, and therefore we put a much stronger focus on performance earlier in the development process. The results were impressive: we saw a four-fold increase in the concurrent members we can support and we recorded response times on average of under two seconds. We are very satisfied with these results, considering the volume of applications going through the system.”
SHORTER TIME TO MARKET WITH SILK TOOLS
Use of the Silk tools has drastically improved the time to market for AAA Life. End-to-end testing for a project of this size and complexity would normally require at least six to eight weeks. On the ExpressTerm project, Ellis and his team were able to execute a complete testing cycle in less than five days, allowing a full test, bug fix, and regression test iteration within a single two-week development sprint. Ellis illustrates the overall impact of Silk with an example: “On previous projects, we could only say ‘it feels like it’s ready to go live,’ which was a subjective viewpoint, and so our way of guaranteeing quality was to run additional test cycles and extend the testing process further. Making full use of the Silk tools, our final go-live approval is now based on fact: have all test cases been executed, are all critical defects resolved, and has all end-to-end testing been completed? All of these metrics are stored in Silk and tied to the exact release build of the code so that it can be reviewed by our Audit team at any time in the future to verify that processes were correctly followed and appropriate controls were in place.”
REUSE OF TEST CASES
Changes to the applications’ functionality can be made much more quickly as a suite of automated regression tests can validate the new functionality has not adversely affected existing functionality. Barlow comments, “Silk has been a key tool in AAA Life becoming more agile. Ongoing regression testing will happen with minimal additional effort thanks to effective test case reuse through Silk Test and Silk Central.”
VISIBILITY IN THE TESTING CYCLE
The standardization of testing processes has brought huge benefits to AAA Life, as Ellis comments: “The lead up to deployment, sign-off, and approval of any new project was always complex. Now, the ability to look at each of the builds, and check specific defects and their sign-off gives us a level of detail that we haven’t had before. All our key business processes are covered, and I feel confident in introducing this QA approach for all of our upcoming projects.”
Compared to previous projects of its size, ExpressTerm has been delivered faster, vitally important for a revenue-generating product. In addition, the team has seen cost savings through increased efficiency and productivity during the project development phase.
Ellis concludes: “Prior to our Agile transformation, all too frequently projects suffered from last-minute scope changes, functionality gaps uncovered during the final testing phase, or missing requirements that needed to be addressed in follow-on releases. With Silk at the heart of our new Agile testing process, the feedback received at each two-week sprint in this project ensured that functionality and quality issues were identified early on, resulting in a professional application where design and functionality work seamlessly and effectively together.”
ROI is represented in the increased quality of the application, the robust processes in place, and the ability of the teams to meet aggressive delivery deadlines.