Cerner collects billions of RTMS records every month, analyzes them, and uses that information to address performance issues in Millennium that may impact care delivery. As more and more health care providers around the world began using Millennium, Cerner’s legacy data warehouse solution was not able to process the volume of data as quickly as it needed to.
“We couldn’t analyze the data fast enough to proactively optimize Millennium timer data at an optimal rate. There was just too much data to successfully process it as it was generated,” explains Dan Woicke, director of Enterprise System Management at Cerner.
It became clear that, in order to sustain its ability to proactively address solution performance, Cerner needed a better approach to Big Data analysis.
When Cerner’s legacy data warehousing solution could no longer support the need for near-real-time data analysis, it evaluated several alternatives to address its analytics needs. Ultimately, the company chose the Vertica Analytics Platform based on best-in-class performance and concurrency, with performance the most important.
“As our solutions scaled, we began to approach the upper limits of our analytical capacity, given the volume of data we were collecting,” says Bill Graff, senior vice president, Cerner Technology Services. “Now that problem is behind us. The Vertica Analytics Platform delivers the speed and massive scalability we need to maintain the high levels of service availability and system performance health care clients deserve.”
Why Vertica? Performance Concurrency
“That’s where Vertica really stands out,” says Woicke. “A large client produces six million RTMS timers per day. Analyzing that in our legacy database took 20 minutes. With Vertica, it can be done in 20 seconds,” which is effectively a 6,000% improvement.
Concurrency—the number of Cerner associates analyzing data at a given time—also favored the Vertica Analytics Platform. During the evaluation, more than 450 concurrent users were able to conduct analysis with Vertica at one time.
Another key consideration is scalability. During the evaluation at Cerner, Vertica ran on a halfrack of eight HPE ProLiant BL460c Server Blades. Today, with Vertica running on a full rack of HPE server blades at Cerner, the software can handle even more concurrent users and process more data.
That scalability has already proven useful. At the time of the evaluation, Cerner was collecting 6 billion RTMS timers per month; since then, the number has grown to 10 billion. And the sky is the limit.
“The unlimited scalability of Vertica gives us confidence that it will grow with us and enable us to use the platform in more ways in the future,” says Woicke. “We know we can add more capacity at any time.”
From reactive to proactive IT Management
For Cerner, analysis through the Vertica Analytics Platform goes beyond identifying solution performance issues. It enables Cerner to become more proactive in managing its hosted Millennium service.
Graff notes that Cerner’s hosting environment generates more than two million event alerts per day on average. “Daily, we distill two million alerts down, send them to our Immediate Response Center, and deal with particular issues that threaten to degrade service.”
With the rapid analysis enabled by Vertica, Cerner wants a more proactive approach. It is looking at the Vertica platform to do predictive analytics across a wide array of performance parameters—from CPU utilization and disk I/O to network I/O, memory utilization and beyond.
“What we want is a wider view that lets us know what will happen to server, storage and network performance long before an event occurs,” explains Graff. “We can then head off problems before they happen.” The ultimate goal, he notes, is a self-healing system that automatically anticipates issues and spins up additional resources to avoid any negative end-user impact.
Coaching to improve user workflow
Another compelling outcome for Cerner is Vertica’s ability to analyze the RTMS timers as “breadcrumbs” to track the workflow of clinicians as they treat patients. Millennium offers numerous configuration options, which are designed to enable users to customize the solution. Unfortunately, not all users make the best of those options.
Millennium’s RTMS timers provide the raw data for Cerner to gain insight into how individual physicians and other users actually use Millennium and how they might use it better.
“The RTMS timers let us examine the clinician’s workflow,” explains Woicke. “We can virtually sit over their shoulder and see how they use the application, and make suggestions about using it more efficiently.”
For example, a user might routinely search through common orders in a hospital or clinic, rather than creating a folder or list of favorites. Using the folder could reduce a clinician’s time spent searching—and reduce the risk of error— ultimately enabling a clinician to spend more time caring for patients.
“If a user isn’t using a time-saving feature effectively, we can contact them and suggest a more efficient workflow for them,” says Woicke. “We can ‘coach’ our customers into being better users of Millennium.” And by using Millennium more efficiently, the users become more efficient physicians.
Improving patient safety and quality of care
Analysis of user workflow in Millennium also offers the promise of improving quality of care. Using Cerner’s Health Facts application, which applies predictive analytics to build a database of patient outcomes based on past treatments, clinicians can steer toward a course of treatment with greater likelihood of success. This is key as health care providers support government regulations, such as the U.S. Patient Protection and Affordable Care Act, designed in part to increase efficiency in health care delivery.
Effective use of Health Facts requires rapid analysis, and that means Vertica. “In the past, some Health Facts users would issue a query at 5 p.m. as they leave for the day, hoping they would have a result when they return at 8 a.m. the next morning,” Woicke explains. “With Vertica, those query times are down to two or three minutes.”
What lies ahead? More data and more analysis. Cerner has already nearly doubled the number of records it analyzes on a daily basis. That growth will continue.
As Cerner hosts Millennium for the majority of its clients, a comprehensive view of user activity is vital to the incident, event and problem management processes used to drive continual service improvement. If a client has a performance problem, Cerner can use realtime usage and performance data to roll back the clock and to replay a transaction. With this information, Cerner can identify the root cause of the problem, resolve it quickly, and then use that analysis to prevent similar issues among other users.
“We can use the analysis to determine if someone changed a particular setting that led to an incident, then go back and audit that setting across the client base to prevent something from happening to another client,” Woicke explains.
The result: proactive performance management of Cerner’s patient-centric health care solutions, which ultimately translates to improved health care delivery, better outcomes, and healthier, happier patients. Exactly what health care providers, and Cerner, hope to achieve.