Provide KPI-focused data analytics to everyone in a user-friendly self-service model. This required a more robust approach, focused on concurrency and performance
A data analytics platform of Netlink, a leading technology solutions provider, Lumenore considers itself the anti-dashboard team. Lumenore created a platform that empowers human intelligence with insights that are simple yet consumable – a results-driven analytics platform with a top-down KPI-focused approach. It supports informed and data-driven decision making, leveraging artificial intelligence (AI), business intelligence (BI), and robotic process automation.
The Lumenore team is committed to the use of open source technology wherever possible. When the Lumenore platform was developed initially, MySQL, an open source RDBMS, was introduced for the back-end data processing. However, Saurabh Mishra, Director of Analytics and BI Architecture for Lumenore, soon saw issues: “The Lumenore platform grew in popularity with our customers, and with that, the data volumes increased significantly. Fairly simple data queries were taking too long, and overall the platform performance was suffering as a result. We just couldn’t get the analytics performance we needed. With a view to future growth, we realized we needed a more robust data analytics solution.”
Lumenore’s vision of data democratization delivers analytics to anyone within an organization, from C-level to individual contributor. This means that at any given time, 1,000+ people could be querying the platform. User concurrency and performance were key criteria in the search for a new solution. Overall, the team looked for a data analytics solution capable of managing high data volumes in a performant environment with a flexible feature set. A horizontally scalable solution that supports columnar storage with massively parallel processing (MPP) capabilities would ensure there was no single point of failure.
Varun Gaur, Director of Product Management for Lumenore, evaluated MongoDB, Cassandra, Greenplum, and InfoBright as well as the Vertica Analytics Platform: “I had prior experience with Vertica, and was confident it could work for Lumenore. But we were determined to make a fair comparison with available open source options in an extensive proof-of-concept so that we could make an informed decision that would carry us into the future. In the same hardware environment, it soon became clear that Vertica was the winner for us with better management and higher performance, as well as an impressive out-of-the-box feature set.”
He continues: “Vertica’s high compression ratio requires less storage space, which saves on budget, and its flexible deployment opens up various cloud options for us, including Amazon Web Services (AWS). We were also able to deploy a multi-tenant architecture with one large Vertica cluster hosting all our customers, for better scalability and simplified management.”
Vertica is embedded in the Lumenore platform. It helps identify the hidden patterns within the data and, based on user preferences, generates insights in natural language. Vertica’s many data connectors enable Lumenore customers to connect all data sources they need to prepare a data-driven KPI dashboard. Vertica’s performance is essential for the popular ‘Ask me’ feature offered in Lumenore. This powerful AI-driven natural language-based query tool understands the intent of questions asked, builds the correct search algorithms, and returns the desired results.
“We love the flexibility Vertica offers,” comments Gaur. “Even though it comes with many features as standard, if we have a particular use case, we can easily write our own functions in any language – Java, Python, or R – and deploy this in Vertica, harnessing the power of parallel computing. Machine learning and advanced analytics capabilities are becoming increasingly important to us, and there is still so much to explore in Vertica.”
“One of our customers uses Lumenore in its call center facility,” says Mishra. “Several thousand call agents generate 30GB of data hourly which is sent to Lumenore for processing and analytics via Vertica. Every hour, over 200,000 queries are executed on the platform. Lumenore delivers near real-time performance metrics with customized views for every individual agent within the organization. Ingesting such large data sets and delivering the concurrency of complex analytical queries through dashboards is a massive Vertica advantage for us.”
When the COVID-19 pandemic hit, the importance of analytics and business intelligence became even more clear with companies trying to optimize their operational cost model while adjusting to new remote working practices. Data is a vital decision-making tool, and companies need to have clear insight. Rather than capitalizing on the increased market opportunity, Lumenore decided to reduce its price point as much as possible to enable a wider access to the platform.
Gaur concludes: “Vertica’s performance is just awesome and one of the many reasons we love it. Queries that were taking up to four minutes in our previous environment are now completed in less than 30 seconds – a performance boost of more than 90 percent. The combination of Vertica and our Lumenore platform gives our customers the added value we want to deliver.”
