Improve the delivery of time-sensitive data and analytics of semiconductor/electronics manufacturing data.
The company’s legacy database, Microsoft SQL Server, was adequate for basic operational needs. However, the largest customer was approaching 60 TBs of compressed data (300-400 TBs of uncompressed data), which pushed SQL Server past its limits. The IT team had to decide which data to delete or shorten retention times, limiting the amount of historical information accessible for analytics. In addition to resolving data retention issues, Optimal Plus also sought to improve its predictive algorithms to ensure supreme uptime of plant assets. The company considered EXASOL, ParAccel, and Vertica solutions.
Headquartered in Holon, Israel, Optimal Plus last year processed data on 50 billion ICs and boards, which was up 15 billion year over year. Compared to homegrown data-collection alternatives, the company’s holistic software enables seamless visibility across distributed supply chains. Developed for the Industrial Internet of Things (IIoT), the software connects to various plant assets, sensors, and devices to record, store, and analyze continuous streams of data.
“We chose Vertica, the market leader in column store compressed databases,” says Michael Schuldenfrei, Chief Technology Officer for Optimal Plus.
Michael Schuldenfrei – CHIEF TECHNOLOGY OFFICER
Optimal Plus
During tests, Schuldenfrei and Alon Malki, the company’s Chief Architect, saw that they could load and process data in the database at speeds which were significantly faster than with their legacy database.
Michael Schuldenfrei – CHIEF TECHNOLOGY OFFICER
Optimal Plus
Regarding the solution’s fast query speeds, Schuldenfrei explains, “We can answer every query in scales that we couldn’t before. A million data points would take 30 seconds in SQL and processing a billion would take forever. We calculated 17 different statistical functions on 2 billion data points in less than a minute, which is faster than our previous system would have taken or any other system I’m aware of would have taken just to retrieve the data.”
Optimal Plus is now able to run analytics which would have never completed execution in SQL Server, according to Malki.
“We’ve removed the limitations we had before with the amount of data we could store and the size of queries we can do for customers,” Malki says.
Optimal Plus addressed its scalability requirements, as Vertica uses commodity servers instead of expensive, specialized brands. That will make horizontal scalability cost efficient and practically endless. Additionally, the high compression rate of Vertica can generate significant savings.
“The compression rate is so good in Vertica that databases are much smaller than the real data you inject into them. That can lead to significant savings in hardware and licensing costs,” Schuldenfrei says.
Using Vertica, Optimal Plus can compare historical data with almost-instant test information to predict faults and prevent downtime. Optimal Plus analysts create historical baselines, form models from the history, and launch the models into customer facilities to receive actionable insights.
Malki notes that 95% of customer data resides in Vertica, which functions as an index on top of Hadoop software. “We’ve successfully piloted the combined Vertica-Hadoop solution for a customer,” Malki says. “We’ll drive this out fully over the next couple of months.”
The company plans to create a “hot” environment by storing recent data for fast ad hoc queries. Older data will be stored in a “cold” environment, utilizing Hadoop for longer retention. “This arrangement will leverage Vertica’s ability to query data off Hadoop, which it does extremely well,” Malki says.
In production environments where errors can cause profit loss, Optimal Plus customers are enjoying prompt analytics with a solution that integrates numerous data islands.
“Companies demand that analytical tools integrate with their ecosystems and we deliver that. We’re delighted with the platform and Vertica team. They’ve supported us each step of the way,” Schuldenfrei concludes.
