About Propeller Ads
Propeller Ads, a multichannel advertising platform, was founded in 2011, to create an optimal match between advertisers and content publishers. Advertisers are offered a wide range of modern ad formats over web and mobile, leveraging advanced marketing performance analytics to optimize marketing spend for over 100 million users in 195 countries. Every day, the company handles about 3 billion ad requests and generates 2 terabytes of new data.
Propeller Ads started to test Vertica capabilities by using the free Vertica Community Edition, which was used to implement a few prototype data marts to boost the speed and agility of end-customer reports without changing the underlying enterprise data warehouse. Jacob Becker, Vice President of Propeller Ads, explains: “The solution demonstrated significant advantages in processing speed. Request execution time reduced ten-fold and some requests were executed 100 times faster compared to the previous technology”.
However, when the company decided to modernize and upscale the entire data warehouse, Vertica was not automatically selected, but compared against other candidates, including Hadoop and MongoDB. “Our main requirements were speed, scalability, and ease of use and maintenance through a small and agile engineering team,” explains Sergey Nikolenko, Head of BI Development at Propeller Ads.
Another important selection criterion was the ability to work natively with other key product of the business intelligence ecosystem. Since Tableau was already used to generate reports for Propeller Ads customers and internal users, the new platform had to provide efficient integration and support for advanced visualization without introducing additional tools and software. Equally important was an efficient connection with existing ETL systems to load data into the new repository, while maintaining know-how and work from the previous implementation.
Propeller Ads launched a proof-of-concept using all analytical engine candidates, but Vertica surpassed competitors in all key benchmarks. Easydata, a key local Micro Focus partner and Vertica system integrator, supported Propeller Ads for the full Vertica deployment and the solution administration. During the migration, Vertica specialists worked alongside Easydata to provide value-add consultancy during the migration and the performance optimization. As a result, the data warehouse modernization was a smooth process, lasting only six months.
However, a growing need for advanced analytics demanded not only a better performance on the existing architecture, but also to fully exploit new Vertica capabilities. The next step was to introduce a lambda architecture for advanced analytics based on Vertica. Lambda architecture is increasingly deployed by customer requiring high performance data processing, because it consolidates batch and online data processing into a single data storage unit, accelerating data processing across different tasks and analytical workloads. The lambda aggregated data layer is designed to extract specific metrics from the data, analyze them, and generate reports in real or near real-time. Raw and unstructured data is collected as is, without changing formats, and stored for further processing.
A significant drawback of many lambda implementations, though, is that they are designed with very heterogeneous systems. The multiple integrations of heterogeneous components can often significantly affect the processing speed benefits for which the architecture was selected in the first place. Many lambda architecture implementations in the market do not allow SQL queries or easy integration with traditional BI systems. This makes it difficult for BI analysts to work and integrate with BI solutions.
Propeller Ads implemented a lambda architecture entirely on Vertica, and this helped to avoid such disadvantages. All components are leveraging the same storage. The aggregated data layer generates reports for advertisers and publishers, giving them dashboards with up-to-date, detailed statistics to analyze the effectiveness of ad impressions and the value of advertising campaigns. Since Vertica supports SQL query syntax and seamlessly integrates with BI systems, the new advanced analytics are not changing the existing processes of BI analysts.
Report generation is now carried out in near real-time mode, with no more than a minute delay for the most complex applications. Sergey Nikolenko comments: “Vertica storage capacity enables us to provide much more metrics to customers. Before Vertica, we collected only a few aggregated metrics, now any important analysis parameter can be stored in the repository and be easily available to generate new metrics very easily. Thanks to the new architecture and Vertica, our advertising solutions now deliver approximately one hundred metrics, a ten-fold increase from before.”
Propeller Ads also leverages Vertica FlexZone technology to ingest semi-structured data in the same data storage, thus realizing a very effective Data Lake of up to 90 Terabytes over a single technology. Vertical FlexZone allows you to ingest into Vertica semi-structured data in its original format, while accessing it with SQL queries. The ability to work on both structured and unstructured data in the same way as a traditional DBMS is one of the key differentiators that convinced Propeller Ads to consolidate the data lake over a single technology.
Vertica provides Propeller Ads with a high performance, reliable, and scalable framework for the implementation of the lambda architecture, through which the company improves its advertising network every day, increasing customer satisfaction.