The online store is just one element of Lamoda’s omni-channel marketing and logistic operations, which also include three call centers, a major distribution facility outside Moscow and a delivery service to over 600+ cities and towns. A photo studio, London-based design office, an application development center in Vilnius and other smaller teams complete the lineup.
An IT department of 250 creates and maintains advanced business intelligence (BI) applications for both the online and offline operations. Using transactional data from its Microsoft Dynamics ERP system, it develops various in-house software systems for all customer interactions. These support the online store, call centers, mobile apps, order fulfillment, warehousing and logistics.
“We are expected to give the business an up-to-date, holistic picture of how the company performs in each area,” says Sergey Pavlov, Head of Business Intelligence at Lamoda.
The corporate data warehouse amasses information received from relational databases that power various corporate applications, from marketing to customer support, store operation and logistics. The existing data warehouse contained both raw unprocessed data and the normalized detailed data layer for BI organized into specialized data marts for business, while SAP BusinessObjects delivered the historical report dashboarding. The online e-store collects customer clickstreams for behavioral analytics, storing data into Hadoop.
Discussions to replace the BI platform started because it was no longer economically feasible to expand the existing data warehouse in line with the pace of E-commerce growth. As customer data increased, expansion costs were growing at a disproportionately faster pace than actual data volumes.
In fact, every change carried a price tag approximately twice as much as before.
Another major challenge was the extremely long time it took to process business intelligence queries. “Users constantly complained about the insufficient performance of our system,” Pavlov recalls. “It took dozens of minutes and sometimes hours to process any query that was slightly more complex than usual. Our business clients were growing progressively dissatisfied with this slow report generation while online operations commanded actual real-time decisions based on most current data.”
Lamoda had to migrate the entire data warehouse to another database platform in one shot or to adopt a staged approach. This would involve migrating the functional data-marts over to a new analytics platform then improve the core BI schema at a later date.
Lamoda experts had begun searching for a new analytical platform three years-ago when they initially considered migrating the data warehouse and the data marts in one shot. Lamoda received bids from Teradata, Oracle Exadata, Greenplum, Netezza, Vertica, and Exasol. The management eventually concluded that such monolithic migration would take two years, during which time new development of existing BI tools would have to be frozen.
The continuous growth of online traffic aggravated the pressure for getting faster reporting sooner. Therefore, Lamoda opted for a different strategy: store new raw data in Hadoop, keep the detailed data layer in the old warehouse, but start migrating the data marts immediately to Vertica.
This way, users gained faster business reporting out of the Vertica columnar architecture straightaway, while IT could start shifting investments onto the newer analytics platform.
Lamoda experts started by testing the Vertica Community Edition. “Installing the system took just a couple of hours. After running the first tests and measurements, I was pleasantly surprised by its high performance. It took Vertica a matter of seconds to process queries while a competing software needed tens of minutes on the same hardware,” says Pavlov. As a result the company decided to purchase the enterprise-grade Vertica Analytics Platform.
The first phase involved migrating all data marts to Vertica, optimizing the extract, transform, and load procedures. Integration with other platforms went smoothly as the company had upgraded its ETL to copy data marts from the central data warehouse to Vertica incrementally. Whenever problems arose, Lamoda experts consulted Vertica representatives and its partner company EasyData, which specializes in Vertica deployments.
As of now, Lamoda maintains the legacy ETL procedures to feed the existing data warehouse, which is then used to feed data into Vertica for analysis. They are enriched with various system logs and other sparse data marts. SAP BusinessObjects is still the standard reporting tool, and Lamoda is also introducing Tableau as a new visualization platform, profiting from Vertica projections technology.
“We uphold the self-service approach when it comes to analytics and reporting. Any user can access the SAP BusinessObjects system and generate reports based on data from Vertica,” says Pavlov.
Vertica is already being used to analyze Lamoda sales data, with more than half of all sales reports now generated via Vertica. Some 150 Lamoda employees, both IT Department analysts and representatives of business units, use data from data marts built using Vertica.
The Vertica platform delivers a major boost to the speed of analytical query processing. “Now that we have distributed data mart calculation and data sampling for reports between central storage and Vertica, analytical queries are processed an order of magnitude faster compared to our old monolithic architecture. Our business clients are very happy with the results of this migration,” says Pavlov.
Further proof of the project’s success is the rapidly growing number of business users who have given up the old analytical tools and spreadsheets and are now accessing new reporting over Vertica. “The new data visualization tools offer much better performance and functional diversity. Vertica has helped to promote the popularity of data visualization amongst analysts.”
Pavlov points to the platform’s reliability and low administration costs as two of its key advantages: “Vertica requires next to zero administration. We do not have a dedicated admin for this database and so far we have had no need to contact the support service.”
Warehousing and logistical operations analytics will become available soon. “It is extremely important for us to keep track of the speed and quality of order picking and packing, as well as the performance of our delivery service,” explains Pavlov. Future plans include creating Vertica data marts for the marketing department and other teams.
Lamoda also plans to exploit Vertica’s in-database machine learning capabilities for predictive analytics and Vertica ability to analyze data residing in Hadoop. “Vertica features are largely aligned with our company’s technology evolution strategy, which is why we plan to put them to good use,” adds Sergey Pavlov.
“You do not necessarily need to be versed in big data in order to benefit from using the Vertica Analytics Platform. A pragmatic approach that involves upgrading the BI infra - structure can yield tangible business results quickly. If the existing system cannot keep up with the speed of your business, Vertica will considerably speed up the generation of analytical reports. This is enabling companies to handle the growing volumes of data and velocity of data streams and use them effectively to achieve continued business growth,” comments Evgeny Stepanov, Head of Vertica Sales, Micro Focus Russia.