Create a platform to analyze Big Data.
Shanghai Shengfutong Electronic Business Co., Ltd. (ShengPay) is a leading Chinese independent third-party payment platform. It was founded by the Shengda Group and focuses on providing safe, convenient and stable payment services to internet and commercial users. Its online and offline comprehensive payment system enables users to fully experience ‘pay anytime and anywhere’ services.
With the number of internet financial users rapidly increasing and the continuous innovation of internet financial applications, there has been explosive growth in the amount of business data ShengPay must deal with. While this growth tests the capacity of ShengPay’s IT systems, it also provides a rare opportunity for the company to use this data to further innovate its financial business. Using Big Data, ShengPay wanted to unearth user demands from massive amounts of business information. This would enable it to improve customer sales and increase risk management and operational strategy, which in turn would help to strengthen its core competitiveness.
To achieve this, ShengPay decided to deploy a Big Data analysis platform to enhance the competitiveness of its financial business.
However, during early discussions on the project, ShengPay discovered that it was not easy to find a suitable Big Data analysis platform. Firstly, there is a massive amount of data in ShengPay’s business systems, and this was growing by hundreds of gigabytes every day, putting huge demand on the inquiry and compression capabilities of the Big Data platform. If the data was not processed quickly, it would not be able to provide a fast and efficient response when servicing and supporting the company’s business data, possibly delaying valuable business opportunities.
Furthermore, ShengPay’s internet financial data is growing exponentially, putting huge demand on the scalability of the Big Data analysis platform. A traditional database is limited by its system framework design. As its performance cannot achieve growth alongside the equipment, it will experience performance bottlenecks once it has expanded to a certain level. If it moves to a new platform, not only will there be a dramatic increase in IT costs but the sustainability of the business may also be affected.
Lastly, the security and usability of the Big Data analysis platform were major areas of focus for ShengPay. Due to the high value and sensitivity of internet financial services, the company needs to guarantee the integrity and usability of its financial data. This means that even if some of the data nodes are damaged, it can still guarantee data security and provide sustainable services. Additionally, ShengPay hoped that the Big Data analysis platform would have automatic optimization functionality. This would not only reduce the workload for operation and maintenance staff but also support the stable and efficient operation of the database.
TECHNICAL DEPARTMENT SUPERINTENDENT
Shanghai Shengfutong Electronic Business Co., Ltd.
Chinese payment platform, ShengPay, amasses huge amounts of financial data and the volume continues to grow at a fast rate. It wanted to analyze this information to gain valuable insights that it could use to improve services and become more competitive. A powerful platform was needed and ShengPay chose Vertica.
In order to deploy a high performance, highly-scalable, and highly-usable Big Data analysis platform, ShengPay ultimately chose Vertica. This platform is purpose-built for Big Data analytics. It is designed for use in data warehouses and other Big Data workloads where speed, scalability, simplicity, and openness are crucial to the success of analytics. Vertica relies on a tested, reliable distributed architecture and columnar compression to deliver fast speed.
Vertica was chosen because it could help ShengPay resolve and refine the three major challenges of sales, risk management, and operation strategy, as well as build and drive the healthy development of its financial Big Data environment.
Following the deployment of Vertica, Hewlett Packard Enterprise (now part of Micro Focus®) helped ShengPay build an efficient Big Data analysis platform. Vertica uses balanced memory and disk distributed compressed columnar architecture with clustered Big Data storage. Compared to traditional technology, there is exponential growth in the speed data can be analyzed, which significantly enhances the inquiry capability of the system. Operational data shows that once all the historical data is moved to Vertica, ShengPay can directly summarize this data from many years, with the inquiry speed reduced from two hours to just seconds or minutes; enhancing efficiency more than ten-fold.
Vertica has a unique high compression feature which can greatly reduce the data processing load at the same time as lessening demand on storage and other hardware equipment. Currently, ShengPay’s daily data growth amounts to between ten and several hundred gigabytes. With the Vertica high compression and batch loading features, data loading can be completed in under an hour, and in some instances just minutes. This significantly increases loading speed, thus providing sufficient buffer processing time for the second integration of downstream data.
In terms of scalability, Vertica uses the Massively Parallel Processing (MPP) framework design. This can be used on a cluster of economical and efficient commercial servers which can be easily expanded. This provides flexible and simple scalability for ShengPay’s business development, guaranteeing that with the company’s anticipated rapid growth, Vertica will still support performance enhancement and expansion without huge costs or any suspension of services.
After deploying Vertica, ShengPay can also guarantee the integrity of data. Its design has no assistant node or single point of failure, and even if some of the nodes break down, the system can still provide outstanding service. Vertica also uses Database Designer (DBD) automatic optimization design tools which significantly reduce the workload of database administrators, enabling ShengPay’s database to be operated in a more stable and efficient way.
