Implement a platform for big data analysis.
Innovative finance needs the support of big data analysis and that’s what is delivered by The Merchants-Unicom Consumer Finance Company (MUCFC) which is a joint venture between China Merchants Bank Hong Kong subsidiary Wing Lung Bank and China Unicom. With a registered capital of two billion yuan, MUCFC uses the concept of micro-finances and new interconnectivity as the basis for its developments in the internet era.
The increasing interconnectivity of devices requires new rules to be understood and new partners to be found. New technology needs to be used and new models must be sought. Systems must be created for credit-staging, financing, mobile payment, and other products to create a new consumer financial experience and to continually increase core competitiveness.
MUCFC is focused on providing financial services to online clients so building a big data analysis platform has become the top priority. The organization hoped that, through the integration of financial and telecoms data, it would be able to carry out real-time analysis and evaluation of customer’s credit and risk. In this way, it would gradually form a user portrait based on the user’s asset changes, consumption habits, social networking and other information. This would not only allow the control of financial risks, it would also help MUCFC to explore new business models and improve the accuracy of financial services.
MUCFC encountered a lot of challenges in its construction of a big data analysis platform. First was how to ensure the security and availability of financial services. MUCFC’s big data analysis platform supports huge volumes of financial data and the failure of an important node could lead to system downtime or even data loss. That could lead to business stagnation and other systemic risks, bringing incalculable losses. MUCFC needed to ensure the security of the financial data so that a user’s error query would not lead to the paralysis of the system and so that, even if some nodes were damaged, the overall availability of the business platform would be protected.
Secondly, the performance of the big data analysis platform is critical to the financial sector.
“There is never enough time for data analysis, as the longer the data analysis takes, the lower the value, especially for financial services. Quickly completing data analysis means that we can respond to user needs as soon as possible, reducing business waiting times and bringing about great innovation,” says MUCFC financial research and development director, Jiang Lianglei. “MUCFC has now accumulated more than 10,000 credit customers so huge amounts of data need to be analyzed by a node in a short time. Only a high-performing data analysis platform can ensure the timely response to demand and enable business flexibility.”
With the rapid development of startups, MUCFC is also very concerned about the future expansion of its big data analysis platform. Jiang stresses: “A platform that was not scalable would prevent business growth so would have to be replaced, creating significant IT costs and potentially causing business interruption. The impact on MUCFC would be huge.”
As a result, selecting a Massive Parallel Processing (MPP) architecture for strong scalability was the first choice for MUCFC, to achieve big data analysis along with the potential for online and connectivity expansion. It hoped for future linear performance expansion.
Finally, MUCFC hoped to be able to simplify the operation and maintenance of the big data platform to reduce IT costs.
“We hoped that the big data platform would have standardized, automated management tools, as this would not only be conducive to fast big data analysis but would also increase system stability. In addition, we also wanted a platform that would fully integrate resources and tools available on the internet, making them easier to use,” adds Jiang.
Jiang Lianglei – FINANCIAL RESEARCH AND DEVELOPMENT DIRECTOR
MUCFC
Formed as a joint venture between Wing Lung Bank and China Unicom, MUCFC uses big data analysis to control financial risk and improve business efficiency for online clients. It has deployed the Vertica relational database for its data consistency, availability, and stability.
During the selection process, MUCFC conducted Proof of Concept tests on the mainstream data analytics platforms available on the market, finally selecting the deployment of the big data analysis platform, Vertica. In actual deployment, MUCFC deployed a server cluster consisting of four to nine ordinary PCs, using Vertica to analyze the users’ transaction data.
To control its financial risks, improve service efficiency, and accuracy, MUCFC conducted detailed analysis of the different technical options – traditional minicomputer with shared storage or single integrated units. Both were costly and this meant that, with business growth and the introduction of external data, costs would increase quickly. Open source Hadoop technology lacked the required performance for operation and maintenance requirements, multi-table related inquiries and other aspects.
“Vertica is a relational database that meets the requirements for data consistency, availability, stability and so on,” says Jiang. “It is also an open platform which can integrate a variety of resources and tools, so will meet our needs for the analysis of financial data.”
Vertica is a solution that addresses the challenges of big data analysis, using a highly scalable MPP architecture design which can be run on a cost-effective PC server cluster. This supports the requirements for the platform to allow smooth expansion of the customer base either through online expansion or connectivity expansion and a server cluster structure allows linear or even geometric growth.
Vertica is a good fit for the financial performance, security, availability, scalability, ease of use and other needs, and helps MUCFC gain a rapid insight into user characteristics, controlling financial risks through the analysis of large amounts of data. This has helped form a big data environment that enables healthy development, providing users with more competitive financial services.
MUCFC decided to start with the deployment of a platform comprising four server nodes. Due to business growth, this was extended to six nodes then to eight. This not only allowed smooth expansion but also protected the value of IT investments.
In terms of performance, Vertica delivers a large-scale parallel processing system, using memory that can be stored in columns and using a distributed column architecture with an internal memory and magnetic disk to provide ultra-fast big data analysis.
Jiang says: “During data analysis, MUCFC used PC servers equipped with Vertica. This allowed 280,000,000 users to be extracted from the business environment, making a total of about one billion user behavior data points. It took only 10 minutes to complete the analysis. In terms of efficiency, this is comparable to running a big data analysis on the server, but the performance to price ratio is greatly improved.”
After deploying Vertica, MUCFC could ensure the high availability of the platform. In the tests, a node was manually turned off to simulate failure. The result was that the performance declined but the platform maintained overall availability. This is because it features a ready-to-use design, so the data capability of a single node can be allocated to two adjacent nodes. With this design, even in the case when some nodes collapse, the platform is still available. Vertica also has cross-cabinet backup and other functions which can enhance platform availability.
