Spark Summit: SequoiaDB x Spark Architecture Scales Operational Data Lake in China

SAN FRANCISCO, June 30, 2017 /PRNewswire/ -- From June 5 to 7, Spark Summit SF 2017 came to meet the Silicon Valley again. SequoiaDB, the top distributed database vendor in China and also one of the 14 certified Spark distributors over the world, was invited to the summit and share a talk about how SequoiaDB and Spark build a large scale operational data lake in China.

SequoiaDB is the top distributed Database vendor in China. Their main product, also named SequoiaDB, is the only distributed NewSQL database built in China. The company has received The Red Herring Global 100 reward in two consecutive years. It was recognized as one of the 50 most innovative companies in China by Fast Company.

The sharing was brought by Yang Peng, VP of Solutions in SequoiaDB. The talk started from giving the audience a glance of the increasing demand of high performance data processing from the financial industry and was followed by showing the technical solutions of building an operational data lake integrated with Spark to meet the challenges.

Big Data in China Scale Shocks the Silicon Valley

China has been through a rapid growth with the Internet and mobilized technology. The rise of the Internet and Mobilization has been named as one of the national economic movements for the large and traditional enterprises in China.

Especially in the financial industry, in 2016, for all the urban residents over 13 years old, the E-banking and mobile-banking user ratios were 46% and 42% of the China population respectively. The payment services such as WeChat Pay and AliPay have over 400 million users. In 2017 the mobile banking will become the Number 1 banking channel. Besides banking, insurance, securities and stocks they are in the same pace of the transforming technology.

The business innovations grow rapidly as well such as facial and finger recognition in remote account opening, self-insurance processing with mobile phone, real-time risk management, responsive products and service recommendation. Following the E-Commerce, China is determined to move much faster and further in technology adoptions than other countries.

To build these systems in such a large scale brings up huge technical challenges especially in data processing. Hence, Big Data has been placed under the spotlight to be the major power to push the innovation forward. It requires multi-model data handling to cover pictures, audios, videos, etc. It also requires high performance of data storage, data processing and data analyzing in both operational and analytical areas.

SequoiaDB is Taking on the Big Data Challenges

The big data challenges can be categorized as:

Data Scale: With the huge Internet and mobile app user population, China enterprises will face hundreds of millions of user accounts. The data volume is large and grows quickly. Scalability and elasticity are the first priority at beginning. Averagely, large enterprises in China have to deal with PB-Level of data volume.

Mobilization: Over 90% of the end users are using their mobile phone to manage their banking operations. High concurrency capability, fast response time and low latency are definitely the most important requirements of data processing.

Multi-Model: Besides the huge scale of data, more users also means more complicated business scenarios and functions, which uses different types of data, from structured data, semi-structured data to unstructured data. Multi-model capability that supports SQL, NoSQL and Object processing becomes a necessity for data platform.

With the brand new business scenarios and complicated functions of applications, China data technology products and vendors have been through far more critical 'field test' which push their products and services to develop and iterate more quickly than oversea competitors. It requires strong local services to support and work with clients together to achieve the performance, scalability and stability. This is one of the main reasons of shrinking down for the international companies in China.

SequoiaDB Innovate the Big Data Management

SequoiaDB is China's leading database and big data platform vendor, to satisfy the China big data applications, SequoiaDB provides two products:

Distributed multi-model database -- SequoiaDB, and distributed content management software -- SequoiaCM. The two products can cover all the data models from structured data to unstructured data, and reform the big data management.

With the leading technology and products, SequoiaDB has also withstand the critical challenges of hundreds of enterprise customers especially banking customers.

SequoiaDB's products and services are in a leading position in China against both domestic and international competitors. The data lake solution with Spark and SequoiaDB has also been extensively applied in Top 500 level customers.

SOURCE SequoiaDB Ltd.