EasyStack Collaborates with Tsinghua University to Create the Largest Chinese Research Cloud Platform

BEIJING, May 4, 2016 /PRNewswire/ -- As expected, OpenStack has taken the world by surprise. In the Research field alone, CERN (European Organization for Nuclear Research) has utilized OpenStack in theorizing about the birth of the universe and NASA has employed OpenStack to look into the possibility of human habitation on other planets.

Recently, the Institution for Interdisciplinary Information Sciences at the world-renowned Tsinghua University successfully completed the creation of the largest research-purposed cloud platform, consisting of 16TB memory, 540TB storage and single cluster capable of holding up to 6,000 cores, performing advanced research in emerging areas such as DNA sequencing, Social Networking and Natural Language Analytics, this project has also reduced the university's data center CAPEX by 60%; IT management tasks by 33%, and electricity costs by 80%.

The Tsinghua University Institution of Interdisciplinary Information Sciences

The Tsinghua University Institution of Interdisciplinary Information Sciences was founded on Dec. 30, 2010 and is led by Dr. Qingzhi Yao, the internationally respected computer scientist, Turing Award winner for the year 2000; Fellow at the U.S. National Academies of Sciences, Engineering and Medicine; and the Chinese Academy of Sciences. This institute is the first educational research organization on interdisciplinary Information Technology, and is dedicated to the correlation and cross-buildup of Computer Sciences and Physics. It is also the leading research institute in China on the research of quantum information.

New Research needs and modeling on institutional cloud platform

As computer clusters are needed more and more in research projects, the good old days of using a single HPC cluster for processing single research projects are gone. Traditional IT architecture has grown obsolete as it distracts scientists from conducting researches itself.

"It shouldn't be the scientists who demands for IT to be more flexible. Instead IT systems should be mobile as needed," renowned biologist John Boyle expressed the urgent need for a flexible IT system in an article published in the Nature Magazine titled "The Need for Biology to Develop its own Big Data System."

Today, what's draining on the IT Operations staff is that computer clusters are needed in all sorts of research-related computations and analysis. Scientific image processing, Cryo-EM, Protein Structure, online education data analysis, social network analysis, natural language processing, DNA sequencing are just some of the projects Tsinghua University's Institution for Interdisciplinary Information Sciences need support with. Although the Institution has over 200 x86 servers, different projects often require different computing sources, generating bulky configurations burdening the IT operations and maintenance teams.

Finally resting on recognition of professional and technical expertise, the Tsinghua University's Institution for interdisciplinary Information Sciences chose EasyStack as its partner in building of this OpenStack Cloud Platform.

Requirement and Proposal Design

To design a suitable cloud platform with existing hardware for the Institution's data center; consisting of network cards/SSD/SATA discs, etc., all of which meeting OpenStack hardware configuration standards. The time frame was one week of downtime during the summer break, to complete the deployment and roll out of the cloud platform.

HA

As an internal public cloud platform within the Tsinghua University, the first round of OpenStack deployment had 125 physical nodes, considering cluster size and access load, a five-nodes HA solution was used, scalable according to future needs. By initiating practices such as clustering/host backup/load balance, MySQL/Message Queue/API services/Web access and other service were all configured to achieve HA.

Storage

Improvements on OpenStack services resulted in Nova/Glance/Cinder all using Ceph RBD as the storage backend, allowing cloud resources such as host/snapshot/cloud hard drives to be accessible within seconds. The Ceph storage cluster was programmed to have three copies, ensuring data HA. To fully utilize physical machine resources, SSD/SATA discs and quintillion network cards are added to the computing nodes and storage-compute convergence employed through Cgroup/taskset and other practices to perform resource isolation of computing and storage processes, ensuring their stability and avoiding interference.

Network

Configured with the WLAN+OVS method to reasonably allocate traffic for different network planes and reserve enough VLAN ID domains for the production network, allowing professors and students to build their own private networks. L3 agents were assigned to distribute workloads onto the five controller nodes. The configurations on it will automatically reschedule to other nodes when L3's down to minimize network breakdown. South-North and East-West network traffics were also restrained reasonably.

Account Management

After the OpenStack cluster deployment is complete, Keystone configuration files were fine-tuned to dock with LDAP, which will then present them in the user logs. Adjustments to existing LDAP user accesses would result in users seeing only the authorized pages upon login. The OpenStack correlation scripts will allow for newly-created LDAP users instant access to the cloud platform after ID creation. Differentiating between faculties and schools by grouping in the LDAP assigned one "enterprise" administrator for each faculty or school to manage the particular user/project/billing, etc.

Billing

With enhancement on Ceilometer allowed for the billing system to perform precise cloud resource calculations instantaneously. Operating with the ID management system, each faculty or school is billed separately as a billing unit. Pricing is adjustable by the cloud administrator at any time, and can be designated to take effect on any set date. AliPay and other online payment methods are used.

Node deployment plan

Through coordination with network plane traffic and isolating access loads from different networks, access fluidity and speed is ensured. Specifically, on the compute-storage convergent node, two quintillion network cards are used for load balance bundles to increase network throughput and ensure Ceph backup copy's write-in speed.

Project Characteristics

High Availability

HA, as well as EasyStack ESCloud's highly reliable design and its subsequent deployment has differentiated and set itself as the leader to other OpenStack releases. This offering can be used to carry large-scale cloud computing environments in critical business operations, easily meeting business challenges.

Open, compatible, device-salvaging

The ESCloud cloud management platform is highly open, and is independent of the different vendors at the device layer, allowing the Institution to still leverage elements of the old infrastructure.

Performance optimization

The ESCloud from EasyStack digs deep in optimizing platform computing, storage and network capabilities, begin from base-layer technologies such as KVM, OVS, and Ceph to conduct source code level optimization, finally providing hardware capabilities on computing, storage and network close to that of physical machines, providing to this project high-performance services on Hadoop, Spark, Docker and invasion tests, fulfilling all research needs.

Multi-level access control

Operation and maintenance wise, ESCloud provided three levels of access management. Each research group or school/faculty can allocate resources according to actual needs, then manage it. Coupled with precise and detailed billing functions, this greatly lowered overall operational costs.

The end product, the OpenStack production-educational-research cloud platform project at the Tsinghua University Institution for Interdisciplinary Information Sciences had 6,000 cores, with 16TB memory and 540TB storage, making it the largest production/education/research all-in-one OpenStack cloud platform.

This project has reduced the Institution's data center procurement cost by 60%; its time spent on daily IT management by 33%, and utility costs by 80%.

In Closing

"EasyStack helped the Institution to use the ESCloud all-open sourced cloud computing solution to build the largest production/education/research all-in-one OpenStack cloud environment. This environment is now being used in big data & analytics, teaching and researches of distributed educational system while providing paid cloud services to other faculties and schools. The ESCloud has greatly reduced procurement and operational costs, enabling precious staff manpower for more worthwhile research projects," said the Assistant Dean of the Tsinghua University Institution for Interdisciplinary Information Sciences.

SOURCE OpenStack.cn