DDN Annual High Performance Computing Trends Survey Reveals that Data Storage Has Become the Most Strategic Part of the HPC Data Center

SANTA CLARA, Calif., Nov. 12, 2015 /PRNewswire/ -- DataDirect Networks (DDN) today announced the results of its annual High Performance Computing (HPC) Trends survey, which showed that HPC end-users view data storage as the most strategic part of the HPC data center and that managing mixed I/O performance and rapid data growth remain the biggest challenges for HPC organizations going into 2016.

Conducted by DDN for the third consecutive year, the survey polled a cross-section of end-users managing data intensive infrastructures worldwide representing hundreds of petabytes of storage investment. Respondents included individuals responsible for high performance compute, networking and storage systems from Financial Services, Government, Higher Education, Life Sciences, Manufacturing, National Labs and Oil & Gas organizations. The data under management in each of these organizations is staggering!  Of organizations surveyed:

• 68 percent manage or use more than one petabyte of data storage; and
• More than 25 percent manage or use more than 10 PBs of data storage, which represents a 69 percent increase versus 2014 survey results.

Storage is the fastest growing segment in IT spending in HPC¹, and according to an overwhelming majority of survey respondents (77 percent) data and data storage has now become the most strategic part of the HPC data center as end-users seek to solve data access, workflow and analytics challenges to accelerate time to results.

Tweet This: Mixed I/O is No. 1 #BigData challenge; Data #storage deemed Most Strategic by #HPC in @DDN_Limitless customer survey http://bit.ly/1Pp9RLd

The diverse set of applications driving this ever growing data in large scale environment and analytical workflows places rigorous demands on storage infrastructures and creates unique challenges for HPC users. As the graph below highlights:

• Performance ranks as the number one storage and Big Data challenge by approximately two-thirds (66 percent) of those polled; and
• Mixed I/O performance was cited as the biggest concern by more than half (53 percent) of the respondents, which represented an eight percentage point increase compared with last year's survey results.

As shown below, more than half of respondents (56 percent) identify storage I/O as the main bottleneck in analytics workflows.

These responses illustrate how performance issues have surged as Big Data environments grapple with a proliferation of diverse applications creating mixed IO patterns and stifling the performance of their storage infrastructure.   

Only a small percent of respondents believe today's file systems and data management technologies will be able to scale to Exascale levels, while almost 2/3 of respondents believe new innovation will be required. This belief is exemplified in respondents' views on addressing performance issues:

• 58 percent view burst buffers to be the most likely technology to push storage to the next level as users seek faster and more efficient ways to offload I/O from compute resources, to separate storage bandwidth acquisition from capacity acquisition and to support parallel file systems to meet Exascale requirements.  

Given the intense performance requirements of HPC environments as well as security, data access, workflow management and cost considerations, it is not surprising that according to 75 percent of those surveyed, security and data sharing complexity remain the largest impediments to increased collaboration across sites. This concern is underscored by respondents' overwhelming preference for private clouds:

• By a ratio of 3:1 private clouds were preferred over public clouds for HPC users' existing cloud deployments.

With storage performance a critical requirement for today's large scale and petascale-level data centers, site-wide file systems like those at Oakridge National Lab (ORNL), National Energy Research Scientific Computing Center (NERSC) and Texas Advanced Computing Center (TACC) continue to be a significant infrastructure trend in HPC environments, according to more than two-thirds (67 percent) of HPC customers polled. Site-wide file systems allow architects to either consolidate multiple computers on the same system and/or provide the flexibility to upgrade storage and servers as needed.

"The results of DDN's annual HPC Trends Survey reaffirms the key issues and requisite requirements we hear from HPC users regularly. Handling mixed I/O workloads and resolving I/O bottlenecks in data intensive workloads are critical issues DDN has worked fervently to address within our suite of high performance end-to-end solutions," said Molly Rector, CMO, executive vice president product management and worldwide marketing, DDN. "Survey respondents were clear that data storage has become the most strategic part of the data center, and users rely on DDN to deliver continued technology innovation that enables research and business organizations to accelerate strategic insights and to speed time to results. DDN remains committed to the relentless innovation that raises the bar to meet the surging I/O performance, scale, content distribution and collaboration demands of high performance computing – in ways that are faster, smarter, more efficient and more cost effective than anyone else in the market."

Upcoming SC15 Conference
DDN executives, senior level engineering staff and industry workflow experts will be available next week at the SC15 conference in Austin, Texas, at DDN booth #633 to discuss the latest HPC trends and the DDN technology products and innovations that give HPC users the power to achieve their highest ambitions. To schedule a meeting, click here. 

Supporting Resources

• Survey results available upon request at [email protected]
• Connect with DDN at SC15
• More on DDN HPC Storage Products
• Follow DDN via Blog and Twitter

¹ Intersect360 Research. "Worldwide High Performance Computing 2014 Total Market Model and 2015–2019 Forecast", June 2015.

About DDN
DataDirect Networks (DDN) is the world's leading big data storage supplier to data-intensive, global organizations. For more than 15 years, DDN has designed, developed, deployed and optimized systems, software and solutions that enable enterprises, service providers, universities and government agencies to generate more value and to accelerate time to insight from their data and information, on premise and in the cloud. Organizations leverage the power of DDN technology and the deep technical expertise of its team to capture, store, process, analyze, collaborate and distribute data, information and content at largest scale in the most efficient, reliable and cost effective manner. DDN customers include many of the world's leading financial services firms and banks, healthcare and life science organizations, manufacturing and energy companies, government and research facilities, and web and cloud service providers. For more information, go to www.ddn.com or call 1-800-837-2298.

Media Contacts:
DDN
Michael King
Sr. Director of Marketing
Email: [email protected]
Work: 408-419-2892

IGNITE Consulting, on behalf of DDN
Linda Dellett, 303-439-9398
Kathleen Sullivan, 303-439-9365
[email protected]

©2015 All rights reserved. DDN Storage and DDN are trademarks owned by DataDirect Networks. All other trademarks are the property of their respective owners.

Logo - http://photos.prnewswire.com/prnh/20130223/MM65263LOGO 
Photo - http://photos.prnewswire.com/prnh/20151111/286611
Photo - http://photos.prnewswire.com/prnh/20151111/286612
Photo - http://photos.prnewswire.com/prnh/20151111/286613

SOURCE DataDirect Networks (DDN)

Related Links

http://www.ddn.com