Expanding HPC Capacity, Performance at Oklahoma State University

When Dr. Dana Brunson of Oklahoma State University sought to find a new HPC solution for the university in 2012 with a newly awarded grant from the National Science Foundation, she faced a number of challenges.

First and foremost, she needed to expand the university’s HPC capacity to support a rapidly growing community of HPC users at Oklahoma State University and across Oklahoma.

Second, she needed to increase HPC performance at the university to help researchers accelerate projects and solve larger, more complex problems in less time.

Third and finally, Dr. Brunson needed to control HPC’s footprint, enriching HPC resources while controlling the physical space, power consumption and cooling required for the infrastructure.

Advanced Clustering Technologies worked with Dr. Brunson to find the best solution for the Oklahoma State University High Performance Computing Center. That solution is “Cowboy,” a 3,048-core cluster built by Advanced Clustering that uses servers based on the Intel Xeon processor E5 family. “Cowboy” provides the center with nine times the capacity of its previous cluster in only two times the physical space.

Expanding HPC Resources to Help Researchers Solve More Complex Problems

The number of researchers using HPC resources at the Oklahoma State University High Performance Computing Center had increased more than eight times in the three years preceding the introduction of “Cowboy.” This rapid growth in HPC users, coupled with the end of hardware support for the previous 512-core cluster, forced the center to expand.

“We needed a new cluster to help us better accommodate current users and eliminate the long wait times they were experiencing, while also planning for projected growth,” said Brunson. “Some workloads have run up to twice as fast on this cluster as on the previous cluster while using the same number of cores. These performance gains help researchers complete projects faster and expand the scope of their research by enabling them to run larger workloads and address more complex problems.”

Oklahoma State University’s new cluster had to meet a diverse set of technical requirements.

“After talking with our users, we realized we had to select a hardware platform that could deliver outstanding performance for both compute- and memory-bound workloads,” Brunson said. “At the same time, we needed a dense architecture that could help us control data center power, cooling and real estate.”

Dr. Brunson’s team secured funds for the new cluster with a substantial grant from the National Science Foundation. Then they began to evaluate processing architectures and other system components. Vendor-provided benchmarks from Advanced Clustering for key applications were crucial in that process.

“We needed to find the optimal combination of raw compute performance, large-scale memory bandwidth, and core density,” says Brunson. “We need to provide large memory capacity per server node so we can accommodate memory-bound applications with the fewest possible nodes. But we also need to provide the raw compute performance per core for compute-bound applications and sufficient core counts for scalable codes.”

Advanced Clustering Technologies recommended servers with the Intel Xeon processor E5 family. The center decided to equip the compute nodes with the Intel Xeon processor E5-2600 product family.

“Prior to the purchase, we worked with Dr. Brunson to understand what her key concerns were with the existing infrastructure and tested different technologies to understand their performance and how it related to the various applications that would be run on Cowboy,” said Jim Paugh, Director of Sales and Marketing at Advanced Clustering. “Those results were provided to Dr. Brunson, who selected what was best for her researchers.”

The 3,048-core cluster build by Advanced Clustering that comprises “Cowboy” improved performance for HPC users at Oklahoma State University, where some biology and physics applications run up to twice as fast on these systems as on the center’s previous-generation Intel Xeon processor–based cluster.

Helping Researchers by Providing a Wide Range of HPC Applications

“Cowboy” uses the CentOS* Linux operating system and offers users a wide range of HPC applications. The center optimizes those applications for the Intel platform with Intel Software Development Tools from Intel® Cluster Studio.

“Our goal is to make this as easy as possible for users,” says Brunson. “By compiling numerous applications for the Intel® architecture, we can help researchers get to work quickly and make the most of their applications while also ensuring that applications are easily available for the broadest possible user base.”

The new cluster is helping to make Oklahoma State University even more attractive to prospective undergraduate and graduate students, postdoctoral candidates, and professors.

“Many of the prospective faculty members are surprised to learn that we are providing a free resource that can be available to them from day one. They don’t have to secure grants to purchase cycles, buy their own machines, or ask graduate students to provide systems administration,” Brunson said. “The center can even help them write grant proposals that incorporate Cowboy into their research projects. Our center exists because of the researchers, and with a powerful new cluster like Cowboy, we can help them reach their goals.”