Student Cluster Competition

Benchmarks

Teams will run a set of benchmarks selected from a suite that includes the following:

LINPACK Benchmark
http://top500.org/project/linpack/

The Linpack Benchmark is a measure of a computer’s floating-point rate of execution. It is determined by running a computer program that solves a dense system of linear equations. It is used by the TOP 500 as a tool to rank peak performance. The benchmark allows the user to scale the size of the problem and to optimize the software in order to achieve the best performance for a given machine. This performance does not reflect the overall performance of a given system, as no single number ever can. It does, however, reflect the performance of a dedicated system for solving a dense system of linear equations. Since the problem is very regular, the performance achieved is quite high, and the performance numbers give a good correction of peak performance.

HPCG Benchmark
http://hpcg-benchmark.org/

The High Performance Conjugate Gradients (HPCG) Benchmark project is an effort to create a new metric for ranking HPC systems. HPCG is intended as a complement to the High Performance LINPACK (HPL) benchmark, currently used to rank the TOP500 computing systems. The computational and data access patterns of HPL are still representative of some important scalable applications, but not all. HPCG is designed to exercise computational and data access patterns that more closely match a different and broad set of important applications, and to give incentive to computer system designers to invest in capabilities that will have impact on the collective performance of these applications.

IO500 Benchmark
http://io500.org

The IO500 benchmark is a benchmark suite for High-Performance IO. It harnesses existing and trusted open-source benchmarks such as IOR and MDTest and bundles execution rules and multiple workloads with the purpose to evaluate and analyze the storage devices for various IO patterns. The IO500 benchmark is designed to provide performance boundaries of the storage for HPC applications regarding data and metadata operations under what are commonly observed to be both easy and difficult IO patterns from multiple concurrent clients. Moreover, there is a phase that scans for previously-created files that match certain conditions using a (possibly file system-specific) parallel find utility to evaluate the speed of namespace traversal and file attribute retrieval. The final score that is used to rank submissions in the list is a combined score across all the executed benchmarks.

Applications

PHASTA
PHASTA performs Parallel Hierarchic Adaptive Stabilized Transient Analysis of compressible and incompressible Navier Stokes equations.

LAMMPS
LAMMPS is a classical molecular dynamics code with a focus on materials modeling. It’s an acronym for Large-scale Atomic/Molecular Massively Parallel Simulator. It can be used to model atoms or, more generically, as a parallel particle simulator at the atomic, meso, or continuum scale.

Mystery Application
At the start of the competition, teams will be given an application and datasets for a mystery application. Students will be expected to build, optimize and run this mystery application all at the competition.

Reproducibility Challenge
One of the applications presented to the student teams is the Reproducibility Challenge, in which students attempt to reproduce results from an accepted paper from the prior year’s Technical Program.

Students have the opportunity to interact directly with the paper’s authors as they attempt to reproduce specific results and conclusions from the paper. As part of this challenge, each student team writes a reproducibility report detailing their experience in reproducing the results from the paper. Authors of the most highly rated reproducibility reports may be invited to submit their reports to a reproducibility special issue.

Full information regarding SCC logistics will be announced at a later date.

Teams are encouraged to include diverse participation including new participants and under-represented groups. To encourage new participants and help new teams participate, “new participant” points will be applied to Team Application evaluations.

Teams must qualify for at least ten (10) “new participant” points in order to be eligible to participate in SCC22, as follows:

• Two (2) points for each first-time SC SCC participant.
• One (1) point for each second-time SC SCC participant.
• Eight (8) points if this will be the team’s first time in any of the major Student Cluster Competitions (SC, ASC and ISC, considering only 2019 and later).
• Three (3) points if this will be the team’s second time in any of the major Student Cluster Competitions (SC, ASC and ISC, considering only 2019 and later).

A full description of the “new participant” points, including some examples, is provided at the Student Cluster Competition website.

Teams will be evaluated according to their responses to the following questions in the Team Application Form:

Strength of Team

Explain why this team would do well in the SCC. Comment on the team’s technical expertise, interdisciplinary and/or scientific background, interest in HPC, and previous experience in similar competitions.

Considerations:

• What HPC and/or computational science experience do team members have?
• How is the team interdisciplinary? What non-computing or domain science disciplines are represented?
• How do members create a team with a broad background of experience relevant to the competition?
• Have any team members participated before?
• How will HPC help team members in their academic careers? List specific reasons for wanting to participate in the competition.
• Introduce the advisor and advisor’s background.

Strength of Hardware and Software Approach

Explain the hardware and software architecture and discuss its feasibility.

Considerations:

• Describe the hardware and software architecture in detail.
• Explain why this architecture will be successful.
• Explain the strategy for running applications and/or optimization during the competition.
• How is the architecture suited for the competition applications?
• How will you manage your system’s administration and application workflow.

Strength of Approach for Software and Cloud Administration

Teams will be given a fixed cloud budget for testing and development and a separate fixed cloud budget for the competition.

Considerations:

• Provide a detailed description of the software architecture you plan to deploy.
• How do you plan to manage your cloud resources?
• Include details on how you will manage the system and your strategy for running the applications.

Strength of Vendor/Institution Relationship

Describe in detail what support will be given by the vendor and supporting institution before and during the competition (e.g., training, travel expenses).

Virtual-only teams who do not have a vendor partner should answer this question with regard to the support your institution will provide.

For in-person teams:

Discuss how the hardware will be provided by the vendor or supporting institution:

• Will the vendor and/or institution provide hardware and fund equipment shipping costs as well as fund other travel and travel-related expenses that are not covered by the competition?
• Will the vendor and/or institution provide resources for pre-competition preparation?
• If the reviewers have questions about the architecture, who can answer those questions from the vendor and team?
• If the team is planning to use new architecture that is not generally available, what is the backup plan if this new architecture is not released or available by the start of the competition?

For all teams:

What other support will the team receive from the vendors and/or the institution?

Strength of Diversity

Describe efforts to broaden participation in under-represented communities for team selection. Because the definition of “under-represented” varies from country to country, authors should include a discussion of the ways in which their team includes under-represented communities in their geographic region. In the United States, for example, some salient issues regarding diversity within advanced computing include the under representation of women as well as racial and ethnic minorities.

Considerations:

• Does the team include meaningful contributions by groups that are traditionally under-represented in the country of the sponsoring institution?
• What efforts were made during the team selection process to approach under represented communities?

If you feel your country or location does not have issues with under-representation in computing, we encourage you to think more deeply or do some research to learn about this issue. Failure to respond to this question, or response with technical breadth of skills will result in points lost.

Team Preparation

Discuss how your team will prepare to use HPC resources and/or learn computational science.

Considerations:

• What courses are available and attended?
• What HPC resources will be used to investigate the applications before vendor hardware arrives? (Please list)
• What is the team’s method for preparing for the competition?
• Do team members have specific roles or duties?
• Provide details on the advisor’s background and the training schedule you have planned to prepare for the competition.

Team Educational Goals

Describe what you hope to learn during the course of your preparation and at the competition.

The full rules are published on the Student Cluster Competition website. Some key details are as follows.

Violation of any rule may result in a team’s disqualification from the competition, or point penalization, at the discretion of the SCC committee. Any unethical conduct not otherwise covered in these rules will also be penalized at the discretion of the SCC Committee.

The following violations will result in immediate disqualification:

• Having anyone other than the 6 registered team members working on the team’s cloud resources during competition hours.
• Any communication between your cloud resource and a network other than the approved cloud networks.

All decisions are the sole discretion of the SCC committee, and SCC committee decisions concerning the rules in a given situation, are final.

Safety First

Equipment configurations, booth layout, and booth occupancy are always subject to safety as first consideration. If a task cannot be done safely, then it is unacceptable. When in doubt, ask an SCC supervisor or team liaison.

Teams

• Advisors are required to be staff, faculty or graduate students of the team’s educational institution(s) or sponsoring HPC center.
• Student Team Members must:
  • Be enrolled in a university or high school
  • Be at least 18 years old by the beginning of the SCC (Monday November 14, 2022)
  • Not have received a bachelor’s degree or equivalent before the beginning of the competition
• Teams are encouraged to include diverse participation including new participants and under-represented groups. To encourage new participants and help new teams participate, “new participant” points will be applied to Team Application evaluations, as described in the full rules on the Student Cluster Competition website.
• Teams must qualify for at least ten (10) “new participant” points in order to be eligible to participate in SCC22.
• Once the competition begins, the six team members must work on the competition tasks with no external assistance – advisors, vendor partners and other supporters must not help the team in any way (other than to occasionally deliver coffee, snacks, etc). Outsourcing of competition tasks to either paid services or unpaid volunteers is not permitted.
• Teams must conduct themselves professionally and adhere to the SC Code of Conduct. Students must compete fairly and ethically.

Hardware

Detailed hardware requirements including specifications for the PDU provided are on the Student Cluster Competition website.

• All hardware used must be commercially available at the time of the start of the competition.
• All competition hardware must be powered through the provided PDU and must stay within the power limit applicable at the time. Other systems (such as laptops and monitors) may be powered from separate non-competition power sources provided by the conference.
• The competition has a dynamic power limit: the power available to each team for their competition hardware at times during the competition may be as high as 4000-watts (but will usually be lower) and may be as low as 1500-watts (but will usually be higher).
• Teams must ensure that their hardware’s power consumption while idle consumes no more than 1500W, and that their hardware can run the applications and benchmarks without consuming more than 4000-watts.
• No changes to the physical configuration are permitted after the start of the competition.

Software

• All system software (operating system, drivers, filesystems, compilers, etc) used in the competition must be publically or commercially available at the start of the competition.
• System software must not be modified after the benchmarking period.
• The benchmark and application executables used in the competition must be built by the team members from open source implementations. Executables may be built in advance by the team members, but teams must provide the URL of the source package (for tarballs etc) or commit hash (for git etc repos). Teams should also be prepared to demonstrate building and running the executable if requested.
• Teams may study and tune the code used in the benchmarks and applications. Any modifications to the source code made by the team must be shared with the SCC committee.

Networking

• A network drop will be provided for outgoing connections only. Teams will NOT be permitted to access their clusters from outside the local network.
• Competition hardware may be connected via wired connections only – wireless access is not permitted.
• Free wireless access for laptops will be available throughout the convention center via SCinet.

Logistics

• Teams are responsible for obtaining their cluster hardware and transporting it to the convention center. (Team vendor partners are encouraged to help their teams with this).
• Teams are responsible for their own travel arrangements to and from the conference, and for daily expenses such as meals (Team vendor partners are encouraged to help their teams with this).

Mandatory Events

• All participants must attend the safety briefing before any unpacking or assembling of hardware, and before participating in the Benchmarking component or any computing tasks in the competition.
• All students must attend the Students@SC Orientation.
• At least one student competitor from each team must attend the daily committee-and-teams stand-up meeting.
• For each team participating in-person, there must be at least 2 student competitors in their team booth at all times while the exhibition floor is open (except during mandatory events scheduled elsewhere).
• For each team participating only in the virtual component, at least 2 student competitors must be in the competition zoom session throughout the competition.