Top U.S. Academic Supercomputing Community Gathers at 2022 Frontera User Meeting

Frontera users showcase scientific discoveries in cosmology, hypersonic flow, climate change, and more

TACC Executive Director Dan Stanzione leads a tour of Frontera, one of the fastest supercomputers in the world. The 2022 Frontera User Meeting consisted of scientific presentations and user feedback panels. Photo credit: TACC.

More than 30 researchers and project staff gathered in Austin for the 2022 Frontera User Meeting held at the Texas Advanced Computing Center (TACC), August 4-5. The community of users from across the nation presented scientific talks highlighting their most recent discoveries and exchanged views on how to make the effective use of one of the most powerful supercomputers in the world.

Funded by the National Science Foundation (NSF) to meet the most demanding computational needs of the Nation's scientists and engineers and operated by TACC, Frontera launched in 2019. Over the past 12 months, the system completed over 1.3 million jobs and delivered over 72 million node hours consumed.

The meeting keynote presentation was given by Frontera user Ping Chang, the Louis & Elizabeth Scherck Chair in Oceanography at Texas A&M University. Professor Chang talked about the computational challenges in generating current and future climate simulations for the Coupled Model Intercomparison of the Intergovernmental Panel on Climate Change (IPCC).

Other researchers presented topics on their research supported by Frontera ranging from black holes and galaxy clusters, to hypersonic turbulence, to organ-protective epithelial cells, to superconductivity, and more. Users also exchanged views about the best ways to use this unique computational resource, and they offered feedback to TACC on how to plan for future operations of the resource.

TACC Executive Director Dan Stanzione also discussed plans for the future of Frontera and took input from attendees on their requirements for the next evolution of the NSF's large-scale computing capabilities.

Frontera is the first step in the creation of an NSF Leadership-Class Computing Facility (LCCF). The LCCF will wield 10 times the computing capabilities of Frontera and is anticipated to begin construction in 2024.

At the meeting, Stanzione outlined 21 CSA projects selected to begin work on Frontera for the next two years prior to the launch of LCCF. The projects cover a wide spectrum of scientific disciplines — astronomy, particle physics, turbulence, climate, natural hazards like earthquakes, viral infection, and deep learning for misinformation detection.

Participants and TACC staff at the 2022 Frontera User Meeting met in Austin at TACC.

TACC Executive Director Dan Stanzione gave a ‘State of Frontera Address' at the 2022 Frontera Users Meeting highlighting the numerous accomplishments and successes of the NSF flagship supercomputer supporting science at TACC.

Astrophysicist Yueying Ni from Carnegie Mellon University has used Frontera to develop AI-assisted "super-resolution" simulations of the cosmic web, encompassing clusters of galaxies at scales of millions of light years. Their deep neural network can enhance the resolution of the cosmic web by 512 times and can greatly expedite the next-generation of large cosmological simulations of the universe.

She helped develop the large cosmological hydrodynamic simulation ASTRID on Frontera. It models the evolution of millions of galaxies and supermassive black holes over cosmic history.

"We performed the ASTRID simulation on the Frontera supercomputer. Currently, this is the largest cosmological simulation up to date that covered the epoch of Cosmic Noon, when star formation and supermassive black holes both reached their peak activity," Ni said. Her work shows that deep learning and cosmological simulations can form a powerful combination to model the universe over its full dynamic range.

"Accompanied by the theoretical modeling and simulations, we will soon grasp a more comprehensive understanding of how galaxies and supermassive black holes evolved to the current day universe," Ni said. Her team published their findings in the Proceedings of the National Academy of Sciences May 2021 and a series of articles in the Monthly Notices of the Royal Astronomical Society from 2021 to 2022.

Physicist Fatemeh Khalili-Araghi of the University of Illinois Chicago researches the flexibility mechanism of tight junctions found at epithelial cells that line and protect organs and blood vessels throughout the body, as well as supporting transport of nutrients, ions and water. The tight junctions are multiprotein complexes in cells that serve as points of contact and control the permeability of ions and small molecules between cells, an essential feature in multicellular organisms. Khalili-Araghi uses molecular dynamics simulations on Frontera to investigate the mechanical properties and morphology of tight junction strands.

Aerospace Engineering Professor Daniel Bodony of the University of Illinois Urbana-Champaign studies hypersonic flows, currently focusing on the computational modeling of a Mach 6 flow interacting with a hypersonic vehicle's deflected control surface with a thermal protection system. Bodony uses Frontera to study the response of ablative materials, which undergo pyrolysis to remove heat through outgassing. He's written a computational fluid dynamics code that has scaled to over 224,000 cores of Frontera, with an eye towards solving the temporal and spatial multi-scale challenges that only exascale computing can tackle.

Oceanographer Ping Chang of Texas A&M University is one of the leading climate scientists in the world and a contributing author to three chapters in the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Chang gave the keynote presentation at the 2022 Frontera Users Meeting. His current project on Frontera aims to improve seasonal-to-decadal forecasts for fisheries management in the California upwelling system that supplies nutrients for fish and is impacted by climate change. Using Frontera, his team was among the first in the world to complete long-range IPCC climate simulations at resolution high enough to discern weather extremes such as tropical cyclones and fine-scale ocean features such as coastal upwelling.

Remote sensing expert Claire Porter of the University of Minnesota outlined the work of the EarthDEM project, which extracts 2-meter-resolution digital elevation models (DEMs) from stereoscopic commercial satellite imagery to produce a worldwide series of time-dependent, high-resolution topographic observations. Frontera has provided some of the heavy computational resources needed for the model extraction, resulting in publicly available maps in exquisite detail.

Professor P.K. Yeung of Georgia Tech presented on his research aimed to advance understanding and improve the ability to model the transport of small particles carried in a turbulent fluid flow, such as from a car tailpipe or a sneeze. He's developed particle-tracking algorithms on Frontera using cubic spline interpolation to achieve high accuracy and has been able to obtain good scalability to at least 114,000 cores. Yeung's proposed simulations are working toward record resolution and accuracy, with a particle count exceeding 1 billion.

Physics PhD candidate Akaxia Cruz of the University of Washington researches the potential self-interactions of dark matter. These interactions could solve a discrepancy that astronomers have inferred between dark matter density profiles observed of low-mass galaxies and the density profiles predicted by cosmological simulations. Her work on Frontera simulates the core collapse of very small ultra-faint dwarf galaxies, whose structure is hypothesized to be dominated by dark matter.

Theoretical physicist Yao Wang of Clemson University studies the strong correlations of quantum materials, exotic behaviors that can't be described by classical methods. Wang discussed his discovery in 2021 of near-neighbor attraction in cuprates and its connection to the material's superconductivity. Wang used Frontera to accurately simulate electron correlations and electron-phonon coupling in low-dimension cuprates.

Research meteorologist Keith Brewster of the University of Oklahoma tests high-resolution numerical weather prediction through his work at the Center for Analysis and Prediction of Storms. This work is in collaboration with the National Weather Service of the National Oceanic and Atmospheric Administration and used in their testbeds to improve forecasts of high-impact weather events across the United States. Brewster outlined the Flash Flood and Intense Rainfall (FFaIR) experiment run on Frontera that generates 84-hour forecasts using ensemble weather prediction methods.

Astrophysicist Tom Quinn of the University of Washington studies galaxy formation and the large-scale structure of galaxy clusters, the largest bound objects in the universe. Quinn has developed on Frontera a suite of high resolution, state-of-the art simulations of galaxy formation that account for that influence of dark matter. He used the Charm Nbody Gravity Solver to scale his code efficiently on Frontera using zoom simulations to capture features such as the gravity hydrodynamics and supernova feedback of galaxy clusters.

Space physicist Nikolai Pogorelov investigates the flow of the solar wind and its interaction with the local interstellar medium. He described his work using the HelioCubed next-generation model of mapped multi-blocked grids on Frontera's heterogeneous processors to overcome the challenges in the modeling of turbulent flows of partially ionized plasma governed by magnetohydrodynamic equations for charged particles coupled with the kinetic Boltzmann equations describing the transport of neutral atoms, and with a special treatment of non-thermal ions. He also outlined future work with Frontera that will help interpret the observations of space missions such as IBEX, New Horizons, Voyager, and the Parker Solar Probe.

Astrophysicist Lev Arzamasskiy of the Institute for Advanced Study studies the dynamics of turbulent space and astrophysical plasmas in phenomena such as the solar wind, black-hole accretion flows, and the intracluster medium of galaxy clusters. His work on Frontera further explores turbulence in multi-ion plasmas; the origin and amplification of cosmic magnetic fields; and convective instabilities and turbulence in dilute, magnetized plasmas.

Research associate Eirik Valseth of the Oden Institute for Computational Engineering and Sciences at UT Austin presented work that modeled the environmental impacts of channel deepening on the Texas Gulf Coast. The study area of Aransas Pass serves as a nursery to larvae of red drum, a protected game fish. Valseth used the Advanced Circulation (ADCIRC) model with carefully designed finite element meshes on Frontera to compute circulation patterns over 60-day time spans for three years to compare the current and a future proposed channel, modeling the fish larvae as passive Lagrangian particles. Valseth also used the same model to ascertain the impacts of a deeper channel on hurricane storm surge for Hurricane Harvey, a major hurricane which significantly impacted Port Aransas in 2017. The model results suggest that the deeper channel does not lead to significantly increased storm surge in the area for a hurricane with the characteristics of Harvey.

A user feedback panel gave researchers the opportunity to shape how TACC operates its resources and to provide requirements for the NSF Leadership-Class Computing Facility.

TACC staff fielded questions during the user feedback panel, designed to help ensure that users have a direct voice in the operations and policies of the center. Left to right: Lars Koesterke, Virginia Trueheart, Bryan Snead, and John Cazes.

TACC Director of Advanced Computing Systems Tommy Minyard led a tour of the machine room housing Frontera.

Frontera meeting attendees got a first-hand look at the inner workings of Frontera.

We at TACC enjoyed hosting our distinguished guests and look forward to powering the systems such as Frontera that help them make scientific discoveries that change the world.