Seminar: DistConv and LBANN: Enabling Large-Scale Scientific Machine Learning

→ Europe/London
Description

Join Pier Fiedorowicz from Lawrence Livermore National Laboratory (LLNL) as he introduces DistConv 2.0, a new PyTorch-based framework that enables large-scale distributed training for 3D scientific datasets. Discover how spatial parallelism can unlock larger models, accelerate research, and support the next generation of AI-driven scientific discovery.

What to expect:

  • An overview of DistConv 2.0 and distributed deep learning
  • How spatial parallelism enables training beyond single-GPU limits
  • Integrating scalable workflows into existing PyTorch models
  • A look at ScaFFold, a new benchmark for scientific machine learning
  • Applications at the intersection of AI and high-performance computing

How to join:

Sign up via the link below and you will receive a link to attend online. A recording will be available after the seminar.

Adam Featherstone
Registration
Seminar: How Data Complexity Shapes Machine Learning Performance
    • Welcome, talk and discussion

      Abstract:
      Practical scientific machine learning (SciML) applications, from surrogate models to scientific data analysis, often require processing large, high-resolution volumes produced by simulations. For such inputs, 3D convolutional neural networks can easily produce intermediate activations exceeding 100 GB of memory per sample, pushing the limits of what a single GPU can hold. To address this, LLNL previously developed DistConv (Dryden et al., 2019), a spatial parallelism capability within the Livermore Big Artificial Neural Network (LBANN) framework that partitions each sample across GPUs and exchanges halo regions between them. This talk introduces DistConv 2.0, a pure-PyTorch rewrite that brings this key capability to the PyTorch ecosystem. DistConv 2.0 transparently performs halo exchanges inside standard convolution layers when applied to inputs spatially distributed via our custom DCTensor class, enabling researchers to add spatial parallelism to existing models with minimal code changes. We also highlight ScaFFold, a new scalable SciML benchmark built using DistConv 2.0.

      Bio:
      Pier Fiedorowicz is a machine learning researcher in Lawrence Livermore National Laboratory’s Computing Directorate, where he works at the intersection of applied machine learning and high-performance computing. His current work focuses on scalable model training and inference, and on developing AI agents that can understand and steer scientific simulations. Before joining LLNL in 2022, Pier completed his Ph.D. in cosmology at the University of Arizona, where he applied Bayesian inference and machine learning to modeling large-scale structure through weak gravitational lensing observations.