Workflow Engines: Synergy between Collaborations and Project

Thursday 08/17
3:30 - 5:00pm
Aster 2

Workflow Engines: DESC and the LSST Project have a mutual interest in workflow engines for running pipelines, both DM stack and science, at scale - but they have different needs regarding the computing architectures they need to run on. We’d like to discuss workflow engine strategy, and explore the possibilities of using the same tools. DESC is already running large production runs, and so the Project may be able to learn from this.

Draft agenda:

  •  intro etc - Richard Dubois
  •  DESC DC1 workflows experiences- Tom Glanzman (PhoSim)/Jim Chiang (ImSim)/Tony Johnson (DM L1/L2)
  •  DESC analysis pipeline designs (in progress) - Katrin Heitmann
  •  Progress with Pegasus at NERSC - Tom Glanzman
  •  NCSA workflow engine plans - Pegasus, DESDM - Michelle Gower, Hsin-Fang Chiang
  • DM plans for SuperTask et al - Jim Bosch
 
And we want to leave a goodly bit of time for discussion in our 90' slot.
Organizer: 
Richard Dubois
Suggested Audience: 
DESC, DM

Slides