This workshop will introduce the Julia package Dagger.jl (https://github.com/JuliaParallel/Dagger.jl).
Designed specifically for HPC and inspired by the Python library Dask (https://docs.dask.org/en/latest/), Dagger is a distributed framework with a scheduler built on top of the Distributed standard library for efficient parallel and out-of-core executions.
Dagger represents operations as a directed acyclic graph or DAG (https://en.wikipedia.org/wiki/Directed_acyclic_graph).
Multi-threading, multiple processes, and GPUs are supported.
When your code is complex with a large number of moving parts and you don't want to spend time optimizing data transfers, workers workloads, and every degree of parallelism possible, the Dagger scheduler does a good job at doing this for you. It allows to scale to any number of processors and to use GPUs when they are available. Checkpoints are also easy to create if you need to interrupt and resume computations. Finally, Dagger allows debugging, visualizing where your tasks are being scheduled, and how well available resources are being utilized.
Speaker: Marie-Helene Burle
