Skip to content

Implement WMS workflow for atmospheric profile production

Summary

We need to prepare the calibration pipeline part of the WMS workflow that should trigger a new simulation based on the updated atmospheric profile. The notes from the relevant discussion are posted here: https://notes.desy.de/uZmwzNRuTqCgnbPmIz6-BQ#

What is the expected correct behavior?

We need to provide the workflow file in CWL and DIRAC YAML dialects. The workflow should be flexible and scalable and may contain the following blocks:

  • [optional] update externally provided atmosphere-related information (e.g. query GDAS)
  • compute a new atmospheric profile
    • side questions: how do we handle atmosphere-data association? To consider later, do we relate atmosphere to an observation block? To a time period?
  • [optional] store the new atmospheric profile in the calibration database. See the side questions above on the DB keys construction
  • [optional] retrieve the new atmospheric profile from the calibration database

Subsequent workflow steps must be handled by WMS/SimPipe

Relevant logs and/or screenshots

CWL documentation: https://www.commonwl.org/user_guide/introduction/quick-start.html#hello-world DIRAC WL examples: https://gitlab.cta-observatory.org/cta-computing/dpps/CTADIRAC/-/tree/main/src/CTADIRAC/ProductionSystem/config DIRAC WL documentation will be provided by the WMS team