Skip to content

Incorrect makedirs invocation in cat-B calibration tool

Summary

Looks like cleanup of the grib files is done incorrectly, or the makedirs tries to recreate a directory with improperly matching permissions (or permissions of the files inside the directory doesn't match to the directory)

Steps to reproduce

Run uc-dpps-cp-110_cat_b.cwl workflow

What is the expected correct behavior?

The workflow shall succeed

Relevant logs and/or screenshots

$ cwltool $WORKFLOW.cwl $WORKFLOW.cfg
INFO /root/micromamba/envs/cta-dev/bin/cwltool 3.1.20230624081518
INFO Resolved 'uc-dpps-cp-110_cat_b.cwl' to 'file:///builds/cta-computing/dpps/calibrationpipeline/calibpipe/calibpipe/tests/workflows/atmosphere/uc-dpps-cp-110_cat_b.cwl'
INFO [job uc-dpps-cp-110_cat_b.cwl] /tmp/qncf2cl9$ select_ref_model_full \
    --das \
    GDAS \
    --date \
    '27-06-2023 02:00:00' \
    --date_format \
    '%d-%m-%Y %H:%M:%S' \
    --degrees_below_horizon \
    18 \
    --grib_download_path \
    /tmp/qncf2cl9/testarea \
    --site \
    north \
    --request_conf \
    /tmp/yblgfwq0/stg8546e028-cc1c-4d90-a672-2e7d32a9b458/control_gdas.json \
    --waiting_time \
    7200 > /tmp/qncf2cl9/output.txt
2023-06-27 14:33:18,904 ERROR [calibpipe.SelectMDP_full_catB] (tool.run): Caught unexpected exception: [Errno 17] File exists: '/tmp/qncf2cl9/testarea'
Traceback (most recent call last):
  File "/root/micromamba/envs/cta-dev/lib/python3.9/site-packages/ctapipe/core/tool.py", line 426, in run
    self.start()
  File "/root/micromamba/envs/cta-dev/lib/python3.9/site-packages/calibpipe/tools/selection_reference_model_cat_b.py", line 38, in start
    status = self.comp.request_rda_data()
  File "/root/micromamba/envs/cta-dev/lib/python3.9/site-packages/calibpipe/atmosphere/molecular_atmosphere_calibrator.py", line 544, in request_rda_data
    os.makedirs(dest_dir, exist_ok=True)
  File "/root/micromamba/envs/cta-dev/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/tmp/qncf2cl9/testarea'
INFO [job uc-dpps-cp-110_cat_b.cwl] Max memory used: 103MiB
WARNING [job uc-dpps-cp-110_cat_b.cwl] exited with status: 1
WARNING [job uc-dpps-cp-110_cat_b.cwl] completed permanentFail

Environment

  • Version used: py 3.9, main
  • Operation System: