Improve DAS requests handling

Description

Requests handling and related logging is improved in order to ensure that requests are properly purged.

Currently, some number of requests are still dangling due to manual purge timeout at NCAR. I will keep an eye on it. Please do not retry test pipelines.

Related Issue

Closes #67 (closed), #52 (closed)

How Has This Been Tested?

Tested manually

> request_das --grib_download_path=download_dir --debug
2023-01-30 12:39:50,255 INFO [calibpipe.DataRequest] (tool.initialize): Loading config from '[]'
2023-01-30 12:39:50,255 INFO [calibpipe.DataRequest] (tool.initialize): ctapipe version 0.17.0
2023-01-30 12:39:50,256 INFO [calibpipe.DataRequest] (tool.run): Starting: DataRequest
2023-01-30 12:39:50,329 INFO [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (calibrator.__init__): Calibrator class of type: MolecularAtmosphereCalibrator has been created.
Calibration method: Undefined
Calibration version: Undefined
2023-01-30 12:39:50,331 DEBUG [calibpipe.DataRequest] (tool.run): CONFIG: {'DataRequest': {'config_files': [], 'log_config': {}, 'log_datefmt': '%Y-%m-%d %H:%M:%S', 'log_file': None, 'log_file_level': 'INFO', 'log_format': '[%(name)s]%(highlevel)s %(message)s', 'log_level': 10, 'logging_config': {}, 'provenance_log': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/DataRequest.provenance.log'), 'quiet': False, 'show_config': False, 'show_config_json': False, 'MolecularAtmosphereCalibrator': {'co2_file': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/macobac.csv'), 'coordinates': [[-24, -70], [29, -18]], 'corsika_mp_file': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/atmotest.dat'), 'das': 'GDAS', 'date': 'not_set', 'end_summer_Atacama': [1, 4], 'end_summer_LaPalma': [1, 10], 'end_winter_Atacama': [1, 10], 'end_winter_LaPalma': [1, 3], 'grib_download_path': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/download_dir'), 'gribfile': 'test.grib2', 'gridstep': [1.0, 0.75], 'keeling_curve': 'https://scrippsco2.ucsd.edu/assets/data/atmospheric/stations/in_situ_co2/monthly/monthly_in_situ_co2_mlo.csv', 'mdp_file': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/contemporary_mdp.dat'), 'merged_grib_file': 'merged_grib_file', 'reqfile': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/test.json'), 'site': 'north', 'start_summer_Atacama': [1, 12], 'start_summer_LaPalma': [1, 5], 'start_winter_Atacama': [1, 6], 'start_winter_LaPalma': [1, 12], 'textfile': 'test.txt', 'timeseries_limits': [18, 0], 'waiting_time': 7200}}}
2023-01-30 12:39:50,331 INFO [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.date_request): 202301291800/to/202301300000
2023-01-30 12:39:50,331 DEBUG [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.create_request): Created DAS request:
{
    "dataset": "ds083.2",
    "date": "202301291800/to/202301300000",
    "datetype": "init",
    "param": "HGT/PRES/TMP/R H/P WAT/A PCP/U GRD/V GRD/T CDC/LANDN/TOZNE",
    "level": "ISBL:1000/975/950/925/900/850/800/750/700/650/600/550/500/450/400/350/300/250/200/150/100/50/20",
    "nlat": 29,
    "slat": 29,
    "elon": -18,
    "wlon": -18,
    "product": "Analysis"
}
2023-01-30 12:39:50,331 DEBUG [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.create_request): Created DAS request:
{
    "dataset": "ds083.2",
    "date": "202301291800/to/202301300000",
    "datetype": "init",
    "param": "HGT/PRES/TMP/R H/P WAT/A PCP/U GRD/V GRD/T CDC/LANDN/TOZNE",
    "level": "ISBL:1000/975/950/925/900/850/800/750/700/650/600/550/500/450/400/350/300/250/200/150/100/50/20",
    "nlat": 29,
    "slat": 29,
    "elon": -18,
    "wlon": -18,
    "product": "Analysis"
}
2023-01-30 12:39:51,645 CRITICAL [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.request_rda_data): Request ID can't be retrieved, request can't be purged.Manual intervention is required to purge the request!
Response content:
{
    "status": "error",
    "code": 400,
    "messages": [
        "User has more than 10 open requests. Purge requests before trying again."
    ],
    "result": {},
    "request_start": "2023-01-30T04:39:51.580181",
    "request_end": "2023-01-30T04:39:51.644743",
    "request_duration": "0.064562 seconds"
}
2023-01-30 12:39:51,645 DEBUG [calibpipe.DataRequest] (tool.write_provenance): PROVENANCE: '[]'

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.