Skip to content

NCAR data can't be downloaded

Summary

request_das tool fails with Not valid request, please retry. immediately after launch

Steps to reproduce

Run the mentioned tool (reproduced in CI and locally)

What is the expected correct behavior?

The tool should download the requested data or produce a better error message. The current one doesn't help much.

Relevant logs and/or screenshots

JSON request content:

{"dataset": "ds083.2", "date": "202301291800/to/202301300000", "datetype": "init", "param": "HGT/PRES/TMP/R H/P WAT/A PCP/U GRD/V GRD/T CDC/LANDN/TOZNE", "level": "ISBL:1000/975/950/925/900/850/800/750/700/650/600/550/500/450/400/350/300/250/200/150/100/50/20", "nlat": 29, "slat": 29, "elon": -18, "wlon": -18, "product": "Analysis"}

Log:

request_das --grib_download_path=download_dir --debug
2023-01-30 11:24:03,392 INFO [calibpipe.DataRequest] (tool.initialize): Loading config from '[]'
2023-01-30 11:24:03,392 INFO [calibpipe.DataRequest] (tool.initialize): ctapipe version 0.17.0
2023-01-30 11:24:03,392 INFO [calibpipe.DataRequest] (tool.run): Starting: DataRequest
2023-01-30 11:24:03,448 INFO [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (calibrator.__init__): Calibrator class of type: MolecularAtmosphereCalibrator has been created.
Calibration method: Undefined
Calibration version: Undefined
2023-01-30 11:24:03,449 DEBUG [calibpipe.DataRequest] (tool.run): CONFIG: {'DataRequest': {'config_files': [], 'log_config': {}, 'log_datefmt': '%Y-%m-%d %H:%M:%S', 'log_file': None, 'log_file_level': 'INFO', 'log_format': '[%(name)s]%(highlevel)s %(message)s', 'log_level': 10, 'logging_config': {}, 'provenance_log': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/calibpipe/tests/workflows/atmosphere/DataRequest.provenance.log'), 'quiet': False, 'show_config': False, 'show_config_json': False, 'MolecularAtmosphereCalibrator': {'co2_file': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/calibpipe/tests/workflows/atmosphere/macobac.csv'), 'coordinates': [[-24, -70], [29, -18]], 'corsika_mp_file': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/calibpipe/tests/workflows/atmosphere/atmotest.dat'), 'das': 'GDAS', 'date': 'not_set', 'end_summer_Atacama': [1, 4], 'end_summer_LaPalma': [1, 10], 'end_winter_Atacama': [1, 10], 'end_winter_LaPalma': [1, 3], 'grib_download_path': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/calibpipe/tests/workflows/atmosphere/download_dir'), 'gribfile': 'test.grib2', 'gridstep': [1.0, 0.75], 'keeling_curve': 'https://scrippsco2.ucsd.edu/assets/data/atmospheric/stations/in_situ_co2/monthly/monthly_in_situ_co2_mlo.csv', 'mdp_file': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/calibpipe/tests/workflows/atmosphere/contemporary_mdp.dat'), 'merged_grib_file': 'merged_grib_file', 'reqfile': PosixPath('/Users/mdalchen/work/dpps/code/code-prototypes/calibpipe/calibpipe/tests/workflows/atmosphere/test.json'), 'site': 'north', 'start_summer_Atacama': [1, 12], 'start_summer_LaPalma': [1, 5], 'start_winter_Atacama': [1, 6], 'start_winter_LaPalma': [1, 12], 'textfile': 'test.txt', 'timeseries_limits': [18, 0], 'waiting_time': 7200}}}
2023-01-30 11:24:03,449 INFO [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.date_request): 202301291800/to/202301300000
2023-01-30 11:24:03,450 INFO [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.create_request): {'dataset': 'ds083.2', 'date': '202301291800/to/202301300000', 'datetype': 'init', 'param': 'HGT/PRES/TMP/R H/P WAT/A PCP/U GRD/V GRD/T CDC/LANDN/TOZNE', 'level': 'ISBL:1000/975/950/925/900/850/800/750/700/650/600/550/500/450/400/350/300/250/200/150/100/50/20', 'nlat': 29, 'slat': 29, 'elon': -18, 'wlon': -18, 'product': 'Analysis'}
2023-01-30 11:24:03,451 INFO [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.create_request): {'dataset': 'ds083.2', 'date': '202301291800/to/202301300000', 'datetype': 'init', 'param': 'HGT/PRES/TMP/R H/P WAT/A PCP/U GRD/V GRD/T CDC/LANDN/TOZNE', 'level': 'ISBL:1000/975/950/925/900/850/800/750/700/650/600/550/500/450/400/350/300/250/200/150/100/50/20', 'nlat': 29, 'slat': 29, 'elon': -18, 'wlon': -18, 'product': 'Analysis'}
2023-01-30 11:24:04,601 ERROR [calibpipe.DataRequest.MolecularAtmosphereCalibrator] (molecular_atmosphere_calibrator.request_rda_data): Not valid request, please retry. Exiting. Response status error
2023-01-30 11:24:04,601 DEBUG [calibpipe.DataRequest] (tool.write_provenance): PROVENANCE: '[]'

Environment

  • Version used: b761a0eb
  • Operation System: OS X