Skip to content

DB integration

Mykhailo.Dalchenko requested to merge db-integration into main

Description

The aim of this MR is to bring the necessary DB functionality for the Molecular Atmosphere calibration, as well as to provide templates and guidelines for DB integration for the other modules of the calibration pipeline.

A current draft of the DB schema to be implemented looks as follows:

calibpipe_db_mol_atmo

To start, only the two rightmost tables are considered: Observatory and Season. These two tables make use of almost all of the added features: they use dimensional data, dimensionless data, implement primary and foreign keys and have one-to-many relationship. In order to provide this implementation, the following functionality have been implemented:

  • Update units handling and types
    • Store units in PG column comment
    • Convert units to those indicated in the SQLTableInfo
      • N.B.: the order of quantities in the Container and SQLTableInfo must be strictly the same!
    • Add 1,2, and 3-D Float arrays to types
  • Implement constraints handling at the SQLTableInfo level
  • SCD policy with start/end effective dates

Current implementation provides (and tests) only writing functionality, but not reading. The reading will be added in subsequent commits. However, a current draft is sufficient to review the foreseen functionality.

Related Issue

Closes #70 (closed) and #71 (closed)

How Has This Been Tested?

Tested in CI.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Merge request reports