Use pydantic for metadata serialization?
Lots of complex code is used to ensure metadata is serialized properly, isn't overwritten, and has some type conversion. Particularly this applies to InputDataset
, which I already refactored a bit into a dataclass
. However, this could be a lot simpler if pydanic was used, and would allow for a proper schema for all metadata.