Configuration

hydrodatasource uses a central YAML configuration file to manage data paths and connections to remote services. It shares the same file and storage.* format as hydrodataset and hydromodel, so all three repositories resolve against one deterministic contract.

The hydro_setting.yml File

The configuration is managed through a file named hydro_setting.yml located in your user home directory (e.g., C:\Users\YourUser on Windows or /home/YourUser on Linux).

If this file does not exist, hydrodatasource will use a default directory named hydrodatasource_data in your home directory (with a warning).

Unified storage.* Format

storage:
  default_source: local      # 'local' or 'cloud' — the fallback when `source` is not given
  local:
    root: 'D:\data\hydrodatasource'   # The main directory for all your data
  cache: data\cache
  s3:                        # Optional — cloud (MinIO/S3) access
    endpoint_url: 'http://minio:9000'
    key: 'access_key'
    secret: 'secret_key'
    bucket: hydro-data
    prefix: hydromodel

• storage.default_source: The storage backend used by resolve_data_path() / open_dataset() when the caller does not pass source. Either local or cloud.
• storage.local.root: The top-level directory for local data. Registered datasets resolve relative to this root.
• storage.cache: Where cached NetCDF files (generated by SelfMadeHydroDataset) are stored. This speeds up data loading on subsequent runs.
• storage.s3: Cloud (MinIO/S3) credentials and bucket. Used when default_source: cloud (or source="cloud").

How Resolution Uses It

resolve_data_path(dataset_id) / open_dataset(dataset_id) follow this contract, shared with hydrodataset:

1. Look up the dataset id in the registry (HDS_DATASETS + hydrodataset's datasets) to get the reader alias and relative path.
2. Resolve the storage root from storage.local.root (local) or storage.s3 (cloud), per source / default_source.
3. Validate that the resulting absolute path exists.

from hydrodatasource.configs.data_resolver import (
    open_dataset,
    resolve_data_path,
    ResolverContext,
)

# Uses ~/hydro_setting.yml storage.local.root (or storage.s3 when default_source: cloud)
uri = resolve_data_path("songliao_event")
ds = open_dataset("songliao_event")

# Override the storage root without editing hydro_setting.yml
ctx = ResolverContext(storage={"local": {"root": "/your/data/root"}})
ds = open_dataset("songliao_event", ctx=ctx)

Legacy Config

The old local_data_path.root / datasets-origin / datasets-interim format, and the minio.* / postgres.* blocks, are no longer supported. A non-empty hydro_setting.yml that is missing the storage section is rejected with a warning and falls back to the default data root.