Enhancing CrocoLakeTools with IOOS Data Sync from ERDDAP
CrocoLakeTools currently downloads ocean datasets from static sources, either a fixed URL or a fixed file list. This works for stable snapshots like GLODAP, but not for live ERDDAP servers like the IOOS Glider DAC, which hosts hundreds of delayed-mode glider deployment datasets that grow and change over time. This project builds an incremental sync layer targeting the IOOS Glider DAC. The implementation follows a three-layer architecture discussed with the mentor: a CLI script for user interaction, a dataset-specific downloader (DownloaderIOOSGliders) that handles ERDDAP query logic and URL construction using erddapy, and the base Downloader class that provides shared tools used by all downloaders in the package. The sync tool queries the server, compares server-side timestamps against local files, and downloads only what is new or updated. Files are stored exactly as served by ERDDAP with no schema conversion. Deliverables: (1) download_parallel() added to the base Downloader class, (2) DownloaderIOOS base class wrapping erddapy, (3) DownloaderIOOSGliders with incremental sync logic, (4) download_ioos_gliders.py CLI script, (5) unit and integration tests, (6) animal telemetry subclass as a stretch goal.
Project details
Technologies
Not listed in the archive