Back to DBpedia
GSoC 2026

Databus Python Client: Manifest-Driven Reproducible Workflow Orchestration

The DBpedia Databus Python Client currently supports downloading, deploying, and deleting datasets but lacks reproducibility, workflow orchestration, and format interoperability. This project extends the client with three tightly integrated features: First, a Format and Mapping Conversion Layer (Layers 2 and 3 of the Java client pipeline) that enables on-the-fly conversion between RDF triples, RDF quads, and CSV/TSV formats using rdflib — bringing the Python client to feature parity with the Java Databus Client for the first time. Second, a structured Run Manifest System that records every operation as a fixed-format JSON-LD file using the DataID vocabulary. Manifests support replay, summary, and structured error capture — making any Databus operation fully reproducible from a single file. Third, a declarative Workflow Engine that executes multi-step pipelines defined as YAML files, with dynamic step chaining, per-step error recovery (fail, continue, retry), and automatic manifest generation for every run. Deliverables include a working --convert-format flag with 14 round-trip tests, --manifest flag on all three existing commands with manifest replay and summary commands, a databusclient workflow run command with step chaining and error recovery, five tested real-world example workflows, and full documentation — all fully backward compatible.

Project details

Contributor

Dhanashree Petare

Mentors

Not available

Technologies

Not listed in the archive