Back to Cuneiform Digital Library Initiative (CDLI)
GSoC 2026

Search & Discovery Improvements

CDLI hosts over 340k cuneiform artifacts used by researchers worldwide. The current search stack runs a nightly Logstash pipeline that rebuilds the entire index from scratch, meaning approved edits can take up to 24 hours to appear in search results. Publications, collections, proveniences, and periods have no search infrastructure at all they query the database directly. This project replaces the nightly rebuild with event-driven incremental indexing using a Transactional Outbox pattern, so approved edits appear in search within minutes. It migrates the stack from Elasticsearch to OpenSearch, builds a PHP document builder to replace the existing Ruby transform, brings publications, collections, proveniences, and periods into OpenSearch, and implements cascade hooks so changes to shared entities automatically trigger reindexing of all affected artifacts. A full reindex command and admin UI, handle the initial migration and ongoing recovery scenarios.

Project details

Contributor

harsh_dev

Mentors

Not available

Technologies

Not listed in the archive