Introduction
The Datahub is an open source metadata aggregator which enables cultural organisations to share records, information and knowledge on the Web using open, standardised exchange formats such as Dublin Core.
Key features
- A JSON based REST API with HATEOS support
- XML Schema validation of ingested records.
- Supports publishing records via OAI-PMH
- Support for LIDO XML and Dublin Core formats
- OAuth support for fine grained access control
Use case
Cultural organisations curate and store expertise and knowledge about the collections in digital formats and databases such as record management systems or collection registration systems. However, making these metadata accessible and reusable for the benefit of their audiences is a challenge.
The lack of an adaptable, maintainable, secure and cost effective infrastructure is a key obstacle. A digital architecture based on direct transfer of metadata between databases and end-user applications like websites and mobile applications, proprietary exchange formats and closed source software tools is a brittle solution.
A durable solution consists of a service oriented architecture based on a loose connection between backoffice systems and databases used to manage and preserve metadata records, and the wide array of end user applications - websites, mobile applications, visualisations, print,... - used to cater to the everchanging needs of cultural audiences.
The Datahub is an open source, web based application that sits between these backoffice systems and end-user applications and ...
- ... ingests records structured using standardised, open formats such as Dublin Core or LIDO XML, from one or multiple datasources.
- ... persistently stores the ingested records in an internal database.
- ... publishes the stored records on the Web via a RESTful API and an OAI-PMH API.
Technical changes at the level of the datasource (i.e. a migration to a different product for records management) do not affect the audience facing consumer applications that reuse metadata to inform, to educate, to tell stories, to create new insights, to bring new user experiences.
Design principles
- The Datahub is build API's first. The application is not meant to cater directly to the specific needs of end-users through userfriendly interfaces. The output are records which can easily be (re)used in consumer applications.
- The Dathahub is focussed on durability. The application should be simple to use and simple to maintain.
- The Datahub has a transitive purpose, not a transformative purpose. Individual records are not changed nor can be edited. The ingested records will be published as-is through. Neither does the Datahub generate new repressentations of the ingested records.
- The Datahub is extensible. Support for new formats and datamodels beyond those used in the cultural domain is easily added.
Technologies
The Datahub is a web based application build with well established open source technologies.
