Technology

MongoDB

Document Oriented Database should be well suited to the type of data we are using.

The dynamic nature of the database (absence of required DB schemas) makes developing with it much easier, as we don’t have to adhere to rigid structures.

This also means, we will start without fixed schemas, to make it easier to test which data shapes work best for our use cases

At a later stage we want to introduce fixed schemas with validation to make the data consistent and create a fail-early environment to prevent subtle bugs

Dashboard Database frontend_data

Collections:

• dataSets: Identifier for dataset generated by pipeline
• projects: Individual projects funded by FFG and FWF
• organizations: Universities or companies participating in the projects
• grants: Grants providing funding to the projects

Schemas

datasets

projects

key_technologies

technology_fields

organisations

grants

Pipeline Database pipelines

Stores status of created pipelines and schedules

Collections:

• pipelines: Pipelines created manually of by schedule
• steps: Individual steps executed by the pipeline
• schedules: Created schedules that will create pipelines in configurable time intervals