https://github.com/moosi312-master-thesis/architecture/blob/main/architecture.drawio
https://viewer.diagrams.net/?tags={}&lightbox=1&highlight=0000ff&layers=1&nav=1&title=architecture.drawio&dark=auto#Uhttps%3A%2F%2Fraw.githubusercontent.com%2Fmoosi312-master-thesis%2Farchitecture%2Fmain%2Farchitecture.drawio
We want to visualise the data generated by the Data
Backend to handle queries made by the UI
UI to display data in map/details view
Gather information from different sources and save them in an intermediary database
Take scraped data and normalize it for the dashboard (combine duplicates, bring to standardized form, gather/flag missing/additional data)
Used to start runs of Scraper and Pipeline with different configurations
We want data to be versioned for each scraper/pipeline run, so we can easily compare the results from different configurations.
Data pipeline should be able to choose from different Raw-Data versions Dashboard should be able to choose from different Pipeline runs