Creating a Data Pipeline to Monitor Local Crime Trends

A data pipeline is created to extract, transform, and load local crime data for visualization. The process uses police log data from the Cambridge, MA police department, accessed via the Socrata Open Data API. The ETL pipeline is orchestrated with Prefect, using Python to extract the data, pandas for validation and transformation, and PostgreSQL for storage. Finally, the processed data is visualized in a Metabase dashboard to monitor crime trends over time. This walkthrough provides code examples and explains the data flow from raw data to the final dashboard.