I Built a Pipeline to Track the Job Market I'm Trying to Break Into.

A data engineering enthusiast documents the process of building a data pipeline to track job market trends for roles like data engineer and analytics engineer. The project involves extracting data from Hacker News and a job board API, using Python for extraction and filtering. The author details challenges encountered, such as pivoting from BigQuery to DuckDB and dealing with a non-functional API before settling on RemoteOK. The core learning objective is implementing dbt snapshots to create a Slowly Changing Dimension (SCD) Type 2 table, which tracks the history of job listings over time.