Data observability is so hot right now...but do you know what's also hot? Using some tried and tested ingredients like Apache Airflow and PyOD to perform painless anomaly detection on your key business metrics. You don't need to run off and buy an (expensive!) subscription for the latest hot data observability Sass offering (there is … Continue reading Painless Anomaly Detection with Apache Airflow
Category: airflow
Airflow “Trigger Dags” Python Script
You have some dag that runs multiple times a day but you need to do a manual backfill of last 30 days. It's 2022 and this is still surprisingly painful with Airflow. The "new" REST API helps and mean's all the building blocks are there but, as I found out today, there can often still … Continue reading Airflow “Trigger Dags” Python Script