Recipe
Airflow DAG skeleton writer
Generate a complete, lint-clean Airflow DAG scaffold from a natural-language pipeline description.
Prompt
Write an Airflow 2.x DAG in Python that: - Runs daily at 06:00 UTC - Extracts data from a Postgres source - Transforms it with a PythonOperator calling pandas - Loads the result into BigQuery - Sends a Slack notification on failure - Uses catchup=False and max_active_runs=1
Output
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.providers.slack.operators.slack import SlackAPIPostOperator
default_args = {
"owner": "meridian",
"retries": 2,
"retry_delay": timedelta(minutes=5),
"on_failure_callback": None,
}
def extract(**ctx):
pass
def transform(**ctx):
pass
def load(**ctx):
pass
with DAG(
dag_id="etl_postgres_to_bigquery",
default_args=default_args,
schedule="0 6 * * *",
start_date=datetime(2025, 1, 1),
catchup=False,
max_active_runs=1,
tags=["meridian", "etl"],
) as dag:
extract_task = PythonOperator(task_id="extract", python_callable=extract)
transform_task = PythonOperator(task_id="transform", python_callable=transform)
load_task = PythonOperator(task_id="load", python_callable=load)
extract_task >> transform_task >> load_taskVariations
- Swap Postgres for MySQL, S3, or Kafka source
- Add a Great Expectations data-quality gate
- Replace Slack with PagerDuty or Opsgenie
- Parameterize schedule with Airflow variables