Simple Example
More Robust Example
Production-ready with validation and downstream impact analysis:- Added columns: [added_columns]
- Removed columns: [removed_columns]
- Type changes: [modified_columns]
name: schema-sync
description: Auto-update staging models when source schemas change
triggers:
- type: event
source: fivetran
event_type: schema_change
tools:
preset: standard # Needs to write files
prompt: |
Source schema changed: [table_name from event]
Added columns: [added_columns from event]
1. Find the staging model for this table
2. Add new columns with our naming conventions
3. Run dbt parse to validate
4. Create PR with changes
name: schema-change-handler
description: Automatically adapt staging models to source schema changes
triggers:
- type: event
source: fivetran
event_type: schema_change
tools:
preset: standard
restrictions:
files:
allow: ["models/staging/"] # Only touch staging
prompt: |
Schema change detected in [connector_name].[table_name]
## 1. Understand the change
Changes from event:
- Added columns: [added_columns]
- Removed columns: [removed_columns]
- Type changes: [modified_columns]
## 2. Find the staging model
Look for: models/staging/[connector_name]/stg_[table_name].sql
If not found:
- Create new staging model from template
- Include all columns from source
## 3. Update the model
For ADDED columns:
- Add to SELECT with proper aliasing
- Follow naming conventions:
* snake_case
* Prefix booleans: is_, has_
* Suffix dates: _at, _date
- Add comment explaining column
For REMOVED columns:
- Check if used downstream (search marts/)
- If used: Comment out + add deprecation notice
- If not used: Remove from SELECT
For TYPE CHANGES:
- Update CAST if needed
- Add comment about type change
- Check downstream for compatibility
## 4. Update documentation
Update models/staging/[connector_name]/_[table_name].yml:
- Add descriptions for new columns
- Mark removed columns as deprecated
- Update data types
## 5. Validate
Run: dbt parse --models stg_[table_name]
If errors:
- Fix syntax issues
- Retry validation
## 6. Check downstream impact
Search for ref('stg_[table_name]') in marts/
If removed columns are used downstream:
- Add to PR description
- Create follow-up issue
- Alert team in Slack
## 7. Create PR
Title: "feat(staging): Sync [table_name] with source schema"
Description:
## 8. Notify team
Send Slack to #data-pipeline:
"📊 Schema change: [table_name]
PR created: [link]
Review needed: [impact summary]"
Was this page helpful?