Skip to main content
This agent automatically detects when source schemas change and updates your dbt staging models to match. When tools like Fivetran or Airbyte send schema change webhooks, the agent updates your staging models with new columns, handles removed columns, regenerates documentation, and creates a pull request for your review.

Simple Example

name: schema-sync
description: Auto-update staging models when source schemas change

triggers:
  - type: event
    source: fivetran
    event_type: schema_change

tools:
  preset: standard  # Needs to write files

prompt: |
  Source schema changed: [table_name from event]
  Added columns: [added_columns from event]
  
  1. Find the staging model for this table
  2. Add new columns with our naming conventions
  3. Run dbt parse to validate
  4. Create PR with changes

More Robust Example

Production-ready with validation and downstream impact analysis:
name: schema-change-handler
description: Automatically adapt staging models to source schema changes

triggers:
  - type: event
    source: fivetran
    event_type: schema_change

tools:
  preset: standard

restrictions:
  files:
    allow: ["models/staging/"]  # Only touch staging

prompt: |
  Schema change detected in [connector_name].[table_name]
  
  ## 1. Understand the change
  Changes from event:
  - Added columns: [added_columns]
  - Removed columns: [removed_columns]
  - Type changes: [modified_columns]
  
  ## 2. Find the staging model
  Look for: models/staging/[connector_name]/stg_[table_name].sql
  
  If not found:
    - Create new staging model from template
    - Include all columns from source
  
  ## 3. Update the model
  
  For ADDED columns:
    - Add to SELECT with proper aliasing
    - Follow naming conventions:
      * snake_case
      * Prefix booleans: is_, has_
      * Suffix dates: _at, _date
    - Add comment explaining column
  
  For REMOVED columns:
    - Check if used downstream (search marts/)
    - If used: Comment out + add deprecation notice
    - If not used: Remove from SELECT
  
  For TYPE CHANGES:
    - Update CAST if needed
    - Add comment about type change
    - Check downstream for compatibility
  
  ## 4. Update documentation
  Update models/staging/[connector_name]/_[table_name].yml:
    - Add descriptions for new columns
    - Mark removed columns as deprecated
    - Update data types
  
  ## 5. Validate
  Run: dbt parse --models stg_[table_name]
  
  If errors:
    - Fix syntax issues
    - Retry validation
  
  ## 6. Check downstream impact
  Search for ref('stg_[table_name]') in marts/
  
  If removed columns are used downstream:
    - Add to PR description
    - Create follow-up issue
    - Alert team in Slack
  
  ## 7. Create PR
  Title: "feat(staging): Sync [table_name] with source schema"
  
  Description:
Source schema changed: [connector_name].[table_name] Changes made:
  • Added columns: [added_columns]
  • Removed columns: [removed_columns]
  • Type changes: [modified_columns]
Downstream impact: [List any marts affected] Automated by Buster schema-sync agent. Review and merge to update staging layer.

## 8. Notify team
Send Slack to #data-pipeline:
"📊 Schema change: [table_name]
 PR created: [link]
 Review needed: [impact summary]"