An agent is a YAML configuration file that defines when and how to automate a specific data engineering task. Agents can respond to pull requests, run on schedules, react to data stack events, and take actions like creating PRs, posting comments, or sending notifications.This guide covers everything you need to create, configure, and deploy effective agents for your workflows.
The unique identifier for your agent. Used in logs, the web interface, and file naming.Must be lowercase with hyphens (kebab-case). Example: schema-change-handler
A brief explanation of what the agent does. Appears in the agent list and helps team members understand the agent’s purpose.Example: Adapts staging models to source schema changes
Instructions for the agent. Write this like you’re instructing a colleague—be clear about goals but let the agent determine implementation details.The agent has access to:
Your repository files (can read, analyze, and modify)
Your data warehouse (can run queries and analyze data)
Git operations (can create branches, commits, and PRs)
Structure your prompt with clear steps when you want a specific workflow. Use open-ended instructions when you want the agent to figure out the best approach.
List of files to always include in the agent’s context. Useful for style guides, conventions, or reference documentation.Example: [".github/STYLE_GUIDE.md", "docs/conventions.md"]
name: schema-change-handlerdescription: Adapts staging models to source schema changestriggers: - type: event event_name: schema_change_detected source: fivetran filters: schema: raw.salesforceprompt: | A schema change was detected in raw.salesforce. 1. Check what changed (new columns, removed columns, renamed tables) 2. Find the corresponding staging model in models/staging/salesforce/ 3. Update the model to match our naming conventions: - Snake case for all columns - Prefix boolean columns with "is_" or "has_" - Use full words, no abbreviations 4. Run dbt parse to validate the changes 5. Create a PR titled "fix(staging): Adapt to Salesforce schema change" In the PR description, list what changed and why you made each update. If a column was removed, explain the impact on downstream models.restrictions: files: allow: ["models/staging/salesforce/"] git_operations: branch_prefix: "agent/schema-change-"notifications: slack: channel: "#data-alerts" on_success: true on_failure: true
Instead of “check for issues,” specify what issues to look for and what to do when found. Include examples of good and bad patterns when relevant.
Copy
Ask AI
# Vagueprompt: Check this PR# Specificprompt: | Check if new columns in staging models follow our naming convention: - Boolean columns must start with "is_" or "has_" - Date columns must end with "_date" or "_at" - Amount columns must end with "_amount" or "_value"
Provide context and examples
Reference your team’s style guides, documentation standards, or example implementations. The agent can read these files and apply the patterns consistently.
For multi-step workflows, use numbered lists. This helps the agent understand the sequence and dependencies between actions.
Copy
Ask AI
prompt: | 1. First, identify all changed models 2. Then, run dbt parse to validate syntax 3. If validation passes, check downstream dependencies 4. Finally, create a summary comment with findings
Specify the output format
Tell the agent how to present results—should it create a PR, post a comment, update a file, or send a notification? Include what information to include.
Copy
Ask AI
prompt: | Generate a markdown report with: - Summary statistics at the top - Detailed findings grouped by severity - Code snippets showing the issues - Suggested fixes for each item Post this report as a PR comment.
After deploying an agent, monitor its first few runs carefully:
1
Check the run logs
Go to the Runs page in the web interface to see exactly what the agent did. Review the files it accessed, queries it ran, and decisions it made.
2
Refine the prompt
If the agent didn’t behave as expected, update the prompt with more specific instructions. Look for places where your instructions were ambiguous or missing important details.
3
Adjust restrictions if needed
If the agent tried to access files or perform actions it shouldn’t, add restrictions to constrain its behavior. Start permissive and tighten as you understand the agent’s needs.
4
Monitor outcomes over time
Track how often the agent succeeds vs. fails, and whether its actions are helpful. Adjust the prompt and configuration based on real-world usage patterns.
Every agent run is logged with full transparency. You can always audit what happened and why.
Agent configurations must be on your default branch (usually main) to be active. Changes in feature branches won’t trigger agents until merged.
Once pushed to your default branch, agents activate automatically. You’ll see them listed in the web interface and they’ll start responding to their configured triggers.