Simple Example
More Robust Example
Production-ready with anomaly detection and real-time monitoring:2. Identify expensive queries
For each query, classify: CRITICAL (> $50):- Single query cost > $50
- Immediate action needed
- Single query cost > $10
- Should optimize
- Scanned > 1 TB
- Might have optimization opportunities
3. Detect patterns
Look for: a) Runaway queries:- Same query running repeatedly
- Check: GROUP BY query_text, COUNT(*)
- If > 100 times/hour: Likely broken
- No WHERE clause on large tables
- Pattern: SELECT * FROM large_table
- JOIN without ON condition
- Result: Massive row explosion
- User queries > 500 times/hour
- Likely auto-refresh set too aggressive
4. Alert based on severity
For CRITICAL issues: Send Slack to #data-incidents: ”🚨 EXPENSIVE QUERY ALERT Cost: $[cost] ([credits] credits) User: [user] Warehouse: [warehouse] Duration: [seconds]s Data scanned: [tb_scanned] TB Query:- Check if query is necessary
- Consider killing: CALL SYSTEM$CANCEL_QUERY(‘[query_id]’)
- Optimize query (add filters, use materialized views)
- Contact user: [user]
5. Daily cost report (9 AM trigger)
Calculate yesterday’s totals:- Queries: [total_queries]
- Data scanned: [total_tb_scanned] TB
- Avg cost/query: $[avg_cost]
- [count_full_scans] queries doing full table scans
- [count_no_limits] queries without LIMIT clauses
- [count_repeated] queries that could be cached”
6. Proactive optimization suggestions
For common patterns, suggest fixes: Full table scan detected: “Your query scans 5 TB but returns 10 rows. Consider adding WHERE clause on date column. Current: SELECT * FROM events Optimized: SELECT * FROM events WHERE event_date >= DATEADD(‘day’, -7, CURRENT_DATE())” Dashboard overuse: “User dashboard_viewer ran same query 500 times today. Consider:- Increasing refresh interval (5 min → 30 min)
- Materializing results
- Using incremental model”
7. Track trends
Compare to last week:- Cost up/down %
- New expensive queries
- Users with cost increases