Databricks Agentic Data Engineering Workshop
2-day onsite intensive · Up to 20 data engineers per cohort
Overview
This workshop teaches data engineering teams to use agentic coding assistants—Claude Code, GitHub Copilot, Codex CLI, Cursor, and others—to build, maintain, and evolve data pipelines, dashboards, and data applications on the Databricks Data Intelligence Platform. We teach the full agentic development lifecycle through the lens of data engineering: Unity Catalog context injection, Databricks AI Dev Kit skills, MCP-based tool access, event-driven backpressure (deterministic tools wired to agent events for imperative self-correction), and production-grade deployment patterns.
The curriculum is tool-agnostic by design—agents come and go, but the context-engineering and backpressure patterns are durable.
Key Outcomes
- Configure the Databricks AI Dev Kit (skills, MCP server, Unity Catalog integration)
- Use agents to build Delta Live Tables, SQL dashboards, and data apps from natural language specs
- Enforce data-specific backpressure: deterministic tools intercept agent output and return imperative schema, quality, and lineage directives
- Deploy data pipelines through staging schemas → blue/green data diff → canary with rollback
- Implement agentic data review with evidence-first PRs (data diffs, lineage impact, cost estimates)
- Build observability into agentic data workflows with MLflow tracking and Unity Catalog audit trails
Who Should Attend
- Data Platform Leads — modernizing data engineering with agent-assisted pipelines
- VP of Engineering / CTO — shipping faster without sacrificing data quality or governance
- Principal / Staff Data Engineers — establishing agentic data engineering standards
- DevSecOps / Platform Engineering — building CI/CD pipelines for agent-generated data systems
Delivery Format
| Element | Detail |
|---|---|
| Pre-Workshop Discovery | 2 × 2-hour online sessions (scoping, customization, stakeholder alignment) |
| Format | Onsite, instructor-led, hands-on |
| Duration | 2 days (16 hours instruction + 4 hours guided lab) |
| Cohort Size | 12–20 data engineers |
| Prerequisites | Python + SQL proficiency; active Databricks workspace; Git fluency; basic CI/CD familiarity |
| Deliverables | Databricks agentic dev environment config, BACKPRESSURE.md spec, agentic CI/CD pipeline |
| Post-Workshop | 30-day Slack access; Databricks-specific follow-up; tool configuration guides |
Packages
| Package | Includes |
|---|---|
| Core 2-Day Workshop | 2 × 2-hr discovery sessions + prep + customization + 2-day onsite + materials + 30-day support |
| Core + Capstone Day | Above + guided build of production-ready agentic data workflow on your stack |
| Multi-Cohort Bundle | 3 workshops within 12 months (mix workshops), each with discovery & customization |
Travel expenses billed at cost for locations outside major metro areas. Discovery sessions conducted online via Zoom/Teams.
Curriculum at a Glance
| Day | Focus | Modules |
|---|---|---|
| Day 1 | Databricks Agentic Environment & The Agentic Data Loop | DB Agentic Landscape, Context Engineering for Data, The Data Loop, Backpressure I |
| Day 2 | Data V&V, DataOps & Production | Backpressure II (E2E/Visual/Perf), Agentic Review & DataOps, DataOps & Blue/Green, Production Workflow |
Platform Requirements
| Databricks Workspace | Unity Catalog enabled; Serverless SQL + Jobs available |
| AI Dev Kit | pip install databricks-ai-dev-kit or equivalent |
| Agent Tools | Claude Code, Copilot, and/or Codex CLI installed locally |
| Git | Repository connected to Databricks Git integration |
| CI/CD | GitHub Actions or Azure DevOps for pipeline exercises |
| Data | Workshop data provided; customer data may be substituted for capstone |
Email us to schedule a discovery call and discuss your data platform's agentic engineering goals.