Project
Agentic Data Quality on Databricks
A metadata-driven data quality pattern on Databricks Free Edition, using AI-assisted checks while keeping the architecture close to production concepts.
- Context
- Public technical project documented on Medium.
- Problem
- Many data quality examples are isolated scripts. The goal was to explore a more platform-oriented approach that can be described through metadata and connected to repeatable execution.
- Solution
- Built a production-aligned pattern around Databricks, metadata-driven rules and AI-assisted validation, keeping the project small enough to explain but close to real platform concerns.
- Impact
- Shows the direction of a reusable data quality framework rather than a one-off validation notebook.
Stack
DatabricksMetadata-driven architectureData qualitySparkAI-assisted validation
Links
This project is useful as a public example of how Rafael approaches data platform work: start with the operating model, express behavior through metadata, and keep implementation details traceable.