[Avg. reading time: 3 minutes]
Medallion Architecture
This is also called as Multi-Hop architecture.
Bronze Layer (Raw Data)
- Typically just a raw copy of ingested data.
- Replaces traditional data lake.
- Provides efficient storage and querying of unprocessed history of data.
Silver Layer (Cleansed and Conformed Data)
- Reduces data storage complexity, latency, and redundancy.
- Optimizes ETL throughput and analytic query performance.
- Preserves grain of original data.
- Eliminates Duplicate records.
- Production schema is enforced.
- Data quality checks and corrupt data are quarantined.
Gold Layer (Curated Business-level tables)
- Powers ML applications, reporting, dashboards, and ad-hoc analytics.
- Refined views of data, typically with aggregations.
- Optimizes query performance for business-critical data.
Different Personas
- Data Engineer
- Data Analysts
- Data Scientists