Update bronze backfill models to use source() function instead of direct
table references. This improves maintainability and allows for environment-
specific configuration through the existing bronze_streamline source definition.
Changes:
- Add checkpoints_backfill, transactions_backfill, events_backfill to sources.yml
- Update all bronze backfill models to use {{ source('bronze_streamline', '...') }}
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This change introduces a separate backfill data pipeline to ingest historical
Sui blockchain data from an alternative source while preserving the existing
real-time ingestion pipeline.
Architecture:
- Bronze layer: View-based models pointing to streamline.sui backfill tables
- Silver layer: Table-materialized transformations with backfill-specific JSON parsing
- Gold layer: UNION ALL queries combining real-time and backfill data streams
The backfill pipeline handles different JSON structures from the alternative
data source, particularly for transaction and checkpoint parsing. Gold models
now merge both data streams without modifying the incremental processing logic
for the primary real-time pipeline.
Current scope: Epoch 628, checkpoints < 96605300
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
new file: models/bronze/prices/bronze__complete_token_asset_metadata.sql
new file: models/bronze/prices/bronze__complete_token_prices.sql
modified: models/descriptions/columns.md
new file: models/descriptions/prices.md
new file: models/gold/prices/price__ez_asset_metadata.sql
new file: models/gold/prices/price__ez_asset_metadata.yml
new file: models/gold/prices/price__ez_prices_hourly.sql
new file: models/gold/prices/price__ez_prices_hourly.yml
new file: models/silver/prices/silver__complete_token_asset_metadata.sql
new file: models/silver/prices/silver__complete_token_asset_metadata.yml
new file: models/silver/prices/silver__complete_token_prices.sql
new file: models/silver/prices/silver__complete_token_prices.yml
modified: models/sources.yml
