mirror of
https://github.com/FlipsideCrypto/sui-models.git
synced 2026-02-06 15:16:45 +00:00
8f7e87b71a
This change introduces a separate backfill data pipeline to ingest historical Sui blockchain data from an alternative source while preserving the existing real-time ingestion pipeline. Architecture: - Bronze layer: View-based models pointing to streamline.sui backfill tables - Silver layer: Table-materialized transformations with backfill-specific JSON parsing - Gold layer: UNION ALL queries combining real-time and backfill data streams The backfill pipeline handles different JSON structures from the alternative data source, particularly for transaction and checkpoint parsing. Gold models now merge both data streams without modifying the incremental processing logic for the primary real-time pipeline. Current scope: Epoch 628, checkpoints < 96605300 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| core | ||
| defi | ||
| labels | ||
| prices | ||