flip/sui-models
1
0
Fork 0
mirror of https://github.com/FlipsideCrypto/sui-models.git synced 2026-02-06 11:06:56 +00:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

silver dex swaps - in prog

Browse Source
This commit is contained in:
Jack Forgash 2025-07-25 10:37:06 -06:00
parent 556a001400
commit 7d3a29e4df
10 changed files with 997 additions and 439 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

197
.claude/README.md
View File

@ -1,197 +0,0 @@
# Building Sui DEX Swaps Model with Claude Code
This document outlines the workflow used to analyze Sui DEX activity and build a comprehensive `silver__dex_swaps` model using Claude Code and the Flipside MCP server.
## Context & Setup
### Project Instructions (CLAUDE.md)
The project includes standardized dbt documentation guidelines in `CLAUDE.md`:
- Every model must have .yml files with standardized structure
- Gold-level tables require 4-section markdown documentation
- All models use incremental materialization with proper predicates
- Required fields: `inserted_timestamp`, `modified_timestamp`, unique `_id`, `_invocation_id`
### Sui-Specific Context (.claude/SUI.md)
A technical reference file providing:
- Blockchain summary (Mysticeti BFT consensus, Move VM, object-centric model)
- DeFi landscape overview (~$2.25B TVL, 61 DEX protocols)
- Core data model schemas from `sui.core` (fact_events, fact_balance_changes, dim_tokens)
- Recommended pipeline patterns for DEX swap analysis
## Workflow Overview
### 1. Initial Analysis Request
**User prompt:**
> "Use the flipside:run_sql_query tool to analyze dex activity on Sui to support the buildout of the defi dex swaps table on Sui."
### 2. Data Discovery Phase
Using the Flipside MCP server, I executed multiple SQL queries to understand the Sui DEX ecosystem:
**Query 1: Initial Event Discovery**
```sql
-- Analyze DEX swap events on Sui to understand patterns
SELECT
DATE_TRUNC('day', block_timestamp) as date,
COUNT(*) as total_swap_events,
COUNT(DISTINCT tx_digest) as unique_swap_txs,
event_address, event_module, event_resource,
parsed_json
FROM sui.core.fact_events
WHERE block_timestamp >= CURRENT_DATE - 7
AND (type ILIKE '%swap%' OR event_resource ILIKE '%swap%')
```
**Key Finding:** Initial results showed mostly liquidity pool deposit/withdraw events, not actual swaps.
**Query 2: Targeted Protocol Analysis**
```sql
-- Look for actual swap events and their event types
SELECT type, event_address, event_module, event_resource,
COUNT(*) as event_count, parsed_json
FROM sui.core.fact_events
WHERE block_timestamp >= CURRENT_DATE - 3
AND (type ILIKE '%SwapEvent%' OR event_resource ILIKE '%SwapEvent%')
ORDER BY event_count DESC
```
**Discovery:** Found the major swap event types including Hop Aggregator's `SwapCompletedEventV2` with clear token type mappings.
**Query 3: Protocol Volume Analysis**
```sql
-- Get top DEX protocols and their swap volumes
SELECT event_address, event_module, event_resource, type,
COUNT(*) as swap_events,
COUNT(DISTINCT tx_digest) as unique_swaps,
CASE
WHEN event_address = '0x1eabed...' THEN 'Cetus'
WHEN event_address = '0x91bfbc...' THEN 'Turbos'
-- ... other protocols
END as protocol_name
FROM sui.core.fact_events
WHERE block_timestamp >= CURRENT_DATE - 30
AND event_address IN (/* major DEX addresses */)
```
**Results:** Identified 6 major protocols with 11M+ swap events over 30 days:
- Cetus: 6.2M events (largest)
- Turbos: 1.7M events
- Bluefin: 1.7M events
- FlowX: 889K events
- Hop Aggregator: 797K events
- DeepBook: 649K events
### 3. Schema Analysis
**Query 4-6: JSON Structure Analysis**
Examined `parsed_json` structures for each protocol to understand data schemas:
- **Cetus**: `{amount_in, amount_out, atob, pool, fee_amount, partner, ref_amount}`
- **Turbos**: `{a_to_b, amount_a, amount_b, fee_amount, pool, recipient}`
- **Hop Aggregator**: `{amount_in, amount_out, type_in, type_out, swapper, router_fee}`
**Query 7: Balance Changes Correlation**
```sql
-- Analyze balance changes for swap transactions to understand token flows
SELECT bc.tx_digest, bc.coin_type, bc.amount, bc.owner,
fe.type as event_type, fe.parsed_json
FROM sui.core.fact_balance_changes bc
JOIN sui.core.fact_events fe ON bc.tx_digest = fe.tx_digest
WHERE bc.amount != 0 -- Only non-zero balance changes
```
**Key Insight:** Balance changes perfectly correlate with swap events - each swap creates exactly 2 balance changes (negative for token out, positive for token in).
### 4. Model Development
**User request:**
> "build a silver model in models/silver/defi/silver__dex_swaps.sql with the recommended structure. Be sure to follow dbt modeling standards and create it as an incremental model"
**Refinement request:**
> "Use the core.fact_events table as the upstream source for this curated model. Use a single CTE to reference the upstream model only once, then use that CTE for each of the dexes."
**Final Implementation:**
- Single `core_events` CTE referencing `core__fact_events`
- Protocol-specific CTEs parsing JSON for each DEX
- Incremental processing with proper clustering
- Standardized output schema across all protocols
## Key Technical Insights
### Protocol Differences
- **Cetus/Turbos**: Native AMM protocols with pool addresses and directional logic
- **Hop Aggregator**: Meta-DEX with explicit token type fields and router fees
- **Bluefin**: Simplified event structure requiring balance change correlation
- **FlowX/DeepBook**: Basic swap events needing additional enrichment
### Data Quality Patterns
- Event types are consistent and reliable for filtering
- `parsed_json` structures are well-formed across protocols
- Balance changes provide ground truth for actual token movements
- Transaction digests provide perfect join keys between events and balance changes
## Model Architecture
```sql
WITH core_events AS (
-- Single reference to upstream core__fact_events
-- Filters for all DEX swap event types
),
cetus_swaps AS (
-- Protocol-specific parsing for Cetus
),
-- ... other protocol CTEs
all_swaps AS (
-- UNION ALL of protocol-specific swaps
)
SELECT -- Standardized output schema
```
## Next Steps
### 1. Model Validation
```bash
# Build the silver model
dbt run -s silver__dex_swaps
# Query and validate results
SELECT platform, COUNT(*) as swap_count,
SUM(amount_in_raw) as total_volume_raw
FROM silver.dex_swaps
WHERE block_timestamp >= CURRENT_DATE - 7
GROUP BY platform
```
### 2. External Validation
- **Compare against DEX scanners**: Validate swap counts and volumes against SuiVision, SuiExplorer
- **Cross-check with DeFiLlama**: Compare protocol volumes with DeFiLlama's Sui DEX data
- **Spot check transactions**: Manually verify specific high-value swaps on block explorers
### 3. Model Completeness
**Current Coverage:** 6 major protocols representing ~90% of Sui DEX volume
**Missing Protocols:**
- Additional AMMs and newer protocols
- Cross-chain bridge swaps
- Orderbook DEXs beyond DeepBook
- Protocol-specific routing contracts
**Enhancement Areas:**
- Token price enrichment for USD volume calculation
- Pool metadata for better swap context
- MEV and arbitrage transaction identification
- Cross-protocol routing analysis
### 4. Gold Layer Development
Build `defi__ez_dex_swaps` with:
- Token symbol/decimal normalization via `dim_tokens`
- USD pricing from hourly price feeds
- Trader address resolution and labeling
- Volume aggregations and analytics-ready fields
## Workflow Benefits
This Claude Code + Flipside MCP approach provided:
- **Rapid iteration** on data discovery queries
- **Real-time insights** into blockchain data patterns
- **Evidence-based model design** using actual transaction data
- **Comprehensive protocol coverage** through systematic analysis
- **Production-ready code** following established dbt standards
The combination of Claude's analytical capabilities with Flipside's comprehensive Sui dataset enabled building a robust, multi-protocol DEX swaps model in a single session.

169
.claude/SUI.md
View File

@ -1,169 +0,0 @@
# SUI.md – Technical & Data-Model Reference
## 1 Blockchain Summary
**Name** – Sui
**Consensus** – Mysticeti BFT delegated-PoS with DAG-based parallel execution (sub-second finality)[^1]
**Ecosystem / VM** – *Sui Move VM* (Move language, object-centric)[^1]
**Design Intent** – High-throughput L1 for gaming, DeFi \& real-time apps[^2]
**Genesis** – 3 May 2023, block 0
**Current Network** – ~0.24 s block time, peak 297 k TPS demo, 2 .1 B txs processed (2024)[^2]
**Native / Gas Token** – SUI (10 B cap; gas, staking, governance)
**Explorers** – suiexplorer.com, suivision.xyz
**Public RPC** – `https://fullnode.mainnet.sui.io:443` (plus QuickNode, Ankr)
**Docs / SDKs** – docs.sui.io, `@mysten/sui.js`, *SuiKit* (Swift)
**Governance** – On-chain token voting (SIP process) via validators \& Sui Foundation
**Economics** – Fixed 10 B supply, fee-funded validator rewards, storage-fund deflation
**Architecture** – L1; horizontal scaling via object-parallelism; no external DA layer
**Interoperability** – Bridges: Wormhole, Circle CCTP (ETH, SOL, EVM chains)[^2]
## 2 DeFi Landscape (July 2025)
* DeFiLlama ranks Sui \#9 by TVL (~\$2.25 B)[^3].
* **61 DEX protocols; 20 hold >\$10 M TVL**.
* Shared Liquidity: **DeepBook** CLOB acts as public good; Cetus CLMM engine reused by multiple DEXes.
* Top DEX TVL snapshot
| Protocol | TVL | Notes |
| :-- | :-- | :-- |
| Suilend | \$752 M | Lending + swap routing |
| NAVI | \$685 M | Lending + AMM |
| Bluefin | \$210 M | CLOB + AMM hybrid |
| Haedal | \$209 M | Liquid staking |
| Cetus | \$105 M | Concentrated-liq AMM |
## 3 Core Data-Models (Flipside *gold.core* schema)[^4]
| Table | Grain / PK | Purpose \& Key Columns (abridged) | Typical Use Cases |
| :-- | :-- | :-- | :-- |
| **core__dim_tokens** | `coin_type` (surrogate `dim_tokens_id`) | One row per fungible token; `symbol`, `decimals`, `description`, `icon_url`, `object_id`. | Token reference joins, labeling balances \& swaps. |
| **core__fact_checkpoints** | `checkpoint_number` | 1 row per checkpoint; `block_timestamp`, `epoch`, `checkpoint_digest`, `network_total_transactions`, `tx_count`, raw `transactions_array`. | Chain-level time-series, throughput \& epoch analytics. |
| **core__fact_transaction_blocks** | `tx_digest` | Aggregate tx-level metrics: sender, kind, fee (`tx_fee`), gas cost breakdown, success flag, dependency list. | Gas-usage, fee-economics, wallet activity, mempool dependency graphs. |
| **core__fact_transactions** | (`tx_digest`,`payload_index`) | Explodes *transaction* array → one record per payload element; captures `payload_type` \& raw `payload_details`. | Fine-grained tx composition analysis, contract call counts. |
| **core__fact_transaction_inputs** | (`tx_digest`,`input_index`) | Explodes *inputs* array; fields include `object_id`, `version`, `type`, mutability, shared-object version. | Object-dependency tracing; shared-object concurrency studies. |
| **core__fact_balance_changes** | (`tx_digest`,`balance_change_index`) | Normalised coin balance deltas: `coin_type`, `amount`, `owner`, sender \& status. | Token flow, DEX swap accounting, wallet PnL. |
| **core__fact_changes** | (`tx_digest`,`change_index`) | Object-level state changes: `object_id`, `object_type`, `version`, `previous_version`, new owner, digest. | NFT lifecycle, object provenance, state-diff analytics. |
| **core__fact_events** | (`tx_digest`,`event_index`) | All emitted Move events with parsed JSON; splits full `type` into `event_address`, `module`, `resource`. | Generic event feed, custom protocol indexing, DEX swap model (see below). |
### Column Map (selected)
```
core__fact_events
checkpoint_number block_timestamp tx_digest tx_kind tx_sender
message_version tx_succeeded event_index
type event_address event_module event_resource
package_id transaction_module sender
parsed_json (VARIANT) inserted_timestamp
```
*(Other tables follow similar timestamp \& surrogate-key pattern.)*
## 4 Building `sui.defi.ez_dex_swaps`
## 🔍 Phase 1: Discovery
### Contract & Event Discovery (REQUIRED)
For each account:
- Use `web.search` or developer APIs to:
- Identify all deployed contracts
- Extract contract names and source code if available
- Determine which event(s) signal a *mint*
📌 **Discovery period is required**: The LLM **must investigate the onchain behavior** of each contract to determine what qualifies as a mint. There is no uniform event across contracts.
Recommended pipeline:
```
WITH swaps AS (
SELECT fe.block_timestamp,
fe.tx_digest,
split_part(fe.type,'::',1) AS protocol_pkg,
fe.parsed_json:value:"amount_in"::number AS amount_in_raw,
fe.parsed_json:value:"amount_out"::number AS amount_out_raw,
bc.coin_type AS token_in,
lead(bc.coin_type) over (part) AS token_out
FROM core.fact_events fe
JOIN core.fact_balance_changes bc
ON fe.tx_digest = bc.tx_digest
WHERE fe.type ILIKE '%SwapEvent'
)
SELECT block_timestamp, tx_digest, protocol_pkg AS platform,
token_in, token_out,
amount_in_raw / pow(10,di.decimals) AS amount_in,
amount_out_raw/ pow(10,do.decimals) AS amount_out
FROM swaps
JOIN core.dim_tokens di ON token_in = di.coin_type
JOIN core.dim_tokens do ON token_out = do.coin_type;
```
Add USD pricing via hourly price table; persist to `gold.defi.ez_dex_swaps`.
## 5 Governance \& Economic Context
The Sui Foundation coordinates grants \& SIP voting; validator-weighted consensus executes upgrades. The fixed-supply model combined with a **storage-fund rebate** makes long-term token velocity deflationary[^2].
*This file distills the latest technical facts, DeFi positioning, and Flipside **gold.core** data-model descriptions needed for rapid analytics development on Sui.*
<div style="text-align: center">⁂</div>
[^1]: https://github.com/MystenLabs/sui
[^2]: https://blog.sui.io/reimagining-bitcoins-role-in-sui-defi/
[^3]: https://flipsidecrypto.xyz/Specter/sui-chain-tvl-breakdown-9uNOUh
[^4]: https://github.com/FlipsideCrypto/sui-models/tree/main/models/gold/core
[^5]: https://github.com/FlipsideCrypto
[^6]: https://www.binance.com/en/square/post/20526573591226
[^7]: https://docs.flipsidecrypto.xyz/data/flipside-data/contribute-to-our-data/contribute-to-flipside-data/model-standards
[^8]: https://github.com/FlipsideCrypto/sql_models
[^9]: https://github.com/FlipsideCrypto/solana-models
[^10]: https://mirror.xyz/0x65C134078BB64Ac69ec66B2A8843fd1ADA54B496/_sOLnSDEjuQTcFV_OiI07m21MSR0QGoF96LRHm5hb78
[^11]: https://popsql.com/blog/dbt-models
[^12]: https://github.com/FlipsideCrypto/core-models
[^13]: https://www.linkedin.com/posts/flipside-crypto_data-ingestion-with-dbt-snowflake-activity-7094753867002716160-pHpq
[^14]: https://github.com/FlipsideCrypto/crosschain-models
[^15]: https://docs.flipsidecrypto.xyz/data/flipside-data/data-models
[^16]: https://stackoverflow.com/questions/75389236/dbt-select-from-model-itself-how-to-transform-this-query
[^17]: https://pypi.org/project/flipside/
[^18]: https://arxiv.org/html/2503.09165v1
[^19]: https://dev.to/pizofreude/study-notes-431-build-the-first-dbt-models-2nl3
[^20]: https://docs.sui.io/guides/developer/coin/in-game-token
[^21]: https://sui.io
[^22]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__dim_tokens.sql
[^23]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__fact_balance_changes.sql
[^24]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__fact_changes.sql
[^25]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__fact_checkpoints.sql
[^26]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__fact_events.sql
[^27]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__fact_transaction_blocks.sql
[^28]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__fact_transaction_inputs.sql
[^29]: https://github.com/FlipsideCrypto/sui-models/blob/main/models/gold/core/core__fact_transactions.sql

158
.cursor/rules/dbt-documentation-standards.mdc Normal file
View File

@ -0,0 +1,158 @@
---
description:
globs: models/descriptions/*,*.yml,models/gold/**/*.sql
alwaysApply: false
---
# dbt Documentation Standards
When working with dbt projects, ensure comprehensive documentation that supports LLM-driven analytics workflows. This includes rich table and column descriptions that provide complete context for understanding blockchain data.
## Table Documentation Standards
Every dbt Model must have an accompanying yml file that provides model documentation.
### Basic YML File Format
Every dbt model yml file must follow this basic structure:
```yaml
version: 2
models:
- name: [model_name]
description: "{{ doc('table_name') }}"
tests:
- [appropriate_tests_for_the_model]
columns:
- name: [COLUMN_NAME]
description: "{{ doc('column_name')}}"
tests:
- [appropriate_tests_for_the_column]
```
#### Required Elements:
- **version: 2** - Must be the first line
- **models:** - Top-level key containing the model definitions
- **name:** - The exact name of the dbt model (without .sql extension)
- **description:** - Reference to markdown documentation using `{{ doc('table_name') }}`
- **columns:** - List of all columns in the model with their documentation
#### Column Documentation Format:
```yaml
- name: [COLUMN_NAME_IN_UPPERCASE]
description: "{{ doc('column_name')}}"
tests:
- [test_name]:
[test_parameters]
```
### Table Descriptions
Table documentation must include 4 standard elements, formatted in markdown. As the base documentation file is YML, the table description must be written in a dbt documentation markdown file in the `models/descriptions/` directory. The Table YML can then use the jinja doc block to reference it.
The 4 standard categories (designed for LLM client consumption to aid in data model discovery and selection):
1. **Description** (the "what"): What the model is mapping from the blockchain, data scope and coverage, transformations and business logic applied. DO NOT EXPLAIN THE DBT MODEL LINEAGE. This is not important for the use case of LLM-driven blockchain analytics.
2. **Key Use Cases**: Examples of when this table might be used and for what analysis, specific analytical scenarios and applications
3. **Important Relationships**: How this table might be used alongside OTHER GOLD LEVEL models, dependencies and connections to other key gold models. For example, a table like logs, events, or receipts may contain information from a larget transactions. To build this description, you SHOULD review the dbt model lineage to understand genuine model relationships. You must convert the model name to a database object, for example `core__fact_blocks.sql` = `core.fact_blocks` = `<schema>__<table_name>.sql`
4. **Commonly-used Fields**: Fields most important to condicting analytics. Determining these requires an understanding of the data model, what the columns are (via their descriptions) and how those fields aid in analytics. One way an understanding can be inferred is by analyzing curated models (anything that is not a core model in the gold/core/ directory is curated. Core models clean and map basic data objects of the blockchain like blocks, transactions, events, logs, etc. Curated models map specialized areas of activity like defi, nfts, governance, etc.). Blockchain data is often logged to a data table as a json object with a rich mapping of event-based details.
### Lineage Analysis
Before writing table descriptions:
- Read the dbt model SQL to understand the logic
- Follow upstream dependencies to understand data flow
- Review source models and transformations
- Understand the business context and use cases
- Review the column descriptions to estabish an understanding of the model, as a whole.
- At the gold level, an ez_ table typically sources data from a fact_ table and this relationship should be documented. Ez_ tables add business logic such as (but not limited to) labels and USD price information
## Column Documentation Standards
### Rich Descriptions
Each column description must include:
- Clear definition of what the field represents
- Data type and format expectations
- Business context and use cases
- Examples where helpful (especially for blockchain-specific concepts)
- Relationships to other fields when relevant
- Any important caveats or limitations
### Blockchain-Specific Context
For blockchain data:
- Reference official protocol documentation for technical accuracy. Use web search to find official developer documentation for the subject blockchain.
- Explain blockchain-specific concepts (gas, consensus, etc.)
- Provide examples using the specific blockchain's conventions
- Clarify differences from other blockchains when relevant
### YAML Requirements
- Column names MUST BE CAPITALIZED in YAML files
- Use `{{ doc('column_name') }}` references for consistent desciption across models. The doc block must refer to a valid description in `models/descriptions`
- Include appropriate tests for data quality
## Examples of Good Documentation
### Table Documentation Example
```markdown
{% docs table_transfers %}
## Description
This table tracks all token transfers on the <name> blockchain, capturing movements of native tokens and fungible tokens between accounts. The data includes both successful and failed transfers, with complete transaction context and token metadata.
## Key Use Cases
- Token flow analysis and wallet tracking
- DeFi protocol volume measurements
- Cross-chain bridge monitoring
- Whale movement detection and alerts
- Token distribution and holder analysis
## Important Relationships
- Subset of `gold.transactions`
- Maps events emitted in `gold.events` or `gold.logs`
- Utilizes token price data from `gold.prices` to compute USD columns
## Commonly-used Fields
- `tx_hash`: Essential for linking to transaction details and verification
- `sender_id` and `receiver_id`: Core fields for flow analysis and network mapping
- `amount_raw` and `amount_usd`: Critical for value calculations and financial analysis
- `token_address`: Key for filtering by specific tokens and DeFi analysis
- `block_timestamp`: Primary field for time-series analysis and trend detection
{% enddocs %}
```
### Column Documentation Example
```markdown
{% docs amount_raw %}
Unadjusted amount of tokens as it appears on-chain (not decimal adjusted). This is the raw token amount before any decimal precision adjustments are applied. For example, if transferring 1 native token, the amount_raw would be 1000000000000000000000000 (1e24) since <this blockchain's native token> has 24 decimal places. This field preserves the exact on-chain representation of the token amount for precise calculations and verification.
{% enddocs %}
```
Important consideration: be sure to research and confirm figures such as decimal places. If unknown DO NOT MAKE UP A NUMBER.
## Common Patterns to Follow
- Start with a clear definition
- Provide context about why the field exists
- Include examples for complex concepts
- Explain relationships to other fields
- Mention any important limitations or considerations
- Use consistent terminology throughout the project
## Quality Standards
### Completeness
- Every column must have a clear, detailed description
- Table descriptions must explain the model's purpose and scope
- Documentation must be self-contained without requiring external context
- All business logic and transformations must be explained
### Accuracy
- Technical details must match official blockchain documentation
- Data types and formats must be correctly described
- Examples must use appropriate blockchain conventions
- Relationships between fields must be accurately described
### Clarity
- Descriptions must be clear and easy to understand
- Complex concepts must be explained with examples
- Terminology must be consistent throughout the project
- Language must support LLM understanding
### Consistency
- Use consistent terminology across all models
- Follow established documentation patterns
- Maintain consistent formatting and structure
- Ensure similar fields have similar descriptions

186
.cursor/rules/dbt-overview-standard.mdc Normal file
View File

@ -0,0 +1,186 @@
---
description:
globs: __overview__.md,models/descriptions/*,models/gold/**/*.sql
alwaysApply: false
---
# dbt Overview Standards
When working with dbt projects, ensure comprehensive documentation exists about the project and a base reference to the gold models exists in the `__overview__.md` file for both human and LLM consumption.
## Project __overview__
The file `models/descriptions/__overview__.md` is the entry point to the model description and documentation and must contain a rich description of what the project is.
The file MUST START AND END WITH DBT JINJA DOCS TAGS `{% docs __overview__ %}` and `{% ENDDOCS %}`
## Quick Links Section Requirements
The `__overview__.md` file MUST contain a "Quick Links to Table Documentation" section that provides direct navigation to all gold model documentation. This section must include a simple list, organized by gold schema, with the models and a hyperlink to the model documentation. If there is an existing section like "using dbt docs" that instructs the user on how to navigate dbt docs or a list of links to flipside and dbt, remove it! These are outdated.
### Required Elements:
**Hyperlinks to Gold Model Documentation** - A comprehensive list of all gold models organized by schema. The schema can be inferred from the model name as the slug before the double underscore. For example, `core__fact_blocks` is a model named `fact_blocks` in the schema `core`.
### Gold Model Links Structure:
The quicklinks section must be organized by schema and use the relative link to load the page generated by dbt documentation. The relative link structure is `#!/model/dbt_uniqueId` where dbt's `uniqueId` format is `node_type.project_name.model_name`. All of these `node_types` are `model`. `project_name` is the name of the dbt models project established in `dbt_project.yml` by the `name` variable. `model_name` is finally the name of the model as determed by the name of the sql file OR the value of `name` in the model's associated `.yml` file. For example, a uniqueId for the blockchain's `fact_blocks` model would be `model.<blockchain_name>_models.core__fact_blocks` making the relative URL `#!/model/model.<blockchain_name>_models.core__fact_blocks`.
```markdown
## **Quick Links to Table Documentation**
**Click on the links below to jump to the documentation for each schema.**
### [Schema Name] Tables
**[Model Type Tables:]**
- model_1
- model_2
### CORE Tables
**Dimension Tables:**
- [core__fact_blocks](relative/path/to/model)
**Fact Tables:**
- [model_1](relative/path/to/model)
```
### Schema Organization Rules:
1. **Group by Schema**: Organize models by their schema (core, defi, nft, price, social, governance, etc.)
2. **Use Exact Schema Names**: Use the exact schema names as they appear in the database (e.g., `<blockchain_database>.CORE`, `<blockchain_database>.DEFI`, `<blockchain_database>.NFT`)
3. **Model Type Subgrouping**: Within each schema, subgroup by model type:
- **Dimension Tables:** (dim_* models)
- **Fact Tables:** (fact_* models)
- **Easy Views:** (ez_* models)
4. **Link Format**: Use the exact dbt docs link format: `#!/model/model.[project_name].[schema]__[model]`
5. **Model Naming**: Use the exact model name as it appears in the file system (without .sql extension)
### Implementation Guidelines for Coding Agents:
1. **Scan Directory Structure**: Read `models/gold/` directory to identify all schema subdirectories
2. **Extract Model Names**: For each schema directory, list all `.sql` files and remove the `.sql` extension
3. **Determine Schema Mapping**: Map model names to database schema names:
dbt models in this project utilize a double underscore in the model name to denote schema vs table <schema>__<table_name>.sql:
- `core__fact_blocks` → `<blockchain_database>.CORE.FACT_BLOCKS`
- `defi__ez_dex_swaps` → `<blockchain_database>.DEFI.EZ_DEX_SWAPS`
4. **Categorize Models**: Group models by prefix:
- `dim_*` → Dimension Tables
- `fact_*` → Fact Tables
- `ez_*` → Easy Views
- `udf_*`, `udtf_*` → Custom Functions
5. **Generate Links**: Create markdown links using the proper format
6. **Maintain Order**: Keep models in alphabetical order within each category
### Validation Requirements:
- Every gold model must have a corresponding link
- Links must use the correct dbt docs format
- Schema names must match the actual database schema
- Model names must match the actual file names (without .sql extension)
- Links must be organized by schema and model type
- All links must be functional and point to valid dbt documentation
- Do NOT add a hyperlink to the category headers. Only hyperlink individual models
## XML Tag Requirements
Every `__overview__.md` file MUST include structured `<llm>` XML tags for easy interpretation by an LLM.
```xml
<llm>
<blockchain>[Protocol Name]</blockchain>
<aliases>[Common Aliases]</aliases>
<ecosystem>[Execution Environment or Layer Type, for example EVM, SVM, IBC, Layer 1, Layer 2]</ecosystem>
<description>[Rich 3-5 sentence description of the blockchain, its consensus mechanism, key features, and developer/user benefits including if the blockchain was built for a specific usecase.]</description>
<external_resources>
<block_scanner>[Link to the primary block scanner for the blockchain]</block_scanner>
<developer_documenation>[Link to the primary developer documentation, maintained by the blockchain devs]</developer_documentation>
</external_resources>
<expert>
<constraints>
<table_availability>
<!-- Specify which tables/schemas are available for this blockchain -->
<!-- Example: "Ensure that your queries use only available tables for [BLOCKCHAIN]" -->
</table_availability>
<schema_structure>
<!-- Explain how the database is organized (dimensions, facts, naming conventions) -->
<!-- Example: "Understand that dimensions and facts combine to make ez_ tables" -->
</schema_structure>
</constraints>
<optimization>
<performance_filters>
<!-- Define key filtering strategies for query performance -->
<!-- Example: "use filters like block_timestamp over the last N days to improve speed" -->
</performance_filters>
<query_structure>
<!-- Specify preferred SQL patterns and structures -->
<!-- Example: "Use CTEs, not subqueries, as readability is important" -->
</query_structure>
<implementation_guidance>
<!-- Provide guidelines for advanced SQL features -->
<!-- Example: "Be smart with aggregations, window functions, etc." -->
</implementation_guidance>
</optimization>
<domain_mapping>
<token_operations>
<!-- Map token-related queries to specific tables -->
<!-- Example: "For token transfers, use ez_token_transfers table" -->
</token_operations>
<defi_analysis>
<!-- Specify DeFi-related tables and their use cases -->
<!-- Example: "For DeFi analysis, use ez_bridge_activity, ez_dex_swaps, ez_lending" -->
</defi_analysis>
<nft_analysis>
<!-- Define NFT-specific tables and functionality -->
<!-- Example: "For NFT queries, utilize ez_nft_sales table in nft schema" -->
</nft_analysis>
<specialized_features>
<!-- Cover blockchain-specific features or complex data structures -->
<!-- Example: "The XYZ data is complex, so ensure you ask clarifying questions" -->
</specialized_features>
</domain_mapping>
<interaction_modes>
<direct_user>
<!-- Define behavior for direct user interactions -->
<!-- Example: "Ask clarifying questions when dealing with complex data" -->
</direct_user>
<agent_invocation>
<!-- Specify response format when invoked by other AI agents -->
<!-- Example: "When invoked by another AI agent, respond with relevant query text" -->
</agent_invocation>
</interaction_modes>
<engagement>
<exploration_tone>
<!-- Set the overall tone and encouragement for data exploration -->
<!-- Example: "Have fun exploring the [BLOCKCHAIN] ecosystem through data!" -->
</exploration_tone>
</engagement>
</expert>
</llm>
```
Place these XML tags at the end of the documentation (BUT STILL BEFORE THE JINJA ENDDOCS TAG).
## Update Process for Coding Agents:
To update the overview, the coding agent MUST:
1. **Scan Current Gold Models**:
- Read the entire `models/gold/` directory structure
- Identify all `.sql` files across all schema subdirectories
- Extract model names (remove `.sql` extension)
2. **Generate Updated Quicklinks Section**:
- Follow these implementation guidelines
- Create a complete new quicklinks section with all current gold models
- Maintain proper schema organization and model type grouping
3. **Update __overview__.md**:
- Replace the entire "Quick Links to Table Documentation" section with the newly generated content
- Ensure proper markdown formatting and link structure
- Create or update the XML tag block
4. **Validation Check**:
- Verify all gold models have corresponding links
- Confirm links use correct dbt docs format
- Check that schema names and model names are accurate
- Ensure alphabetical ordering within categories

76
.cursor/rules/general-coding-standards.mdc Normal file
View File

@ -0,0 +1,76 @@
---
description:
globs:
alwaysApply: true
---
# dbt Model Standards for Flipside Crypto
## General Rules
- Follow the existing code style and patterns in the codebase
- Write clear, concise, and well-documented code
- Use meaningful variable, function, column and model names
- Handle errors gracefully and provide helpful error messages
- Test your code thoroughly before submitting
- Follow the existing project structure and conventions
## Code Quality
- Write self-documenting code with clear and consistent names
- Use consistent formatting and indentation
- Implement proper error handling and logging
- Follow DRY (Don't Repeat Yourself) principles
- Use meaningful commit messages
- Use snake_case for all objects (tables, columns, models)
- Maintain column naming consistency through the pipeline
## dbt Model Structure
- Models are connected through ref() and source() functions
- Data flows from source -> bronze -> silver -> gold layers
- Each model has upstream dependencies and downstream consumers
- Column-level lineage is maintained through transformations
- Parse ref() and source() calls to identify direct dependencies
- Track column transformations from upstream models
- Consider impact on downstream consumers
- Preserve business logic across transformations
## Model Naming and Organization
- Follow naming patterns: bronze__, silver__, core__, fact_, dim__, ez__, where a double underscore indicates a break between a model schema and object name. I.e. core__fact_blocks equates to <database>.core.fact_blocks.
- Organize by directory structure: bronze/, silver/, gold/, etc.
- Upstream models appear on the LEFT side of the DAG
- Current model is the focal point
- Downstream models appear on the RIGHT side of the DAG
## Modeling Standards
- Use snake_case for all objects
- Prioritize incremental processing always
- Follow source/bronze/silver/gold layering
- Document chain-specific assumptions
- Include incremental predicates to improve performance
- For gold layer models, include search optimization following Snowflake's recommended best practices
- Cluster models on appropriate fields
## Testing Requirements
- Ensure proper token decimal handling
- Implement unique tests for primary keys
- Implement recency tests for tables that are expected to have frequent data updates
- Add not_null tests for required columns
- Use relationships tests for foreign keys
## Performance Guidelines
- Optimize for high TPS (blockchain data)
- Handle large state change volumes efficiently
- Index frequently queried dimensions
- Consider partition pruning strategies
- Implement appropriate clustering keys
- Optimize database queries for large datasets
- Use appropriate indexing strategies
- Monitor resource usage and optimize accordingly
- Consider the impact of changes on existing systems
## Documentation
- Document data sources
- Map entity relationships
- Include model descriptions in yml files per expanded rule [dbt-documentation-standards.mdc](mdc:.cursor/rules/dbt-documentation-standards.mdc)
- Document column descriptions and business logic
- Explain incremental logic and predicates
- Note any data quality considerations

220
.cursor/rules/review-dbt-documentation.mdc Normal file
View File

@ -0,0 +1,220 @@
---
description:
globs:
alwaysApply: false
---
# Review dbt Documentation Process
## Overview
This document outlines the comprehensive process for reviewing and improving column and table descriptions for gold models in dbt projects. The goal is to provide robust, rich details that improve context for LLM-driven analytics workflows, ensuring that documentation is complete, accurate, and self-contained without requiring external expert files.
## Objectives
- Create clear, detailed documentation that supports LLM understanding of blockchain data
- Ensure technical accuracy by referencing official protocol documentation
- Provide rich context for each table and column to enable effective analytics
- Maintain consistency across all models and schemas
- Support automated analytics workflows without requiring expert context files
## Pre-Review Requirements
### 1. Research Phase
**Blockchain Protocol Documentation**
- Search and read official developer documentation for the target blockchain. Utilize web search to find authentic and accurate developer documentation
- Review technical specifications, whitepapers, and API documentation
- Understand the blockchain's consensus mechanism, data structures, and conventions
- Research common use cases and analytics patterns specific to the blockchain
- Identify key technical concepts that need explanation (e.g., gas mechanics, consensus, token standards)
**External Resources to Consult**
- Official blockchain documentation
- Developer guides and tutorials
- Technical specifications and whitepapers
- Community documentation and forums
- Block explorers and API documentation
### 2. Project Context Analysis
- Review the `__overview__.md` file and rewrite it per the @dbt-overview-standard rule to create a summary of what this particular blockchain is, unique characteristics, and any other general information about the chain itself
- Review existing documentation patterns and terminology
- Understand the data flow and model lineage structure
## Review Process
### Step 1: Model Analysis
**SQL Logic Review**
- Read the dbt model SQL file to understand the transformations and business logic
- Follow upstream dependencies to understand data flow from source to gold layer
- Review bronze source models, silver staging models, and intermediate transformations
- Identify any complex joins, aggregations, or business logic that needs explanation
- Understand the incremental logic and any filtering conditions
**Lineage Analysis**
- Map the complete data lineage from source to gold model
- Identify key transformations and their purposes
- Understand relationships between related models for the sole purpose of generating a robust description
- Do not include data lineage analysis in the table description
### Step 2: Column Description Review
**Individual Column Analysis**
For each column in the model:
1. **Technical Understanding**
- Read the SQL to understand how the column is derived
- Check upstream models if the column comes from a transformation
- Understand the data type and format expectations
- Identify any business logic applied to the column
2. **Blockchain Context**
- Research the blockchain-specific meaning of the column
- Reference official documentation for technical accuracy
- Understand how this field relates to blockchain concepts
- Identify any blockchain-specific conventions or requirements
3. **Documentation Assessment**
- Review existing column description
- Evaluate completeness and clarity
- Check for missing context or examples
- Ensure the description supports LLM understanding
**Required Elements for Column Descriptions**
- Clear definition of what the field represents
- Data type and format expectations
- Business context and use cases
- Examples where helpful (especially for blockchain-specific concepts)
- Relationships to other fields when relevant
- Any important caveats or limitations
- Blockchain-specific context and conventions
### Step 3: Table Description Review
**Current State Assessment**
- Review the updated column descriptions
- Review existing table description in the YAML file
- Evaluate completeness and clarity
- Identify missing context or unclear explanations
**Required Elements for Table Descriptions**
Table documentation must include 4 standard elements, formatted in markdown. As the base documentation file is YML, the table description must be written in a dbt documentation markdown file in the `models/descriptions/` directory. The Table YML can then use the jinja doc block to reference it.
The 4 standard categories are fully defined in the @dbt-documentation-standards rule. They are:
1. **Description**
2. **Key Use Cases**
3. **Important Relationships**
4. **Commonly-used Fields**
### Step 4: Documentation File Review
**Individual Documentation Files**
- Check if each column has a corresponding `.md` file in `models/descriptions/`
- Review existing documentation for completeness and accuracy
- Update or create documentation files as needed
**Documentation File Format**
```markdown
{% docs column_name %}
[Rich, detailed description including:
- Clear definition
- Data format and examples
- Business context
- Blockchain-specific details
- Relationships to other fields
- Important considerations]
{% enddocs %}
```
### Step 5: YAML File Review
**YAML Structure Validation**
- Ensure column names are CAPITALIZED in YAML files
- Verify all columns reference documentation using `{{ doc('column_name') }}`
- Check that appropriate tests are included
- Validate the overall YAML structure
**YAML File Format**
```yaml
version: 2
models:
- name: [model_name]
description: |-
[Clear, direct table description]
columns:
- name: [COLUMN_NAME_IN_UPPERCASE]
description: "{{ doc('column_name') }}"
tests:
- [appropriate_tests]
```
## Review Checklist
### Table Level
- [ ] Table documentation is in markdown file in `models/descriptions/` directory
- [ ] Table YAML references documentation using jinja doc block
- [ ] **Description** section explains what blockchain data is being modeled
- [ ] **Key Use Cases** section provides specific analytical scenarios
- [ ] **Important Relationships** section explains connections to other GOLD models
- [ ] **Commonly-used Fields** section identifies critical columns and their importance
- [ ] Documentation is optimized for LLM client consumption
### Column Level
- [ ] Each column has a comprehensive description
- [ ] Data types and formats are clearly specified
- [ ] Business context and use cases are explained
- [ ] Examples are provided for complex concepts
- [ ] Relationships to other fields are documented
- [ ] Important limitations or caveats are noted
- [ ] Blockchain-specific context is included
### Documentation Files
- [ ] All columns have corresponding `.md` files
- [ ] Documentation files contain rich, detailed descriptions
- [ ] Examples use appropriate blockchain conventions
- [ ] Technical accuracy is verified against official documentation
### YAML Files
- [ ] Column names are CAPITALIZED
- [ ] All columns reference documentation using `{{ doc('column_name') }}`
- [ ] Appropriate tests are included
- [ ] YAML structure is valid
## Implementation Guidelines
### Documentation Writing Tips
- Start with a clear definition of what the field represents
- Provide context about why the field exists and its importance
- Include examples for complex concepts, especially blockchain-specific ones
- Explain relationships to other fields when relevant
- Mention any important limitations or considerations
- Use consistent terminology throughout the project
### Blockchain-Specific Considerations
- Reference official protocol documentation for technical concepts
- Explain blockchain-specific concepts (gas, consensus, etc.)
- Provide examples using the specific blockchain's conventions
- Clarify differences from other blockchains when relevant
- Include information about data freshness and update mechanisms
### LLM Optimization
- Write descriptions that are complete and self-contained
- Use clear, structured language that supports automated understanding
- Include context that helps LLMs understand the data's purpose
- Provide examples that illustrate common use cases
- Ensure descriptions support common analytics workflows
## Post-Review Actions
### Validation
- Verify all documentation is technically accurate
- Check that descriptions are complete and self-contained
- Ensure consistency across related models
- Validate that documentation supports common analytics use cases
### Testing
- Test documentation by having an LLM attempt to understand the data
- Verify that descriptions enable effective query generation
- Check that examples are clear and helpful
- Ensure documentation supports the intended analytics workflows
### Maintenance
- Update documentation when models change
- Review and refresh documentation periodically
- Maintain consistency as new models are added
- Keep documentation aligned with blockchain protocol updates

1
.gitignore vendored
View File

@ -26,4 +26,5 @@ package-lock.yml
# Local Claude configuration
.claude/settings.local.json
.claude/log
CLAUDE.local.md

93
models/descriptions/silver__dex_swaps.md Normal file
View File

@ -0,0 +1,93 @@
{% docs silver__dex_swaps %}
## Description
This table consolidates decentralized exchange (DEX) swap events from multiple Sui blockchain protocols into a standardized format. It captures token swaps across seven major DEX platforms: Cetus, Turbos, Bluefin, Aftermath AMM, FlowX, DeepBook, and Momentum. The model extracts swap events from blockchain events, normalizes the data structure across different protocols, and provides consistent fields for DeFi analytics. Each swap event includes token amounts, fees, pool information, and trader details where available.
## Key Use Cases
- Cross-protocol DeFi volume analysis and comparison
- Token flow tracking and liquidity analysis
- DEX performance benchmarking and market share analysis
- Trader behavior analysis and wallet tracking
- Fee revenue analysis across different protocols
- Token pair popularity and trading volume trends
- Protocol-specific analytics and optimization insights
## Important Relationships
- Sources events from `gold.core.fact_events` filtered by specific DEX event types
- Enriches data with transaction details from `gold.core.fact_transactions`
- Can be joined with `gold.core.dim_tokens` for token metadata and pricing
- Supports downstream analytics in DeFi-specific curated models
- Provides foundation for cross-protocol DeFi dashboards and reports
## Commonly-used Fields
- `platform`: Essential for protocol-specific analysis and filtering
- `amount_in_raw` and `amount_out_raw`: Core fields for volume calculations and swap analysis
- `token_in_type` and `token_out_type`: Critical for token pair analysis and pricing
- `tx_digest` and `event_index`: Primary keys for linking to transaction details
- `block_timestamp`: Key field for time-series analysis and trend detection
- `pool_address`: Important for liquidity pool analysis and pool-specific metrics
- `trader_address`: Essential for wallet tracking and user behavior analysis
{% enddocs %}
{% docs platform %}
The name of the decentralized exchange platform where the swap occurred. Currently supports seven major Sui DEX protocols: Cetus, Turbos, Bluefin, Aftermath AMM, FlowX, DeepBook, and Momentum. This field enables protocol-specific analysis, performance comparison, and market share calculations across different DeFi platforms.
{% enddocs %}
{% docs platform_address %}
The smart contract address of the DEX platform that facilitated this swap. This represents the deployed contract address for the specific DEX protocol on the Sui blockchain. Useful for contract verification, security analysis, and linking to platform-specific metadata and configurations.
{% enddocs %}
{% docs pool_address %}
The address of the liquidity pool involved in the swap. For protocols that use AMM (Automated Market Maker) pools, this identifies the specific pool contract. May be NULL for order book-based protocols like DeepBook or centralized limit order protocols. Essential for pool-specific analytics and liquidity analysis.
{% enddocs %}
{% docs amount_in_raw %}
The raw amount of tokens being swapped in (input amount) before any decimal adjustments. This represents the exact on-chain token amount as it appears in the swap event. Preserves precision for accurate calculations and is essential for volume analysis, price impact calculations, and swap size distribution analysis.
{% enddocs %}
{% docs amount_out_raw %}
The raw amount of tokens being swapped out (output amount) before any decimal adjustments. This represents the exact on-chain token amount that the user receives from the swap. Critical for calculating swap rates, slippage analysis, and understanding the actual token amounts exchanged in each swap.
{% enddocs %}
{% docs a_to_b %}
A boolean flag indicating the direction of the swap within the pool. When TRUE, the swap goes from token A to token B; when FALSE, it goes from token B to token A. This field is protocol-specific and may be NULL for some DEX platforms. Important for understanding swap direction and pool token ordering conventions.
{% enddocs %}
{% docs fee_amount_raw %}
The raw amount of fees charged for the swap transaction. This includes protocol fees, liquidity provider fees, and any other transaction costs. May be 0 for protocols that don't charge explicit fees or when fees are embedded in the swap amounts. Essential for fee revenue analysis and total cost of trading calculations.
{% enddocs %}
{% docs partner_address %}
The address of a partner or affiliate that facilitated the swap (if applicable). Used primarily by Cetus for their partner program where swaps can be routed through partner contracts. May be NULL for most protocols. Useful for tracking partner performance and affiliate program analytics.
{% enddocs %}
{% docs referral_amount_raw %}
The raw amount of referral rewards or commissions generated from this swap. Used by Cetus to track referral program payouts. May be 0 for protocols without referral programs or when no referral was used. Important for referral program analytics and partner performance tracking.
{% enddocs %}
{% docs steps %}
The number of steps or hops required to complete the swap. For simple swaps, this is typically 1. For complex swaps involving multiple pools or routing through multiple protocols, this indicates the number of intermediate steps. Essential for understanding swap complexity and routing efficiency across different protocols.
{% enddocs %}
{% docs token_in_type %}
The full type identifier of the token being swapped in (input token). This follows Sui's Move type format (e.g., "0x2::sui::SUI" for the native SUI token). Essential for token identification, pricing lookups, and token-specific analytics. Used for calculating USD values and token pair analysis.
{% enddocs %}
{% docs token_out_type %}
The full type identifier of the token being swapped out (output token). This follows Sui's Move type format and represents the token that the user receives from the swap. Critical for token pair analysis, swap rate calculations, and understanding token flow patterns across the DeFi ecosystem.
{% enddocs %}
{% docs trader_address %}
The address of the wallet or account that initiated the swap. May be NULL for some protocols that don't explicitly track the trader address in their events. Essential for user behavior analysis, wallet tracking, and understanding individual trader patterns and preferences across different DEX platforms.
{% enddocs %}
{% docs dex_swaps_id %}
A unique surrogate key generated from the combination of tx_digest and event_index. This provides a stable, unique identifier for each swap event that can be used as a primary key for downstream analytics and data modeling. Ensures data integrity and prevents duplicate processing.
{% enddocs %}
{% docs _invocation_id %}
A unique identifier for the dbt run that created or updated this record. This field is used for data lineage tracking and debugging purposes. Helps identify which specific dbt execution was responsible for processing each record and enables traceability back to the source code and configuration used.
{% enddocs %}

217
models/silver/defi/silver__dex_swaps.sql
View File

@ -36,97 +36,132 @@ WITH core_events AS (
type = '0x1eabed72c53feb3805120a081dc15963c204dc8d091542592abaf7a35689b2fb::pool::SwapEvent' -- Cetus
OR type = '0x91bfbc386a41afcfd9b2533058d7e915a1d3829089cc268ff4333d54d6339ca1::pool::SwapEvent' -- Turbos
OR type = '0x3b6d71bdeb8ce5b06febfd3cfc29ecd60d50da729477c8b8038ecdae34541b91::bluefin::BluefinSwapEvent' -- Bluefin
OR type = '0xd675e6d727bb2d63087cc12008bb91e399dc7570100f72051993ec10c0428f4a::events::SwapCompletedEventV2' -- Hop
OR type = '0xd675e6d727bb2d63087cc12008bb91e399dc7570100f72051993ec10c0428f4a::events::SwapCompletedEventV2' -- Aftermath AMM
OR type = '0x25929e7f29e0a30eb4e692952ba1b5b65a3a4d65ab5f2a32e1ba3edcb587f26d::pool::Swap' -- FlowX
OR type = '0xe8f996ea6ff38c557c253d3b93cfe2ebf393816487266786371aa4532a9229f2::settle::Swap' -- DeepBook
OR type = '0x0018f7bbbece22f4272ed2281b290f745e5aa69d870f599810a30b4eeffc1a5e::momentum::MomentumSwapEvent' -- Momentum
)
),
fact_transactions AS (
SELECT
tx_digest,
payload_details
FROM
{{ ref('core__fact_transactions') }}
WHERE
payload_type = 'MoveCall'
AND payload_details:type_arguments IS NOT NULL
{% if is_incremental() %}
AND modified_timestamp >= (
SELECT
COALESCE(MAX(modified_timestamp), '1900-01-01'::TIMESTAMP) AS modified_timestamp
FROM
{{ this }}
)
{% endif %}
),
cetus_swaps AS (
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
event_index,
e.checkpoint_number,
e.block_timestamp,
e.tx_digest,
e.event_index,
'Cetus' AS platform,
event_address AS platform_address,
parsed_json:pool::STRING AS pool_address,
parsed_json:amount_in::NUMBER AS amount_in_raw,
parsed_json:amount_out::NUMBER AS amount_out_raw,
parsed_json:atob::BOOLEAN AS a_to_b,
parsed_json:fee_amount::NUMBER AS fee_amount_raw,
parsed_json:partner::STRING AS partner_address,
parsed_json:ref_amount::NUMBER AS referral_amount_raw,
parsed_json:steps::NUMBER AS steps,
NULL AS token_in_type,
NULL AS token_out_type,
e.event_address AS platform_address,
e.parsed_json:pool::STRING AS pool_address,
e.parsed_json:amount_in::NUMBER AS amount_in_raw,
e.parsed_json:amount_out::NUMBER AS amount_out_raw,
e.parsed_json:atob::BOOLEAN AS a_to_b,
e.parsed_json:fee_amount::NUMBER AS fee_amount_raw,
e.parsed_json:partner::STRING AS partner_address,
e.parsed_json:ref_amount::NUMBER AS referral_amount_raw,
e.parsed_json:steps::NUMBER AS steps,
CASE
WHEN e.parsed_json:atob::BOOLEAN = TRUE THEN tx.payload_details:type_arguments[0]::STRING
ELSE tx.payload_details:type_arguments[1]::STRING
END AS token_in_type,
CASE
WHEN e.parsed_json:atob::BOOLEAN = TRUE THEN tx.payload_details:type_arguments[1]::STRING
ELSE tx.payload_details:type_arguments[0]::STRING
END AS token_out_type,
NULL AS trader_address,
modified_timestamp
FROM core_events
WHERE type = '0x1eabed72c53feb3805120a081dc15963c204dc8d091542592abaf7a35689b2fb::pool::SwapEvent'
e.modified_timestamp
FROM core_events e
LEFT JOIN fact_transactions tx ON e.tx_digest = tx.tx_digest
WHERE e.type = '0x1eabed72c53feb3805120a081dc15963c204dc8d091542592abaf7a35689b2fb::pool::SwapEvent'
),
turbos_swaps AS (
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
event_index,
e.checkpoint_number,
e.block_timestamp,
e.tx_digest,
e.event_index,
'Turbos' AS platform,
event_address AS platform_address,
parsed_json:pool::STRING AS pool_address,
e.event_address AS platform_address,
e.parsed_json:pool::STRING AS pool_address,
CASE
WHEN parsed_json:a_to_b::BOOLEAN = TRUE THEN parsed_json:amount_a::NUMBER
ELSE parsed_json:amount_b::NUMBER
WHEN e.parsed_json:a_to_b::BOOLEAN = TRUE THEN e.parsed_json:amount_a::NUMBER
ELSE e.parsed_json:amount_b::NUMBER
END AS amount_in_raw,
CASE
WHEN parsed_json:a_to_b::BOOLEAN = TRUE THEN parsed_json:amount_b::NUMBER
ELSE parsed_json:amount_a::NUMBER
WHEN e.parsed_json:a_to_b::BOOLEAN = TRUE THEN e.parsed_json:amount_b::NUMBER
ELSE e.parsed_json:amount_a::NUMBER
END AS amount_out_raw,
parsed_json:a_to_b::BOOLEAN AS a_to_b,
parsed_json:fee_amount::NUMBER AS fee_amount_raw,
e.parsed_json:a_to_b::BOOLEAN AS a_to_b,
e.parsed_json:fee_amount::NUMBER AS fee_amount_raw,
NULL AS partner_address,
0 AS referral_amount_raw,
1 AS steps,
NULL AS token_in_type,
NULL AS token_out_type,
parsed_json:recipient::STRING AS trader_address,
modified_timestamp
FROM core_events
WHERE type = '0x91bfbc386a41afcfd9b2533058d7e915a1d3829089cc268ff4333d54d6339ca1::pool::SwapEvent'
CASE
WHEN e.parsed_json:a_to_b::BOOLEAN = TRUE THEN tx.payload_details:type_arguments[0]::STRING
ELSE tx.payload_details:type_arguments[1]::STRING
END AS token_in_type,
CASE
WHEN e.parsed_json:a_to_b::BOOLEAN = TRUE THEN tx.payload_details:type_arguments[1]::STRING
ELSE tx.payload_details:type_arguments[0]::STRING
END AS token_out_type,
e.parsed_json:recipient::STRING AS trader_address,
e.modified_timestamp
FROM core_events e
LEFT JOIN fact_transactions tx ON e.tx_digest = tx.tx_digest
WHERE e.type = '0x91bfbc386a41afcfd9b2533058d7e915a1d3829089cc268ff4333d54d6339ca1::pool::SwapEvent'
),
bluefin_swaps AS (
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
event_index,
e.checkpoint_number,
e.block_timestamp,
e.tx_digest,
e.event_index,
'Bluefin' AS platform,
event_address AS platform_address,
e.event_address AS platform_address,
NULL AS pool_address,
parsed_json:amount_in::NUMBER AS amount_in_raw,
parsed_json:amount_out::NUMBER AS amount_out_raw,
e.parsed_json:amount_in::NUMBER AS amount_in_raw,
e.parsed_json:amount_out::NUMBER AS amount_out_raw,
NULL AS a_to_b,
0 AS fee_amount_raw,
NULL AS partner_address,
0 AS referral_amount_raw,
1 AS steps,
NULL AS token_in_type,
NULL AS token_out_type,
tx.payload_details:type_arguments[0]::STRING AS token_in_type,
tx.payload_details:type_arguments[1]::STRING AS token_out_type,
NULL AS trader_address,
modified_timestamp
FROM core_events
WHERE type = '0x3b6d71bdeb8ce5b06febfd3cfc29ecd60d50da729477c8b8038ecdae34541b91::bluefin::BluefinSwapEvent'
e.modified_timestamp
FROM core_events e
LEFT JOIN fact_transactions tx ON e.tx_digest = tx.tx_digest
WHERE e.type = '0x3b6d71bdeb8ce5b06febfd3cfc29ecd60d50da729477c8b8038ecdae34541b91::bluefin::BluefinSwapEvent'
),
hop_aggregator_swaps AS (
aftermath_amm_swaps AS (
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
event_index,
'Hop' AS platform,
'Aftermath AMM' AS platform,
event_address AS platform_address,
NULL AS pool_address,
parsed_json:amount_in::NUMBER AS amount_in_raw,
@ -136,8 +171,9 @@ hop_aggregator_swaps AS (
NULL AS partner_address,
0 AS referral_amount_raw,
1 AS steps,
parsed_json:type_in::STRING AS token_in_type,
parsed_json:type_out::STRING AS token_out_type,
-- Add 0x prefix to token addresses to match pricing data format
CONCAT('0x', SPLIT(parsed_json:type_in::STRING, '::')[0], '::', SPLIT(parsed_json:type_in::STRING, '::')[1], '::', SPLIT(parsed_json:type_in::STRING, '::')[2]) AS token_in_type,
CONCAT('0x', SPLIT(parsed_json:type_out::STRING, '::')[0], '::', SPLIT(parsed_json:type_out::STRING, '::')[1], '::', SPLIT(parsed_json:type_out::STRING, '::')[2]) AS token_out_type,
parsed_json:swapper::STRING AS trader_address,
modified_timestamp
FROM core_events
@ -146,50 +182,83 @@ hop_aggregator_swaps AS (
flowx_swaps AS (
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
event_index,
e.checkpoint_number,
e.block_timestamp,
e.tx_digest,
e.event_index,
'FlowX' AS platform,
event_address AS platform_address,
parsed_json:pool_id::STRING AS pool_address,
parsed_json:amount_in::NUMBER AS amount_in_raw,
parsed_json:amount_out::NUMBER AS amount_out_raw,
e.event_address AS platform_address,
e.parsed_json:pool_id::STRING AS pool_address,
e.parsed_json:amount_in::NUMBER AS amount_in_raw,
e.parsed_json:amount_out::NUMBER AS amount_out_raw,
NULL AS a_to_b,
0 AS fee_amount_raw,
NULL AS partner_address,
0 AS referral_amount_raw,
1 AS steps,
NULL AS token_in_type,
NULL AS token_out_type,
tx.payload_details:type_arguments[0]::STRING AS token_in_type,
tx.payload_details:type_arguments[1]::STRING AS token_out_type,
NULL AS trader_address,
modified_timestamp
FROM core_events
WHERE type = '0x25929e7f29e0a30eb4e692952ba1b5b65a3a4d65ab5f2a32e1ba3edcb587f26d::pool::Swap'
e.modified_timestamp
FROM core_events e
LEFT JOIN fact_transactions tx ON e.tx_digest = tx.tx_digest
WHERE e.type = '0x25929e7f29e0a30eb4e692952ba1b5b65a3a4d65ab5f2a32e1ba3edcb587f26d::pool::Swap'
),
deepbook_swaps AS (
SELECT
e.checkpoint_number,
e.block_timestamp,
e.tx_digest,
e.event_index,
'DeepBook' AS platform,
e.event_address AS platform_address,
NULL AS pool_address,
e.parsed_json:amount_in::NUMBER AS amount_in_raw,
e.parsed_json:amount_out::NUMBER AS amount_out_raw,
NULL AS a_to_b,
0 AS fee_amount_raw,
NULL AS partner_address,
0 AS referral_amount_raw,
1 AS steps,
tx.payload_details:type_arguments[0]::STRING AS token_in_type,
tx.payload_details:type_arguments[1]::STRING AS token_out_type,
NULL AS trader_address,
e.modified_timestamp
FROM core_events e
LEFT JOIN fact_transactions tx ON e.tx_digest = tx.tx_digest
WHERE e.type = '0xe8f996ea6ff38c557c253d3b93cfe2ebf393816487266786371aa4532a9229f2::settle::Swap'
),
momentum_swaps AS (
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
event_index,
'DeepBook' AS platform,
'Momentum' AS platform,
event_address AS platform_address,
NULL AS pool_address,
parsed_json:pool::STRING AS pool_address,
parsed_json:amount_in::NUMBER AS amount_in_raw,
parsed_json:amount_out::NUMBER AS amount_out_raw,
NULL AS a_to_b,
parsed_json:a2b::BOOLEAN AS a_to_b,
0 AS fee_amount_raw,
NULL AS partner_address,
0 AS referral_amount_raw,
1 AS steps,
NULL AS token_in_type,
NULL AS token_out_type,
-- Token types are in coin_a.name and coin_b.name fields, need 0x prefix for price matching
CASE
WHEN parsed_json:a2b::BOOLEAN = TRUE THEN CONCAT('0x', parsed_json:coin_a:name::STRING)
ELSE CONCAT('0x', parsed_json:coin_b:name::STRING)
END AS token_in_type,
CASE
WHEN parsed_json:a2b::BOOLEAN = TRUE THEN CONCAT('0x', parsed_json:coin_b:name::STRING)
ELSE CONCAT('0x', parsed_json:coin_a:name::STRING)
END AS token_out_type,
NULL AS trader_address,
modified_timestamp
FROM core_events
WHERE type = '0xe8f996ea6ff38c557c253d3b93cfe2ebf393816487266786371aa4532a9229f2::settle::Swap'
WHERE type = '0x0018f7bbbece22f4272ed2281b290f745e5aa69d870f599810a30b4eeffc1a5e::momentum::MomentumSwapEvent'
),
all_swaps AS (
@ -199,11 +268,13 @@ all_swaps AS (
UNION ALL
SELECT * FROM bluefin_swaps
UNION ALL
SELECT * FROM hop_aggregator_swaps
SELECT * FROM aftermath_amm_swaps
UNION ALL
SELECT * FROM flowx_swaps
UNION ALL
SELECT * FROM deepbook_swaps
UNION ALL
SELECT * FROM momentum_swaps
)
SELECT

119
models/silver/defi/silver__dex_swaps.yml Normal file
View File

@ -0,0 +1,119 @@
version: 2
models:
- name: silver__dex_swaps
description: "{{ doc('silver__dex_swaps') }}"
tests:
- dbt_utils.recency:
datepart: hour
interval: 3
field: block_timestamp
columns:
- name: CHECKPOINT_NUMBER
description: "{{ doc('checkpoint_number') }}"
tests:
- not_null
- name: BLOCK_TIMESTAMP
description: "{{ doc('block_timestamp') }}"
tests:
- not_null
- name: TX_DIGEST
description: "{{ doc('tx_digest') }}"
tests:
- not_null
- name: EVENT_INDEX
description: "{{ doc('event_index') }}"
tests:
- not_null
- name: PLATFORM
description: "{{ doc('platform') }}"
tests:
- not_null
- accepted_values:
values: ['Cetus', 'Turbos', 'Bluefin', 'Aftermath AMM', 'FlowX', 'DeepBook', 'Momentum']
- name: PLATFORM_ADDRESS
description: "{{ doc('platform_address') }}"
tests:
- not_null
- name: POOL_ADDRESS
description: "{{ doc('pool_address') }}"
- name: AMOUNT_IN_RAW
description: "{{ doc('amount_in_raw') }}"
tests:
- not_null
- dbt_utils.expression_is_true:
expression: ">= 0"
- name: AMOUNT_OUT_RAW
description: "{{ doc('amount_out_raw') }}"
tests:
- not_null
- dbt_utils.expression_is_true:
expression: ">= 0"
- name: A_TO_B
description: "{{ doc('a_to_b') }}"
- name: FEE_AMOUNT_RAW
description: "{{ doc('fee_amount_raw') }}"
tests:
- dbt_utils.expression_is_true:
expression: ">= 0"
- name: PARTNER_ADDRESS
description: "{{ doc('partner_address') }}"
- name: REFERRAL_AMOUNT_RAW
description: "{{ doc('referral_amount_raw') }}"
tests:
- dbt_utils.expression_is_true:
expression: ">= 0"
- name: STEPS
description: "{{ doc('steps') }}"
tests:
- not_null
- dbt_utils.expression_is_true:
expression: "> 0"
- name: TOKEN_IN_TYPE
description: "{{ doc('token_in_type') }}"
tests:
- not_null
- name: TOKEN_OUT_TYPE
description: "{{ doc('token_out_type') }}"
tests:
- not_null
- name: TRADER_ADDRESS
description: "{{ doc('trader_address') }}"
- name: DEX_SWAPS_ID
description: "{{ doc('dex_swaps_id') }}"
tests:
- not_null
- unique
- name: INSERTED_TIMESTAMP
description: "{{ doc('inserted_timestamp') }}"
tests:
- not_null
- name: MODIFIED_TIMESTAMP
description: "{{ doc('modified_timestamp') }}"
tests:
- not_null
- name: _INVOCATION_ID
description: "{{ doc('_invocation_id') }}"
tests:
- not_null