flip/sui-models
1
0
Fork 0
mirror of https://github.com/FlipsideCrypto/sui-models.git synced 2026-02-06 13:46:42 +00:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

Merge branch 'main' into prices

Browse Source
This commit is contained in:
Eric Laurello 2025-07-29 16:33:35 -04:00
commit 04898362f5
22 changed files with 1214 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

158
.cursor/rules/dbt-documentation-standards.mdc Normal file
View File

@ -0,0 +1,158 @@
---
description:
globs: models/descriptions/*,*.yml,models/gold/**/*.sql
alwaysApply: false
---
# dbt Documentation Standards
When working with dbt projects, ensure comprehensive documentation that supports LLM-driven analytics workflows. This includes rich table and column descriptions that provide complete context for understanding blockchain data.
## Table Documentation Standards
Every dbt Model must have an accompanying yml file that provides model documentation.
### Basic YML File Format
Every dbt model yml file must follow this basic structure:
```yaml
version: 2
models:
- name: [model_name]
description: "{{ doc('table_name') }}"
tests:
- [appropriate_tests_for_the_model]
columns:
- name: [COLUMN_NAME]
description: "{{ doc('column_name')}}"
tests:
- [appropriate_tests_for_the_column]
```
#### Required Elements:
- **version: 2** - Must be the first line
- **models:** - Top-level key containing the model definitions
- **name:** - The exact name of the dbt model (without .sql extension)
- **description:** - Reference to markdown documentation using `{{ doc('table_name') }}`
- **columns:** - List of all columns in the model with their documentation
#### Column Documentation Format:
```yaml
- name: [COLUMN_NAME_IN_UPPERCASE]
description: "{{ doc('column_name')}}"
tests:
- [test_name]:
[test_parameters]
```
### Table Descriptions
Table documentation must include 4 standard elements, formatted in markdown. As the base documentation file is YML, the table description must be written in a dbt documentation markdown file in the `models/descriptions/` directory. The Table YML can then use the jinja doc block to reference it.
The 4 standard categories (designed for LLM client consumption to aid in data model discovery and selection):
1. **Description** (the "what"): What the model is mapping from the blockchain, data scope and coverage, transformations and business logic applied. DO NOT EXPLAIN THE DBT MODEL LINEAGE. This is not important for the use case of LLM-driven blockchain analytics.
2. **Key Use Cases**: Examples of when this table might be used and for what analysis, specific analytical scenarios and applications
3. **Important Relationships**: How this table might be used alongside OTHER GOLD LEVEL models, dependencies and connections to other key gold models. For example, a table like logs, events, or receipts may contain information from a larget transactions. To build this description, you SHOULD review the dbt model lineage to understand genuine model relationships. You must convert the model name to a database object, for example `core__fact_blocks.sql` = `core.fact_blocks` = `<schema>__<table_name>.sql`
4. **Commonly-used Fields**: Fields most important to condicting analytics. Determining these requires an understanding of the data model, what the columns are (via their descriptions) and how those fields aid in analytics. One way an understanding can be inferred is by analyzing curated models (anything that is not a core model in the gold/core/ directory is curated. Core models clean and map basic data objects of the blockchain like blocks, transactions, events, logs, etc. Curated models map specialized areas of activity like defi, nfts, governance, etc.). Blockchain data is often logged to a data table as a json object with a rich mapping of event-based details.
### Lineage Analysis
Before writing table descriptions:
- Read the dbt model SQL to understand the logic
- Follow upstream dependencies to understand data flow
- Review source models and transformations
- Understand the business context and use cases
- Review the column descriptions to estabish an understanding of the model, as a whole.
- At the gold level, an ez_ table typically sources data from a fact_ table and this relationship should be documented. Ez_ tables add business logic such as (but not limited to) labels and USD price information
## Column Documentation Standards
### Rich Descriptions
Each column description must include:
- Clear definition of what the field represents
- Data type and format expectations
- Business context and use cases
- Examples where helpful (especially for blockchain-specific concepts)
- Relationships to other fields when relevant
- Any important caveats or limitations
### Blockchain-Specific Context
For blockchain data:
- Reference official protocol documentation for technical accuracy. Use web search to find official developer documentation for the subject blockchain.
- Explain blockchain-specific concepts (gas, consensus, etc.)
- Provide examples using the specific blockchain's conventions
- Clarify differences from other blockchains when relevant
### YAML Requirements
- Column names MUST BE CAPITALIZED in YAML files
- Use `{{ doc('column_name') }}` references for consistent desciption across models. The doc block must refer to a valid description in `models/descriptions`
- Include appropriate tests for data quality
## Examples of Good Documentation
### Table Documentation Example
```markdown
{% docs table_transfers %}
## Description
This table tracks all token transfers on the <name> blockchain, capturing movements of native tokens and fungible tokens between accounts. The data includes both successful and failed transfers, with complete transaction context and token metadata.
## Key Use Cases
- Token flow analysis and wallet tracking
- DeFi protocol volume measurements
- Cross-chain bridge monitoring
- Whale movement detection and alerts
- Token distribution and holder analysis
## Important Relationships
- Subset of `gold.transactions`
- Maps events emitted in `gold.events` or `gold.logs`
- Utilizes token price data from `gold.prices` to compute USD columns
## Commonly-used Fields
- `tx_hash`: Essential for linking to transaction details and verification
- `sender_id` and `receiver_id`: Core fields for flow analysis and network mapping
- `amount_raw` and `amount_usd`: Critical for value calculations and financial analysis
- `token_address`: Key for filtering by specific tokens and DeFi analysis
- `block_timestamp`: Primary field for time-series analysis and trend detection
{% enddocs %}
```
### Column Documentation Example
```markdown
{% docs amount_raw %}
Unadjusted amount of tokens as it appears on-chain (not decimal adjusted). This is the raw token amount before any decimal precision adjustments are applied. For example, if transferring 1 native token, the amount_raw would be 1000000000000000000000000 (1e24) since <this blockchain's native token> has 24 decimal places. This field preserves the exact on-chain representation of the token amount for precise calculations and verification.
{% enddocs %}
```
Important consideration: be sure to research and confirm figures such as decimal places. If unknown DO NOT MAKE UP A NUMBER.
## Common Patterns to Follow
- Start with a clear definition
- Provide context about why the field exists
- Include examples for complex concepts
- Explain relationships to other fields
- Mention any important limitations or considerations
- Use consistent terminology throughout the project
## Quality Standards
### Completeness
- Every column must have a clear, detailed description
- Table descriptions must explain the model's purpose and scope
- Documentation must be self-contained without requiring external context
- All business logic and transformations must be explained
### Accuracy
- Technical details must match official blockchain documentation
- Data types and formats must be correctly described
- Examples must use appropriate blockchain conventions
- Relationships between fields must be accurately described
### Clarity
- Descriptions must be clear and easy to understand
- Complex concepts must be explained with examples
- Terminology must be consistent throughout the project
- Language must support LLM understanding
### Consistency
- Use consistent terminology across all models
- Follow established documentation patterns
- Maintain consistent formatting and structure
- Ensure similar fields have similar descriptions

186
.cursor/rules/dbt-overview-standard.mdc Normal file
View File

@ -0,0 +1,186 @@
---
description:
globs: __overview__.md,models/descriptions/*,models/gold/**/*.sql
alwaysApply: false
---
# dbt Overview Standards
When working with dbt projects, ensure comprehensive documentation exists about the project and a base reference to the gold models exists in the `__overview__.md` file for both human and LLM consumption.
## Project __overview__
The file `models/descriptions/__overview__.md` is the entry point to the model description and documentation and must contain a rich description of what the project is.
The file MUST START AND END WITH DBT JINJA DOCS TAGS `{% docs __overview__ %}` and `{% ENDDOCS %}`
## Quick Links Section Requirements
The `__overview__.md` file MUST contain a "Quick Links to Table Documentation" section that provides direct navigation to all gold model documentation. This section must include a simple list, organized by gold schema, with the models and a hyperlink to the model documentation. If there is an existing section like "using dbt docs" that instructs the user on how to navigate dbt docs or a list of links to flipside and dbt, remove it! These are outdated.
### Required Elements:
**Hyperlinks to Gold Model Documentation** - A comprehensive list of all gold models organized by schema. The schema can be inferred from the model name as the slug before the double underscore. For example, `core__fact_blocks` is a model named `fact_blocks` in the schema `core`.
### Gold Model Links Structure:
The quicklinks section must be organized by schema and use the relative link to load the page generated by dbt documentation. The relative link structure is `#!/model/dbt_uniqueId` where dbt's `uniqueId` format is `node_type.project_name.model_name`. All of these `node_types` are `model`. `project_name` is the name of the dbt models project established in `dbt_project.yml` by the `name` variable. `model_name` is finally the name of the model as determed by the name of the sql file OR the value of `name` in the model's associated `.yml` file. For example, a uniqueId for the blockchain's `fact_blocks` model would be `model.<blockchain_name>_models.core__fact_blocks` making the relative URL `#!/model/model.<blockchain_name>_models.core__fact_blocks`.
```markdown
## **Quick Links to Table Documentation**
**Click on the links below to jump to the documentation for each schema.**
### [Schema Name] Tables
**[Model Type Tables:]**
- model_1
- model_2
### CORE Tables
**Dimension Tables:**
- [core__fact_blocks](relative/path/to/model)
**Fact Tables:**
- [model_1](relative/path/to/model)
```
### Schema Organization Rules:
1. **Group by Schema**: Organize models by their schema (core, defi, nft, price, social, governance, etc.)
2. **Use Exact Schema Names**: Use the exact schema names as they appear in the database (e.g., `<blockchain_database>.CORE`, `<blockchain_database>.DEFI`, `<blockchain_database>.NFT`)
3. **Model Type Subgrouping**: Within each schema, subgroup by model type:
- **Dimension Tables:** (dim_* models)
- **Fact Tables:** (fact_* models)
- **Easy Views:** (ez_* models)
4. **Link Format**: Use the exact dbt docs link format: `#!/model/model.[project_name].[schema]__[model]`
5. **Model Naming**: Use the exact model name as it appears in the file system (without .sql extension)
### Implementation Guidelines for Coding Agents:
1. **Scan Directory Structure**: Read `models/gold/` directory to identify all schema subdirectories
2. **Extract Model Names**: For each schema directory, list all `.sql` files and remove the `.sql` extension
3. **Determine Schema Mapping**: Map model names to database schema names:
dbt models in this project utilize a double underscore in the model name to denote schema vs table <schema>__<table_name>.sql:
- `core__fact_blocks` → `<blockchain_database>.CORE.FACT_BLOCKS`
- `defi__ez_dex_swaps` → `<blockchain_database>.DEFI.EZ_DEX_SWAPS`
4. **Categorize Models**: Group models by prefix:
- `dim_*` → Dimension Tables
- `fact_*` → Fact Tables
- `ez_*` → Easy Views
- `udf_*`, `udtf_*` → Custom Functions
5. **Generate Links**: Create markdown links using the proper format
6. **Maintain Order**: Keep models in alphabetical order within each category
### Validation Requirements:
- Every gold model must have a corresponding link
- Links must use the correct dbt docs format
- Schema names must match the actual database schema
- Model names must match the actual file names (without .sql extension)
- Links must be organized by schema and model type
- All links must be functional and point to valid dbt documentation
- Do NOT add a hyperlink to the category headers. Only hyperlink individual models
## XML Tag Requirements
Every `__overview__.md` file MUST include structured `<llm>` XML tags for easy interpretation by an LLM.
```xml
<llm>
<blockchain>[Protocol Name]</blockchain>
<aliases>[Common Aliases]</aliases>
<ecosystem>[Execution Environment or Layer Type, for example EVM, SVM, IBC, Layer 1, Layer 2]</ecosystem>
<description>[Rich 3-5 sentence description of the blockchain, its consensus mechanism, key features, and developer/user benefits including if the blockchain was built for a specific usecase.]</description>
<external_resources>
<block_scanner>[Link to the primary block scanner for the blockchain]</block_scanner>
<developer_documenation>[Link to the primary developer documentation, maintained by the blockchain devs]</developer_documentation>
</external_resources>
<expert>
<constraints>
<table_availability>
<!-- Specify which tables/schemas are available for this blockchain -->
<!-- Example: "Ensure that your queries use only available tables for [BLOCKCHAIN]" -->
</table_availability>
<schema_structure>
<!-- Explain how the database is organized (dimensions, facts, naming conventions) -->
<!-- Example: "Understand that dimensions and facts combine to make ez_ tables" -->
</schema_structure>
</constraints>
<optimization>
<performance_filters>
<!-- Define key filtering strategies for query performance -->
<!-- Example: "use filters like block_timestamp over the last N days to improve speed" -->
</performance_filters>
<query_structure>
<!-- Specify preferred SQL patterns and structures -->
<!-- Example: "Use CTEs, not subqueries, as readability is important" -->
</query_structure>
<implementation_guidance>
<!-- Provide guidelines for advanced SQL features -->
<!-- Example: "Be smart with aggregations, window functions, etc." -->
</implementation_guidance>
</optimization>
<domain_mapping>
<token_operations>
<!-- Map token-related queries to specific tables -->
<!-- Example: "For token transfers, use ez_token_transfers table" -->
</token_operations>
<defi_analysis>
<!-- Specify DeFi-related tables and their use cases -->
<!-- Example: "For DeFi analysis, use ez_bridge_activity, ez_dex_swaps, ez_lending" -->
</defi_analysis>
<nft_analysis>
<!-- Define NFT-specific tables and functionality -->
<!-- Example: "For NFT queries, utilize ez_nft_sales table in nft schema" -->
</nft_analysis>
<specialized_features>
<!-- Cover blockchain-specific features or complex data structures -->
<!-- Example: "The XYZ data is complex, so ensure you ask clarifying questions" -->
</specialized_features>
</domain_mapping>
<interaction_modes>
<direct_user>
<!-- Define behavior for direct user interactions -->
<!-- Example: "Ask clarifying questions when dealing with complex data" -->
</direct_user>
<agent_invocation>
<!-- Specify response format when invoked by other AI agents -->
<!-- Example: "When invoked by another AI agent, respond with relevant query text" -->
</agent_invocation>
</interaction_modes>
<engagement>
<exploration_tone>
<!-- Set the overall tone and encouragement for data exploration -->
<!-- Example: "Have fun exploring the [BLOCKCHAIN] ecosystem through data!" -->
</exploration_tone>
</engagement>
</expert>
</llm>
```
Place these XML tags at the end of the documentation (BUT STILL BEFORE THE JINJA ENDDOCS TAG).
## Update Process for Coding Agents:
To update the overview, the coding agent MUST:
1. **Scan Current Gold Models**:
- Read the entire `models/gold/` directory structure
- Identify all `.sql` files across all schema subdirectories
- Extract model names (remove `.sql` extension)
2. **Generate Updated Quicklinks Section**:
- Follow these implementation guidelines
- Create a complete new quicklinks section with all current gold models
- Maintain proper schema organization and model type grouping
3. **Update __overview__.md**:
- Replace the entire "Quick Links to Table Documentation" section with the newly generated content
- Ensure proper markdown formatting and link structure
- Create or update the XML tag block
4. **Validation Check**:
- Verify all gold models have corresponding links
- Confirm links use correct dbt docs format
- Check that schema names and model names are accurate
- Ensure alphabetical ordering within categories

76
.cursor/rules/general-coding-standards.mdc Normal file
View File

@ -0,0 +1,76 @@
---
description:
globs:
alwaysApply: true
---
# dbt Model Standards for Flipside Crypto
## General Rules
- Follow the existing code style and patterns in the codebase
- Write clear, concise, and well-documented code
- Use meaningful variable, function, column and model names
- Handle errors gracefully and provide helpful error messages
- Test your code thoroughly before submitting
- Follow the existing project structure and conventions
## Code Quality
- Write self-documenting code with clear and consistent names
- Use consistent formatting and indentation
- Implement proper error handling and logging
- Follow DRY (Don't Repeat Yourself) principles
- Use meaningful commit messages
- Use snake_case for all objects (tables, columns, models)
- Maintain column naming consistency through the pipeline
## dbt Model Structure
- Models are connected through ref() and source() functions
- Data flows from source -> bronze -> silver -> gold layers
- Each model has upstream dependencies and downstream consumers
- Column-level lineage is maintained through transformations
- Parse ref() and source() calls to identify direct dependencies
- Track column transformations from upstream models
- Consider impact on downstream consumers
- Preserve business logic across transformations
## Model Naming and Organization
- Follow naming patterns: bronze__, silver__, core__, fact_, dim__, ez__, where a double underscore indicates a break between a model schema and object name. I.e. core__fact_blocks equates to <database>.core.fact_blocks.
- Organize by directory structure: bronze/, silver/, gold/, etc.
- Upstream models appear on the LEFT side of the DAG
- Current model is the focal point
- Downstream models appear on the RIGHT side of the DAG
## Modeling Standards
- Use snake_case for all objects
- Prioritize incremental processing always
- Follow source/bronze/silver/gold layering
- Document chain-specific assumptions
- Include incremental predicates to improve performance
- For gold layer models, include search optimization following Snowflake's recommended best practices
- Cluster models on appropriate fields
## Testing Requirements
- Ensure proper token decimal handling
- Implement unique tests for primary keys
- Implement recency tests for tables that are expected to have frequent data updates
- Add not_null tests for required columns
- Use relationships tests for foreign keys
## Performance Guidelines
- Optimize for high TPS (blockchain data)
- Handle large state change volumes efficiently
- Index frequently queried dimensions
- Consider partition pruning strategies
- Implement appropriate clustering keys
- Optimize database queries for large datasets
- Use appropriate indexing strategies
- Monitor resource usage and optimize accordingly
- Consider the impact of changes on existing systems
## Documentation
- Document data sources
- Map entity relationships
- Include model descriptions in yml files per expanded rule [dbt-documentation-standards.mdc](mdc:.cursor/rules/dbt-documentation-standards.mdc)
- Document column descriptions and business logic
- Explain incremental logic and predicates
- Note any data quality considerations

220
.cursor/rules/review-dbt-documentation.mdc Normal file
View File

@ -0,0 +1,220 @@
---
description:
globs:
alwaysApply: false
---
# Review dbt Documentation Process
## Overview
This document outlines the comprehensive process for reviewing and improving column and table descriptions for gold models in dbt projects. The goal is to provide robust, rich details that improve context for LLM-driven analytics workflows, ensuring that documentation is complete, accurate, and self-contained without requiring external expert files.
## Objectives
- Create clear, detailed documentation that supports LLM understanding of blockchain data
- Ensure technical accuracy by referencing official protocol documentation
- Provide rich context for each table and column to enable effective analytics
- Maintain consistency across all models and schemas
- Support automated analytics workflows without requiring expert context files
## Pre-Review Requirements
### 1. Research Phase
**Blockchain Protocol Documentation**
- Search and read official developer documentation for the target blockchain. Utilize web search to find authentic and accurate developer documentation
- Review technical specifications, whitepapers, and API documentation
- Understand the blockchain's consensus mechanism, data structures, and conventions
- Research common use cases and analytics patterns specific to the blockchain
- Identify key technical concepts that need explanation (e.g., gas mechanics, consensus, token standards)
**External Resources to Consult**
- Official blockchain documentation
- Developer guides and tutorials
- Technical specifications and whitepapers
- Community documentation and forums
- Block explorers and API documentation
### 2. Project Context Analysis
- Review the `__overview__.md` file and rewrite it per the @dbt-overview-standard rule to create a summary of what this particular blockchain is, unique characteristics, and any other general information about the chain itself
- Review existing documentation patterns and terminology
- Understand the data flow and model lineage structure
## Review Process
### Step 1: Model Analysis
**SQL Logic Review**
- Read the dbt model SQL file to understand the transformations and business logic
- Follow upstream dependencies to understand data flow from source to gold layer
- Review bronze source models, silver staging models, and intermediate transformations
- Identify any complex joins, aggregations, or business logic that needs explanation
- Understand the incremental logic and any filtering conditions
**Lineage Analysis**
- Map the complete data lineage from source to gold model
- Identify key transformations and their purposes
- Understand relationships between related models for the sole purpose of generating a robust description
- Do not include data lineage analysis in the table description
### Step 2: Column Description Review
**Individual Column Analysis**
For each column in the model:
1. **Technical Understanding**
- Read the SQL to understand how the column is derived
- Check upstream models if the column comes from a transformation
- Understand the data type and format expectations
- Identify any business logic applied to the column
2. **Blockchain Context**
- Research the blockchain-specific meaning of the column
- Reference official documentation for technical accuracy
- Understand how this field relates to blockchain concepts
- Identify any blockchain-specific conventions or requirements
3. **Documentation Assessment**
- Review existing column description
- Evaluate completeness and clarity
- Check for missing context or examples
- Ensure the description supports LLM understanding
**Required Elements for Column Descriptions**
- Clear definition of what the field represents
- Data type and format expectations
- Business context and use cases
- Examples where helpful (especially for blockchain-specific concepts)
- Relationships to other fields when relevant
- Any important caveats or limitations
- Blockchain-specific context and conventions
### Step 3: Table Description Review
**Current State Assessment**
- Review the updated column descriptions
- Review existing table description in the YAML file
- Evaluate completeness and clarity
- Identify missing context or unclear explanations
**Required Elements for Table Descriptions**
Table documentation must include 4 standard elements, formatted in markdown. As the base documentation file is YML, the table description must be written in a dbt documentation markdown file in the `models/descriptions/` directory. The Table YML can then use the jinja doc block to reference it.
The 4 standard categories are fully defined in the @dbt-documentation-standards rule. They are:
1. **Description**
2. **Key Use Cases**
3. **Important Relationships**
4. **Commonly-used Fields**
### Step 4: Documentation File Review
**Individual Documentation Files**
- Check if each column has a corresponding `.md` file in `models/descriptions/`
- Review existing documentation for completeness and accuracy
- Update or create documentation files as needed
**Documentation File Format**
```markdown
{% docs column_name %}
[Rich, detailed description including:
- Clear definition
- Data format and examples
- Business context
- Blockchain-specific details
- Relationships to other fields
- Important considerations]
{% enddocs %}
```
### Step 5: YAML File Review
**YAML Structure Validation**
- Ensure column names are CAPITALIZED in YAML files
- Verify all columns reference documentation using `{{ doc('column_name') }}`
- Check that appropriate tests are included
- Validate the overall YAML structure
**YAML File Format**
```yaml
version: 2
models:
- name: [model_name]
description: |-
[Clear, direct table description]
columns:
- name: [COLUMN_NAME_IN_UPPERCASE]
description: "{{ doc('column_name') }}"
tests:
- [appropriate_tests]
```
## Review Checklist
### Table Level
- [ ] Table documentation is in markdown file in `models/descriptions/` directory
- [ ] Table YAML references documentation using jinja doc block
- [ ] **Description** section explains what blockchain data is being modeled
- [ ] **Key Use Cases** section provides specific analytical scenarios
- [ ] **Important Relationships** section explains connections to other GOLD models
- [ ] **Commonly-used Fields** section identifies critical columns and their importance
- [ ] Documentation is optimized for LLM client consumption
### Column Level
- [ ] Each column has a comprehensive description
- [ ] Data types and formats are clearly specified
- [ ] Business context and use cases are explained
- [ ] Examples are provided for complex concepts
- [ ] Relationships to other fields are documented
- [ ] Important limitations or caveats are noted
- [ ] Blockchain-specific context is included
### Documentation Files
- [ ] All columns have corresponding `.md` files
- [ ] Documentation files contain rich, detailed descriptions
- [ ] Examples use appropriate blockchain conventions
- [ ] Technical accuracy is verified against official documentation
### YAML Files
- [ ] Column names are CAPITALIZED
- [ ] All columns reference documentation using `{{ doc('column_name') }}`
- [ ] Appropriate tests are included
- [ ] YAML structure is valid
## Implementation Guidelines
### Documentation Writing Tips
- Start with a clear definition of what the field represents
- Provide context about why the field exists and its importance
- Include examples for complex concepts, especially blockchain-specific ones
- Explain relationships to other fields when relevant
- Mention any important limitations or considerations
- Use consistent terminology throughout the project
### Blockchain-Specific Considerations
- Reference official protocol documentation for technical concepts
- Explain blockchain-specific concepts (gas, consensus, etc.)
- Provide examples using the specific blockchain's conventions
- Clarify differences from other blockchains when relevant
- Include information about data freshness and update mechanisms
### LLM Optimization
- Write descriptions that are complete and self-contained
- Use clear, structured language that supports automated understanding
- Include context that helps LLMs understand the data's purpose
- Provide examples that illustrate common use cases
- Ensure descriptions support common analytics workflows
## Post-Review Actions
### Validation
- Verify all documentation is technically accurate
- Check that descriptions are complete and self-contained
- Ensure consistency across related models
- Validate that documentation supports common analytics use cases
### Testing
- Test documentation by having an LLM attempt to understand the data
- Verify that descriptions enable effective query generation
- Check that examples are clear and helpful
- Ensure documentation supports the intended analytics workflows
### Maintenance
- Update documentation when models change
- Review and refresh documentation periodically
- Maintain consistency as new models are added
- Keep documentation aligned with blockchain protocol updates

2
.github/workflows/dbt_run_streamline_realtime.yml vendored
View File

@ -6,7 +6,7 @@ on:
branches:
- "main"
schedule:
- cron: '10 */2 * * *' # Run every hour at 10 minutes past the hour
- cron: '10 * * * *' # Run every hour at 10 minutes past the hour
env:
DBT_PROFILES_DIR: "${{ vars.DBT_PROFILES_DIR }}"

7
.gitignore vendored
View File

@ -18,4 +18,9 @@ package-lock.yml
.vscode/
.env
.DS_Store
.user.yml
.user.yml
.claude/log/*
.claude/settings.local.json
.cursor/mcp.json

50
models/bronze/bronze_api__coin_metadata.sql
View File

@ -3,37 +3,53 @@
unique_key = "coin_type",
merge_exclude_columns = ["inserted_timestamp"],
full_refresh = false,
tags = ['silver','core']
tags = ['silver']
) }}
WITH coins AS (
SELECT
coin_type
A.coin_type,
x
FROM
{{ ref('silver__coin_types') }}
(
SELECT
A.coin_type,
COUNT(1) x
FROM
{{ ref('core__fact_balance_changes') }} A
GROUP BY
1
) A
{% if is_incremental() %}
EXCEPT
SELECT
coin_type
FROM
{{ this }}
LEFT JOIN (
SELECT
coin_type
FROM
{{ this }}
{# WHERE
decimals IS NOT NULL --rerun if decimals is null and inserted_timestamp is within the last 7 days (if the token still doesnt have decimals after 7 day then we will stop trying)
OR (
decimals IS NULL
AND inserted_timestamp < CURRENT_DATE -7
) #}
) b
ON A.coin_type = b.coin_type
WHERE
decimals IS NOT NULL --rerun if decimals is null and inserted_timestamp is within the last 7 days (if the token still doesnt have decimals after 7 day then we will stop trying)
OR (
decimals IS NULL
AND inserted_timestamp < CURRENT_DATE -7
)
b.coin_type IS NULL
{% endif %}
ORDER BY
x DESC
LIMIT
100
10
), lq AS (
SELECT
coin_type,
{{ target.database }}.live.udf_api(
'POST',
'{Service}/{Authentication}',
{# '{Service}/{Authentication}', #}
'https://sui-mainnet-endpoint.blockvision.org/',
OBJECT_CONSTRUCT(
'Content-Type',
'application/json',
@ -51,8 +67,8 @@ LIMIT
ARRAY_CONSTRUCT(
coin_type
)
),
'Vault/prod/sui/quicknode/mainnet'
) {# ,
'Vault/prod/sui/quicknode/mainnet' #}
) :data: "result" AS DATA
FROM
coins

12
models/descriptions/columns.md
View File

@ -94,6 +94,18 @@ Type/category of object state modification or event. Enum: created, modified, de
Sui address (32-byte hex) representing the transaction or event sender. Used for authorization, security analysis, and user activity tracking. Example: '0xabc123...'.
{% enddocs %}
{% docs receiver %}
Sui address (32-byte hex) representing the transaction or event receiver. Used for tracking destination addresses, transfer flows, and recipient analytics. In transfer contexts, this is the address receiving tokens or assets. Example: '0xdef456...'.
{% enddocs %}
{% docs ez_transfers_id %}
Surrogate key for the enhanced transfers table. Generated unique identifier by combining transaction digest and balance change index, ensuring each transfer event enriched with token metadata is uniquely addressable. Used as the primary key for user-friendly transfer analytics, dashboard queries, and cross-model joins. In Sui, this supports transfer analysis with normalized amounts and token symbols, enabling easy identification and comparison of token movements.
{% enddocs %}
{% docs fact_transfers_id %}
Surrogate key for the core fact transfers table. Generated unique identifier by combining transaction digest and balance change index, ensuring each transfer event is uniquely addressable. Used as the primary key for transfer tracking, analytics workflows, and cross-model joins. In Sui, this supports precise transfer analysis, portfolio tracking, and compliance reporting by enabling unique identification of each token movement between addresses.
{% enddocs %}
{% docs digest %}
32-byte cryptographic hash (hex) of object contents, using SHA-256. Used for content verification, integrity checking, and unauthorized modification detection. Example: 'a1b2c3...'.
{% enddocs %}

8
models/descriptions/tables.md
View File

@ -2,6 +2,14 @@
Dimension table providing authoritative metadata for all fungible and non-fungible tokens on the Sui blockchain. Includes decimals, symbols, names, descriptions, and icon URLs sourced from on-chain metadata and Move package definitions. Serves as the canonical reference for token identification, decimal normalization, and UI display across analytics workflows. Data is sourced from bronze_api__coin_metadata and cross-referenced with on-chain Move modules, covering both native SUI and custom tokens. Essential for accurate balance calculations, token flow analysis, and user-facing applications. Supports lineage tracing from raw on-chain metadata to analytics-ready token attributes.
{% enddocs %}
{% docs core__fact_transfers %}
Fact table capturing all token and coin transfers on the Sui blockchain at the finest granularity. Each row represents a single transfer event between a sender and receiver, including the raw amount, coin type, and transaction context. Serves as the foundational transfer table that filters out null amounts from the silver layer and provides clean transfer data for downstream analytics. Includes checkpoint metadata, transaction success status, and balance change indexing to support precise tracking of asset movements across the network. Enables reconstruction of transfer flows, portfolio analysis, and compliance reporting by providing complete visibility into all token movements between addresses. Data is derived from transaction execution effects and object state transitions, following Sui's explicit ownership model where transfers represent actual balance changes between distinct owners.
{% enddocs %}
{% docs core__ez_transfers %}
Enhanced fact table providing user-friendly transfer analytics by joining core transfer data with token metadata. Each row represents a single transfer event enriched with normalized amounts (decimal-adjusted), token symbols, and human-readable identifiers. Serves as the primary table for transfer analysis, portfolio tracking, and user-facing applications by converting raw blockchain amounts into meaningful values. The amount_normalized field automatically applies decimal precision adjustments based on token metadata, enabling direct comparison across different token types and simplifying balance calculations. Includes all transfer metadata from the base fact table while adding token symbols for easy identification and normalized amounts for accurate financial analysis. Essential for dashboards, reporting tools, and analytics workflows that require human-readable token information and precise decimal calculations.
{% enddocs %}
{% docs core__fact_balance_changes %}
Fact table recording every token and coin balance change event on the Sui blockchain at the finest granularity. Each row represents a single balance delta (positive or negative) for a specific owner, coin type, and transaction, capturing the full flow of assets across wallets and contracts. Includes object IDs, transaction context, and ownership metadata, supporting precise tracking of token movements, portfolio changes, and treasury operations. Enables reconstruction of wallet balances, detection of large transfers, and analysis of token velocity. Data is derived from transaction execution effects and object state transitions, following Sui's explicit ownership and versioning model.
{% enddocs %}

34
models/gold/core/core__dim_labels.sql Normal file
View File

@ -0,0 +1,34 @@
{{ config(
materialized = 'incremental',
unique_key = ['address'],
incremental_strategy = 'merge',
merge_exclude_columns = ["inserted_timestamp"],
post_hook = ["ALTER TABLE {{ this }} ADD SEARCH OPTIMIZATION ON EQUALITY(address)", "DELETE FROM {{ this }} WHERE address in (select address from {{ ref('silver__crosschain_labels') }} where _is_deleted = TRUE);",],
tags = ['gold','core']
) }}
SELECT
blockchain,
creator,
address,
label_type,
label_subtype,
project_name AS label,
address_name,
labels_combined_id AS dim_labels_id,
inserted_timestamp,
modified_timestamp
FROM
{{ ref('silver__crosschain_labels') }}
{% if is_incremental() %}
WHERE
modified_timestamp >= (
SELECT
MAX(
modified_timestamp
)
FROM
{{ this }}
)
{% endif %}

42
models/gold/core/core__ez_transfers.sql Normal file
View File

@ -0,0 +1,42 @@
{{ config(
materialized = 'incremental',
unique_key = ['tx_digest','balance_change_index'],
incremental_strategy = 'merge',
incremental_predicates = ["dynamic_range_predicate", "block_timestamp::DATE"],
merge_exclude_columns = ["inserted_timestamp"],
cluster_by = ['block_timestamp::DATE','modified_timestamp::DATE'],
post_hook = "ALTER TABLE {{ this }} ADD SEARCH OPTIMIZATION ON EQUALITY(tx_digest, sender, receiver, coin_type, symbol);",
tags = ['core'],
enabled = false
) }}
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
balance_change_index,
tx_succeeded,
sender,
receiver,
ft.coin_type,
symbol,
amount,
ROUND(NULLIFZERO(DIV0NULL(amount, POWER(10, dt.decimals))), 3) as amount_normalized,
{{ dbt_utils.generate_surrogate_key(
['tx_digest','balance_change_index']
) }} AS ez_transfers_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp
FROM
{{ ref('core__fact_transfers') }} ft
LEFT JOIN
{{ ref('core__dim_tokens') }} dt
ON ft.coin_type = dt.coin_type
{% if is_incremental() %}
WHERE ft.modified_timestamp >= (
SELECT
MAX(modified_timestamp)
FROM
{{ this }}
)
{% endif %}

6
models/gold/core/core__fact_balance_changes.sql
View File

@ -24,7 +24,8 @@ WITH base AS (
b.value AS bc_value,
bc_value :"amount" :: bigint AS amount,
bc_value :"coinType" :: STRING AS coin_type,
bc_value :"owner" :"AddressOwner" :: STRING AS owner
bc_value :"owner" :"AddressOwner" :: STRING AS address_owner,
bc_value :"owner" :"ObjectOwner" :: STRING AS object_owner,
FROM
{{ ref('silver__transactions') }} A,
LATERAL FLATTEN(
@ -51,7 +52,8 @@ SELECT
balance_change_index,
coin_type,
amount,
owner,
object_owner,
address_owner,
{{ dbt_utils.generate_surrogate_key(['tx_digest','balance_change_index']) }} AS fact_transaction_balance_changes_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp

12
models/gold/core/core__fact_changes.sql
View File

@ -27,9 +27,13 @@ WITH base AS (
change_value :"digest" :: STRING AS digest,
change_value :"objectId" :: STRING AS object_id,
change_value :"objectType" :: STRING AS object_type,
change_value :"version" :: BIGINT AS version,
change_value :"previousVersion" :: BIGINT AS previous_version,
change_value :"version" :: bigint AS version,
change_value :"previousVersion" :: bigint AS previous_version,
change_value :"owner" :"ObjectOwner" :: STRING AS object_owner,
change_value :"owner" :"AddressOwner" :: STRING AS address_owner,
change_value :"owner" :"Shared" AS shared_owner,
change_value :"packageId" :: STRING AS package_id,
change_value :"modules" :: STRING AS modules
FROM
{{ ref('silver__transactions') }} A,
LATERAL FLATTEN(
@ -62,6 +66,10 @@ SELECT
version,
previous_version,
object_owner,
address_owner,
shared_owner,
package_id,
modules,
{{ dbt_utils.generate_surrogate_key(['tx_digest','change_index']) }} AS fact_changes_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp

2
models/gold/core/core__fact_events.sql
View File

@ -21,7 +21,7 @@ WITH base AS (
ELSE TRUE
END AS tx_succeeded,
b.value AS event_value,
event_value :"id" :"eventSeq" :: STRING AS event_index,
event_value :"id" :"eventSeq" :: INT AS event_index,
event_value :"packageId" :: STRING AS package_id,
event_value :"transactionModule" :: STRING AS transaction_module,
event_value :"sender" :: STRING AS sender,

42
models/gold/core/core__fact_transfers.sql Normal file
View File

@ -0,0 +1,42 @@
{{ config(
materialized = 'incremental',
unique_key = ['tx_digest','balance_change_index'],
incremental_strategy = 'merge',
incremental_predicates = ["dynamic_range_predicate", "block_timestamp::DATE"],
merge_exclude_columns = ["inserted_timestamp"],
cluster_by = ['block_timestamp::DATE','modified_timestamp::DATE'],
post_hook = "ALTER TABLE {{ this }} ADD SEARCH OPTIMIZATION ON EQUALITY(tx_digest, sender, receiver, coin_type);",
tags = ['core'],
enabled = false
) }}
SELECT
checkpoint_number,
block_timestamp,
tx_digest,
tx_succeeded,
sender,
receiver,
balance_change_index,
coin_type,
amount,
{{ dbt_utils.generate_surrogate_key(
['tx_digest','balance_change_index']
) }} AS fact_transfers_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp
FROM
{{ ref(
'silver__transfers'
) }}
WHERE
amount is not null
{% if is_incremental() %}
AND modified_timestamp >= (
SELECT
MAX(modified_timestamp)
FROM
{{ this }}
)
{% endif %}

208
models/gold/core/gold_core.yml
View File

@ -107,8 +107,11 @@ models:
data_type: NUMBER
tests:
- not_null
- name: OWNER
description: "{{ doc('owner') }}"
- name: OBJECT_OWNER
description: "{{ doc('object_owner') }}"
data_type: VARCHAR
- name: ADDRESS_OWNER
description: "{{ doc('address_owner') }}"
data_type: VARCHAR
- name: FACT_TRANSACTION_BALANCE_CHANGES_ID
description: "{{ doc('fact_transaction_balance_changes_id') }}"
@ -200,6 +203,18 @@ models:
- name: OBJECT_OWNER
description: "{{ doc('object_owner') }}"
data_type: VARCHAR
- name: ADDRESS_OWNER
description: "{{ doc('address_owner') }}"
data_type: VARCHAR
- name: SHARED_OWNER
description: "{{ doc('shared_owner') }}"
data_type: VARIANT
- name: PACKAGE_ID
description: "{{ doc('package_id') }}"
data_type: VARCHAR
- name: MODULES
description: "{{ doc('modules') }}"
data_type: VARCHAR
- name: FACT_CHANGES_ID
description: "{{ doc('fact_changes_id') }}"
data_type: VARCHAR
@ -338,7 +353,7 @@ models:
- not_null
- name: EVENT_INDEX
description: "{{ doc('event_index') }}"
data_type: VARCHAR
data_type: NUMBER
tests:
- not_null
- name: TYPE
@ -644,4 +659,189 @@ models:
field: block_timestamp
interval: 12
severity: error
tags: ['test_recency']
tags: ['test_recency']
- name: core__dim_labels
description: "The labels table is a store of one-to-one address identifiers, or an address name. Labels are broken out into a \"type\" (such as cex, dex, dapp, games, etc.) and a \"subtype\" (ex: contract_deployer, hot_wallet, token_contract, etc.) in order to help classify each address name into similar groups. Our labels are sourced from many different places, but can primarily be grouped into two categories: automatic and manual. Automatic labels are continuously labeled based on certain criteria, such as a known contract deploying another contract, behavior based algorithms for finding deposit wallets, and consistent data pulls of custom protocol APIs. Manual labels are done periodically to find addresses that cannot be found programmatically such as finding new protocol addresses, centralized exchange hot wallets, or trending addresses. Labels can also be added by our community by using our add-a-label tool (https://science.flipsidecrypto.xyz/add-a-label/).A label can be removed by our labels team if it is found to be incorrect or no longer relevant; this generally will only happen for mislabeled deposit wallets."
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- ADDRESS
columns:
- name: ADDRESS
description: "Raw address string"
- name: BLOCKCHAIN
description: "In this table, always ton. Used to join to cross-chain tables."
- name: CREATOR
description: "Name of the label creator - for now, this will always be \"Flipside.\""
- name: LABEL_TYPE
description: "A broad category that describes what a label is representing."
- name: LABEL_SUBTYPE
description: "Adds more detail to the label type."
- name: LABEL
description: "The label or name of the address."
- name: ADDRESS_NAME
description: "Name of the labeled address"
- name: DIM_LABELS_ID
description: "Unique primary key for the dimension table, used as the main identifier for each record."
- name: INSERTED_TIMESTAMP
description: "Timestamp when this record was inserted."
- name: MODIFIED_TIMESTAMP
description: "Timestamp when this record was last modified."
- name: core__fact_transfers
description: "{{ doc('core__fact_transfers') }}"
config:
contract:
enforced: true
columns:
- name: CHECKPOINT_NUMBER
description: "{{ doc('checkpoint_number') }}"
data_type: NUMBER
tests:
- not_null
- name: BLOCK_TIMESTAMP
description: "{{ doc('block_timestamp') }}"
data_type: TIMESTAMP_NTZ
tests:
- not_null
- name: TX_DIGEST
description: "{{ doc('tx_digest') }}"
data_type: VARCHAR
tests:
- not_null
- name: TX_SUCCEEDED
description: "{{ doc('tx_succeeded') }}"
data_type: BOOLEAN
tests:
- not_null
- name: SENDER
description: "{{ doc('sender') }}"
data_type: VARCHAR
tests:
- not_null
- name: RECEIVER
description: "{{ doc('receiver') }}"
data_type: VARCHAR
tests:
- not_null
- name: BALANCE_CHANGE_INDEX
description: "{{ doc('balance_change_index') }}"
data_type: NUMBER
tests:
- not_null
- name: COIN_TYPE
description: "{{ doc('coin_type') }}"
data_type: VARCHAR
tests:
- not_null
- name: AMOUNT
description: "{{ doc('amount') }}"
data_type: NUMBER
tests:
- not_null
- name: FACT_TRANSFERS_ID
description: "{{ doc('fact_transfers_id') }}"
data_type: VARCHAR
tests:
- not_null
- unique
- name: INSERTED_TIMESTAMP
description: "{{ doc('inserted_timestamp') }}"
data_type: TIMESTAMP_NTZ
tests:
- not_null
- name: MODIFIED_TIMESTAMP
description: "{{ doc('modified_timestamp') }}"
data_type: TIMESTAMP_NTZ
tests:
- not_null
tests:
- dbt_utils.recency:
datepart: hour
field: block_timestamp
interval: 12
severity: error
tags: ['test_recency']
- name: core__ez_transfers
description: "{{ doc('core__ez_transfers') }}"
config:
contract:
enforced: true
columns:
- name: CHECKPOINT_NUMBER
description: "{{ doc('checkpoint_number') }}"
data_type: NUMBER
tests:
- not_null
- name: BLOCK_TIMESTAMP
description: "{{ doc('block_timestamp') }}"
data_type: TIMESTAMP_NTZ
tests:
- not_null
- name: TX_DIGEST
description: "{{ doc('tx_digest') }}"
data_type: VARCHAR
tests:
- not_null
- name: BALANCE_CHANGE_INDEX
description: "{{ doc('balance_change_index') }}"
data_type: NUMBER
tests:
- not_null
- name: TX_SUCCEEDED
description: "{{ doc('tx_succeeded') }}"
data_type: BOOLEAN
tests:
- not_null
- name: SENDER
description: "{{ doc('sender') }}"
data_type: VARCHAR
tests:
- not_null
- name: RECEIVER
description: "{{ doc('receiver') }}"
data_type: VARCHAR
tests:
- not_null
- name: COIN_TYPE
description: "{{ doc('coin_type') }}"
data_type: VARCHAR
tests:
- not_null
- name: SYMBOL
description: "{{ doc('symbol') }}"
data_type: VARCHAR
- name: AMOUNT
description: "{{ doc('amount') }}"
data_type: NUMBER
tests:
- not_null
- name: AMOUNT_NORMALIZED
description: "{{ doc('amount_normalized') }}"
data_type: FLOAT
- name: EZ_TRANSFERS_ID
description: "{{ doc('ez_transfers_id') }}"
data_type: VARCHAR
tests:
- not_null
- unique
- name: INSERTED_TIMESTAMP
description: "{{ doc('inserted_timestamp') }}"
data_type: TIMESTAMP_NTZ
tests:
- not_null
- name: MODIFIED_TIMESTAMP
description: "{{ doc('modified_timestamp') }}"
data_type: TIMESTAMP_NTZ
tests:
- not_null
tests:
- dbt_utils.recency:
datepart: hour
field: block_timestamp
interval: 12
severity: error
tags: ['test_recency']

4
models/silver/core/silver__transactions.sql
View File

@ -48,7 +48,9 @@ SELECT
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
bronze_txs qualify ROW_NUMBER() over (
bronze_txs
WHERE
block_timestamp IS NOT NULL qualify ROW_NUMBER() over (
PARTITION BY tx_digest
ORDER BY
_inserted_timestamp DESC

69
models/silver/core/silver__transfers.sql Normal file
View File

@ -0,0 +1,69 @@
{{ config(
materialized = 'incremental',
unique_key = ['tx_digest','balance_change_index'],
incremental_strategy = 'merge',
merge_exclude_columns = ['inserted_timestamp'],
cluster_by = ['block_timestamp::DATE'],
incremental_predicates = ["dynamic_range_predicate", "block_timestamp::date"],
tags = ['core','transfers']
) }}
WITH
allowed_tx AS (
SELECT
tx_digest
FROM
{{ ref('core__fact_transactions') }}
WHERE
(payload_type IN ('TransferObjects','SplitCoins','MergeCoins'))
OR
(payload_type = 'MoveCall' AND payload_details :package = '0x0000000000000000000000000000000000000000000000000000000000000002')
{% if is_incremental() %}
AND modified_timestamp >= (SELECT COALESCE(MAX(modified_timestamp),'1970-01-01') FROM {{ this }})
{% endif %}
),
filtered as (
SELECT
fbc.checkpoint_number,
fbc.block_timestamp,
fbc.tx_digest,
fbc.tx_succeeded,
case
when fbc.amount < 0
and fbc.address_owner IS NOT NULL
and fbc.address_owner <> fbc.tx_sender
then fbc.address_owner
else fbc.tx_sender end as sender,
coalesce(fbc.address_owner, fbc.object_owner) as receiver,
fbc.balance_change_index,
fbc.coin_type,
fbc.amount
FROM
{{ ref('core__fact_balance_changes') }} fbc
JOIN
allowed_tx at
ON fbc.tx_digest = at.tx_digest
WHERE
fbc.tx_succeeded
AND fbc.tx_sender != coalesce(fbc.address_owner, fbc.object_owner)
AND NOT (balance_change_index = 0 AND amount < 0) -- remove mints, self-splits, proofs, flash loans
{% if is_incremental() %}
AND fbc.modified_timestamp >= (SELECT COALESCE(MAX(modified_timestamp),'1970-01-01') FROM {{ this }})
{% endif %}
)
SELECT DISTINCT
checkpoint_number,
block_timestamp,
tx_digest,
tx_succeeded,
sender,
receiver,
balance_change_index,
coin_type,
amount,
{{ dbt_utils.generate_surrogate_key(['tx_digest','balance_change_index']) }} AS transfers_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
filtered

62
models/silver/core/silver_core.yml
View File

@ -65,3 +65,65 @@ models:
data_type: TIMESTAMP_NTZ
- name: _invocation_id
data_type: VARCHAR
- name: silver__transfers
config:
contract:
enforced: true
columns:
- name: CHECKPOINT_NUMBER
description: "{{ doc('checkpoint_number') }}"
data_type: NUMBER
tests:
- not_null
- name: BLOCK_TIMESTAMP
description: "{{ doc('block_timestamp') }}"
data_type: TIMESTAMP_NTZ
tests:
- not_null
- name: TX_DIGEST
description: "{{ doc('tx_digest') }}"
data_type: VARCHAR
tests:
- not_null
- name: TX_SUCCEEDED
description: "{{ doc('tx_succeeded') }}"
data_type: BOOLEAN
tests:
- not_null
- name: SENDER
description: "{{ doc('sender') }}"
data_type: VARCHAR
tests:
- not_null
- name: RECEIVER
description: "{{ doc('receiver') }}"
data_type: VARCHAR
tests:
- not_null
- name: BALANCE_CHANGE_INDEX
description: "{{ doc('balance_change_index') }}"
data_type: NUMBER
tests:
- not_null
- name: COIN_TYPE
description: "{{ doc('coin_type') }}"
data_type: VARCHAR
tests:
- not_null
- name: AMOUNT
description: "{{ doc('amount') }}"
data_type: NUMBER
tests:
- not_null
- name: TRANSFERS_ID
data_type: VARCHAR
- name: INSERTED_TIMESTAMP
description: "{{ doc('inserted_timestamp') }}"
data_type: TIMESTAMP_NTZ
- name: MODIFIED_TIMESTAMP
description: "{{ doc('modified_timestamp') }}"
data_type: TIMESTAMP_NTZ
- name: _INVOCATION_ID
data_type: VARCHAR

24
models/silver/labels/silver__crosschain_labels.sql Normal file
View File

@ -0,0 +1,24 @@
{{ config(
materialized = 'view',
tags = ['scheduled_core']
) }}
SELECT
blockchain,
address,
creator,
label_type,
label_subtype,
address_name,
project_name,
_is_deleted,
labels_combined_id,
inserted_timestamp,
modified_timestamp
FROM
{{ source(
'crosschain_silver',
'labels_combined'
) }}
WHERE
blockchain = 'sui'

16
models/silver/labels/silver__crosschain_labels.yml Normal file
View File

@ -0,0 +1,16 @@
version: 2
models:
- name: silver__crosschain_labels
columns:
- name: BLOCKCHAIN
- name: ADDRESS
- name: CREATOR
- name: LABEL_TYPE
- name: LABEL_SUBTYPE
- name: ADDRESS_NAME
- name: PROJECT_NAME
- name: _IS_DELETED
- name: LABELS_COMBINED_ID
- name: INSERTED_TIMESTAMP
- name: MODIFIED_TIMESTAMP

6
models/streamline/core/realtime/streamline__transactions_realtime.sql
View File

@ -4,10 +4,10 @@
func = 'streamline.udf_bulk_rest_api_v2',
target = "{{this.schema}}.{{this.identifier}}",
params ={ "external_table" :"transactions",
"sql_limit" :"1200000",
"sql_limit" :"800000",
"producer_batch_size" :"100000",
"worker_batch_size" :"2500",
"async_concurrent_requests" :"10",
"async_concurrent_requests" :"8",
"sql_source" :"{{this.identifier}}",
'exploded_key': '["result"]',
"order_by_column": "checkpoint_number DESC" }
@ -114,4 +114,4 @@ FROM
ORDER BY
block_timestamp DESC
LIMIT
1200000
800000