solana-models/README.md at f2cda1ccac6d740462505d521985fd0ad62bfe50

mirror of https://github.com/FlipsideCrypto/solana-models.git synced 2026-02-06 13:56:50 +00:00

* snowflake tagging macros

* wip

* apply tags on run end

* wip

* add var for controlling when to run tags, remove col lvl support

* update db tag values

* update model tag values

* update readme for how to apply tags

* don't log to stdout by default

* hardcode schema

* bad schema value example

* fix db ref

2022-10-21 11:00:13 -07:00

2.2 KiB

Raw Blame History

Profile Set Up

Use the following within profiles.yml

solana:
  target: dev
  outputs:
    dev:
      type: snowflake
      account: <ACCOUNT>
      role: <ROLE>
      user: <USERNAME>
      password: <PASSWORD>
      region: <REGION>
      database: SOLANA_DEV
      warehouse: <WAREHOUSE>
      schema: SILVER
      threads: 4
      client_session_keep_alive: False
      query_tag: <TAG>

Notes

block_id column in our data models matches what Solcan classifies as Block. This value is same as what is referred to as Slot.

Resources:

Learn more about dbt in the docs
Check out Discourse for commonly asked questions and answers
Join the chat on Slack for live discussions and support
Find dbt events near you
Check out the blog for the latest news on dbt's development and best practices

Applying Model Tags

Database / Schema level tags

Database and schema tags are applied via the add_database_or_schema_tags macro. These tags are inherited by their downstream objects. To add/modify tags call the appropriate tag set function within the macro.

{{ set_database_tag_value('SOME_DATABASE_TAG_KEY','SOME_DATABASE_TAG_VALUE') }}
{{ set_schema_tag_value('SOME_SCHEMA_TAG_KEY','SOME_SCHEMA_TAG_VALUE') }}

Model tags

To add/update a model's snowflake tags, add/modify the meta model property under config. Only table level tags are supported at this time via DBT.

{{ config(
    ...,
    meta={
        'database_tags':{
            'table': {
                'PURPOSE': 'SOME_PURPOSE'
            }
        }
    },
    ...
) }}

By default, model tags are not pushed to snowflake on each load. You can push a tag update for a model by specifying the UPDATE_SNOWFLAKE_TAGS project variable during a run.

dbt run --var '{"UPDATE_SNOWFLAKE_TAGS":True}' -s models/core/core__fact_swaps.sql

Querying for existing tags on a model in snowflake

select *
from table(solana.information_schema.tag_references('solana.core.fact_blocks', 'table'));

2.2 KiB Raw Blame History