Go to file
tarikceric 2cc12c6960
An 2296/intermediate swaps tables 2 (#157)
* create testing data

* table w failed and successful txfers

* initial model

* create transfers2 and testing data

* add expected results

* testing additions

* check-in

* remove not-null requirement

* add business test cases, simplify, need acct owners

* wip

* clean tests and update model

* update model for edge cases

* fix expected test case

* check in

* logic for unordered swaps

* create single model for all dex swaps

* check in updates

* remove mint actions

* general model

* remove jup v4

* clean up

* update transfers2

* update name

* test that specific tx are excluded

* initial changes per pr comments

* add program_id

* Add base_events and combine union

* uncomment incremental logic

* move block_id filter to base_events

* remove flatten, map accounts directly to transfer to/from

* filter out non-swap actions and fix join logic

* remove raydium logic and rename

* remove raydium swap in test

* Include TRPY swaps

* update table config

* add program_id column to yml

* add additional test records

* updates per PR comments

* remove dependency of excluded tx test on csv

* remove unnecessary program_id

* remove orca dex references

* additional inner join logic

* query cleanup/formatting

* add test cases

* add back delegate mapping for saber + test cases

* An 2455/intermediate swaps raydium (#161)

* Intermediate swaps model for raydium

* add test cases, remove unnecessary filter and hook

* tweak event filters and join optimizations

* revisions per pr comments

* update excluded tx tests and remove csv dependency

* add token accts

* add test cases, round amts to 8 decimals

* use associated accts given in swap instruction

* handle multisig authorities

* add test case

Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>

* add udf to get swapper from multi signer txs

python udf for getting swapper if multiple signers

* use udf for getting multi signer swapper

* add more multi signer test cases

* add mappings and test cases for solrise initiated swaps

Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2022-12-05 08:28:14 -08:00
.github/workflows An 2510/GitHub dbt workflows (#167) 2022-11-30 15:50:35 -08:00
analysis dbt proj files and sp setup 2022-02-25 14:49:40 -08:00
data An 2296/intermediate swaps tables 2 (#157) 2022-12-05 08:28:14 -08:00
docs update docs 2022-11-03 08:39:25 -07:00
macros An 2296/intermediate swaps tables 2 (#157) 2022-12-05 08:28:14 -08:00
models An 2296/intermediate swaps tables 2 (#157) 2022-12-05 08:28:14 -08:00
snapshots dbt proj files and sp setup 2022-02-25 14:49:40 -08:00
tests remove ; 2022-10-17 10:39:33 -07:00
.env.sample An 2510/GitHub dbt workflows (#167) 2022-11-30 15:50:35 -08:00
.gitignore add community curation resources (#138) 2022-10-18 13:09:55 -07:00
dbt_project.yml missed quotes 2022-12-01 18:01:55 -08:00
docker-compose.yml add community curation resources (#138) 2022-10-18 13:09:55 -07:00
Dockerfile add community curation resources (#138) 2022-10-18 13:09:55 -07:00
LICENSE Initial commit 2022-02-25 14:28:59 -08:00
Makefile add community curation resources (#138) 2022-10-18 13:09:55 -07:00
packages.yml force dbt to 0.9.2 2022-10-27 07:22:10 -07:00
profiles.yml An 2510/GitHub dbt workflows (#168) 2022-11-30 16:24:58 -08:00
README.md An 2294/create sf tags (#141) 2022-10-21 11:00:13 -07:00

Profile Set Up

Use the following within profiles.yml


solana:
  target: dev
  outputs:
    dev:
      type: snowflake
      account: <ACCOUNT>
      role: <ROLE>
      user: <USERNAME>
      password: <PASSWORD>
      region: <REGION>
      database: SOLANA_DEV
      warehouse: <WAREHOUSE>
      schema: SILVER
      threads: 4
      client_session_keep_alive: False
      query_tag: <TAG>

Notes

  • block_id column in our data models matches what Solcan classifies as Block. This value is same as what is referred to as Slot.

Resources:

  • Learn more about dbt in the docs
  • Check out Discourse for commonly asked questions and answers
  • Join the chat on Slack for live discussions and support
  • Find dbt events near you
  • Check out the blog for the latest news on dbt's development and best practices

Applying Model Tags

Database / Schema level tags

Database and schema tags are applied via the add_database_or_schema_tags macro. These tags are inherited by their downstream objects. To add/modify tags call the appropriate tag set function within the macro.

{{ set_database_tag_value('SOME_DATABASE_TAG_KEY','SOME_DATABASE_TAG_VALUE') }}
{{ set_schema_tag_value('SOME_SCHEMA_TAG_KEY','SOME_SCHEMA_TAG_VALUE') }}

Model tags

To add/update a model's snowflake tags, add/modify the meta model property under config. Only table level tags are supported at this time via DBT.

{{ config(
    ...,
    meta={
        'database_tags':{
            'table': {
                'PURPOSE': 'SOME_PURPOSE'
            }
        }
    },
    ...
) }}

By default, model tags are not pushed to snowflake on each load. You can push a tag update for a model by specifying the UPDATE_SNOWFLAKE_TAGS project variable during a run.

dbt run --var '{"UPDATE_SNOWFLAKE_TAGS":True}' -s models/core/core__fact_swaps.sql

Querying for existing tags on a model in snowflake

select *
from table(solana.information_schema.tag_references('solana.core.fact_blocks', 'table'));