Go to file
desmond-hui 780e4a5ef1
Tensorswap models using logs (#600)
* udf to grab program data logs from log messages

* output table with encoded program data in tx logs

* add comments

* setup views and source

* update tag

* wip realtime view

* wip

* fix bad join

* wip

* decoded logs

* wip

* add event_type, index, inner_index and refactor existing

* handle additional nested calls

* add columns, add to SO

* use backfill macro

* wip

* handle non-contiguous program log messaging

* tmp upperbound

* minor changes

* reset child index when a new top level program is invoked

* rebuild transaction logs program data

* remove backfill

* still need to use min price as backup

* add post PR TODO

* revert tmp full-refresh changes

* add comment

* add tests

* update columns

* add docs and tests, fix col name

* update col description
2024-07-09 09:27:57 -07:00
.github/workflows Streamline decoded logs backfill (#580) 2024-06-13 06:52:18 -07:00
analysis dbt proj files and sp setup 2022-02-25 14:49:40 -08:00
data Tensorswap models using logs (#600) 2024-07-09 09:27:57 -07:00
docs update docs 2022-11-03 08:39:25 -07:00
macros reset child index when a new top level program is invoked (#597) 2024-07-05 07:13:20 -07:00
models Tensorswap models using logs (#600) 2024-07-09 09:27:57 -07:00
snapshots dbt proj files and sp setup 2022-02-25 14:49:40 -08:00
tests update partition pointer for test 2024-07-05 08:12:27 -07:00
.env.sample An 2510/GitHub dbt workflows (#167) 2022-11-30 15:50:35 -08:00
.gitignore Hotfix/reingest full parts to fix partial load (#205) 2023-01-11 09:20:34 -08:00
.user.yml Add solana parser jobs and models (#316) 2023-07-21 18:33:08 +02:00
dbt_project.yml An 4390/retry views for failed old decoder txs (#441) 2024-01-11 09:37:23 -08:00
docker-compose.yml add community curation resources (#138) 2022-10-18 13:09:55 -07:00
Dockerfile add community curation resources (#138) 2022-10-18 13:09:55 -07:00
LICENSE Initial commit 2022-02-25 14:28:59 -08:00
Makefile add community curation resources (#138) 2022-10-18 13:09:55 -07:00
package-lock.yml An 4508/integrate fsc utils dynamic predicate macro (#482) 2024-02-09 07:00:54 -08:00
packages.yml An 4508/integrate fsc utils dynamic predicate macro (#482) 2024-02-09 07:00:54 -08:00
profiles.yml An 2510/solana incrementals (#183) 2022-12-08 15:25:36 -08:00
README.md Cleanup/decrease test time and docs update (#584) 2024-06-17 10:46:30 -07:00
requirements.txt pin (#513) 2024-03-13 15:48:31 -04:00

Profile Set Up

Use the following within profiles.yml


solana:
  target: dev
  outputs:
    dev:
      type: snowflake
      account: <ACCOUNT>
      role: <ROLE>
      user: <USERNAME>
      password: <PASSWORD>
      region: <REGION>
      database: SOLANA_DEV
      warehouse: <WAREHOUSE>
      schema: SILVER
      threads: 4
      client_session_keep_alive: False
      query_tag: <TAG>

Notes

  • block_id column in our data models matches what Solscan classifies as Block. This value is same as what is referred to as Slot.

Resources:

  • Learn more about dbt in the docs
  • Check out Discourse for commonly asked questions and answers
  • Join the chat on Slack for live discussions and support
  • Find dbt events near you
  • Check out the blog for the latest news on dbt's development and best practices

Applying Model Tags

Database / Schema level tags

Database and schema tags are applied via the add_database_or_schema_tags macro. These tags are inherited by their downstream objects. To add/modify tags call the appropriate tag set function within the macro.

{{ set_database_tag_value('SOME_DATABASE_TAG_KEY','SOME_DATABASE_TAG_VALUE') }}
{{ set_schema_tag_value('SOME_SCHEMA_TAG_KEY','SOME_SCHEMA_TAG_VALUE') }}

Model tags

To add/update a model's snowflake tags, add/modify the meta model property under config. Only table level tags are supported at this time via DBT.

{{ config(
    ...,
    meta={
        'database_tags':{
            'table': {
                'PURPOSE': 'SOME_PURPOSE'
            }
        }
    },
    ...
) }}

By default, model tags are pushed to Snowflake on each DBT run. You can disable this by setting the UPDATE_SNOWFLAKE_TAGS project variable to False during a run.

dbt run --var '{"UPDATE_SNOWFLAKE_TAGS":False}' -s models/gold/defi/defi__fact_swaps.sql

Querying for existing tags on a model in snowflake

select *
from table(solana.information_schema.tag_references('solana.core.fact_blocks', 'table'));