Commit Graph

215 Commits

Author SHA1 Message Date
stanz
211547b74b
run_query (#496)
Some checks failed
docs_update / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
2025-10-28 00:17:42 +07:00
stanz
27e3414619
AN-6514 Streamline 2.0 Upgrade (#465)
Some checks failed
docs_update / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / notify-failure (push) Has been cancelled
* rm topshot

* upd yml

* deploy udf

* upd blocks_realtime

* sl func

* bronze upd

* v1 namespace

* bronze v2

* prod endpoint

* define api integrations for v2

---------

Co-authored-by: Jack Forgash <58153492+forgxyz@users.noreply.github.com>
2025-10-24 01:15:13 +07:00
Jack Forgash
5c8c2d9afa
fix min blockdate issue (#495)
Some checks are pending
docs_update / run_dbt_jobs (push) Waiting to run
docs_update / notify-failure (push) Blocked by required conditions
2025-10-22 16:21:09 -06:00
Austin
470917b806
move tag (#491)
Some checks failed
docs_update / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / notify-failure (push) Has been cancelled
2025-10-03 09:55:10 -04:00
Austin
0e91031c85
account storage (#490)
Some checks are pending
docs_update / run_dbt_jobs (push) Waiting to run
docs_update / notify-failure (push) Blocked by required conditions
2025-10-02 22:39:19 -04:00
Mike Stepanovic
7549b06024
fix syntax that dbt doesn't like (#489)
Some checks failed
docs_update / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / notify-failure (push) Has been cancelled
dbt_run_scheduled_daily / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_daily / notify-failure (push) Has been cancelled
Co-authored-by: Mike Stepanovic <mike.stepanovic@flipsidecrypto.com>
2025-09-26 11:06:10 -06:00
tarikceric
64ce354fbd
add quotes to date (#488)
Some checks failed
docs_update / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
2025-09-18 11:28:38 -07:00
Jack Forgash
b6bb25d7a6
add fallback (#487) 2025-09-17 10:45:41 -06:00
Jack Forgash
05da02b5fe
AN-6550/flow testnet v2 (#483)
* flow testnet v2

* set batch sizes

* add QN node_url to sl testnet model

* upd node_url in testnet rt models

* add gha workflows for testnet

* testnet gold models

* upd tag on gold

* upd tests

* upd tests and set min to 280mm

* upd test - rm null

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2025-09-16 20:32:27 -06:00
Jack Forgash
45a68debed
AN-6231/modernize incremental predicates (#442)
Some checks failed
dbt_run_streamline_evm_daily_silver / run_dbt_jobs (push) Has been cancelled
dbt_run_evm_decoded_logs / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_transactions / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_transaction_results / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_external_realtime / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_external_points_balances_realtime / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_evm_realtime / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_blocks / run_dbt_jobs (push) Has been cancelled
dbt_run_evm / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_non_core / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_collections / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
dbt_run_streamline_evm_daily_realtime / notify-failure (push) Has been cancelled
dbt_run_moments_metadata / notify-failure (push) Has been cancelled
dbt_observability_models / notify-failure (push) Has been cancelled
dbt_run_streamline_evm_daily_silver / notify-failure (push) Has been cancelled
dbt_run_evm_decoded_logs / notify-failure (push) Has been cancelled
dbt_run_streamline_transactions / notify-failure (push) Has been cancelled
dbt_run_streamline_transaction_results / notify-failure (push) Has been cancelled
dbt_run_streamline_external_realtime / notify-failure (push) Has been cancelled
dbt_run_streamline_external_points_balances_realtime / notify-failure (push) Has been cancelled
dbt_run_streamline_evm_realtime / notify-failure (push) Has been cancelled
dbt_run_scheduled / notify-failure (push) Has been cancelled
dbt_run_streamline_blocks / notify-failure (push) Has been cancelled
dbt_run_evm / notify-failure (push) Has been cancelled
dbt_run_scheduled_non_core / notify-failure (push) Has been cancelled
dbt_run_streamline_collections / notify-failure (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / notify-failure (push) Has been cancelled
* upd incr predicate on silver core models

* upd rest of the models

* rm predicate from d+i model
2025-08-06 12:40:17 -04:00
SAI
5194f813dd
Swaps increment factory (#453)
Some checks failed
docs_update / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / notify-failure (push) Has been cancelled
dbt_run_streamline_evm_daily_silver / run_dbt_jobs (push) Has been cancelled
dbt_test_recent_evm / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_evm_daily_silver / notify-failure (push) Has been cancelled
dbt_test_recent_evm / notify-failure (push) Has been cancelled
dbt_run_scheduled_streamline_non_core / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_scores / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_daily / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_streamline_non_core / notify-failure (push) Has been cancelled
dbt_run_scheduled_scores / notify-failure (push) Has been cancelled
dbt_run_scheduled_daily / notify-failure (push) Has been cancelled
dbt_test_scheduled / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_non_core / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_abis / run_dbt_jobs (push) Has been cancelled
dbt_run_evm / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_non_core / notify-failure (push) Has been cancelled
dbt_run_scheduled_abis / notify-failure (push) Has been cancelled
dbt_run_evm / notify-failure (push) Has been cancelled
* added

* swaps final

* added

* reorg files

* reorg + rename

---------

Co-authored-by: SAI <sairaj@flipsidecrypto.com>
Co-authored-by: gregoriustanleyy <gstanleytejakusuma@gmail.com>
2025-07-28 23:10:55 +07:00
Austin
197c506d8c
transfers (#460) 2025-06-24 16:12:56 -04:00
eric-laurello
68f25187d6
add stargate to complete (#457) 2025-06-18 13:27:09 -04:00
SAI
8cf0157fb4
unique key fixed (#449)
Co-authored-by: SAI <sairaj@flipsidecrypto.com>
2025-06-02 10:05:30 -05:00
SAI
bf21d191fb
Layerzero bridge (#446)
* add

* added

* aded

* updated

* updated

* csv

* updated

* added

* updated name

* add createpool add inc predicates

* changes pool file

* added

* removed limit

* updates

* removed commented code

---------

Co-authored-by: SAI <sairaj@flipsidecrypto.com>
Co-authored-by: Eric Laurello <eric.laurello@flipsidecrypto.com>
2025-05-29 11:25:17 -05:00
SAI
3d13e03a45
Topshot data (#440)
* livequery UDF model

* streamline models

* Add streamline model for TopShot metadata real-time processing

* buyback table

* add

* topshot and all day

* added streamline changes

* realtime

* update

* updated

* SL work - proposed changes (#441)

* updates

* .

* rev

* view to table

* fixed a few bugs

* sql limit

---------

Co-authored-by: eric-laurello <102970824+eric-laurello@users.noreply.github.com>
Co-authored-by: Eric Laurello <eric.laurello@flipsidecrypto.com>
2025-05-28 15:40:14 -05:00
SAI
5dffab25ab
Refactor streamline transaction results SQL to remove data size const… (#437)
* Refactor streamline transaction results SQL to remove data size constraint

* Removed the OCTET_LENGTH(DATA) < 16777216 condition from the query.
* Cleaned up unnecessary comments and code related to data size checks.

* update

* update

* from
2025-04-23 17:15:15 -05:00
RAI
47995aab12
added retry window to streamline transactions final (#434) 2025-04-14 10:11:27 -05:00
RAI
a3e481dd80
added manual fix (#432)
* added manual fix

* ffixed
2025-04-10 15:10:18 -05:00
Jack Forgash
ccf2b2600e
set max data size (#428) 2025-04-08 19:23:00 -06:00
Jack Forgash
1ae1816673
Modify Transaction Results (#427)
* rm data from complete txres model

* comment

* rm data from complete tx

* add manual load to txres

* add RETRY_WINDOW var

* fix tx final RETRY_WINDOW
2025-04-08 11:22:07 -06:00
Jack Forgash
24eded92bc
upd block range for latest overflow error (#425) 2025-04-05 10:53:08 -06:00
Jack Forgash
95aca6fa9a
add back field (#418) 2025-03-31 12:14:46 -06:00
RAI
dd2b111960
adding a bandaid solution to avoid truncation error (#417)
* adding a bandaid solution to avoid truncation error

* added here too
2025-03-28 12:16:02 -05:00
Jack Forgash
e6f71d4e49
add block filter to tx results models (#415) 2025-03-27 11:05:18 -06:00
Jack Forgash
42bf946bf0
move streamline dir out of silver (#379)
* move streamline dir out of silver

* move evm streamline from silver->evm

* upd makefile and readme
2024-11-18 14:04:46 -07:00
Jack Forgash
22fe605582
AN-5413/Snag API (#376)
* init api models

* rm

* add vault path

* fix typo json key

* rm qualify from first silver model

* points spend silver model

* upd silver spend columns

* gha workflow and tags

* disable workflow

* disable correct gha

* add yml for silver model

* tag w streamline_non_core

* upd tags, gha workflow, final tweaks

* upd so, cluster, unique_key
2024-11-18 12:35:25 -07:00
Jack Forgash
4df472bace
AN-5459/point transfers api integration (#378)
* point transfers api integration

* upd 2 silver models

* gha workflow with auth

* upd gha workflow

* add gha on push config for testing

* add print step

* chg echo

* upd auth.py

* rm run on commit from gha

* upd model tags

* merge 2 silver into 1, add yml, upd gha

* upd auth return vals

* add exit 1 on failure

* upd return on success True

* add gha on-run config for final test

* add backup default to env_var(JWT)

* add backup default to env_var(JWT) - 2

* rm run on commit from gha and upd sql limit for balance call

* add yml for evm address model

* add evm address threshold test

* upd gha workflow

* upd per CRs
2024-11-15 09:43:52 -07:00
Jack Forgash
7edd60669c
AN-5418/Flow x EVM address mapping (#377)
* address mapping

* gold

* upd comment, recency threshold, kill distinct
2024-11-11 12:50:01 -07:00
Jack Forgash
bf0f515937
rm line (#375) 2024-11-04 13:32:19 -07:00
Jack Forgash
6ed4afda47
incr strategy to delete+insert (#374) 2024-11-04 12:27:29 -07:00
Jack Forgash
64a009f0ab
AN-5252/points api integration (#368)
* basic query

* external points api request

* rn REWARD_POINTS and add supplementary models

* upd vault

* add gha workflows

* add headers incl user-agent

* upd header

* rm content type header

* addback secret

* use target

* rn _realtime

* upd silver and add FR view
2024-10-11 14:16:25 -06:00
Jack Forgash
96755a007d
AN-5236/new transfers model (#364)
* new transfers models - still a few edge cases

* fix 1 withdrawal 2 deposited edge case

* upd filter in legacy model

* core union all and view -> incremental

* rm silver so
2024-09-19 13:31:36 -06:00
Jack Forgash
2988813d42
upd tag to non-core (#362) 2024-09-13 13:21:24 -06:00
Jack Forgash
9a7151bfc1
AN-5209/Upd staking model (#358)
* Upd staking model

* drop _s suffix

* rm event_data col

* upd desc
2024-09-13 11:07:37 -06:00
Jack Forgash
ccf595e199
AN-5100/tx actors model (#361)
* tx actors model

* upd silver

* gold ez tbl

* segment out actors from tx-level participants

* Add alias

* add rn to de-duplicate join in ez

* add incremental_predicates

* final add mod ts cluster, tx_succeeded
2024-09-13 09:40:45 -06:00
Jack Forgash
70d0393cfa
AN-5196 coreevm final (#356)
* an-5198-upd-block-gap-test

* evm block gap test

* evm gaps tests

* blocks and txs tests

* logs tests

* add evm gap test to daily test gha

* run evm job in separate gha workflow
2024-09-09 12:02:03 -06:00
Jack Forgash
610f0174fd
swaps final merge strat d+i (#355) 2024-09-07 10:57:24 -04:00
Jack Forgash
e37db5668f
d+i (#352) 2024-09-04 18:45:33 -06:00
Jack Forgash
433e62a63c
AN-4553/deploy new defi gold views (#345)
* bridge complete model

* depr notice on ez. Tests on fact bridge. TODOs - naming convention for amount and use chain IDs??

* ez and fact bridging

* dex swaps

* fix id col name. add platform to dex

* rm adj
2024-08-01 10:22:24 -06:00
Jack Forgash
3e471404e5
AN-5069/expand onchain metadata mapping (#346) 2024-07-31 08:24:25 -06:00
Jack Forgash
51607b2222
add tag back (#344) 2024-07-26 13:49:04 -06:00
eric-laurello
61c60b99c1
recorded_hour (#342) 2024-07-15 14:33:45 -04:00
eric-laurello
f6b98de014
so & clsutering (#341) 2024-07-15 09:28:47 -06:00
Jack Forgash
30ce62f6a0
AN-4936/Backfill Migration Final (#335)
* del cw models

* core views

* defi views

* gov view

* nft_views

* rm ref to old cw models

* del internal cols

* nv 13-15

* nv 13-15

* nv 17-18

* nv 20

* del temp logic

* upd tests on allday metadata

* upd to warn
2024-07-01 13:27:42 -06:00
Jack Forgash
9127e6de86
AN-4852/Testnet Integration (#326)
* source, bronze, complete, rt views

* complete testnet_ dependency

* upd sources.yml

* upd models and bronze

* upd models.sql

* upd model in bronze views

* adding chainhead_testnet

* add jobs

* fix: hour

* upd api integration

* various fixes. useast2, gha workflows, bad refs, etc

* update udtf_get_base_table

---------

Co-authored-by: WHYTEWYLL <juanmiblanco1@gmail.com>
2024-06-25 12:12:51 -06:00
Jack Forgash
12bf4e2e86
AN-4799/Backfill Migration (#329)
* move models to bronze

* blocks backfill logic ez ez ez

* streamline_load models for all 4 methods

* add back a set of views for the complete models to use

* del dbt run history gha workflow

* migrate txs final to modified ts

* upd events to modified

* set default val for LOAD_BACKFILL_VERSION in project yml

* update curated model incr logic
2024-06-12 12:10:27 -06:00
Jack Forgash
f3e6d0bfab
del deprecating price models (#332) 2024-06-12 12:02:08 -06:00
WHYTEWYLL
f9e451debd
stats authorizer replacement (#328) 2024-06-04 17:43:30 -06:00
Jack Forgash
0b10065d52
incr batches (#325) 2024-05-23 14:55:23 -06:00