Jack Forgash
45a68debed
AN-6231/modernize incremental predicates ( #442 )
...
dbt_run_streamline_evm_daily_silver / run_dbt_jobs (push) Has been cancelled
dbt_run_evm_decoded_logs / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_transactions / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_transaction_results / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_external_realtime / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_external_points_balances_realtime / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_evm_realtime / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_blocks / run_dbt_jobs (push) Has been cancelled
dbt_run_evm / run_dbt_jobs (push) Has been cancelled
dbt_run_scheduled_non_core / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_collections / run_dbt_jobs (push) Has been cancelled
docs_update / notify-failure (push) Has been cancelled
dbt_run_streamline_evm_daily_realtime / notify-failure (push) Has been cancelled
dbt_run_moments_metadata / notify-failure (push) Has been cancelled
dbt_observability_models / notify-failure (push) Has been cancelled
dbt_run_streamline_evm_daily_silver / notify-failure (push) Has been cancelled
dbt_run_evm_decoded_logs / notify-failure (push) Has been cancelled
dbt_run_streamline_transactions / notify-failure (push) Has been cancelled
dbt_run_streamline_transaction_results / notify-failure (push) Has been cancelled
dbt_run_streamline_external_realtime / notify-failure (push) Has been cancelled
dbt_run_streamline_external_points_balances_realtime / notify-failure (push) Has been cancelled
dbt_run_streamline_evm_realtime / notify-failure (push) Has been cancelled
dbt_run_scheduled / notify-failure (push) Has been cancelled
dbt_run_streamline_blocks / notify-failure (push) Has been cancelled
dbt_run_evm / notify-failure (push) Has been cancelled
dbt_run_scheduled_non_core / notify-failure (push) Has been cancelled
dbt_run_streamline_collections / notify-failure (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / run_dbt_jobs (push) Has been cancelled
dbt_run_streamline_decoded_logs_history / notify-failure (push) Has been cancelled
* upd incr predicate on silver core models
* upd rest of the models
* rm predicate from d+i model
2025-08-06 12:40:17 -04:00
Austin
e119e85c07
An 5719/flow rework ( #396 )
...
* core models
* tests
* decoder / abis
* history process
* workflows
* end points
* workflow
2025-01-30 14:11:04 -05:00
Jack Forgash
88df737fa2
AN-5203/Deploy flowEVM ( #343 )
...
* flow evm init
* flow evm testnet - blocks
* deploy dev udfs, update typo
* send block height as hex
* dev limit
* use stg external table
* some links
* add query to side doc
* upd external tbl to stg & testnet
* silver and evm txs
* testnet silver models pt1
* testnet events final, decode hash udf
* udf get evm chainhead
* upd resource list
* new udf. receipts pipeline. upd testnet model cols. expand readme
* receipts silver
* traces
* vault
* reorg to evm dir
* rm testnet, blocks lookback
* lookbacks, move qualify
* macro and align naming
* del testnet, reset namespace to gen
* silver_evm - receipts and txs
* core_evm fact_blocks & fact_transactions
* fix vault path in get blocks. Logs v1 (need sample)
* core_evm fact_logs
* CR updates
* del readme and add workflow
* upd vault path, batch size
* incr logic on modified to core_evm
* use local utils.udf, add blockNumber col to complete blocks check
* rm col
* correct _invocation_id
* upd nv csv
* incr batch limit due to late start
* add evm tag to model run
2024-09-05 20:43:33 -06:00
Jack Forgash
e27aa3b83b
use us-east-2 integration ( #339 )
2024-06-25 14:11:59 -06:00
Jack Forgash
965d037877
upd aws_flow_api_dev_2 ( #338 )
...
* upd aws_flow_api_dev_2
* update everywhere
2024-06-25 12:45:19 -06:00
Jack Forgash
9127e6de86
AN-4852/Testnet Integration ( #326 )
...
* source, bronze, complete, rt views
* complete testnet_ dependency
* upd sources.yml
* upd models and bronze
* upd models.sql
* upd model in bronze views
* adding chainhead_testnet
* add jobs
* fix: hour
* upd api integration
* various fixes. useast2, gha workflows, bad refs, etc
* update udtf_get_base_table
---------
Co-authored-by: WHYTEWYLL <juanmiblanco1@gmail.com>
2024-06-25 12:12:51 -06:00
Shah Newaz Khan
fd840c2c3e
add flow evm v2 udfs ( #337 )
2024-06-25 07:49:43 -07:00
WHYTEWYLL
537a8b57c0
v2 to v1 ( #296 )
2024-03-14 19:25:31 -06:00
WHYTEWYLL
022390ebdc
Fix target ( #295 )
...
* hardcoding v2
* v2
2024-03-14 18:58:58 -06:00
Jack Forgash
668b22097b
expliticly call putstanding NV and methods ( #268 )
2024-02-13 14:50:37 -07:00
Shah Newaz Khan
dbebfbf0a0
fix udf 2 api integration ( #267 )
...
Co-authored-by: shah <info@shahnewazkhan.ca>
2024-02-13 08:26:57 -08:00
WHYTEWYLL
2fe9333eea
add: user agent to not get ban ( #257 )
2024-01-10 14:29:37 -03:00
Jack Forgash
971d555dff
update prod clone macro ( #240 )
2023-12-06 13:12:46 -07:00
WHYTEWYLL
9dae9f240f
Pipeline metadata ( #238 )
...
* add: if condition
* removed row_list
* add: gold view, fix macro, and silver models
* gold view extension
* add: test bronce
* g
* adding test
* add: test
* add: working test
* delete: tags
* delete: hardcode ref
2023-12-04 07:16:32 -08:00
WHYTEWYLL
6d6f478df6
alldaynft metadata with LQ ( #229 )
...
* base
* clean up; add: silver model
* add: new standards
* fix: refactor
* delete: folders
* refactor simplying
* remove no longer needed
* fix: refactor to silver layer
* ready to merge
---------
Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2023-11-27 07:55:44 -08:00
WHYTEWYLL
d17d5f06d7
deprecation ( #222 )
...
* deprecation
* del old udf creation
---------
Co-authored-by: forgash_ <jmfxyz@pm.me>
2023-11-07 10:40:24 -07:00
Jack Forgash
eb40b6d91d
Backfill Model - Incorporate all external tables ( #211 )
...
* upd sources, convert blocks to loop model, fix collections history
* multiple external tbl query macro and blocks update
* del network version.yml output
* add back rt blocks
* lower blocks
* revert str blocks and collections and move change to history view
* bronze history views and move into folders
* history workflow - blocks
* del history step from streamline blocks
* create complete get history models and upd get history streamline views - BLOCKS
* upd remaining method get history views to use history complete tables
* upd txs history dependency note
* upd history gha and del parameterized models
* fix collections external table config
* collections template fix
* upd txs templates
* upd history views
* fix to tx results template / views
* set history schedule to hourly
* del commented out backfill from streamline RT GHAs
* move blocks filter up to cte
2023-10-26 11:31:19 -06:00
desmond-hui
8ff2e1aa07
modify how dbt performs merges by default ( #213 )
2023-10-23 10:04:29 -07:00
Jack Forgash
792f11599f
AN-3903/Migration Reorg ( #196 )
...
* txs final and upd streamline tests w 1 hr inserted timestamp buffer
* copy silver models into silver_cw dir
* txs final, events model, curated model _s copies
* add pending col to txs final
* test config for pending_result_response
* add coalesce to tx id in txs final
* rem tests on inserted timestamp
* uppercase gap test vars
* add retry for null block timestamp and del mainnet23 filter from events
* address adj append 0x, add network ver to blocks, etc
* upd coalesce on events data
* saving event migration test, disabled
* req changes
* upd txs final ingested logic
2023-10-10 13:24:43 -06:00
Shah Newaz Khan
aaacbcf7e7
add UPDATE_LQ_UDFS project var ( #194 )
...
* add UPDATE_LQ_UDFS project var
* rename UPDATE_LQ_UDFS to UPDATE_EPHEMERAL_UDFS
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-13 10:52:15 -07:00
Shah Newaz Khan
c663e7523e
lq & udf_api overload deploy ( #186 )
...
* lq overload deploy success
* fixed make directive line continuation
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-13 10:26:37 -07:00
Shah Newaz Khan
bfa514ed7e
added multi region udfs ( #189 )
...
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-08 18:50:28 -07:00
Jack Forgash
4877011c99
add select tags ( #185 )
2023-09-08 08:33:59 -04:00
Jack Forgash
b1f0925a90
AN-3712/Metadata livequery ( #146 )
...
* topshot metadata pull v1
* before swap from lambda
* ready for stress test / batch sizing tmrw
* define unique key in helper tbl
* typo
* add todo deloy udf_api
* slight format tweak
* named
* migrate to livequery schema
* add parameter yml
* add livequery model yml, need tests
* alter metadata job run
* del old streamline macros
* silver upd - in flight
* clean up py model
* limit comment
* revert silver topshot metadata
* new silver model
* union new lq data into nft metadata table(s)
* del block_timestamp from metadata needed view
* untag silver nft allday metadata
* set limit to 3500
* addback topshot tag
* update schedule to hourly
2023-09-07 10:13:02 -06:00
Jack Forgash
d8c023ba46
An 3897/streamline tests ( #180 )
2023-09-05 16:44:38 -06:00
Shah Newaz Khan
920d811c90
update dev udf urls ( #165 )
...
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-28 15:58:53 -07:00
Shah Newaz Khan
f3a5b863e1
Stream 392 fix prod ( #154 )
...
* node url join optimized | warehouse privilege grants
* added udf api
* revert udtf alias
* fixed incremental complete runs
* node url join optimized | warehouse privilege grants
* added udf api
* reverted case insentive filename changes
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-22 12:01:55 -07:00
Shah Newaz Khan
1beb5f8ad6
Stream 392 lq udf ( #151 )
...
* node url join optimized | warehouse privilege grants
* added udf api
* revert udtf alias
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-22 10:29:42 -07:00
Shah Newaz Khan
2413473512
GHA & Backfill Docs ( #149 )
...
* node url join optimized | warehouse privilege grants
* added string interpolation for env based target.database
* revert to candidate node 7 root height
* collections history success
* removed end height from network_version
* removed EOF
* added prod udf
* added tx & tx_result backfill models
* adds FR models
* add prod api integraton
* switched dev flag off
* added gha & docs
* updated serialization png
* added tx specific gha model selection
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-16 15:57:04 -07:00
Shah Newaz Khan
fe7f688cc2
Stream 98 collection backfill ( #145 )
...
* node url join optimized | warehouse privilege grants
* added string interpolation for env based target.database
* revert to candidate node 7 root height
* collections history success
* removed end height from network_version
* removed EOF
* added prod udf
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-10 14:23:29 -07:00
Shah Newaz Khan
b344373700
Stream 98 node url seed ( #138 )
...
* added clone purge fix make directive
* fixed json parsing for get blocks history
* added aws_lambda_role envar to undo_clone directive | added database grants to lambda role
* get_node_url parsing error
* fixed varchar return type for get_node_url
* node mapping window function success
* removed get_node_url macro | added node_url seed
* removed debug logs | removed create node_url macro
* added filter to node_mapping CTE
* Update models/silver/streamline/core/streamline__blocks.sql
* fixed merge conflict
* Reduce columns for get_blocks_history
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-01 14:56:15 -07:00
Shah Newaz Khan
4a1596b0d0
Stream 97 blocks assembly ( #134 )
...
* added clone purge fix make directive
* fixed json parsing for get blocks history
* added aws_lambda_role envar to undo_clone directive | added database grants to lambda role
* removed debug logs
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-31 12:54:18 -07:00
Shah Newaz Khan
e39845bc3d
Stream 97 streamline blocks ( #129 )
...
* udf_bulk_grpc success
* removed test analysis sql
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-24 09:24:52 -07:00
Shah Newaz Khan
cecced9db1
streamline flow chainhead udf ( #128 )
...
* added makefile udf/api integration directives | added api integration / udf macros
* added reamdme comments
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-18 13:04:21 -07:00
Jack Forgash
eb20a6c924
fix create udfs ( #123 )
...
* fix create udfs
* move_block
2023-06-06 11:33:54 -06:00
Jack Forgash
59d72bfea8
add utils package ( #122 )
...
* add utils package
* add create udfs
* rename core dir to gold
* vars->inputs
2023-06-06 10:42:38 -06:00
Jack Forgash
ea19cd3857
tags fix ( #119 )
2023-05-04 12:39:55 -06:00
Jack Forgash
6e6fbc3bdd
tags ( #104 )
2023-01-23 10:59:46 -07:00
eric-laurello
c5dc787f21
add custom query tags ( #97 )
2022-12-13 16:22:42 -08:00
desmond-hui
ed39c69c7a
add resources for community curation ( #75 )
2022-10-17 16:32:28 -07:00
Jack Forgash
861e667d4b
AN-1870/allday ids ( #51 )
...
* allday nft ids
* silver topshot view for compat
* limit allday to 3000
* str
* add nfl all day metadata integration and refactor streamline process
* no longer needed
Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2022-08-26 11:07:40 -07:00
Jack Forgash
edb39921fa
AN-1807/store tests ( #53 )
...
* store tests
* dbt 1.2
* test names and off for ingested at
2022-08-19 14:13:02 -06:00
desmond-hui
ab3c73e994
An 1709/make dev refresh dbt cloud op ( #38 )
...
* dev refresh will now run on dbt cloud
* remove unintended file commit
2022-06-30 18:07:25 -07:00
desmond-hui
a1bdcbb529
An 1625/integrate moments minted api ( #34 )
...
* clone stages to dev
* add api integration for topshot metadata
2022-06-28 09:24:09 -07:00
desmond-hui
8279eaac27
clone stages to dev ( #30 )
2022-06-21 09:20:06 -07:00
Jack Forgash
a956618336
AN-1201/tests ( #18 )
...
* tests
* docs
* date filter
* and
* events
2022-05-19 17:53:07 -06:00
Desmond Hui
fdc4e6cf12
initial setup files
2022-04-14 11:34:12 -07:00
