WHYTEWYLL
537a8b57c0
v2 to v1 ( #296 )
2024-03-14 19:25:31 -06:00
WHYTEWYLL
022390ebdc
Fix target ( #295 )
...
* hardcoding v2
* v2
2024-03-14 18:58:58 -06:00
Jack Forgash
668b22097b
expliticly call putstanding NV and methods ( #268 )
2024-02-13 14:50:37 -07:00
Shah Newaz Khan
dbebfbf0a0
fix udf 2 api integration ( #267 )
...
Co-authored-by: shah <info@shahnewazkhan.ca>
2024-02-13 08:26:57 -08:00
WHYTEWYLL
2fe9333eea
add: user agent to not get ban ( #257 )
2024-01-10 14:29:37 -03:00
Jack Forgash
971d555dff
update prod clone macro ( #240 )
2023-12-06 13:12:46 -07:00
WHYTEWYLL
9dae9f240f
Pipeline metadata ( #238 )
...
* add: if condition
* removed row_list
* add: gold view, fix macro, and silver models
* gold view extension
* add: test bronce
* g
* adding test
* add: test
* add: working test
* delete: tags
* delete: hardcode ref
2023-12-04 07:16:32 -08:00
WHYTEWYLL
6d6f478df6
alldaynft metadata with LQ ( #229 )
...
* base
* clean up; add: silver model
* add: new standards
* fix: refactor
* delete: folders
* refactor simplying
* remove no longer needed
* fix: refactor to silver layer
* ready to merge
---------
Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2023-11-27 07:55:44 -08:00
WHYTEWYLL
d17d5f06d7
deprecation ( #222 )
...
* deprecation
* del old udf creation
---------
Co-authored-by: forgash_ <jmfxyz@pm.me>
2023-11-07 10:40:24 -07:00
Jack Forgash
eb40b6d91d
Backfill Model - Incorporate all external tables ( #211 )
...
* upd sources, convert blocks to loop model, fix collections history
* multiple external tbl query macro and blocks update
* del network version.yml output
* add back rt blocks
* lower blocks
* revert str blocks and collections and move change to history view
* bronze history views and move into folders
* history workflow - blocks
* del history step from streamline blocks
* create complete get history models and upd get history streamline views - BLOCKS
* upd remaining method get history views to use history complete tables
* upd txs history dependency note
* upd history gha and del parameterized models
* fix collections external table config
* collections template fix
* upd txs templates
* upd history views
* fix to tx results template / views
* set history schedule to hourly
* del commented out backfill from streamline RT GHAs
* move blocks filter up to cte
2023-10-26 11:31:19 -06:00
desmond-hui
8ff2e1aa07
modify how dbt performs merges by default ( #213 )
2023-10-23 10:04:29 -07:00
Jack Forgash
792f11599f
AN-3903/Migration Reorg ( #196 )
...
* txs final and upd streamline tests w 1 hr inserted timestamp buffer
* copy silver models into silver_cw dir
* txs final, events model, curated model _s copies
* add pending col to txs final
* test config for pending_result_response
* add coalesce to tx id in txs final
* rem tests on inserted timestamp
* uppercase gap test vars
* add retry for null block timestamp and del mainnet23 filter from events
* address adj append 0x, add network ver to blocks, etc
* upd coalesce on events data
* saving event migration test, disabled
* req changes
* upd txs final ingested logic
2023-10-10 13:24:43 -06:00
Shah Newaz Khan
aaacbcf7e7
add UPDATE_LQ_UDFS project var ( #194 )
...
* add UPDATE_LQ_UDFS project var
* rename UPDATE_LQ_UDFS to UPDATE_EPHEMERAL_UDFS
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-13 10:52:15 -07:00
Shah Newaz Khan
c663e7523e
lq & udf_api overload deploy ( #186 )
...
* lq overload deploy success
* fixed make directive line continuation
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-13 10:26:37 -07:00
Shah Newaz Khan
bfa514ed7e
added multi region udfs ( #189 )
...
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-08 18:50:28 -07:00
Jack Forgash
4877011c99
add select tags ( #185 )
2023-09-08 08:33:59 -04:00
Jack Forgash
b1f0925a90
AN-3712/Metadata livequery ( #146 )
...
* topshot metadata pull v1
* before swap from lambda
* ready for stress test / batch sizing tmrw
* define unique key in helper tbl
* typo
* add todo deloy udf_api
* slight format tweak
* named
* migrate to livequery schema
* add parameter yml
* add livequery model yml, need tests
* alter metadata job run
* del old streamline macros
* silver upd - in flight
* clean up py model
* limit comment
* revert silver topshot metadata
* new silver model
* union new lq data into nft metadata table(s)
* del block_timestamp from metadata needed view
* untag silver nft allday metadata
* set limit to 3500
* addback topshot tag
* update schedule to hourly
2023-09-07 10:13:02 -06:00
Jack Forgash
d8c023ba46
An 3897/streamline tests ( #180 )
2023-09-05 16:44:38 -06:00
Shah Newaz Khan
920d811c90
update dev udf urls ( #165 )
...
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-28 15:58:53 -07:00
Shah Newaz Khan
f3a5b863e1
Stream 392 fix prod ( #154 )
...
* node url join optimized | warehouse privilege grants
* added udf api
* revert udtf alias
* fixed incremental complete runs
* node url join optimized | warehouse privilege grants
* added udf api
* reverted case insentive filename changes
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-22 12:01:55 -07:00
Shah Newaz Khan
1beb5f8ad6
Stream 392 lq udf ( #151 )
...
* node url join optimized | warehouse privilege grants
* added udf api
* revert udtf alias
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-22 10:29:42 -07:00
Shah Newaz Khan
2413473512
GHA & Backfill Docs ( #149 )
...
* node url join optimized | warehouse privilege grants
* added string interpolation for env based target.database
* revert to candidate node 7 root height
* collections history success
* removed end height from network_version
* removed EOF
* added prod udf
* added tx & tx_result backfill models
* adds FR models
* add prod api integraton
* switched dev flag off
* added gha & docs
* updated serialization png
* added tx specific gha model selection
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-16 15:57:04 -07:00
Shah Newaz Khan
fe7f688cc2
Stream 98 collection backfill ( #145 )
...
* node url join optimized | warehouse privilege grants
* added string interpolation for env based target.database
* revert to candidate node 7 root height
* collections history success
* removed end height from network_version
* removed EOF
* added prod udf
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-10 14:23:29 -07:00
Shah Newaz Khan
b344373700
Stream 98 node url seed ( #138 )
...
* added clone purge fix make directive
* fixed json parsing for get blocks history
* added aws_lambda_role envar to undo_clone directive | added database grants to lambda role
* get_node_url parsing error
* fixed varchar return type for get_node_url
* node mapping window function success
* removed get_node_url macro | added node_url seed
* removed debug logs | removed create node_url macro
* added filter to node_mapping CTE
* Update models/silver/streamline/core/streamline__blocks.sql
* fixed merge conflict
* Reduce columns for get_blocks_history
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-01 14:56:15 -07:00
Shah Newaz Khan
4a1596b0d0
Stream 97 blocks assembly ( #134 )
...
* added clone purge fix make directive
* fixed json parsing for get blocks history
* added aws_lambda_role envar to undo_clone directive | added database grants to lambda role
* removed debug logs
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-31 12:54:18 -07:00
Shah Newaz Khan
e39845bc3d
Stream 97 streamline blocks ( #129 )
...
* udf_bulk_grpc success
* removed test analysis sql
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-24 09:24:52 -07:00
Shah Newaz Khan
cecced9db1
streamline flow chainhead udf ( #128 )
...
* added makefile udf/api integration directives | added api integration / udf macros
* added reamdme comments
---------
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-18 13:04:21 -07:00
Jack Forgash
eb20a6c924
fix create udfs ( #123 )
...
* fix create udfs
* move_block
2023-06-06 11:33:54 -06:00
Jack Forgash
59d72bfea8
add utils package ( #122 )
...
* add utils package
* add create udfs
* rename core dir to gold
* vars->inputs
2023-06-06 10:42:38 -06:00
Jack Forgash
ea19cd3857
tags fix ( #119 )
2023-05-04 12:39:55 -06:00
Jack Forgash
6e6fbc3bdd
tags ( #104 )
2023-01-23 10:59:46 -07:00
eric-laurello
c5dc787f21
add custom query tags ( #97 )
2022-12-13 16:22:42 -08:00
desmond-hui
ed39c69c7a
add resources for community curation ( #75 )
2022-10-17 16:32:28 -07:00
Jack Forgash
861e667d4b
AN-1870/allday ids ( #51 )
...
* allday nft ids
* silver topshot view for compat
* limit allday to 3000
* str
* add nfl all day metadata integration and refactor streamline process
* no longer needed
Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2022-08-26 11:07:40 -07:00
Jack Forgash
edb39921fa
AN-1807/store tests ( #53 )
...
* store tests
* dbt 1.2
* test names and off for ingested at
2022-08-19 14:13:02 -06:00
desmond-hui
ab3c73e994
An 1709/make dev refresh dbt cloud op ( #38 )
...
* dev refresh will now run on dbt cloud
* remove unintended file commit
2022-06-30 18:07:25 -07:00
desmond-hui
a1bdcbb529
An 1625/integrate moments minted api ( #34 )
...
* clone stages to dev
* add api integration for topshot metadata
2022-06-28 09:24:09 -07:00
desmond-hui
8279eaac27
clone stages to dev ( #30 )
2022-06-21 09:20:06 -07:00
Jack Forgash
a956618336
AN-1201/tests ( #18 )
...
* tests
* docs
* date filter
* and
* events
2022-05-19 17:53:07 -06:00
Desmond Hui
fdc4e6cf12
initial setup files
2022-04-14 11:34:12 -07:00
