Commit Graph

340 Commits

Author SHA1 Message Date
eric-laurello
5734a9b743
fix inc stats models (#281) 2024-03-01 14:40:47 -05:00
WHYTEWYLL
989fdb6cfa
add stats flow, gas pending (#279)
* add stats flow, gas pending

* fee to table

* stats flow

* fix float, schedule
2024-02-26 16:13:58 -03:00
WHYTEWYLL
3cc7463512
semicolumn (#278) 2024-02-21 07:21:05 -03:00
WHYTEWYLL
6b65655da8
Finetune (#277)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del

* increasing producer batch size

* increasing batch size and worker

* workers limit

* reducing producer

* add fin

* fix doubled key values
2024-02-20 12:07:10 -03:00
WHYTEWYLL
7384530ccf
Finetune (#276)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del

* increasing producer batch size

* increasing batch size and worker

* workers limit

* reducing producer
2024-02-15 20:03:40 -03:00
WHYTEWYLL
4a723f90c5
Finetune (#275)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del

* increasing producer batch size

* increasing batch size and worker

* workers limit
2024-02-15 19:18:36 -03:00
WHYTEWYLL
99d1e0812a
Finetune (#274)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del

* increasing producer batch size

* increasing batch size and worker
2024-02-15 19:07:37 -03:00
WHYTEWYLL
7c5cca0278
Finetune (#273)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del

* increasing producer batch size
2024-02-15 18:09:34 -03:00
WHYTEWYLL
1043c83d4f
Finetune (#272)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del
2024-02-15 17:08:45 -03:00
WHYTEWYLL
298a2530b9
Finetune (#271)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18
2024-02-15 13:10:51 -03:00
WHYTEWYLL
17191eacd0
Finetune (#270)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error
2024-02-14 15:07:30 -03:00
Jack Forgash
668b22097b
expliticly call putstanding NV and methods (#268) 2024-02-13 14:50:37 -07:00
WHYTEWYLL
35c6053bcd
Finetune (#266)
* test a from the base line on env var

* quotes
2024-02-13 11:30:01 -03:00
WHYTEWYLL
759836fc8d
An 4437 add or account for all day burned moments (#263)
* a

* filter previus attempts

* fix empty pack

* del filt

* severity adjust
2024-02-05 15:07:51 -03:00
WHYTEWYLL
6d53230f0a
Fix labels (#265)
* add deleted

* hotfix
2024-01-31 16:11:00 -03:00
WHYTEWYLL
39d17e3f12
add deleted (#264) 2024-01-31 14:20:56 -03:00
WHYTEWYLL
e7615a0cc6
Update labels (#262)
* a

* lavels source updated

* add core view

* unique id

* delete coalesces
2024-01-31 13:44:59 -03:00
Jack Forgash
cb95faf856
beta view w rosters (#259) 2024-01-20 14:32:15 -07:00
Jack Forgash
e0818966ad
AN-4428/add user agent to topshot and upd metadata job (#258)
* add user agent to topshot and upd metadata job

* upd agent

* upd limit to 600
2024-01-18 09:19:18 -07:00
WHYTEWYLL
59b3a5b4b5
fix: parse (#248)
* fix: parse

* fix: hex parse

* fix: format

* add: fix

* delete: last month filter

* a

* fix: parser

* delete: date filter
2024-01-16 12:09:37 -07:00
Jack Forgash
7c87cd4225
Beta allday espn id view (#255) 2024-01-05 12:57:55 -07:00
Jack Forgash
074c11c55a
fix id col in dim allday metadata (#253) 2024-01-04 09:03:25 -07:00
Jack Forgash
b26bf67732
upd desc on views (#251) 2023-12-19 17:26:19 -05:00
Jack Forgash
4c4ef12ca8
AN-4333/beta views for archiving espn ids for livequery use (#249)
* beta views for archiving espn ids for livequery use

* list cols
2023-12-19 13:40:30 -05:00
WHYTEWYLL
50962af5a5
fix: usdc mint transfers (#247)
* fix: usdc mint transfers

* format

* refactor
2023-12-18 11:57:18 -03:00
WHYTEWYLL
5c4e7bfdf8
fix: errors (#246)
* fix: errors

* format

* fix: typo
2023-12-15 07:21:42 -08:00
WHYTEWYLL
e8851383c9
Flow - add pk/inserted & modified dates/_invocation id (#239)
* add: keys core

* add: silver labels jk

* add: silver metadata jk

* add silver prices pk

* add: silver staking jk

* add: silver swaps and transfers jk

* fix: doc

* gold core pk

* gold: defi keys

* price and half nfts

* add missing comma in silver__streamline_blocks

* add: nfts

* fix: models

* list col names in swaps_events_s

* add underscore _invocation

* fix: gold layer

* delete: description

* fix: order and names

* delete: unecesarry qualify

* fix: errors

* fix: name

* dix: name

* rename id

* add: pk to topshot

* add: dim rename

* fix dim metadata views

---------

Co-authored-by: Jack Forgash <58153492+forgxyz@users.noreply.github.com>
2023-12-12 11:49:18 -08:00
Jack Forgash
28e4dd0981
upd lookback in tx res cte (#241)
* upd lookback in tx res cte

* autofmt
2023-12-06 16:13:17 -07:00
WHYTEWYLL
9dae9f240f
Pipeline metadata (#238)
* add: if condition

* removed row_list

* add:  gold view, fix macro, and silver models

* gold view extension

* add: test bronce

* g

* adding test

* add: test

* add: working test

* delete: tags

* delete: hardcode ref
2023-12-04 07:16:32 -08:00
WHYTEWYLL
150fbea1ab
fix: dedeuplicates (#237) 2023-11-29 08:44:07 -08:00
WHYTEWYLL
3e5e0a4bdd
Update silver__nft_allday_metadata_s.sql (#236) 2023-11-29 08:29:39 -08:00
WHYTEWYLL
6d6f478df6
alldaynft metadata with LQ (#229)
* base

* clean up; add: silver model

* add: new standards

* fix: refactor

* delete: folders

* refactor simplying

* remove no longer needed

* fix: refactor to silver layer

* ready to merge

---------

Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2023-11-27 07:55:44 -08:00
Jack Forgash
b33dc89a39
swap set description col (#232)
* swap set description col

* cast str
2023-11-20 16:39:47 -07:00
Jack Forgash
c82c04ad5d
del invocation id from gold (#231) 2023-11-20 16:30:38 -07:00
Jack Forgash
121c3e6a2c
AN-4203/ufc strike metadata (#230)
* ufc strike metadata, plus organized nft folder better

* upd db tag on gold

* upd col name to serial number

* del listing id

* add back listing id
2023-11-20 16:26:39 -07:00
Jack Forgash
821214294c
AN-4096/DEX Swaps Bug (#228)
* add event counts col to easily ignore infra events

* excl txs where to/from is null

* add tests
2023-11-15 17:15:56 -07:00
Jack Forgash
cff19a7bb6
upd test window (#227) 2023-11-14 17:42:25 -07:00
Jack Forgash
80c2462ac4
core v non-core tags and gha workflow (#225) 2023-11-10 14:30:57 -07:00
Jack Forgash
5b096dd83f
upd labels source (#223) 2023-11-07 11:48:11 -07:00
WHYTEWYLL
d17d5f06d7
deprecation (#222)
* deprecation

* del old udf creation

---------

Co-authored-by: forgash_ <jmfxyz@pm.me>
2023-11-07 10:40:24 -07:00
Jack Forgash
cc6f797d71
alter test on bitquery job (#217) 2023-10-31 11:48:27 -06:00
Jack Forgash
eb40b6d91d
Backfill Model - Incorporate all external tables (#211)
* upd sources, convert blocks to loop model, fix collections history

* multiple external tbl query macro and blocks update

* del network version.yml output

* add back rt blocks

* lower blocks

* revert str blocks and collections and move change to history view

* bronze history views and move into folders

* history workflow - blocks

* del history step from streamline blocks

* create complete get history models and upd get history streamline views - BLOCKS

* upd remaining method get history views to use history complete tables

* upd txs history dependency note

* upd history gha and del parameterized models

* fix collections external table config

* collections template fix

* upd txs templates

* upd history views

* fix to tx results template / views

* set history schedule to hourly

* del commented out backfill from streamline RT GHAs

* move blocks filter up to cte
2023-10-26 11:31:19 -06:00
desmond-hui
63ab57f844
An 4065/incremental load tuning (#215)
* modify how dbt performs merges by default

* require at least dbt 1.4

* adjust cluster keys, add incremental pred

* ensure query is being pruned on timestamp

* add cluster key
2023-10-26 07:02:53 -07:00
Jack Forgash
45a07a1da8
An 4050/txs final test (#214)
* test on str txs final for pending response

* add events final gap test

* upd test to check gaps
2023-10-24 11:52:01 -06:00
Jack Forgash
856ac1629f
AN-3935/backfill views (#208)
* archive parameterized model

* create views for each network ver

* network version views

* newline

* switch tx results realtime back to us-east-1

* del blocks history mainnet23

* add backfill step to streamline GHA

* newline

* rm dup WHERE

* upd name and del dup where from template
2023-10-19 13:12:13 -06:00
Jack Forgash
19bbac48a2
Quickfix - Update retry_tx_ids in txs final (#209)
* adj greatest to handle null val

* del if null from retry_tx_ids cte

* add not null test to ins timestamp
2023-10-18 13:23:11 -06:00
Jack Forgash
a18123405e
add pending res to txs final lookback (#207) 2023-10-17 16:55:45 -06:00
Jack Forgash
ea46f3ee02
Adj Test on Beta and Upd Staking Actions (#206) 2023-10-16 18:17:45 -06:00
Jack Forgash
75901993ad
del tests from deprecating core models (#205) 2023-10-16 13:13:28 -06:00
Jack Forgash
bab9e01077
upd collections cte w lookback (#204)
* upd collections cte w lookback

* move lookback to cte
2023-10-13 16:00:16 -06:00
Jack Forgash
753c91d4fa
An 4030/obs model migration (#203)
* upd to streamline models

* add start block var
2023-10-13 13:29:24 -06:00
Jack Forgash
be73c9ba99
Upd workflows for streamline models. Disables CW models. (#202)
* upd workflows for streamline models. Disables CW models.

* repo var

* upd tx_count null test

* ch to 6 hrs
2023-10-12 17:35:59 -06:00
Jack Forgash
bd08ee7f8f
An 4015/streamline cutover (#200)
* core views

* defi schema

* bridge test upd

* gov schema

* nft schema

* price schema

* move tx count to blocks model

* add collection_count_agg to blocks

* upd core views to UNION ALL

* upd collections cluster_by

* upd dim moment metadata to qualify

* upd dim moment metadata test

* upd core dim contract labels to qualify and silver (cw) to incr

* upd str blocks lookback`
2023-10-12 16:23:20 -06:00
Jack Forgash
964f14ab16
upd expiring model to incl usdc (#201) 2023-10-12 09:35:00 -06:00
Jack Forgash
af9a6912ab
del+ins (#199) 2023-10-10 15:56:28 -06:00
Jack Forgash
17bb26999c
Upd tags on workflow to include silver__streamline_events (#198)
* upd tags on scheduled run

* cron to 30 mins
2023-10-10 15:35:43 -06:00
Jack Forgash
50b61ee6d8
add col (#197) 2023-10-10 13:35:09 -06:00
Jack Forgash
792f11599f
AN-3903/Migration Reorg (#196)
* txs final and upd streamline tests w 1 hr inserted timestamp buffer

* copy silver models into silver_cw dir

* txs final, events model, curated model _s copies

* add pending col to txs final

* test config for pending_result_response

* add coalesce to tx id in txs final

* rem tests on inserted timestamp

* uppercase gap test vars

* add retry for null block timestamp and del mainnet23 filter from events

* address adj append 0x, add network ver to blocks, etc

* upd coalesce on events data

* saving event migration test, disabled

* req changes

* upd txs final ingested logic
2023-10-10 13:24:43 -06:00
Jack Forgash
d28d09f53e
AN-3939/add fix gap logic and obs models tests (#195)
* add fix gap logic and obs models tests

* block height recency test on bitquery table

* add tag

* upd workflow

* explicit cast as int
2023-09-15 13:45:28 -06:00
Shah Newaz Khan
c663e7523e
lq & udf_api overload deploy (#186)
* lq overload deploy success

* fixed make directive line continuation

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-13 10:26:37 -07:00
Jack Forgash
9386c2af39
Streamline rt - flip blocks priority to desc (#191)
* add desc to RT model to prioritize CH

* flip to desc
2023-09-11 17:09:00 -06:00
Jack Forgash
ad70599d38
upd rt tx results to ohio (#190) 2023-09-08 20:03:46 -06:00
Shah Newaz Khan
bfa514ed7e
added multi region udfs (#189)
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-09-08 18:50:28 -07:00
Jack Forgash
fe7c2b724b
AN-3890/migrate udf (#188)
* upd to live

* migrate to new udf_api deployment

* adjust metadata schedule to daily
2023-09-08 12:52:40 -06:00
Jack Forgash
b1f0925a90
AN-3712/Metadata livequery (#146)
* topshot metadata pull v1

* before swap from lambda

* ready for stress test / batch sizing tmrw

* define unique key in helper tbl

* typo

* add todo deloy udf_api

* slight format tweak

* named

* migrate to livequery schema

* add parameter yml

* add livequery model yml, need tests

* alter metadata job run

* del old streamline macros

* silver upd - in flight

* clean up py model

* limit comment

* revert silver topshot metadata

* new silver model

* union new lq data into nft metadata table(s)

* del block_timestamp from metadata needed view

* untag silver nft allday metadata

* set limit to 3500

* addback topshot tag

* update schedule to hourly
2023-09-07 10:13:02 -06:00
Jack Forgash
f3f362ef54
Add 3 Mainnets 19-21 to Blocks (#182) 2023-09-05 16:44:07 -06:00
Jack Forgash
aca459a005
AN-3898/fix qualify in silver load models (#181)
* fix qualify

* del txs alias
2023-09-05 13:42:49 -06:00
Jack Forgash
92089f7b84
incr limit to 1m (#178) 2023-09-01 18:27:07 -06:00
Jack Forgash
7608c1c611
add last_3_days filter, rem tbl (#177)
* add last_3_days filter, rem tbl

* max

* rem max
2023-09-01 15:03:15 -06:00
Jack Forgash
eca7813bae
Tx Result Status + Run RT Only, at increased interval (#176)
* add tx status filter commented and comment out blocks filter

* incr frequency of RT models

* typo

* format block
2023-08-31 16:26:04 -06:00
Jack Forgash
feded0669f
disregard unsealed current mainnet blocks (#175)
* disregard unsealed current mainnet blocks

* root height from seeds

* move alias
2023-08-31 15:45:06 -06:00
Jack Forgash
caf9f4804c
An 3780/schema reorg (#153)
* folder structures

* new views

* update docs overview

* upd sources

* upd sources and move silver from base to core

* del flipside prod db prices v2

* add scheduled tag to new views
2023-08-31 10:38:17 -06:00
Jack Forgash
b55f82f5ad uncomment RT limit 2023-08-30 18:01:21 -06:00
Jack Forgash
3f3c841115
add limit and col alias to blocks rt (#172)
* add limit and col alias to blocks

* comment out limit
2023-08-30 15:17:28 -06:00
Jack Forgash
38b4e4abc9
upd start height of realtime to mainnet23 root height and add STREAMLINE_RUN_HISTORY to GHA (#171)
* upd start height of realtime to mainnet23 root height and add "STREAMLINE_RUN_HISTORY": True

* add select
2023-08-30 11:06:30 -06:00
WHYTEWYLL
6ce17a7219
Streamline Realtime and Backfill Models (#156)
* add: model to backfill

* fix: node's string format

* add tags

* upd collection model

* mainnet21 hardcoded collections history model

* del tag

* mainnet 21 getblock

* tag for 21

* realtime models

* alias num as height

* realtime tags

* add missing tag and newlines

* backfiller

* backfiller

* move script to folder (renamed) python, upd test accordingly w dir name ch

* upd script to accept model input, update jobs per method call

* error w use_dev arg

* add: silver mdoels

* limit backfill job in python script

* rename silver dbt models to streamline_ and move into silver/core

* explicit casting to silver streamline models

* add documentation to silver streamline models

* run only current mainnet and history mainnet 22 first

* activate schedule for gha

* del hardcoded mainnet models

* move history modes out of subdirs into history dir

* fix GHA vars

* del upstream 1+ from history step

* del tag

---------

Co-authored-by: Jack Forgash <jmfxyz@pm.me>
2023-08-30 08:38:01 -06:00
Jack Forgash
66ba1c1d59
add fr false to obs (#168) 2023-08-28 16:09:25 -06:00
Jack Forgash
d484b0e164
add fr false (#167) 2023-08-28 16:00:47 -06:00
Jack Forgash
6b94cb4234
remove underscore (#166) 2023-08-28 15:59:11 -06:00
Jack Forgash
51951df051
An 3859/obs model on (#163)
* obs models on

* rename workflow
2023-08-28 10:02:13 -06:00
Jack Forgash
45f72e71d8
An 3660/observability models (#155)
* blocks completeness

* block tx count external api model

* txs completeness model

* add col alias

* test result agg cte and temp test range

* curr range test

* revert source change - other pr

* xchain silver

* nuke api key & move to secrets, create workflow

* add scheduled tag

* update workflow w cache

* add obs tag

* rem testing range from txs complete

* comment out schedule and del double comman

* add default null for key var
2023-08-23 13:37:16 -06:00
Shah Newaz Khan
7e98bd6029
bug fix for non block spine coll/tx ingestion (#157)
Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-23 12:13:40 -07:00
Shah Newaz Khan
f3a5b863e1
Stream 392 fix prod (#154)
* node url join optimized | warehouse privilege grants

* added udf api

* revert udtf alias

* fixed incremental complete runs

* node url join optimized | warehouse privilege grants

* added udf api

* reverted case insentive filename changes

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-22 12:01:55 -07:00
Shah Newaz Khan
1beb5f8ad6
Stream 392 lq udf (#151)
* node url join optimized | warehouse privilege grants

* added udf api

* revert udtf alias

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-22 10:29:42 -07:00
Shah Newaz Khan
2413473512
GHA & Backfill Docs (#149)
* node url join optimized | warehouse privilege grants

* added string interpolation for env based target.database

* revert to candidate node 7 root height

* collections history success

* removed end height from network_version

* removed EOF

* added prod udf

* added tx & tx_result backfill models

* adds FR models

* add prod api integraton

* switched dev flag off

* added gha & docs

* updated serialization png

* added tx specific gha model selection

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-16 15:57:04 -07:00
Shah Newaz Khan
fe7f688cc2
Stream 98 collection backfill (#145)
* node url join optimized | warehouse privilege grants

* added string interpolation for env based target.database

* revert to candidate node 7 root height

* collections history success

* removed end height from network_version

* removed EOF

* added prod udf

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-10 14:23:29 -07:00
Shah Newaz Khan
d14843bd0c
Stream 98 fix nodeurl (#141)
* fixed node urls

* limited streamline_blocks to candidate node 7 root height

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-03 12:30:13 -07:00
Jack Forgash
67790931a4
Quickfix 080323/bad cast (#142)
* update cast to str

* add second str cast

* int
2023-08-03 11:07:39 -06:00
Shah Newaz Khan
b344373700
Stream 98 node url seed (#138)
* added clone purge fix make directive

* fixed json parsing for get blocks history

* added aws_lambda_role envar to undo_clone directive | added database grants to lambda role

* get_node_url parsing error

* fixed varchar return type for get_node_url

* node mapping window function success

* removed get_node_url macro | added node_url seed

* removed debug logs | removed create node_url macro

* added filter to node_mapping CTE

* Update models/silver/streamline/core/streamline__blocks.sql

* fixed merge conflict

* Reduce columns for get_blocks_history

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-08-01 14:56:15 -07:00
Jack Forgash
12e951471d
upd slack msg (#139)
* upd slack msg

* Update dbt_test_alert.py

* !

* alter tests
2023-08-01 10:29:14 -06:00
Shah Newaz Khan
4a1596b0d0
Stream 97 blocks assembly (#134)
* added clone purge fix make directive

* fixed json parsing for get blocks history

* added aws_lambda_role envar to undo_clone directive | added database grants to lambda role

* removed debug logs

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-31 12:54:18 -07:00
Jack Forgash
7552fb7dfe
seed and test adj (#133)
* seed and test adj

* rm_typo
2023-07-25 11:00:32 -06:00
Shah Newaz Khan
e39845bc3d
Stream 97 streamline blocks (#129)
* udf_bulk_grpc success

* removed test analysis sql

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-24 09:24:52 -07:00
Shah Newaz Khan
cecced9db1
streamline flow chainhead udf (#128)
* added makefile udf/api integration directives | added api integration / udf macros

* added reamdme comments

---------

Co-authored-by: shah <info@shahnewazkhan.ca>
2023-07-18 13:04:21 -07:00
WHYTEWYLL
de09e6e00f
add: 248 celer id (#127) 2023-06-27 12:07:39 -07:00
Jack Forgash
d25b02df55
Prices from Swaps update (#126)
* workflow_prod and core->gold

* update prices from swaps
2023-06-07 16:42:53 -06:00
Jack Forgash
dce8ba2ec9
move allday error filter to silver (#124) 2023-06-07 13:45:07 -04:00
Jack Forgash
59d72bfea8
add utils package (#122)
* add utils package

* add create udfs

* rename core dir to gold

* vars->inputs
2023-06-06 10:42:38 -06:00
Jack Forgash
0e5f837448
tx gaps off (#121) 2023-05-24 08:24:09 -06:00
Jack Forgash
c4f5ac129d
remove threshold (#120)
* remove threshold

* cast date
2023-05-05 14:04:12 -06:00
Jack Forgash
dce99ad2f7
upd metadata view (#118) 2023-04-26 11:43:39 -06:00
Jack Forgash
23702dfd7c
addl set (#117) 2023-04-18 17:13:16 -06:00
Jack Forgash
769c1f1354
add exception to moment date test (#113) 2023-04-04 10:29:27 -06:00
Jack Forgash
e6b181c95c
adjust swaps for bad txs (#112) 2023-03-13 12:12:00 -06:00
Jack Forgash
6b44229b32 excl another tx 2023-03-12 21:49:46 -06:00
Jack Forgash
13f911ca79 excl tx with dup swap 2023-03-12 19:07:08 -06:00
Jack Forgash
0541445003
excl another tx (#110) 2023-02-14 11:05:17 -07:00
Jack Forgash
f4bc1f2687
docs page (#109) 2023-02-14 11:02:34 -07:00
Jack Forgash
9c775deae5
metadata update (#108) 2023-02-13 17:53:49 -07:00
Jack Forgash
47e679b9d8
tweaks (#107) 2023-02-02 13:38:43 -07:00
Jack Forgash
09aa518880
An 2719/actions labels tweaks (#105)
* use template for GHA

* add custom notification

* move nft models to folder

* put silver models in folder

* swap labels to crosschain

* move script

* move script

* update failed test list if none
2023-01-26 09:24:49 -07:00
Jack Forgash
6e6fbc3bdd
tags (#104) 2023-01-23 10:59:46 -07:00
eric-laurello
d3c20cd76b
AN-2668 topshot packs (#103)
* ts packs

* alpha version topshot packs

* beta schema
2023-01-18 17:33:48 -05:00
desmond-hui
e5863f899a
An 2569/only retry req 3 times (#98)
* model api responses that resulted in no metadata

* dont try to get metadata after 3 retries

* fix PR comment

* use bronze model per PR
2022-12-14 17:05:19 -08:00
Jack Forgash
404b413147
An 2526/prices (#94)
* silver prices

* core view

* deprecating notice

* swap source

* silver source
2022-12-12 11:18:15 -07:00
Jack Forgash
c1d8aabc39
Minor test fixes (#93)
* threshold for gaps

* transfers
2022-12-02 13:08:25 -07:00
desmond-hui
791aeba098
use silver table instead of bronze (#91)
* use silver table instead of bronze

* add allday

* rename to event_contract in topshot

Co-authored-by: Jack Forgash <jmfxyz@pm.me>
2022-12-01 17:16:15 -07:00
Jack Forgash
458895896a
Silver Events Column (#72)
* adjusted bad events table to hold 3 error types

* typo

* clean

* main and tweak

* del bad events table
2022-12-01 16:57:51 -07:00
Jack Forgash
6be5eea928
An-2435/core liga (#92)
* core view

* restructure

* test

* unique key on metadata tbl

* la liga silver models

* la liga metadata

* final

* now its final

* final to table
2022-12-01 10:15:11 -07:00
desmond-hui
f12f90ede9
source external tables from streamline db (#90) 2022-11-30 14:16:13 -08:00
Jack Forgash
cffac5a733
moments to table (#87)
* moments to table

* unique key

* change nfl allday
2022-11-17 16:37:39 -07:00
Jack Forgash
06643d4fd9
An 2434/swap labels (#86)
* need to match poolid to contract

* pier swap pool labels

* metapier labels test & docs

* swaps labels core view

* doc comment
2022-11-17 15:41:06 -05:00
Jack Forgash
bca375668b
comma (#84) 2022-11-08 13:35:16 -07:00
Jack Forgash
c29567fb36
AN-2632/liga play metadata (#83)
* liga

* incr

* typo
2022-11-08 12:01:17 -07:00
Jack Forgash
b9d6ef2823
AN-2310/rewrite swaps (#82)
* add trades and swaps

* revert old swaps model and branch into new

* swaps v2 still some bugs

* dup token0 issue

* swaps2

* final

* tweak

* del todo

* incr
2022-11-08 11:07:42 -07:00
Jack Forgash
009f0dacab
str (#79) 2022-10-28 17:31:36 -06:00
Jack Forgash
33df30f43e
AN-2290/giglabs (#78)
* nft updates, gig mid

* handle erronous records from missing event data

* marketplace

* add to nft union

* test
2022-10-28 09:41:20 -06:00
Jack Forgash
179ebb95f4
del old fact (#77) 2022-10-24 12:25:09 -06:00
Jack Forgash
06e6b4cb78
An-2311/marketplaces (#76)
* nft updates, gig mid

* marketplace update
2022-10-20 19:10:33 -06:00
Jack Forgash
12d2cd0544
silence (#74) 2022-10-17 15:01:51 -06:00
Jack Forgash
e70d5e8fd0
ref (#73) 2022-10-14 12:27:29 -04:00
Jack Forgash
81f6b817fd
AN-2005/reduce dup tests (#61)
* blocks and txs - silver

* fewer tests

* clean-up

* a few more
2022-10-11 15:20:02 -06:00
Jack Forgash
506de6354d
An 2242/nftz (#71)
* nft_two

* fixed - just move the logic over to the main model

* fix 2 marketplace interactions

* findpack nft
2022-10-07 14:07:09 -06:00
Jack Forgash
1c6f55b3e4
oops (#69) 2022-09-28 23:15:00 -06:00
Jack Forgash
2dff91ae4a
gaia (#68) 2022-09-28 22:49:17 -06:00
Jack Forgash
44d45693aa
An 2186/nft markets batch 1 (#67)
* addtional markets

* add to test

* is_purchased fix
2022-09-27 13:47:34 -04:00
Jack Forgash
b541b628cc
AN-2178/events fix (#66)
* test excl bad events

* del date filter

* prices fix

* nft tweaks

* clean
2022-09-26 19:04:17 -04:00
Jack Forgash
78927966ff
AN-2009/nft enhancements (#62)
* secondary mkt first token withdrawal

* new secondary market model structure

* enhancooor

* overhaul with minimal error - todo comments to review

* further omit multi buys

* no date filter

* excl errors
2022-09-19 17:49:04 -04:00
Jack Forgash
ee7efe3851
AN-2168/Add network (#65)
* net data

* add version
2022-09-19 17:48:12 -04:00
Jack Forgash
1d89e84073
AN-2161/events location (#64)
* events location

* its nested

* coalesce approach

* events fix
2022-09-17 00:52:31 -06:00
Jack Forgash
c11814ef95
allday view (#60) 2022-09-02 13:58:10 -06:00
Jack Forgash
505f166046
AN-1945/fixes (#59)
* nft sales and dep tag

* fixes before removing dup tests

* test fixes

* nft sales

* avoid dupes

* return silver blocks test to split out that change in another pr

* last tweak
2022-09-02 13:51:19 -06:00
Jack Forgash
96a5f01ac8
AN-1988/events handle (#58)
* events try 1

* cte

* coalesce tweak
2022-09-01 18:24:04 -06:00
Jack Forgash
a4be724d15
AN-1900/ez (#57)
* ez views

* add comment to description and legacy fact view

* tag ez models

* add dex in yml name
2022-08-31 19:34:52 -06:00
Jack Forgash
317a8ab617
An-1968/allday silver (#55)
* allday metadata silver

* fixes
2022-08-29 11:45:02 -06:00
desmond-hui
bcc90bacc2
bronze table now holds more than just topshot metadata (#54) 2022-08-26 12:27:26 -07:00
Jack Forgash
861e667d4b
AN-1870/allday ids (#51)
* allday nft ids

* silver topshot view for compat

* limit allday to 3000

* str

* add nfl all day metadata integration and refactor streamline process

* no longer needed

Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2022-08-26 11:07:40 -07:00
Jack Forgash
edb39921fa
AN-1807/store tests (#53)
* store tests

* dbt 1.2

* test names and off for ingested at
2022-08-19 14:13:02 -06:00
Jack Forgash
c5e8289666
AN-1865/bridge fix (#52)
* seed update

* usdc fix
2022-08-18 13:55:31 -06:00
Jack Forgash
779baa4122
AN-1780/staking (#50)
* delegators

* staking events

* docs and core view

* date filter
2022-08-17 18:26:33 -06:00
Ava Masucci
417f5df20d
An 1779/ez token transfers (#49)
* Token transfers model and tests

* logic to include transfers only

* Ez token transfers and tests

* Silver incremental model and core view

* Incremental adds and date change
2022-08-17 11:19:09 -07:00
Jack Forgash
123d690227
AN-45/nft models updated (#48)
* nft models updated

* tx_succeeded field

* docs and test
2022-08-11 11:42:29 -06:00
Jack Forgash
264f9103a9
AN-1837/events fix (#47)
* new vertex patch

* prices adjustment
2022-08-09 09:30:45 -06:00
Jack Forgash
a7cbfb04bc
AN-1830/topshot nft ids (#45)
* add sales to list

* distinct

* list columns
2022-07-28 11:35:09 -06:00
Jack Forgash
1f4208dd91
AN-1822/caps for comment (#44)
* caps for comment

* del comma
2022-07-27 10:32:46 -06:00
Jack Forgash
04282e9466
handle escape character (#42) 2022-07-11 18:28:59 -04:00
Jack Forgash
38f0b9d205
docs and core view (#41) 2022-07-07 17:51:34 -04:00
Jack Forgash
f0e071cc6a
AN-1714/prices from swaps (#37)
* init silver model

* cluster by block_timestamp

* column fixes

* prices from swaps, need to update core to add symbol & contract

* token labels seed and updated core view

* docs
2022-07-06 16:36:41 -04:00
Jack Forgash
a952c32600
AN-1690/topshot_metadata (#36)
* topshot metadata

* adjust stats column names

* bronze view and inserted_timestamp
2022-07-06 10:39:24 -04:00
Jack Forgash
d8b0e8d0f0
add assets (#39) 2022-07-01 11:01:37 -06:00
Jack Forgash
6854a94ed1
AN-1638/inserted_timestamp (#35)
* bronze

* del netlify

* append column

* cluster

* incremental logic

* bridge tables

* docs
2022-06-30 13:12:08 -06:00
Jack Forgash
6ceec78b93
An 1346/bridges (#33)
* just a couple ctes

* add array

* blocto inbound

* blocto teleports final

* celer bridge

* tweaks

* core view

* docs

* docs

* list columns, add incremental to celer, move bridge.md to right dir
2022-06-29 10:00:29 -06:00
desmond-hui
a1bdcbb529
An 1625/integrate moments minted api (#34)
* clone stages to dev

* add api integration for topshot metadata
2022-06-28 09:24:09 -07:00
Jack Forgash
3de7b7918d
date to 420 (#32) 2022-06-23 13:05:04 -06:00
Jack Forgash
c31c362d48
An 1640/nft ids (#31)
* update silver

* yml
2022-06-21 12:44:25 -06:00
Jack Forgash
86cbe28096
An 1343/swaps (#29) 2022-06-16 06:06:13 -06:00
Jack Forgash
f9ec13fa9d
AN-1378/nft secondary market sales (#28)
* nft purchase txs v1

* v2 and tokenflow

* v3 and v4

* comments no changes

* blocto transactions

* nfts

* silver models

* topshot tokenflow

* silver nft sales

* small tweak

* excl outlier
2022-06-09 10:44:52 -06:00
Jack Forgash
1eaa67e407
AN-1534/add 0x (#27)
* add 0x

* fix null when 0 auth
2022-06-08 13:59:22 -06:00
Jack Forgash
978df9b8ed
AN-1375/tests (#26)
* nft purchase txs v1

* gap tests

* del nft
2022-06-06 12:05:26 -06:00
Jack Forgash
19841544e8
AN-1344/prices (#24)
* cmc prices

* gecko prices

* adjust $

* core view, no description yet pending swap prices

* prices

* price to price_usd

* address nulls and del mcap

* tests

* test fix
2022-06-02 12:17:34 -06:00
Jack Forgash
a005424621
AN-1339/nft market (#25)
* topshot sales

* add tx_succeeded

* core nft sales

* docs blocks

* del ingested at

* del last line
2022-06-02 11:54:31 -06:00
Jack Forgash
ab821e4154
AN-1348/refactor into silver (#23)
* squish

* del docs until gitignore updated

* where not and
2022-05-26 14:25:38 -06:00
Jack Forgash
445ef689f2
AN-1295/labels (#19)
* core view

* docs;

* validator labels

* del docs files
2022-05-25 12:22:56 -06:00
Jack Forgash
a956618336
AN-1201/tests (#18)
* tests

* docs

* date filter

* and

* events
2022-05-19 17:53:07 -06:00
Jack Forgash
24f877ec55
An 1219/events labels (#17)
* event labels

* docs

* rename contract_labels

* del source.md

* move distinct

* docs
2022-05-18 16:48:59 -06:00
Jack Forgash
e2913e5330
An 1212/hotfix (#16)
* unique key fix

* docs generate
2022-05-17 16:07:27 -06:00
Jack Forgash
79e8096768
An 1212/gold events (#11)
* gold blocks

* fixes

* revert blocks

* events

* address AN-1269

* format and core

* core yml

* revise event_id and event_data in yml
2022-05-17 15:03:40 -06:00
Jack Forgash
37095a607f
An 1221/gold transactions (#14)
* gold

* format
2022-05-17 14:48:57 -06:00
Jack Forgash
14dbded6a7
An 1220/gold blocks (#13)
* gold blocks

* fixes

* format

* test
2022-05-16 16:22:34 -06:00
Jack Forgash
6381b49a6a
An 1215/labels (#10)
* labels

* typo

* cluster by address

* docs

* varchar
2022-05-16 16:21:54 -06:00
Jack Forgash
aa849c8f44
An 1179/docs (#9)
* docs

* attribute descriptions

* attributes docs

* docs generated
2022-05-06 09:06:14 -06:00
Jack Forgash
ce36c6b66f
An 1194/silver event attributes (#8)
* init

* initial model

* convert addresses to hex

* wip

* adjust for missing leading 0s

* some tests

* changes

* fixed upstream
2022-05-05 18:48:25 -06:00
Jack Forgash
6aab9785d1
An 1210/error messages (#7)
* error msg

* format

* error msgs

* cast

* yml for tx index

* index;
2022-05-05 17:52:43 -06:00
Jack Forgash
3c680bc634
remove collection guarantee (#6) 2022-05-05 13:58:11 -06:00
Jack Forgash
80c84fda4a
AN-1122/Silver tx events (#5)
* init. not run or tested

* hard coding silver txs temp

* coalesce, rename type to event_type

* typo

* tests

* formatting

* newline

* nothing

* underscore in config block

* incremental jinga underscore

* upstream underscores

* reconfig cases
2022-05-04 16:40:38 -06:00
Jack Forgash
de575d0b70
AN-1120/Initial Silver blocks merge (#4)
* silver blocks

* config block

* in progress

* yml

* coalesce, tests

* auto-format

* tweak
2022-05-03 12:10:44 -06:00
Jack Forgash
c3a240d429
AN-1121 Initial push of Silver Transactions (#3)
* tx structure

* yml, qualify, ingested_at

* tests

* block_height

* tests, format, rename error column
2022-05-02 16:01:51 -06:00
Jack Forgash
1ac3da072e del alias 2022-04-18 10:34:27 -06:00
Jack Forgash
007226ac31 bronze txs view 2022-04-18 10:19:06 -06:00
Desmond Hui
fdc4e6cf12 initial setup files 2022-04-14 11:34:12 -07:00