Commit Graph

251 Commits

Author SHA1 Message Date
WHYTEWYLL
63271f1214 add: user agent to not get ban 2024-01-10 16:06:24 +07:00
Jack Forgash
7c87cd4225
Beta allday espn id view (#255) 2024-01-05 12:57:55 -07:00
Jack Forgash
074c11c55a
fix id col in dim allday metadata (#253) 2024-01-04 09:03:25 -07:00
Jack Forgash
ed9f306241
upd schedule to hourly (#254)
* upd schedule to hourly

* revert allday metadata
2024-01-03 16:33:56 -07:00
Jack Forgash
b26bf67732
upd desc on views (#251) 2023-12-19 17:26:19 -05:00
Jack Forgash
6e666d29a7
upd history cron - daily (#250) 2023-12-19 16:09:14 -05:00
Jack Forgash
4c4ef12ca8
AN-4333/beta views for archiving espn ids for livequery use (#249)
* beta views for archiving espn ids for livequery use

* list cols
2023-12-19 13:40:30 -05:00
WHYTEWYLL
50962af5a5
fix: usdc mint transfers (#247)
* fix: usdc mint transfers

* format

* refactor
2023-12-18 11:57:18 -03:00
WHYTEWYLL
5c4e7bfdf8
fix: errors (#246)
* fix: errors

* format

* fix: typo
2023-12-15 07:21:42 -08:00
Jack Forgash
9690c3102c
chg schedule to 3x/hour (#244)
* chg schedule to 3x/hour

* blocks

* dbt schd
2023-12-13 19:32:13 -07:00
Jack Forgash
cf26afc365
chg schedule to 4x/hour (#243) 2023-12-13 08:33:54 -07:00
WHYTEWYLL
e8851383c9
Flow - add pk/inserted & modified dates/_invocation id (#239)
* add: keys core

* add: silver labels jk

* add: silver metadata jk

* add silver prices pk

* add: silver staking jk

* add: silver swaps and transfers jk

* fix: doc

* gold core pk

* gold: defi keys

* price and half nfts

* add missing comma in silver__streamline_blocks

* add: nfts

* fix: models

* list col names in swaps_events_s

* add underscore _invocation

* fix: gold layer

* delete: description

* fix: order and names

* delete: unecesarry qualify

* fix: errors

* fix: name

* dix: name

* rename id

* add: pk to topshot

* add: dim rename

* fix dim metadata views

---------

Co-authored-by: Jack Forgash <58153492+forgxyz@users.noreply.github.com>
2023-12-12 11:49:18 -08:00
Jack Forgash
c51d0c38b3
incr to 3x/hr (#242) 2023-12-11 16:52:05 -07:00
Jack Forgash
28e4dd0981
upd lookback in tx res cte (#241)
* upd lookback in tx res cte

* autofmt
2023-12-06 16:13:17 -07:00
Jack Forgash
971d555dff
update prod clone macro (#240) 2023-12-06 13:12:46 -07:00
WHYTEWYLL
9dae9f240f
Pipeline metadata (#238)
* add: if condition

* removed row_list

* add:  gold view, fix macro, and silver models

* gold view extension

* add: test bronce

* g

* adding test

* add: test

* add: working test

* delete: tags

* delete: hardcode ref
2023-12-04 07:16:32 -08:00
WHYTEWYLL
150fbea1ab
fix: dedeuplicates (#237) 2023-11-29 08:44:07 -08:00
WHYTEWYLL
3e5e0a4bdd
Update silver__nft_allday_metadata_s.sql (#236) 2023-11-29 08:29:39 -08:00
WHYTEWYLL
d3b3c9667f
Update dbt_run_moments_metadata.yml (#235) 2023-11-29 06:44:14 -08:00
Jack Forgash
df7b62123c
Streamline Backfill: upd workflow activate txs tx results (#233)
* upd workflow activate txs tx results

* upd schedule to 30 mins
2023-11-27 12:22:55 -07:00
desmond-hui
b0c422cfdf
reduce risk of running livequery_models accidentally (#234) 2023-11-27 08:09:06 -08:00
WHYTEWYLL
6d6f478df6
alldaynft metadata with LQ (#229)
* base

* clean up; add: silver model

* add: new standards

* fix: refactor

* delete: folders

* refactor simplying

* remove no longer needed

* fix: refactor to silver layer

* ready to merge

---------

Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2023-11-27 07:55:44 -08:00
Jack Forgash
b33dc89a39
swap set description col (#232)
* swap set description col

* cast str
2023-11-20 16:39:47 -07:00
Jack Forgash
c82c04ad5d
del invocation id from gold (#231) 2023-11-20 16:30:38 -07:00
Jack Forgash
121c3e6a2c
AN-4203/ufc strike metadata (#230)
* ufc strike metadata, plus organized nft folder better

* upd db tag on gold

* upd col name to serial number

* del listing id

* add back listing id
2023-11-20 16:26:39 -07:00
Jack Forgash
821214294c
AN-4096/DEX Swaps Bug (#228)
* add event counts col to easily ignore infra events

* excl txs where to/from is null

* add tests
2023-11-15 17:15:56 -07:00
Jack Forgash
cff19a7bb6
upd test window (#227) 2023-11-14 17:42:25 -07:00
Jack Forgash
f7fa714ddc
Update dbt_run_history.yml (#226) 2023-11-10 16:29:25 -07:00
Jack Forgash
80c2462ac4
core v non-core tags and gha workflow (#225) 2023-11-10 14:30:57 -07:00
Jack Forgash
fc83740b50
upd nv csv (#224) 2023-11-08 09:37:24 -07:00
Jack Forgash
5b096dd83f
upd labels source (#223) 2023-11-07 11:48:11 -07:00
WHYTEWYLL
d17d5f06d7
deprecation (#222)
* deprecation

* del old udf creation

---------

Co-authored-by: forgash_ <jmfxyz@pm.me>
2023-11-07 10:40:24 -07:00
Jack Forgash
366f25b1d3
fix to 1 line (#220) 2023-10-31 22:06:42 -06:00
Jack Forgash
7f5f9c79b5
upd history to blocks and collections, 5x/hr (#219) 2023-10-31 21:49:01 -06:00
Jack Forgash
59097b6a1f
upd history run to every half hour (#218) 2023-10-31 15:51:24 -06:00
Jack Forgash
cc6f797d71
alter test on bitquery job (#217) 2023-10-31 11:48:27 -06:00
Jack Forgash
cd39800d32
add streamline vars as repo vars and upd history workflow (#216) 2023-10-31 11:24:07 -06:00
Jack Forgash
eb40b6d91d
Backfill Model - Incorporate all external tables (#211)
* upd sources, convert blocks to loop model, fix collections history

* multiple external tbl query macro and blocks update

* del network version.yml output

* add back rt blocks

* lower blocks

* revert str blocks and collections and move change to history view

* bronze history views and move into folders

* history workflow - blocks

* del history step from streamline blocks

* create complete get history models and upd get history streamline views - BLOCKS

* upd remaining method get history views to use history complete tables

* upd txs history dependency note

* upd history gha and del parameterized models

* fix collections external table config

* collections template fix

* upd txs templates

* upd history views

* fix to tx results template / views

* set history schedule to hourly

* del commented out backfill from streamline RT GHAs

* move blocks filter up to cte
2023-10-26 11:31:19 -06:00
desmond-hui
63ab57f844
An 4065/incremental load tuning (#215)
* modify how dbt performs merges by default

* require at least dbt 1.4

* adjust cluster keys, add incremental pred

* ensure query is being pruned on timestamp

* add cluster key
2023-10-26 07:02:53 -07:00
Jack Forgash
45a07a1da8
An 4050/txs final test (#214)
* test on str txs final for pending response

* add events final gap test

* upd test to check gaps
2023-10-24 11:52:01 -06:00
desmond-hui
8ff2e1aa07
modify how dbt performs merges by default (#213) 2023-10-23 10:04:29 -07:00
eric-laurello
fc5045a44a
schedules (#212) 2023-10-20 15:19:04 -04:00
Jack Forgash
62274b6b64
pause backfill (#210) 2023-10-19 16:16:24 -06:00
Jack Forgash
856ac1629f
AN-3935/backfill views (#208)
* archive parameterized model

* create views for each network ver

* network version views

* newline

* switch tx results realtime back to us-east-1

* del blocks history mainnet23

* add backfill step to streamline GHA

* newline

* rm dup WHERE

* upd name and del dup where from template
2023-10-19 13:12:13 -06:00
Jack Forgash
19bbac48a2
Quickfix - Update retry_tx_ids in txs final (#209)
* adj greatest to handle null val

* del if null from retry_tx_ids cte

* add not null test to ins timestamp
2023-10-18 13:23:11 -06:00
Jack Forgash
a18123405e
add pending res to txs final lookback (#207) 2023-10-17 16:55:45 -06:00
Jack Forgash
ea46f3ee02
Adj Test on Beta and Upd Staking Actions (#206) 2023-10-16 18:17:45 -06:00
Jack Forgash
75901993ad
del tests from deprecating core models (#205) 2023-10-16 13:13:28 -06:00
Jack Forgash
bab9e01077
upd collections cte w lookback (#204)
* upd collections cte w lookback

* move lookback to cte
2023-10-13 16:00:16 -06:00
Jack Forgash
753c91d4fa
An 4030/obs model migration (#203)
* upd to streamline models

* add start block var
2023-10-13 13:29:24 -06:00