Commit Graph

116 Commits

Author SHA1 Message Date
Jack Forgash
7abff47b5c
upd actions/upload-artifact@v3 to v4 (#391) 2025-01-16 09:17:17 -07:00
Jack Forgash
d3153ffff8
fix gha call / model name (#390) 2025-01-08 10:20:49 -07:00
Jack Forgash
0b4bb81bd8
AN-5454/ABIs (#388)
* abi streamline pipeline - in prog

* add coalesce

* gha workflow

* CRs
2024-12-04 09:28:25 -07:00
Jack Forgash
794ab78040
upd gha workflow (#386) 2024-12-03 15:35:12 -07:00
Jack Forgash
116ce2028d
activate minting entries gha (#384) 2024-11-21 15:06:18 -07:00
Jack Forgash
6117353b5a
AN-5517/api integration - GET minting/assets (#381)
* api integration - GET minting/assets

* add to gha workflow

* fixes
2024-11-21 14:34:18 -07:00
Jack Forgash
15dcc02476
AN-5528/Flow API Balances - chg evm_address source (#382)
* chg evm_address source

* disable transaction_entries

* change batch
2024-11-21 13:47:12 -07:00
Jack Forgash
22fe605582
AN-5413/Snag API (#376)
* init api models

* rm

* add vault path

* fix typo json key

* rm qualify from first silver model

* points spend silver model

* upd silver spend columns

* gha workflow and tags

* disable workflow

* disable correct gha

* add yml for silver model

* tag w streamline_non_core

* upd tags, gha workflow, final tweaks

* upd so, cluster, unique_key
2024-11-18 12:35:25 -07:00
Jack Forgash
4df472bace
AN-5459/point transfers api integration (#378)
* point transfers api integration

* upd 2 silver models

* gha workflow with auth

* upd gha workflow

* add gha on push config for testing

* add print step

* chg echo

* upd auth.py

* rm run on commit from gha

* upd model tags

* merge 2 silver into 1, add yml, upd gha

* upd auth return vals

* add exit 1 on failure

* upd return on success True

* add gha on-run config for final test

* add backup default to env_var(JWT)

* add backup default to env_var(JWT) - 2

* rm run on commit from gha and upd sql limit for balance call

* add yml for evm address model

* add evm address threshold test

* upd gha workflow

* upd per CRs
2024-11-15 09:43:52 -07:00
Jack Forgash
f6c0455ec2
upd schedule (#373) 2024-10-30 11:35:34 -06:00
Jack Forgash
ea33fe6134
add slack webhook url (#369) 2024-10-11 14:31:39 -06:00
Jack Forgash
64a009f0ab
AN-5252/points api integration (#368)
* basic query

* external points api request

* rn REWARD_POINTS and add supplementary models

* upd vault

* add gha workflows

* add headers incl user-agent

* upd header

* rm content type header

* addback secret

* use target

* rn _realtime

* upd silver and add FR view
2024-10-11 14:16:25 -06:00
Jack Forgash
894a940c4e
rename dbt_run_evm run-name (#360) 2024-09-10 21:41:58 -06:00
Jack Forgash
70d0393cfa
AN-5196 coreevm final (#356)
* an-5198-upd-block-gap-test

* evm block gap test

* evm gaps tests

* blocks and txs tests

* logs tests

* add evm gap test to daily test gha

* run evm job in separate gha workflow
2024-09-09 12:02:03 -06:00
Jack Forgash
2dccd4f5ef
Upd Package Lock, fix core_evm incr (#353)
* fix core_evm

* package lock
2024-09-06 11:50:30 -04:00
Jack Forgash
88df737fa2
AN-5203/Deploy flowEVM (#343)
* flow evm init

* flow evm testnet - blocks

* deploy dev udfs, update typo

* send block height as hex

* dev limit

* use stg external table

* some links

* add query to side doc

* upd external tbl to stg & testnet

* silver and evm txs

* testnet silver models pt1

* testnet events final, decode hash udf

* udf get evm chainhead

* upd resource list

* new udf. receipts pipeline. upd testnet model cols. expand readme

* receipts silver

* traces

* vault

* reorg to evm dir

* rm testnet, blocks lookback

* lookbacks, move qualify

* macro and align naming

* del testnet, reset namespace to gen

* silver_evm - receipts and txs

* core_evm fact_blocks & fact_transactions

* fix vault path in get blocks. Logs v1 (need sample)

* core_evm fact_logs

* CR updates

* del readme and add workflow

* upd vault path, batch size

* incr logic on modified to core_evm

* use local utils.udf, add blockNumber col to complete blocks check

* rm col

* correct _invocation_id

* upd nv csv

* incr batch limit due to late start

* add evm tag to model run
2024-09-05 20:43:33 -06:00
Shah Newaz Khan
1a3ea2c2fc
fix gha workflow env (#350)
* fix gha workflow env

* add dbt_profiles_dir

* bump fsc-utils to 1.30.0 & add tag_integration_test

* add resource path selector to gha dbt command

* revert to fsc-utils package 1.17.2

* add selectors.yml

* add reusable dbt run workflow

* test python 3.8

* pass in python version

* add reusable workflow template and uses python_version

* remove inline dbt.yml in favor of reusable template

* change to dbt.yml reusable workflow

* fix input command and remove python version

* remove external browser authenticator

* fix profiles in ./

* add password envar

* clean up dbt_run_integration_test gha
2024-08-27 18:55:29 -07:00
Shah Newaz Khan
bb864a8e87
Livequery integration test gha (#349)
* add dbt.yml and integration test gha's

* update dbt selector to run all integration tests in lq package

* remove fail fast strategy
2024-08-14 09:35:41 -07:00
Jack Forgash
30ce62f6a0
AN-4936/Backfill Migration Final (#335)
* del cw models

* core views

* defi views

* gov view

* nft_views

* rm ref to old cw models

* del internal cols

* nv 13-15

* nv 13-15

* nv 17-18

* nv 20

* del temp logic

* upd tests on allday metadata

* upd to warn
2024-07-01 13:27:42 -06:00
Jack Forgash
3bf75dfd18
fix typo (#340) 2024-06-25 15:01:38 -06:00
Jack Forgash
9127e6de86
AN-4852/Testnet Integration (#326)
* source, bronze, complete, rt views

* complete testnet_ dependency

* upd sources.yml

* upd models and bronze

* upd models.sql

* upd model in bronze views

* adding chainhead_testnet

* add jobs

* fix: hour

* upd api integration

* various fixes. useast2, gha workflows, bad refs, etc

* update udtf_get_base_table

---------

Co-authored-by: WHYTEWYLL <juanmiblanco1@gmail.com>
2024-06-25 12:12:51 -06:00
Jack Forgash
e41b0411f0
adj schedule and rename non core workflow group (#336) 2024-06-20 14:02:35 -06:00
Jack Forgash
64a996b2c7
new workflow/adhoc with alert (#334)
* new workflow

* print run status

* add workflow dispatch for testing

* add temp default for inputs

* add temp default for inputs in steps

* modify echo status after run

* add outcome

* print data

* pull echo into separate step

* define dbt_run_status

* run again

* try intentional failed dbt command

* add emoji to status

* fix single quote

* pass data directly

* del testing defaults

* del unused imports
2024-06-20 10:40:48 -06:00
Jack Forgash
12bf4e2e86
AN-4799/Backfill Migration (#329)
* move models to bronze

* blocks backfill logic ez ez ez

* streamline_load models for all 4 methods

* add back a set of views for the complete models to use

* del dbt run history gha workflow

* migrate txs final to modified ts

* upd events to modified

* set default val for LOAD_BACKFILL_VERSION in project yml

* update curated model incr logic
2024-06-12 12:10:27 -06:00
Jack Forgash
cd3e24bb92
rm 14 and 15 (#327) 2024-05-24 09:35:44 -06:00
Jack Forgash
cc51f4bcca
add 14-16 back for last few (#323) 2024-05-23 11:30:34 -06:00
Jack Forgash
40ef79f4fc
flip to non-batch method (#322) 2024-05-23 11:17:51 -06:00
Jack Forgash
6e288d5661
del 14-16 (#320) 2024-05-06 13:15:08 -06:00
Jack Forgash
c2353ab0f3
rm transactions history - complete (#316) 2024-04-24 09:55:20 -06:00
Jack Forgash
797cb9a430
hourly (#311) 2024-03-26 09:22:32 -06:00
Jack Forgash
a531ee5c98
upd txs away from batch (#299) 2024-03-18 11:28:23 -06:00
WHYTEWYLL
cdc0d9e668
2h (#298) 2024-03-18 10:50:27 -06:00
Jack Forgash
dd2b42d326
fix indent dbt test gha (#290) 2024-03-15 10:34:18 -06:00
WHYTEWYLL
e90fb88156
Fix job (#293)
* fix job

* key

* space

* format

* one line
2024-03-14 17:11:05 -06:00
WHYTEWYLL
605b43092a
fix job (#292)
* fix job

* key

* space
2024-03-14 16:52:02 -06:00
WHYTEWYLL
0dee1c27e9
adding batch methods for chain and cronjob (#284)
Finetune
2024-03-14 15:38:49 -06:00
Jack Forgash
db39b35ceb
upd workflows to use requirements.txt (#289) 2024-03-13 13:54:26 -06:00
Jack Forgash
490b0cdea3
add deps back (#287) 2024-03-13 13:19:07 -06:00
Jack Forgash
2f356f9403
del dbt deps (#285) 2024-03-13 13:12:22 -06:00
WHYTEWYLL
6b65655da8
Finetune (#277)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del

* increasing producer batch size

* increasing batch size and worker

* workers limit

* reducing producer

* add fin

* fix doubled key values
2024-02-20 12:07:10 -03:00
WHYTEWYLL
1043c83d4f
Finetune (#272)
* test a from the base line on env var

* quotes

* fine tune trasnactions and results

* format

* error

* increasing producer_batch_size

* add: transactions mainnet

* mainet 18

* reducing job

* var del
2024-02-15 17:08:45 -03:00
Jack Forgash
3261a09b4a
fix model selection in history (#269) 2024-02-13 19:06:27 -07:00
Jack Forgash
668b22097b
expliticly call putstanding NV and methods (#268) 2024-02-13 14:50:37 -07:00
WHYTEWYLL
1a6a72f5f2
An 4442 flow dbt1.7 (#261)
* pkg lock and dbt docs

* update: 1.7

* profile

* update docs

---------

Co-authored-by: forgash_ <jmfxyz@pm.me>
2024-01-30 15:44:19 -03:00
Jack Forgash
e0818966ad
AN-4428/add user agent to topshot and upd metadata job (#258)
* add user agent to topshot and upd metadata job

* upd agent

* upd limit to 600
2024-01-18 09:19:18 -07:00
Jack Forgash
ed9f306241
upd schedule to hourly (#254)
* upd schedule to hourly

* revert allday metadata
2024-01-03 16:33:56 -07:00
Jack Forgash
6e666d29a7
upd history cron - daily (#250) 2023-12-19 16:09:14 -05:00
Jack Forgash
9690c3102c
chg schedule to 3x/hour (#244)
* chg schedule to 3x/hour

* blocks

* dbt schd
2023-12-13 19:32:13 -07:00
Jack Forgash
cf26afc365
chg schedule to 4x/hour (#243) 2023-12-13 08:33:54 -07:00
Jack Forgash
c51d0c38b3
incr to 3x/hr (#242) 2023-12-11 16:52:05 -07:00