Commit Graph

63 Commits

Author SHA1 Message Date
desmond-hui
9d30ea2e3f
remove the cnft backfill task (#546) 2024-04-29 07:04:43 -07:00
tarikceric
1cef5247b9
An 4769/me solsniper cnft sales (#540)
* wip

* wip

* get compressed mint

* add schedule, pr comments update

* temp backfill logic

* backfill only
2024-04-25 10:41:45 -07:00
tarikceric
284e0c93ba
fix incremental, logic, fix from/to cols (#539)
* fix incremental, logic, fix from/to cols

* update desc

* update silver desc
2024-04-23 09:54:11 -07:00
tarikceric
ac4d0d8c9e
An 4638/cnft sales (#536)
* tensor swap models

* update code style

* fix

* update tensorswap cnft sales

* update per pr comments

* update logic for mint and transfers, add relationship test

* add relationship test
2024-04-18 07:46:01 -07:00
desmond-hui
7eda8c5cd2
remove cnft backfill from schedule, it is completed (#532) 2024-04-12 09:32:38 -07:00
desmond-hui
38c5d96397
Fix jupv6 test failures (#531)
* fix seed file column order

* add sanctum burn logic

* fix issue with wrong decimals for certain burns

* add and fix tests
2024-04-12 08:58:52 -07:00
tarikceric
d54b300de1
add tests for solsniper view (#522) 2024-03-20 09:20:25 -07:00
desmond-hui
1e9e3abe94
refactor model logic to include missing swaps, add tests (#520)
* refactor model logic to include missing swaps, add tests

* fix hardcoded name
2024-03-20 08:11:06 -07:00
tarikceric
1335883dd1
add solsniper to nft sales (#505)
* add solsniper

* include additional events

* clean up v1 views

* update incremental logic

* left join to transfers

* remove null test

* update code style

* add solsniper relation test

* updates per comments

* select view in core
2024-03-19 10:55:20 -07:00
tarikceric
a73f9134c1
An 4450/add swap protocols (#494)
* add bonk, meteora, phoenix swaps

* update desc

* add meteora

* add test case

* remove solana reference

* add dooar swaps

* remove phoenix; remove solana prefix

* add phoenix

* phoenix fix

* update backfill date

* logic for multi swaps w incorrect join

* add tests, update logic for multi swap txs

* remove db prefix
2024-03-01 11:26:22 -08:00
desmond-hui
263ca35e27
Fix/support new magic eden seller pattern (#497)
* direct transfer from system program to seller

* add test, fix issue w/ full refresh logic followed by incremental
2024-02-29 08:39:05 -08:00
tarikceric
41f0417890
An 4449/staking tables update (#487)
* update jito staking

* update token
2024-02-19 09:13:11 -08:00
Desmond Hui
9886ee4654 update cnft backfill scheduler 2024-02-06 06:39:20 -08:00
desmond-hui
92c57a4e86
An 4435/compressed mints backfill helpers (#475)
* wip

* make the DMLs ACID

* add backfill job, rename existing workflow

* check there is data to backfill before continuing
2024-02-05 07:16:02 -08:00
desmond-hui
bc9637a79b
use orchestrator to trigger decoded realtime and backfill (#466) 2024-01-26 07:51:28 -08:00
desmond-hui
68174b45cc
test async workflow orchestration (#461)
* test async workflow orchestration

* update
2024-01-25 08:17:11 -08:00
tarikceric
4e1da2c97b
An 4280/bridges v2 (#450)
* wip

* fix naming

* revamp bridges

* split models

* updates

* update tests and get inner events

* fix pk

* further exclude mayan

* exclude fee amounts

* add debridge tests

---------

Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>
2024-01-24 10:00:04 -08:00
desmond-hui
bd23649b00
An 4429/setup pyth backfill (#460)
* setup pyth backfill

* wip

* remove dev file

* adjust backfill scheduler, add more runs of complete decoded

* adjust backfill rane
2024-01-24 07:49:59 -08:00
tarikceric
771e3422ad
add missing tokens (#458) 2024-01-23 12:26:13 -08:00
desmond-hui
034e88e344
An 3943/get compressed nft addresses (#439)
* external funcs (wip)

* call compressed nft mint parser with external func

* ignore bronze_api models

* move to match standard location

* fix missing mints due to inconsistent ordering

* add a tag to bronze_api model, exclude accordingly

* update silver table to use new bronze api data

* wip

* handle non-mint events and multi mints per tx_id

* add regression test
2024-01-15 07:46:46 -08:00
tarikceric
19ea1ac5e7
update to use local lq (#432)
* update to use local lq

* create request in udf_api call

* add missing tokens
2024-01-04 14:56:28 -08:00
Desmond Hui
cc14acf494 add decoder backfill gha workflow 2024-01-04 09:21:57 -08:00
desmond-hui
73a2d32f29
An 4323/initial streamline 2.0 setup (#430)
* wip

* change udf name, fix surrogate key for inner instruct calls

* (wip) decoded instructions backfill helpers

* complete backfill helpers, clean up views with 0 requests

* rename

* revert

* temp change will revert

* rename

* use streamline 2.0 verify idl api endpoint

* this is placeholder model, will replace existing when ready

* reorg files

* add workflow for decode instructions real time

* use pip cache

* update with prod endpoints

* update sql limit to real value, should be union all

* use ref for model names

* remove placeholder, replace existing model logic

* add new decoder flow to gha

* temp disable incremental

* add inner index to core tables

* match existing behavior until we properly incorporate inner_instruction txs

* temp disable
2024-01-03 11:40:57 -08:00
Desmond Hui
36ff2314a1 hotfix broken models part 1 2023-12-28 08:19:03 -08:00
tarikceric
42d48b925f
get missing mints (#422) 2023-12-08 10:25:18 -08:00
tarikceric
bded21f30b
add new program and update existing (#420) 2023-12-06 10:15:38 -08:00
desmond-hui
8bf643dcd2
make core run as part of non-core (#407) 2023-11-17 07:40:05 -08:00
desmond-hui
f8e0ca0e51
make it 15mins instead of every 10 (#403) 2023-11-15 11:27:56 -08:00
desmond-hui
d32599e851
An 4090/force paralell execution (#402)
* force paralell execution for each udf request

* increase number of commands per workflow, decrease schedule frequency
2023-11-15 09:51:03 -08:00
desmond-hui
c567d20d7b
add compressed nft backfill job to SF (#400) 2023-11-14 10:16:18 -08:00
tarikceric
95e392d5b1
Update/schedule core tags (#396)
* wip

* add schedule tags

* use snowflake tasks

---------

Co-authored-by: Eric Laurello <eric.laurello@flipsidecrypto.com>
2023-11-10 15:59:21 -05:00
desmond-hui
c5e10794c9
this should be the filename (#389) 2023-11-07 08:38:38 -08:00
desmond-hui
2734d36d7c
Dispatch gha via snowflake (#388)
* update package

* setup GHA dispatches via Snowflake tasks

* remove unused workflows

* add model qualifier to only run models in solana

* include task status monitoring workflow
2023-11-07 08:17:36 -08:00
desmond-hui
ec6a458253
An 4131/an 4152/jupv5 and jupv6 models (#387)
* v5 swaps, first contract

* rename

* tests wip

* update swap_index

* fix duplicates and null mints for marinade deposits

* disable models

* An 4131/add jupv6 (#385)

* wip

* fix index join

* clean up

* adjust cols and union to swaps

* fix null _log_id col

* update swap_index

* remove solana_dev prefix

---------

Co-authored-by: Desmond Hui <desmond@flipsidecrypto.com>

* initial jup v5.2 model (#386)

* initial jup v5.2 model

* testing and swap_index update

* fix duplicates

* fix transfer ordering issue, fix mint actions bad join

---------

Co-authored-by: tarikceric <tarik@flipsidecrypto.com>

* make same fixes as in v5_2 model

* disable v5 models as there are no longer new events for them

* add incremental predicate

* add new udf to be created

* add succeeded column for v5 models

* remove v6 from silver swaps, add to defi.swaps view directly

---------

Co-authored-by: tarikceric <tarik@flipsidecrypto.com>
Co-authored-by: tarikceric <46071768+tarikceric@users.noreply.github.com>
2023-11-07 06:51:44 -08:00
tarikceric
48d39035f6
An 3190/additional stake pools (#371)
* add new pools and get inner_instruction events

* wip

* additional filters

* additional filter

* add test + delete commented code

* add inner_index
2023-10-24 10:42:28 -07:00
desmond-hui
fd8c7fa424
add test cases for ez staking (#366) 2023-10-17 10:18:38 -07:00
tarikceric
95ddab7edf
An 3269/marinade unstake (#303)
* add token col

* fix null mSol amounts

* fix uncomment
2023-06-06 12:54:14 -07:00
tarikceric
92810c7eff
An 2674/mev2 include fees (#280)
* include all fees

* adjust test cases
2023-04-06 09:55:43 -07:00
tarikceric
b9623df826
update action names (#268) 2023-03-06 11:51:16 -08:00
tarikceric
81f5e54399
An 2787/new nft sale program (#265)
* add instant_sell model

* marketplace description

* model creation

* add test cases

* finalize logic

* exclude instant_sell model

* update amm_sell model name
2023-02-28 11:47:57 -08:00
tarikceric
84aec570ce
An 2661/saber lp actions (#210)
* saber model

* Separate inner/outer lp actions

* additional filter and test txs

* add test tx

* exclude saber from incremental
2023-02-16 12:44:56 -08:00
tarikceric
6265b8b73e
An 2662/raydium lp actions (#250)
* initial model creation

* wip

* raydium specific filters

* wip

* correct incremental logic

* update ymls

* add tests

* add qualify to remove duplicates

* filter out specific older pools

* update tests
2023-02-16 10:27:45 -08:00
tarikceric
4bf749bcb7
An 2663/orca lp actions (#249)
* initial models

* descriptions for liquidity pool columns

* WIP

* get mints that occur during swaps

* correct description tag

* update ymls

* correct model references

* update yml

* update model references and column names

* update incremental logic

* update yml definitions

* revise unique col combo

* wip

* remove null amounts

* additional filter

* add index+inner_index to unique key

* create final joined table

* set inner_index to -1 instead of null

* inner_index refactoring

* additional tests

* exclude models from incremental
2023-02-15 08:57:34 -08:00
tarikceric
f2ac04fa83
An 2745/burn and mint actions (#233)
* initial model

* add test cases

* add tests

* add signer/authority

* add mint_authority/signer cols

* tests and cleanup

* exclude model from incremental

* grab entire signers array

* change col name

* add not-null test for col

* load start date
2023-02-02 17:29:37 -08:00
tarikceric
60c4af1e66
Hot fix/nft price exclude burns (#229)
* add test cases

* filter out burn actions

* fix test

* adjust incremental date

* add test cases

* remove null rows

* join post_token_bal using tx_id

* fix distinct logic

* revert distinct

* exclude models from incremental
2023-02-01 14:54:36 -08:00
tarikceric
4cf4b8cfd5
An 2676/same buyer and seller (#223)
* add test cases

* get correct account in sellers cte

* exclude model and downstream from incremental
2023-01-26 09:38:17 -08:00
eric-laurello
e69ac2b269
AN-2722 bug fixes (#218)
* bug fix

* test updates
2023-01-26 10:54:36 -05:00
tarikceric
8bee42b2b1
capture additional metaplex mints (#216)
* capture additional metaplex mints

* add downstream model tests

* add extra tests for downstream tables

* ignore transfer when source = destination

* add test cases

* temp exclude impacted nft models
2023-01-25 13:30:02 -08:00
tarikceric
701bb7f57f
add workflow for get_token_data macro (#190) 2022-12-16 15:18:51 -08:00
tarikceric
e38c676a7c
add filter and tests (#188) 2022-12-13 15:30:44 -08:00