flip/bsc-models
1
0
Fork 0
mirror of https://github.com/FlipsideCrypto/bsc-models.git synced 2026-02-06 10:46:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

AN-4820/heal-logic (#264)

Browse Source 
* vars

* tx success

* bridge heal

* cte name

* lps heal

* var format

* heal logic

* dex swaps heal

* column names and gold
This commit is contained in:
drethereum 2024-05-30 10:14:25 -06:00 committed by GitHub
parent a8f9fee4cb
commit 0cb3f8adf5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
75 changed files with 2706 additions and 1914 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
README.md
View File

@ -35,35 +35,43 @@ bsc:
query_tag: <TAG>
```
### Variables
## Variables
To control the creation of UDF or SP macros with dbt run:
* UPDATE_UDFS_AND_SPS
When True, executes all macros included in the on-run-start hooks within dbt_project.yml on model run as normal
When False, none of the on-run-start macros are executed on model run
* Default values are False
* When True, executes all macros included in the on-run-start hooks within dbt_project.yml on model run as normal
* When False, none of the on-run-start macros are executed on model run
Default values are False
* Usage: `dbt run --vars '{"UPDATE_UDFS_AND_SPS":True}' -m ...`
* Usage:
dbt run --var '{"UPDATE_UDFS_AND_SPS":True}' -m ...
Use a variable to heal a model incrementally:
* HEAL_MODEL
* Default is FALSE (Boolean)
* When FALSE, logic will be negated
* When TRUE, heal logic will apply
* Include `heal` in model tags within the config block for inclusion in the `dbt_run_heal_models` workflow, e.g. `tags = 'heal'`
To reload records in a curated complete table without a full-refresh, such as `silver_bridge.complete_bridge_activity`:
* HEAL_CURATED_MODEL
Default is an empty array []
When item is included in var array [], incremental logic will be skipped for that CTE / code block
When item is not included in var array [] or does not match specified item in model, incremental logic will apply
Example set up: `{% if is_incremental() and 'axelar' not in var('HEAL_CURATED_MODEL') %}`
* Usage: `dbt run --vars '{"HEAL_MODEL":True}' -m ...`
* Usage:
Single CTE: dbt run --var '{"HEAL_CURATED_MODEL":"axelar"}' -m ...
Multiple CTEs: dbt run --var '{"HEAL_CURATED_MODEL":["axelar","across","celer_cbridge"]}' -m ...
Use a variable to negate incremental logic:
* Example use case: reload records in a curated complete table without a full-refresh, such as `silver_bridge.complete_bridge_activity`:
* HEAL_MODELS
* Default is an empty array []
* When item is included in var array [], incremental logic will be skipped for that CTE / code block
* When item is not included in var array [] or does not match specified item in model, incremental logic will apply
* Example set up: `{% if is_incremental() and 'axelar' not in var('HEAL_MODELS') %}`
### Resources:
- Learn more about dbt [in the docs](https://docs.getdbt.com/docs/introduction)
- Check out [Discourse](https://discourse.getdbt.com/) for commonly asked questions and answers
- Join the [chat](https://community.getdbt.com/) on Slack for live discussions and support
- Find [dbt events](https://events.getdbt.com) near you
- Check out [the blog](https://blog.getdbt.com/) for the latest news on dbt's development and best practices
* Usage:
* Single CTE: `dbt run --vars '{"HEAL_MODELS":"axelar"}' -m ...`
* Multiple CTEs: `dbt run --vars '{"HEAL_MODELS":["axelar","across","celer_cbridge"]}' -m ...`
Use a variable to extend the incremental lookback period:
* LOOKBACK
* Default is a string representing the specified time interval e.g. '12 hours', '7 days' etc.
* Example set up: `SELECT MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'`
* Usage: `dbt run --vars '{"LOOKBACK":"36 hours"}' -m ...`
## Applying Model Tags
@ -97,7 +105,7 @@ To add/update a model's snowflake tags, add/modify the `meta` model property und
By default, model tags are pushed to Snowflake on each load. You can disable this by setting the `UPDATE_SNOWFLAKE_TAGS` project variable to `False` during a run.
```
dbt run --var '{"UPDATE_SNOWFLAKE_TAGS":False}' -s models/core/core__fact_blocks.sql
dbt run --vars '{"UPDATE_SNOWFLAKE_TAGS":False}' -s models/core/core__fact_blocks.sql
```
### Querying for existing tags on a model in snowflake
@ -105,4 +113,11 @@ dbt run --var '{"UPDATE_SNOWFLAKE_TAGS":False}' -s models/core/core__fact_blocks
```
select *
from table(bsc.information_schema.tag_references('bsc.core.fact_blocks', 'table'));
```
```
### Resources:
- Learn more about dbt [in the docs](https://docs.getdbt.com/docs/introduction)
- Check out [Discourse](https://discourse.getdbt.com/) for commonly asked questions and answers
- Join the [chat](https://community.getdbt.com/) on Slack for live discussions and support
- Find [dbt events](https://events.getdbt.com) near you
- Check out [the blog](https://blog.getdbt.com/) for the latest news on dbt's development and best practices

2
dbt_project.yml
View File

@ -70,7 +70,7 @@ vars:
BALANCES_START: 0
BALANCES_END: 10000000
HEAL_MODEL: False
HEAL_CURATED_MODEL: []
HEAL_MODELS: []
START_GHA_TASKS: False
API_INTEGRATION: '{{ var("config")[target.name]["API_INTEGRATION"] if var("config")[target.name] else var("config")["dev"]["API_INTEGRATION"] }}'
EXTERNAL_FUNCTION_URI: '{{ var("config")[target.name]["EXTERNAL_FUNCTION_URI"] if var("config")[target.name] else var("config")["dev"]["EXTERNAL_FUNCTION_URI"] }}'

8
models/gold/defi/defi__ez_bridge_activity.sql
View File

@ -33,7 +33,13 @@ SELECT
token_symbol,
amount_unadj,
amount,
amount_usd,
ROUND(
CASE
WHEN amount_usd < 1e+15 THEN amount_usd
ELSE NULL
END,
2
) AS amount_usd,
COALESCE (
complete_bridge_activity_id,
{{ dbt_utils.generate_surrogate_key(

20
models/gold/defi/defi__ez_dex_swaps.sql
View File

@ -24,10 +24,26 @@ SELECT
event_name,
amount_in_unadj,
amount_in,
amount_in_usd,
ROUND(
CASE
WHEN amount_out_usd IS NULL
OR ABS((amount_in_usd - amount_out_usd) / NULLIF(amount_out_usd, 0)) > 0.75
OR ABS((amount_in_usd - amount_out_usd) / NULLIF(amount_in_usd, 0)) > 0.75 THEN NULL
ELSE amount_in_usd
END,
2
) AS amount_in_usd,
amount_out_unadj,
amount_out,
amount_out_usd,
ROUND(
CASE
WHEN amount_in_usd IS NULL
OR ABS((amount_out_usd - amount_in_usd) / NULLIF(amount_in_usd, 0)) > 0.75
OR ABS((amount_out_usd - amount_in_usd) / NULLIF(amount_out_usd, 0)) > 0.75 THEN NULL
ELSE amount_out_usd
END,
2
) AS amount_out_usd,
sender,
tx_to,
event_index,

6
models/silver/core/silver__transfers.sql
View File

@ -35,7 +35,7 @@ AND _inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -169,7 +169,7 @@ heal_model AS (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -195,7 +195,7 @@ heal_model AS (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)

1
models/silver/defi/bridge/allbridge/silver_bridge__allbridge_sent.sql
View File

@ -53,6 +53,7 @@ WITH base_evt AS (
WHERE
topics [0] :: STRING = '0x884a8def17f0d5bbb3fef53f3136b5320c9b39f75afb8985eeab9ea1153ee56d'
AND contract_address = '0xbbbd1bbb4f9b936c3604906d7592a644071de884'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

2
models/silver/defi/bridge/axelar/silver_bridge__axelar_contractcallwithtoken.sql
View File

@ -42,6 +42,7 @@ WITH base_evt AS (
WHERE
topics [0] :: STRING = '0x7e50569d26be643bda7757722291ec66b1be66d8283474ae3fab5a98f878a7a2'
AND contract_address = '0x304acf330bbe08d1e512eefaa92f6a57871fd895'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
@ -89,6 +90,7 @@ native_gas_paid AS (
WHERE
topics [0] :: STRING = '0x999d431b58761213cf53af96262b67a069cbd963499fd8effd1e21556217b841'
AND contract_address = '0x2d5d7d31f671f86c782533cc367f14109a082712'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/celer/silver_bridge__celer_cbridge_send.sql
View File

@ -51,6 +51,7 @@ WITH base_evt AS (
'0x265b25e22bcd7f10a5bd6e6410f10537cc7567e8',
'0xdd90e5e87a2081dcf0391920868ebc2ffb81a1af'
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/dln/silver_bridge__dln_debridge_createdorder.sql
View File

@ -70,6 +70,7 @@ WITH base_evt AS (
WHERE
topics [0] :: STRING = '0xfc8703fd57380f9dd234a89dce51333782d49c5902f307b02f03e014d18fe471' --CreatedOrder
AND contract_address = '0xef4fb24ad0916217251f553c0596f8edc630eb66' --Dln: Source
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/eywa/silver_bridge__eywa_requestsent.sql
View File

@ -40,6 +40,7 @@ WITH base_evt AS (
--PortalV2
'0xbf0b5d561b986809924f88099c4ff0e6bcce60c9' --PortalV2
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/meson/silver_bridge__meson_transfers.sql
View File

@ -126,6 +126,7 @@ dst_info AS (
WHERE
contract_address = '0x25ab3efd52e6470681ce037cd546dc60726948d3'
AND topics [0] :: STRING = '0x5ce4019f772fda6cb703b26bce3ec3006eb36b73f1d3a0eb441213317d9f5e9d'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

244
models/silver/defi/bridge/silver_bridge__complete_bridge_activity.sql
View File

@ -1,12 +1,14 @@
-- depends_on: {{ ref('silver__complete_token_prices') }}
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['block_number','platform','version'],
cluster_by = ['block_timestamp::DATE'],
tags = ['curated','reorg']
tags = ['curated','reorg','heal']
) }}
WITH allbridge AS (
SELECT
block_number,
block_timestamp,
@ -32,11 +34,11 @@ WITH allbridge AS (
FROM
{{ ref('silver_bridge__allbridge_sent') }}
{% if is_incremental() and 'allbridge' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'allbridge' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -68,11 +70,11 @@ axelar AS (
FROM
{{ ref('silver_bridge__axelar_contractcallwithtoken') }}
{% if is_incremental() and 'axelar' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'axelar' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -104,11 +106,11 @@ celer_cbridge AS (
FROM
{{ ref('silver_bridge__celer_cbridge_send') }}
{% if is_incremental() and 'celer_cbridge' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'celer_cbridge' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -140,11 +142,11 @@ dln_debridge AS (
FROM
{{ ref('silver_bridge__dln_debridge_createdorder') }}
{% if is_incremental() and 'dln_debridge' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'dln_debridge' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -176,11 +178,11 @@ eywa AS (
FROM
{{ ref('silver_bridge__eywa_requestsent') }}
{% if is_incremental() and 'eywa' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'eywa' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -212,11 +214,11 @@ meson AS (
FROM
{{ ref('silver_bridge__meson_transfers') }}
{% if is_incremental() and 'meson' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'meson' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -248,11 +250,11 @@ stargate AS (
FROM
{{ ref('silver_bridge__stargate_swap') }}
{% if is_incremental() and 'stargate' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'stargate' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -284,11 +286,11 @@ symbiosis AS (
FROM
{{ ref('silver_bridge__symbiosis_synthesizerequest') }}
{% if is_incremental() and 'symbiosis' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'symbiosis' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -320,11 +322,11 @@ synapse_tb AS (
FROM
{{ ref('silver_bridge__synapse_token_bridge') }}
{% if is_incremental() and 'synapse_tb' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'synapse_tb' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -356,11 +358,11 @@ synapse_tbs AS (
FROM
{{ ref('silver_bridge__synapse_tokenbridgeandswap') }}
{% if is_incremental() and 'synapse_tbs' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'synapse_tbs' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -392,11 +394,11 @@ wormhole AS (
FROM
{{ ref('silver_bridge__wormhole_transfers') }}
{% if is_incremental() and 'wormhole' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'wormhole' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -419,7 +421,7 @@ all_protocols AS (
celer_cbridge
UNION ALL
SELECT
*
*
FROM
dln_debridge
UNION ALL
@ -458,7 +460,7 @@ all_protocols AS (
FROM
wormhole
),
FINAL AS (
complete_bridge_activity AS (
SELECT
block_number,
block_timestamp,
@ -514,7 +516,7 @@ FINAL AS (
2
)
ELSE NULL
END AS amount_usd_unadj,
END AS amount_usd,
_id,
b._inserted_timestamp
FROM
@ -539,6 +541,183 @@ FINAL AS (
) = LOWER(
b.destination_chain
)
),
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
heal_model AS (
SELECT
block_number,
block_timestamp,
origin_from_address,
origin_to_address,
origin_function_signature,
tx_hash,
event_index,
bridge_address,
event_name,
platform,
version,
sender,
receiver,
destination_chain_receiver,
destination_chain_id,
destination_chain,
t0.token_address,
C.token_symbol AS token_symbol,
C.token_decimals AS token_decimals,
amount_unadj,
CASE
WHEN C.token_decimals IS NOT NULL THEN (amount_unadj / pow(10, C.token_decimals))
ELSE amount_unadj
END AS amount_heal,
CASE
WHEN C.token_decimals IS NOT NULL THEN amount_heal * p.price
ELSE NULL
END AS amount_usd_heal,
_id,
t0._inserted_timestamp
FROM
{{ this }}
t0
LEFT JOIN {{ ref('silver__contracts') }} C
ON t0.token_address = C.contract_address
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON t0.token_address = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
WHERE
CONCAT(
t0.block_number,
'-',
t0.platform,
'-',
t0.version
) IN (
SELECT
CONCAT(
t1.block_number,
'-',
t1.platform,
'-',
t1.version
)
FROM
{{ this }}
t1
WHERE
t1.token_decimals IS NULL
AND t1._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__contracts') }} C
WHERE
C._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND C.token_decimals IS NOT NULL
AND C.contract_address = t1.token_address)
GROUP BY
1
)
OR CONCAT(
t0.block_number,
'-',
t0.platform,
'-',
t0.version
) IN (
SELECT
CONCAT(
t2.block_number,
'-',
t2.platform,
'-',
t2.version
)
FROM
{{ this }}
t2
WHERE
t2.amount_usd IS NULL
AND t2._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t2.token_address
AND p.hour = DATE_TRUNC(
'hour',
t2.block_timestamp
)
)
GROUP BY
1
)
),
{% endif %}
FINAL AS (
SELECT
*
FROM
complete_bridge_activity
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
UNION ALL
SELECT
block_number,
block_timestamp,
origin_from_address,
origin_to_address,
origin_function_signature,
tx_hash,
event_index,
bridge_address,
event_name,
platform,
version,
sender,
receiver,
destination_chain_receiver,
destination_chain_id,
destination_chain,
token_address,
token_symbol,
token_decimals,
amount_unadj,
amount_heal AS amount,
amount_usd_heal AS amount_usd,
_id,
_inserted_timestamp
FROM
heal_model
{% endif %}
)
SELECT
block_number,
@ -562,21 +741,18 @@ SELECT
token_decimals,
amount_unadj,
amount,
CASE
WHEN amount_usd_unadj < 1e+15 THEN amount_usd_unadj
ELSE NULL
END AS amount_usd,
amount_usd,
_id,
_inserted_timestamp,
{{ dbt_utils.generate_surrogate_key(
['_id']
['_id']
) }} AS complete_bridge_activity_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
FINAL
WHERE destination_chain <> 'bsc'
qualify (ROW_NUMBER() over (PARTITION BY _id
FINAL
WHERE
destination_chain <> 'bsc' qualify (ROW_NUMBER() over (PARTITION BY _id
ORDER BY
_inserted_timestamp DESC)) = 1

3
models/silver/defi/bridge/stargate/silver_bridge__stargate_createpool.sql
View File

@ -20,7 +20,8 @@ WITH base_contracts AS (
WHERE
from_address = LOWER('0xe7Ec689f432f29383f217e36e680B5C855051f25')
AND TYPE ILIKE 'create%'
AND tx_status ILIKE 'success'
AND tx_status = 'SUCCESS'
AND trace_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/stargate/silver_bridge__stargate_swap.sql
View File

@ -66,6 +66,7 @@ base_evt AS (
ON d.contract_address = p.pool_address
WHERE
topics [0] :: STRING = '0x34660fc8af304464529f48a778e03d03e4d34bcd5f9b6f0cfbf3cd238c642f7f'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/symbiosis/silver_bridge__symbiosis_synthesizerequest.sql
View File

@ -44,6 +44,7 @@ WITH base_evt AS (
'0xb91d3060c90aac7c4c706aef2b37997b3b2a1dcf',
'0x5aa5f7f84ed0e5db0a4a85c3947ea16b53352fd4'
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/synapse/silver_bridge__synapse_token_bridge.sql
View File

@ -46,6 +46,7 @@ WITH base_evt AS (
'0xd123f70ae324d34a9e76b67a27bf77593ba8749f'
)
AND origin_to_address IS NOT NULL
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/bridge/synapse/silver_bridge__synapse_tokenbridgeandswap.sql
View File

@ -53,6 +53,7 @@ WITH base_evt AS (
'0x0efc29e196da2e81afe96edd041bedcdf9e74893',
'0xd123f70ae324d34a9e76b67a27bf77593ba8749f'
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/biswap/silver_dex__biswap_pools.sql
View File

@ -27,6 +27,7 @@ WITH pool_creation AS (
WHERE
contract_address = '0x858e3312ed3a876947ea49d572a7c42de08af7ee'
AND topics [0] :: STRING = '0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9' --PairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/dodo/silver_dex__dodo_v1_pools.sql
View File

@ -24,6 +24,7 @@ WITH pool_events AS (
WHERE
contract_address = '0xca459456a45e300aa7ef447dbb60f87cccb42828' --DODOZoo
AND topics [0] :: STRING = '0x5c428a2e12ecaa744a080b25b4cda8b86359c82d726575d7d747e07708071f93' --DODOBirth
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

2
models/silver/defi/dex/dodo/silver_dex__dodo_v1_swaps.sql
View File

@ -71,6 +71,7 @@ sell_base_token AS (
FROM
proxies
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
@ -124,6 +125,7 @@ buy_base_token AS (
FROM
proxies
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/dodo/silver_dex__dodo_v2_pools.sql
View File

@ -39,6 +39,7 @@ WITH pools AS (
--NewDSP
'0xaf5c5f12a80fc937520df6fcaed66262a4cc775e0f3fceaf7a7cfe476d9a751d' --NewDVM
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/dodo/silver_dex__dodo_v2_swaps.sql
View File

@ -94,6 +94,7 @@ swaps_base AS (
FROM
proxies
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/fraxswap/silver_dex__fraxswap_pools.sql
View File

@ -31,6 +31,7 @@ WITH pool_creation AS (
'0xf89e6ca06121b6d4370f4b196ae458e8b969a011' --v2 factory
)
AND topics [0] :: STRING = '0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9' --pairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/fraxswap/silver_dex__fraxswap_swaps.sql
View File

@ -59,6 +59,7 @@ swaps_base AS (
ON l.contract_address = pool_address
WHERE
l.topics [0] :: STRING = '0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822' --Swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

3
models/silver/defi/dex/hashflow/silver_dex__hashflow_pools.sql
View File

@ -23,7 +23,8 @@ WITH contract_deployments AS (
'0xb5574750a786a37e300a916974ecd63f93fc6754'
)
AND TYPE ILIKE 'create%'
AND tx_status ILIKE 'success'
AND tx_status = 'SUCCESS'
AND trace_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

2
models/silver/defi/dex/hashflow/silver_dex__hashflow_swaps.sql
View File

@ -46,6 +46,7 @@ router_swaps_base AS (
ON l.contract_address = p.pool_address
WHERE
l.topics [0] :: STRING = '0xb709ddcc6550418e9b89df1f4938071eeaa3f6376309904c77e15d46b16066f5' --swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
@ -89,6 +90,7 @@ swaps_base AS (
ON l.contract_address = p.pool_address
WHERE
l.topics [0] :: STRING = '0x8cf3dec1929508e5677d7db003124e74802bfba7250a572205a9986d86ca9f1e' --swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/hashflow/silver_dex__hashflow_v3_pools.sql
View File

@ -21,6 +21,7 @@ WITH contract_deployments AS (
WHERE
contract_address = '0xde828fdc3f497f16416d1bb645261c7c6a62dab5'
AND topics [0] :: STRING = '0xdbd2a1ea6808362e6adbec4db4969cbc11e3b0b28fb6c74cb342defaaf1daada'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/hashflow/silver_dex__hashflow_v3_swaps.sql
View File

@ -51,6 +51,7 @@ swaps AS (
ON l.contract_address = p.pool_address
WHERE
l.topics [0] :: STRING = '0x34f57786fb01682fb4eec88d340387ef01a168fe345ea5b76f709d4e560c10eb' --Trade
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/kyberswap/silver_dex__kyberswap_v1_dynamic_pools.sql
View File

@ -34,6 +34,7 @@ WITH pool_creation AS (
WHERE
contract_address = '0x878dfe971d44e9122048308301f540910bbd934c' --dynamic fee factory
AND topics [0] :: STRING = '0xfc574402c445e75f2b79b67884ff9c662244dce454c5ae68935fcd0bebb7c8ff' --created pool
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/kyberswap/silver_dex__kyberswap_v1_dynamic_swaps.sql
View File

@ -64,6 +64,7 @@ swaps_base AS (
ON p.pool_address = l.contract_address
WHERE
l.topics [0] :: STRING = '0x606ecd02b3e3b4778f8e97b2e03351de14224efaa5fa64e62200afc9395c2499' --Dynamic Swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/kyberswap/silver_dex__kyberswap_v1_static_pools.sql
View File

@ -39,6 +39,7 @@ WITH pool_creation AS (
WHERE
contract_address = LOWER('0x1c758aF0688502e49140230F6b0EBd376d429be5') --static pool factory
AND topics [0] :: STRING = '0xb6bce363b712c921bead4bcc977289440eb6172eb89e258e3a25bd49ca806de6' --create pool
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/kyberswap/silver_dex__kyberswap_v1_static_swaps.sql
View File

@ -64,6 +64,7 @@ swaps_base AS (
ON p.pool_address = l.contract_address
WHERE
l.topics [0] :: STRING = '0x606ecd02b3e3b4778f8e97b2e03351de14224efaa5fa64e62200afc9395c2499' -- static swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/kyberswap/silver_dex__kyberswap_v2_elastic_pools.sql
View File

@ -30,6 +30,7 @@ WITH pool_creation AS (
WHERE
contract_address = '0x5f1dddbf348ac2fbe22a163e30f99f9ece3dd50a' --Elastic Pool Deployer
AND topics [0] :: STRING = '0x783cca1c0412dd0d695e784568c96da2e9c22ff989357a2e8b1d9b2b4e6b7118' --Create pool
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/kyberswap/silver_dex__kyberswap_v2_elastic_swaps.sql
View File

@ -77,6 +77,7 @@ swaps_base AS (
ON p.pool_address = l.contract_address
WHERE
topics [0] :: STRING = '0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67' -- elastic swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/level/silver_dex__levelfi_swaps.sql
View File

@ -74,6 +74,7 @@ WITH swaps_base AS (
WHERE
contract_address = '0xa5abfb56a78d2bd4689b25b8a77fd49bb0675874' --router
AND topics [0] :: STRING = '0xd6d34547c69c5ee3d2667625c188acf1006abb93e0ee7cf03925c67cf7760413' --swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/pancake_swap/silver_dex__pancakeswap_v1_pools.sql
View File

@ -27,6 +27,7 @@ WITH pool_creation AS (
WHERE
contract_address = '0xbcfccbde45ce874adcb698cc183debcf17952812' --factory
AND topics [0] :: STRING = '0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9' --PairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/pancake_swap/silver_dex__pancakeswap_v2_amm_pools.sql
View File

@ -31,6 +31,7 @@ WITH pool_creation AS (
'0x1f830fb91094a0e87c0a80150aa0af3805456090'
) --factory
AND topics [0] :: STRING = '0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9' --PairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/pancake_swap/silver_dex__pancakeswap_v2_mm_swaps.sql
View File

@ -58,6 +58,7 @@ WITH swaps_base AS (
WHERE
contract_address = '0xfeacb05b373f1a08e68235ba7fc92636b92ced01' --router
AND topics [0] :: STRING = '0xe7d6f812e1a54298ddef0b881cd08a4d452d9de35eb18b5145aa580fdda18b26' --swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/pancake_swap/silver_dex__pancakeswap_v2_ss_pools.sql
View File

@ -44,6 +44,7 @@ WITH pools AS (
'0xa9551fb056fc743efe2a0a34e39f9769ad10166520df7843c09a66f82e148b97',
'0x48dc7a1b156fe3e70ed5ed0afcb307661905edf536f15bb5786e327ea1933532'
) -- swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/pancake_swap/silver_dex__pancakeswap_v3_pools.sql
View File

@ -36,6 +36,7 @@ WITH created_pools AS (
WHERE
contract_address = '0x0bfbcf9fa4f9c56b0f40a671ad40e0805a091865' --factory
AND topics [0] :: STRING = '0x783cca1c0412dd0d695e784568c96da2e9c22ff989357a2e8b1d9b2b4e6b7118' --paircreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

976
models/silver/defi/dex/silver_dex__complete_dex_liquidity_pools.sql
View File

File diff suppressed because it is too large Load Diff

1968
models/silver/defi/dex/silver_dex__complete_dex_swaps.sql
View File

File diff suppressed because it is too large Load Diff

1
models/silver/defi/dex/sushi/silver_dex__sushi_pools.sql
View File

@ -27,6 +27,7 @@ WITH pool_creation AS (
WHERE
contract_address = LOWER('0xc35DADB65012eC5796536bD9864eD8773aBc74C4')
AND topics [0] :: STRING = '0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9' --PairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/trader_joe/silver_dex__trader_joe_v1_pools.sql
View File

@ -27,6 +27,7 @@ WITH pool_creation AS (
WHERE
contract_address = LOWER('0x4f8bdc85E3eec5b9dE67097c3f59B6Db025d9986')
AND topics [0] :: STRING = '0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9' --PairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/trader_joe/silver_dex__trader_joe_v1_swaps.sql
View File

@ -59,6 +59,7 @@ swaps_base AS (
ON p.pool_address = l.contract_address
WHERE
l.topics [0] :: STRING = '0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822' --Swap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/trader_joe/silver_dex__trader_joe_v2_1_swaps.sql
View File

@ -81,6 +81,7 @@ swaps_base AS (
WHERE
topics [0] :: STRING = '0xad7d6f97abf51ce18e17a38f4d70e975be9c0708474987bb3e26ad21bd93ca70' --Swap
AND version = 'v2.1'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/trader_joe/silver_dex__trader_joe_v2_pools.sql
View File

@ -37,6 +37,7 @@ WITH pool_creation AS (
'0x43646a8e839b2f2766392c1bf8f60f6e587b6960'
)
AND topics [0] :: STRING = '0x2c8d104b27c6b7f4492017a6f5cf3803043688934ebcaa6a03540beeaf976aff' --LB PairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/trader_joe/silver_dex__trader_joe_v2_swaps.sql
View File

@ -80,6 +80,7 @@ swaps_base AS (
WHERE
topics [0] :: STRING = '0xc528cda9e500228b16ce84fadae290d9a49aecb17483110004c5af0a07f6fd73' --Swap
AND version = 'v2'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/dex/uniswap/silver_dex__univ2_pools.sql
View File

@ -27,6 +27,7 @@ WITH pool_creation AS (
WHERE
contract_address = LOWER('0x8909Dc15e40173Ff4699343b6eB8132c65e18eC6')
AND topics [0] :: STRING = '0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9' --PairCreated
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

6
models/silver/defi/dex/uniswap/silver_dex__univ3_pools.sql
View File

@ -36,13 +36,14 @@ WITH created_pools AS (
WHERE
topics [0] = '0x783cca1c0412dd0d695e784568c96da2e9c22ff989357a2e8b1d9b2b4e6b7118'
AND contract_address = '0xdb1d10011ad0ff90774d0c6bb92e5c5c8b4461f7'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) :: DATE - 2
) - INTERVAL '12 hours'
FROM
{{ this }}
)
@ -62,13 +63,14 @@ initial_info AS (
{{ ref('silver__logs') }}
WHERE
topics [0] :: STRING = '0x98636036cb66a9c19a37435efc1e90142190214e8abeb821bdba3f2990dd4c95'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) :: DATE - 2
) - INTERVAL '12 hours'
FROM
{{ this }}
)

2
models/silver/defi/dex/uniswap/silver_dex__univ3_swaps.sql
View File

@ -46,7 +46,7 @@ AND _inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) :: DATE
) - INTERVAL '12 hours'
FROM
{{ this }}
)

2
models/silver/defi/dex/woofi/silver_dex__woofi_swaps.sql
View File

@ -64,6 +64,7 @@ WITH router_swaps_base AS (
'0xcef5be73ae943b77f9bc08859367d923c030a269' --v2
)
AND topics [0] :: STRING = '0x27c98e911efdd224f4002f6cd831c3ad0d2759ee176f9ee8466d95826af22a1c' --WooRouterSwap
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
@ -136,6 +137,7 @@ swaps_base AS (
FROM
router_swaps_base
)
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

161
models/silver/defi/lending/complete_lending/silver__complete_lending_borrows.sql
View File

@ -1,9 +1,10 @@
-- depends_on: {{ ref('silver__complete_token_prices') }}
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['block_number','platform'],
cluster_by = ['block_timestamp::DATE'],
tags = ['reorg','curated']
tags = ['reorg','curated','heal']
) }}
WITH kinza AS (
@ -30,19 +31,19 @@ WITH kinza AS (
FROM
{{ ref('silver__kinza_borrows') }} A
{% if is_incremental() and 'kinza' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'kinza' not in var('HEAL_MODELS') %}
WHERE
A._inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
radiant as (
radiant AS (
SELECT
tx_hash,
block_number,
@ -65,19 +66,19 @@ radiant as (
FROM
{{ ref('silver__radiant_borrows') }} A
{% if is_incremental() and 'radiant' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'radiant' not in var('HEAL_MODELS') %}
WHERE
A._inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
liqee as (
liqee AS (
SELECT
tx_hash,
block_number,
@ -100,19 +101,19 @@ liqee as (
FROM
{{ ref('silver__liqee_borrows') }} A
{% if is_incremental() and 'liqee' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'liqee' not in var('HEAL_MODELS') %}
WHERE
A._inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{{ this }}
)
{% endif %}
),
dforce as (
dforce AS (
SELECT
tx_hash,
block_number,
@ -135,19 +136,19 @@ dforce as (
FROM
{{ ref('silver__dforce_borrows') }} A
{% if is_incremental() and 'dforce' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'dforce' not in var('HEAL_MODELS') %}
WHERE
A._inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
venus as (
venus AS (
SELECT
tx_hash,
block_number,
@ -171,19 +172,19 @@ venus as (
{{ ref('silver__venus_borrows') }}
l
{% if is_incremental() and 'venus' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'venus' not in var('HEAL_MODELS') %}
WHERE
l._inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
borrow_union as (
borrow_union AS (
SELECT
*
FROM
@ -209,7 +210,7 @@ borrow_union as (
FROM
radiant
),
FINAL AS (
complete_lending_borrows AS (
SELECT
tx_hash,
block_number,
@ -243,8 +244,126 @@ FINAL AS (
'hour',
block_timestamp
) = p.hour
LEFT JOIN {{ ref('silver__contracts') }} C
ON b.token_address = C.contract_address
),
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
heal_model AS (
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
t0.contract_address,
event_name,
borrower,
protocol_market,
t0.token_address,
t0.token_symbol,
amount_unadj,
amount,
ROUND(
amount * p.price,
2
) AS amount_usd_heal,
platform,
t0.blockchain,
t0._LOG_ID,
t0._INSERTED_TIMESTAMP
FROM
{{ this }}
t0
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON t0.token_address = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
WHERE
CONCAT(
t0.block_number,
'-',
t0.platform
) IN (
SELECT
CONCAT(
t1.block_number,
'-',
t1.platform
)
FROM
{{ this }}
t1
WHERE
t1.amount_usd IS NULL
AND t1._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t1.token_address
AND p.hour = DATE_TRUNC(
'hour',
t1.block_timestamp
)
)
GROUP BY
1
)
),
{% endif %}
FINAL AS (
SELECT
*
FROM
complete_lending_borrows
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
UNION ALL
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
contract_address,
event_name,
borrower,
protocol_market,
token_address,
token_symbol,
amount_unadj,
amount,
amount_usd_heal AS amount_usd,
platform,
blockchain,
_LOG_ID,
_INSERTED_TIMESTAMP
FROM
heal_model
{% endif %}
)
SELECT
*,

231
models/silver/defi/lending/complete_lending/silver__complete_lending_deposits.sql
View File

@ -1,9 +1,10 @@
-- depends_on: {{ ref('silver__complete_token_prices') }}
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['block_number','platform'],
cluster_by = ['block_timestamp::DATE'],
tags = ['reorg','curated']
tags = ['reorg','curated','heal']
) }}
WITH kinza AS (
@ -30,17 +31,17 @@ WITH kinza AS (
FROM
{{ ref('silver__kinza_deposits') }}
{% if is_incremental() and 'kinza' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'kinza' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
radiant as (
radiant AS (
SELECT
tx_hash,
block_number,
@ -63,18 +64,17 @@ radiant as (
FROM
{{ ref('silver__radiant_deposits') }}
{% if is_incremental() and 'radiant' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'radiant' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
venus as (
venus AS (
SELECT
tx_hash,
block_number,
@ -97,17 +97,17 @@ venus as (
FROM
{{ ref('silver__venus_deposits') }}
{% if is_incremental() and 'venus' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'venus' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
liqee as (
liqee AS (
SELECT
tx_hash,
block_number,
@ -130,17 +130,17 @@ liqee as (
FROM
{{ ref('silver__liqee_deposits') }}
{% if is_incremental() and 'liqee' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'liqee' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
dforce as (
dforce AS (
SELECT
tx_hash,
block_number,
@ -163,43 +163,43 @@ dforce as (
FROM
{{ ref('silver__dforce_deposits') }}
{% if is_incremental() and 'dforce' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'dforce' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
deposits as (
SELECT
*
FROM
venus
UNION ALL
SELECT
*
FROM
liqee
UNION ALL
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
dforce
UNION ALL
SELECT
*
FROM
radiant
deposits AS (
SELECT
*
FROM
venus
UNION ALL
SELECT
*
FROM
liqee
UNION ALL
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
dforce
UNION ALL
SELECT
*
FROM
radiant
),
FINAL AS (
complete_lending_deposits AS (
SELECT
tx_hash,
block_number,
@ -210,8 +210,15 @@ FINAL AS (
origin_function_signature,
A.contract_address,
CASE
WHEN platform in ('dForce','Liqee','Venus') THEN 'Mint'
WHEN platform in ('Radiant','Kinza') THEN 'Supply'
WHEN platform IN (
'dForce',
'Liqee',
'Venus'
) THEN 'Mint'
WHEN platform IN (
'Radiant',
'Kinza'
) THEN 'Supply'
ELSE 'Deposit'
END AS event_name,
protocol_market,
@ -237,17 +244,135 @@ FINAL AS (
'hour',
block_timestamp
) = p.hour
LEFT JOIN {{ ref('silver__contracts') }} C
ON A.token_address = C.contract_address
),
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
heal_model AS (
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
t0.contract_address,
event_name,
protocol_market,
depositor,
t0.token_address,
t0.token_symbol,
amount_unadj,
amount,
ROUND(
amount * p.price,
2
) AS amount_usd_heal,
platform,
t0.blockchain,
t0._LOG_ID,
t0._INSERTED_TIMESTAMP
FROM
{{ this }}
t0
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON t0.token_address = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
WHERE
CONCAT(
t0.block_number,
'-',
t0.platform
) IN (
SELECT
CONCAT(
t1.block_number,
'-',
t1.platform
)
FROM
{{ this }}
t1
WHERE
t1.amount_usd IS NULL
AND t1._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t1.token_address
AND p.hour = DATE_TRUNC(
'hour',
t1.block_timestamp
)
)
GROUP BY
1
)
),
{% endif %}
FINAL AS (
SELECT
*
FROM
complete_lending_deposits
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
UNION ALL
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
contract_address,
event_name,
protocol_market,
depositor,
token_address,
token_symbol,
amount_unadj,
amount,
amount_usd_heal AS amount_usd,
platform,
blockchain,
_LOG_ID,
_INSERTED_TIMESTAMP
FROM
heal_model
{% endif %}
)
SELECT
*,
{{ dbt_utils.generate_surrogate_key(
['tx_hash','event_index']
) }} AS complete_lending_deposits_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
*,
{{ dbt_utils.generate_surrogate_key(
['tx_hash','event_index']
) }} AS complete_lending_deposits_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
FINAL qualify(ROW_NUMBER() over(PARTITION BY _log_id
ORDER BY

215
models/silver/defi/lending/complete_lending/silver__complete_lending_flashloans.sql
View File

@ -1,9 +1,10 @@
-- depends_on: {{ ref('silver__complete_token_prices') }}
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['block_number','platform'],
cluster_by = ['block_timestamp::DATE'],
tags = ['reorg','curated']
tags = ['reorg','curated','heal']
) }}
WITH kinza AS (
@ -33,17 +34,17 @@ WITH kinza AS (
FROM
{{ ref('silver__kinza_flashloans') }}
{% if is_incremental() and 'kinza' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'kinza' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
radiant as (
radiant AS (
SELECT
tx_hash,
block_number,
@ -62,36 +63,35 @@ radiant as (
initiator_address,
target_address,
platform,
symbol AS token_symbol,
symbol,
blockchain,
_LOG_ID,
_INSERTED_TIMESTAMP
FROM
{{ ref('silver__radiant_flashloans') }}
{% if is_incremental() and 'radiant' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'radiant' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
flashloans as (
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
radiant
flashloans AS (
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
radiant
),
FINAL AS (
complete_lending_flashloans AS (
SELECT
tx_hash,
block_number,
@ -106,7 +106,7 @@ FINAL AS (
initiator_address AS initiator,
target_address AS target,
f.token_address AS flashloan_token,
token_symbol AS flashloan_token_symbol,
f.symbol AS flashloan_token_symbol,
amount_unadj AS flashloan_amount_unadj,
flashloan_amount,
ROUND(
@ -132,8 +132,179 @@ FINAL AS (
'hour',
block_timestamp
) = p.hour
LEFT JOIN {{ ref('silver__contracts') }} C
ON f.token_address = C.contract_address
),
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
heal_model AS (
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
t0.contract_address,
event_name,
protocol_market,
initiator,
target,
flashloan_token,
flashloan_token_symbol,
flashloan_amount_unadj,
flashloan_amount,
ROUND(
flashloan_amount * p.price,
2
) AS flashloan_amount_usd_heal,
premium_amount_unadj,
premium_amount,
ROUND(
premium_amount * p.price,
2
) AS premium_amount_usd_heal,
platform,
t0.blockchain,
t0._LOG_ID,
t0._INSERTED_TIMESTAMP
FROM
{{ this }}
t0
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON t0.flashloan_token = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
WHERE
CONCAT(
t0.block_number,
'-',
t0.platform
) IN (
SELECT
CONCAT(
t1.block_number,
'-',
t1.platform
)
FROM
{{ this }}
t1
WHERE
t1.flashloan_amount_usd IS NULL
AND t1._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t1.flashloan_token
AND p.hour = DATE_TRUNC(
'hour',
t1.block_timestamp
)
)
GROUP BY
1
)
OR CONCAT(
t0.block_number,
'-',
t0.platform
) IN (
SELECT
CONCAT(
t2.block_number,
'-',
t2.platform
)
FROM
{{ this }}
t2
WHERE
t2.premium_amount_usd IS NULL
AND t2._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t2.flashloan_token
AND p.hour = DATE_TRUNC(
'hour',
t2.block_timestamp
)
)
GROUP BY
1
)
),
{% endif %}
FINAL AS (
SELECT
*
FROM
complete_lending_flashloans
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
UNION ALL
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
contract_address,
event_name,
protocol_market,
initiator,
target,
flashloan_token,
flashloan_token_symbol,
flashloan_amount_unadj,
flashloan_amount,
flashloan_amount_usd_heal AS flashloan_amount_usd,
premium_amount_unadj,
premium_amount,
premium_amount_usd_heal AS premium_amount_usd,
platform,
blockchain,
_LOG_ID,
_INSERTED_TIMESTAMP
FROM
heal_model
{% endif %}
)
SELECT
*,

227
models/silver/defi/lending/complete_lending/silver__complete_lending_liquidations.sql
View File

@ -1,9 +1,10 @@
-- depends_on: {{ ref('silver__complete_token_prices') }}
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['block_number','platform'],
cluster_by = ['block_timestamp::DATE'],
tags = ['reorg','curated']
tags = ['reorg','curated','heal']
) }}
WITH venus AS (
@ -34,17 +35,17 @@ WITH venus AS (
{{ ref('silver__venus_liquidations') }}
l
{% if is_incremental() and 'venus' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'venus' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
dforce as (
dforce AS (
SELECT
tx_hash,
block_number,
@ -71,17 +72,17 @@ dforce as (
{{ ref('silver__dforce_liquidations') }}
l
{% if is_incremental() and 'dforce' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'dforce' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
kinza as (
kinza AS (
SELECT
tx_hash,
block_number,
@ -107,17 +108,17 @@ kinza as (
FROM
{{ ref('silver__kinza_liquidations') }}
{% if is_incremental() and 'kinza' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'kinza' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
radiant as (
radiant AS (
SELECT
tx_hash,
block_number,
@ -143,17 +144,17 @@ radiant as (
FROM
{{ ref('silver__radiant_liquidations') }}
{% if is_incremental() and 'radaint' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'radaint' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
venus as (
venus AS (
SELECT
tx_hash,
block_number,
@ -180,17 +181,17 @@ venus as (
{{ ref('silver__venus_liquidations') }}
l
{% if is_incremental() and 'venus' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'venus' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
liqee as (
liqee AS (
SELECT
tx_hash,
block_number,
@ -215,45 +216,44 @@ liqee as (
_INSERTED_TIMESTAMP
FROM
{{ ref('silver__liqee_liquidations') }}
{% if is_incremental() and 'liqee' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'liqee' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
liquidation_union as (
SELECT
*
FROM
venus
UNION ALL
SELECT
*
FROM
liqee
UNION ALL
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
dforce
UNION ALL
SELECT
*
FROM
radiant
liquidation_union AS (
SELECT
*
FROM
venus
UNION ALL
SELECT
*
FROM
liqee
UNION ALL
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
dforce
UNION ALL
SELECT
*
FROM
radiant
),
FINAL AS (
complete_lending_liquidations AS (
SELECT
tx_hash,
block_number,
@ -264,7 +264,11 @@ FINAL AS (
origin_function_signature,
A.contract_address,
CASE
WHEN platform in ('dForce','Liqee','Venus') THEN 'LiquidateBorrow'
WHEN platform IN (
'dForce',
'Liqee',
'Venus'
) THEN 'LiquidateBorrow'
ELSE 'LiquidationCall'
END AS event_name,
liquidator,
@ -286,14 +290,139 @@ FINAL AS (
A._INSERTED_TIMESTAMP
FROM
liquidation_union A
LEFT JOIN {{ ref('price__ez_prices_hourly') }} P
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON collateral_asset = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
LEFT JOIN {{ ref('silver__contracts') }} C
ON collateral_asset = C.contract_address
),
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
heal_model AS (
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
t0.contract_address,
event_name,
liquidator,
borrower,
protocol_market,
collateral_token,
collateral_token_symbol,
amount_unadj,
amount,
ROUND(
amount * p.price,
2
) AS amount_usd_heal,
debt_token,
debt_token_symbol,
platform,
t0.blockchain,
t0._LOG_ID,
t0._INSERTED_TIMESTAMP
FROM
{{ this }}
t0
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON t0.collateral_token = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
WHERE
CONCAT(
t0.block_number,
'-',
t0.platform
) IN (
SELECT
CONCAT(
t1.block_number,
'-',
t1.platform
)
FROM
{{ this }}
t1
WHERE
t1.amount_usd IS NULL
AND t1._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t1.collateral_token
AND p.hour = DATE_TRUNC(
'hour',
t1.block_timestamp
)
)
GROUP BY
1
)
),
{% endif %}
FINAL AS (
SELECT
*
FROM
complete_lending_liquidations
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
UNION ALL
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
contract_address,
event_name,
liquidator,
borrower,
protocol_market,
collateral_token,
collateral_token_symbol,
amount_unadj,
amount,
amount_usd_heal AS amount_usd,
debt_token,
debt_token_symbol,
platform,
blockchain,
_LOG_ID,
_INSERTED_TIMESTAMP
FROM
heal_model
{% endif %}
)
SELECT
*,

227
models/silver/defi/lending/complete_lending/silver__complete_lending_repayments.sql
View File

@ -1,9 +1,10 @@
-- depends_on: {{ ref('silver__complete_token_prices') }}
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['block_number','platform'],
cluster_by = ['block_timestamp::DATE'],
tags = ['reorg','curated']
tags = ['reorg','curated','heal']
) }}
WITH kinza AS (
@ -31,17 +32,17 @@ WITH kinza AS (
FROM
{{ ref('silver__kinza_repayments') }}
{% if is_incremental() and 'kinza' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'kinza' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
radiant as (
radiant AS (
SELECT
tx_hash,
block_number,
@ -65,17 +66,17 @@ radiant as (
FROM
{{ ref('silver__radiant_repayments') }}
{% if is_incremental() and 'radiant' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'radiant' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
venus as (
venus AS (
SELECT
tx_hash,
block_number,
@ -99,17 +100,17 @@ venus as (
FROM
{{ ref('silver__venus_repayments') }}
{% if is_incremental() and 'venus' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'venus' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
dforce as (
dforce AS (
SELECT
tx_hash,
block_number,
@ -133,17 +134,17 @@ dforce as (
FROM
{{ ref('silver__dforce_repayments') }}
{% if is_incremental() and 'dforce' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'dforce' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
liqee as (
liqee AS (
SELECT
tx_hash,
block_number,
@ -167,43 +168,43 @@ liqee as (
FROM
{{ ref('silver__liqee_repayments') }}
{% if is_incremental() and 'liqee' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'liqee' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
repayments as (
SELECT
*
FROM
venus
UNION ALL
SELECT
*
FROM
liqee
UNION ALL
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
dforce
UNION ALL
SELECT
*
FROM
radiant
repayments AS (
SELECT
*
FROM
venus
UNION ALL
SELECT
*
FROM
liqee
UNION ALL
SELECT
*
FROM
kinza
UNION ALL
SELECT
*
FROM
dforce
UNION ALL
SELECT
*
FROM
radiant
),
FINAL AS (
complete_lending_repayments AS (
SELECT
tx_hash,
block_number,
@ -214,7 +215,11 @@ FINAL AS (
origin_function_signature,
A.contract_address,
CASE
WHEN platform in ('dForce','Liqee','Venus') THEN 'RepayBorrow'
WHEN platform IN (
'dForce',
'Liqee',
'Venus'
) THEN 'RepayBorrow'
ELSE 'Repay'
END AS event_name,
protocol_market,
@ -241,17 +246,137 @@ FINAL AS (
'hour',
block_timestamp
) = p.hour
LEFT JOIN {{ ref('silver__contracts') }} C
ON A.token_address = C.contract_address
),
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
heal_model AS (
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
t0.contract_address,
event_name,
protocol_market,
payer,
borrower,
t0.token_address,
t0.token_symbol,
amount_unadj,
amount,
ROUND(
amount * p.price,
2
) AS amount_usd_heal,
platform,
t0.blockchain,
t0._LOG_ID,
t0._INSERTED_TIMESTAMP
FROM
{{ this }}
t0
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON t0.token_address = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
WHERE
CONCAT(
t0.block_number,
'-',
t0.platform
) IN (
SELECT
CONCAT(
t1.block_number,
'-',
t1.platform
)
FROM
{{ this }}
t1
WHERE
t1.amount_usd IS NULL
AND t1._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t1.token_address
AND p.hour = DATE_TRUNC(
'hour',
t1.block_timestamp
)
)
GROUP BY
1
)
),
{% endif %}
FINAL AS (
SELECT
*
FROM
complete_lending_repayments
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
UNION ALL
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
contract_address,
event_name,
protocol_market,
payer,
borrower,
token_address,
token_symbol,
amount_unadj,
amount,
amount_usd_heal AS amount_usd,
platform,
blockchain,
_LOG_ID,
_INSERTED_TIMESTAMP
FROM
heal_model
{% endif %}
)
SELECT
*,
{{ dbt_utils.generate_surrogate_key(
['tx_hash','event_index']
) }} AS complete_lending_repayments_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
*,
{{ dbt_utils.generate_surrogate_key(
['tx_hash','event_index']
) }} AS complete_lending_repayments_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
FINAL qualify(ROW_NUMBER() over(PARTITION BY _log_id
ORDER BY

167
models/silver/defi/lending/complete_lending/silver__complete_lending_withdraws.sql
View File

@ -1,9 +1,10 @@
-- depends_on: {{ ref('silver__complete_token_prices') }}
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['block_number','platform'],
cluster_by = ['block_timestamp::DATE'],
tags = ['reorg','curated']
tags = ['reorg','curated','heal']
) }}
WITH kinza AS (
@ -30,19 +31,19 @@ WITH kinza AS (
FROM
{{ ref('silver__kinza_withdraws') }}
{% if is_incremental() and 'kinza' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'kinza' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
radiant as (
radiant AS (
SELECT
tx_hash,
block_number,
@ -65,19 +66,19 @@ radiant as (
FROM
{{ ref('silver__radiant_withdraws') }}
{% if is_incremental() and 'radiant' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'radiant' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{{ this }}
)
{% endif %}
),
liqee as (
liqee AS (
SELECT
tx_hash,
block_number,
@ -100,19 +101,19 @@ liqee as (
FROM
{{ ref('silver__liqee_withdraws') }}
{% if is_incremental() and 'liqee' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'liqee' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
venus as (
venus AS (
SELECT
tx_hash,
block_number,
@ -135,19 +136,19 @@ venus as (
FROM
{{ ref('silver__venus_withdraws') }}
{% if is_incremental() and 'venus' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'venus' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
dforce as (
dforce AS (
SELECT
tx_hash,
block_number,
@ -170,19 +171,19 @@ dforce as (
FROM
{{ ref('silver__dforce_withdraws') }}
{% if is_incremental() and 'dforce' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'dforce' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
{% endif %}
),
withdraws as (
withdraws AS (
SELECT
*
FROM
@ -208,7 +209,7 @@ withdraws as (
FROM
radiant
),
FINAL AS (
complete_lending_withdraws AS (
SELECT
tx_hash,
block_number,
@ -219,7 +220,11 @@ FINAL AS (
origin_function_signature,
A.contract_address,
CASE
WHEN platform in ('dForce','Liqee','Venus') THEN 'Redeem'
WHEN platform IN (
'dForce',
'Liqee',
'Venus'
) THEN 'Redeem'
ELSE 'Withdraw'
END AS event_name,
protocol_market,
@ -245,8 +250,126 @@ FINAL AS (
'hour',
block_timestamp
) = p.hour
LEFT JOIN {{ ref('silver__contracts') }} C
ON A.token_address = C.contract_address
),
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
heal_model AS (
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
t0.contract_address,
event_name,
protocol_market,
depositor,
t0.token_address,
t0.token_symbol,
amount_unadj,
amount,
ROUND(
amount * p.price,
2
) AS amount_usd_heal,
platform,
t0.blockchain,
t0._log_id,
t0._inserted_timestamp
FROM
{{ this }}
t0
LEFT JOIN {{ ref('price__ez_prices_hourly') }}
p
ON t0.token_address = p.token_address
AND DATE_TRUNC(
'hour',
block_timestamp
) = p.hour
WHERE
CONCAT(
t0.block_number,
'-',
t0.platform
) IN (
SELECT
CONCAT(
t1.block_number,
'-',
t1.platform
)
FROM
{{ this }}
t1
WHERE
t1.amount_usd IS NULL
AND t1._inserted_timestamp < (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
AND EXISTS (
SELECT
1
FROM
{{ ref('silver__complete_token_prices') }}
p
WHERE
p._inserted_timestamp > DATEADD('DAY', -14, SYSDATE())
AND p.price IS NOT NULL
AND p.token_address = t1.token_address
AND p.hour = DATE_TRUNC(
'hour',
t1.block_timestamp
)
)
GROUP BY
1
)
),
{% endif %}
FINAL AS (
SELECT
*
FROM
complete_lending_withdraws
{% if is_incremental() and var(
'HEAL_MODEL'
) %}
UNION ALL
SELECT
tx_hash,
block_number,
block_timestamp,
event_index,
origin_from_address,
origin_to_address,
origin_function_signature,
contract_address,
event_name,
protocol_market,
depositor,
token_address,
token_symbol,
amount_unadj,
amount,
amount_usd_heal AS amount_usd,
platform,
blockchain,
_log_id,
_inserted_timestamp
FROM
heal_model
{% endif %}
)
SELECT
*,

1
models/silver/defi/lending/dforce/silver__dforce_borrows.sql
View File

@ -56,6 +56,7 @@ dforce_borrows AS (
WHERE token_address <> '0xf51422c47c6c3e40cfca4a7b04232aedb7f49948' --excludes qDOT edge case
)
AND topics [0] :: STRING = '0x2dd79f4fccfd18c360ce7f9132f3621bf05eee18f995224badb32d17f172df73'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/dforce/silver__dforce_deposits.sql
View File

@ -53,6 +53,7 @@ dforce_deposits AS (
WHERE token_address <> '0xf51422c47c6c3e40cfca4a7b04232aedb7f49948' --excludes qDOT edge case
)
AND topics [0] :: STRING = '0x2f00e3cdd69a77be7ed215ec7b2a36784dd158f921fca79ac29deffa353fe6ee'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/dforce/silver__dforce_liquidations.sql
View File

@ -56,6 +56,7 @@ dforce_liquidations AS (
WHERE token_address <> '0xf51422c47c6c3e40cfca4a7b04232aedb7f49948' --excludes qDOT edge case
)
AND topics [0] :: STRING = '0x298637f684da70674f26509b10f07ec2fbc77a335ab1e7d6215a4b2484d8bb52'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/dforce/silver__dforce_repayments.sql
View File

@ -51,6 +51,7 @@ dforce_repayments AS (
WHERE token_address <> '0xf51422c47c6c3e40cfca4a7b04232aedb7f49948' --excludes qDOT edge case
)
AND topics [0] :: STRING = '0x6fadbf7329d21f278e724fa0d4511001a158f2a97ee35c5bc4cf8b64417399ef'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/dforce/silver__dforce_withdraws.sql
View File

@ -53,6 +53,7 @@ dforce_redemptions AS (
WHERE token_address <> '0xf51422c47c6c3e40cfca4a7b04232aedb7f49948' --excludes qDOT edge case
)
AND topics [0] :: STRING = '0x3f693fff038bb8a046aa76d9516190ac7444f7d69cf952c4cbdc086fdef2d6fc'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/liqee/silver__liqee_borrows.sql
View File

@ -55,6 +55,7 @@ liqee_borrows AS (
asset_details
)
AND topics [0] :: STRING = '0x2dd79f4fccfd18c360ce7f9132f3621bf05eee18f995224badb32d17f172df73'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/liqee/silver__liqee_deposits.sql
View File

@ -52,6 +52,7 @@ liqee_deposits AS (
asset_details
)
AND topics [0] :: STRING = '0x2f00e3cdd69a77be7ed215ec7b2a36784dd158f921fca79ac29deffa353fe6ee'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/liqee/silver__liqee_liquidations.sql
View File

@ -55,6 +55,7 @@ liqee_liquidations AS (
asset_details
)
AND topics [0] :: STRING = '0x298637f684da70674f26509b10f07ec2fbc77a335ab1e7d6215a4b2484d8bb52'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/liqee/silver__liqee_repayments.sql
View File

@ -50,6 +50,7 @@ liqee_repayments AS (
asset_details
)
AND topics [0] :: STRING = '0x6fadbf7329d21f278e724fa0d4511001a158f2a97ee35c5bc4cf8b64417399ef'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/liqee/silver__liqee_withdraws.sql
View File

@ -52,6 +52,7 @@ liqee_redemptions AS (
asset_details
)
AND topics [0] :: STRING = '0x3f693fff038bb8a046aa76d9516190ac7444f7d69cf952c4cbdc086fdef2d6fc'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

2
models/silver/defi/lending/venus/silver__venus_borrows.sql
View File

@ -55,6 +55,8 @@ venus_borrows AS (
asset_details
)
AND topics [0] :: STRING = '0x13ed6866d4e1ee6da46f845c46d7e54120883d75c5ea9a2dacc1c4ca8984ab80'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
SELECT

1
models/silver/defi/lending/venus/silver__venus_deposits.sql
View File

@ -52,6 +52,7 @@ venus_deposits AS (
asset_details
)
AND topics [0] :: STRING = '0xb4c03061fb5b7fed76389d5af8f2e0ddb09f8c70d1333abbb62582835e10accb'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

1
models/silver/defi/lending/venus/silver__venus_liquidations.sql
View File

@ -55,6 +55,7 @@ venus_liquidations AS (
asset_details
)
AND topics [0] :: STRING = '0x298637f684da70674f26509b10f07ec2fbc77a335ab1e7d6215a4b2484d8bb52'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

2
models/silver/defi/lending/venus/silver__venus_repayments.sql
View File

@ -50,6 +50,8 @@ venus_repayments AS (
asset_details
)
AND topics [0] :: STRING = '0x1a2a22cb034d26d1854bdc6666a5b91fe25efbbb5dcad3b0355478d6f5c362a1'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (
SELECT

1
models/silver/defi/lending/venus/silver__venus_withdraws.sql
View File

@ -52,6 +52,7 @@ venus_redemptions AS (
asset_details
)
AND topics [0] :: STRING = '0xbd5034ffbd47e4e72a94baa2cdb74c6fad73cb3bcdc13036b72ec8306f5a7646'
AND tx_status = 'SUCCESS'
{% if is_incremental() %}
AND _inserted_timestamp >= (

30
models/silver/nft/sales/silver__complete_nft_sales.sql
View File

@ -38,11 +38,11 @@ WITH nft_base_models AS (
FROM
{{ ref('silver__seaport_1_1_sales') }}
{% if is_incremental() and 'seaport_1_1' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'seaport_1_1' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -78,11 +78,11 @@ SELECT
FROM
{{ ref('silver__seaport_1_4_sales') }}
{% if is_incremental() and 'seaport_1_4' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'seaport_1_4' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -118,11 +118,11 @@ SELECT
FROM
{{ ref('silver__seaport_1_5_sales') }}
{% if is_incremental() and 'seaport_1_5' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'seaport_1_5' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -158,11 +158,11 @@ SELECT
FROM
{{ ref('silver__tofunft_sales') }}
{% if is_incremental() and 'tofunft' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'tofunft' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -198,11 +198,11 @@ SELECT
FROM
{{ ref('silver__pancakeswap_sales') }}
{% if is_incremental() and 'pancakeswap' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'pancakeswap' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -238,11 +238,11 @@ SELECT
FROM
{{ ref('silver__seaport_1_6_sales') }}
{% if is_incremental() and 'seaport_1_6' not in var('HEAL_CURATED_MODEL') %}
{% if is_incremental() and 'seaport_1_6' not in var('HEAL_MODELS') %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) - INTERVAL '36 hours'
MAX(_inserted_timestamp) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -541,7 +541,7 @@ heal_model AS (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -569,7 +569,7 @@ heal_model AS (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)
@ -608,7 +608,7 @@ heal_model AS (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)

2
models/silver/nft/silver__nft_transfers.sql
View File

@ -362,7 +362,7 @@ heal_model AS (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '36 hours'
) - INTERVAL '{{ var("LOOKBACK", "4 hours") }}'
FROM
{{ this }}
)