AN-4711/add-blitz-curated-tables (#34)

* add blitz

* update docs and edge model updates

* final blitz updates

* remove staking model

* add custom tests

* missed comma delete
This commit is contained in:
Matt Romano 2024-04-16 13:32:24 -07:00 committed by GitHub
parent 453e57a0d3
commit 14cdc5a1f5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
23 changed files with 1966 additions and 35 deletions

View File

@ -0,0 +1,45 @@
{% test blitz_missing_products(
model,
filter) %}
with recent_records as (
select * from {{model}}
where modified_timestamp >= SYSDATE() - INTERVAL '12 hours'
),
invalid_product_ids as (
select distinct product_id
from {{ ref('silver__blitz_dim_products') }}
where product_id not in (select product_id from recent_records)
{% if filter %}
AND {{ filter }}
{% endif %}
)
select *
from invalid_product_ids
{% endtest %}
{% test blitz_product_level_recency(
model,
filter) %}
with recent_records as (
select distinct(product_id) from {{model}}
where block_timestamp >= SYSDATE() - INTERVAL '1 DAY'
),
invalid_product_ids as (
select *
from {{ ref('silver__blitz_dim_products') }}
where product_id not in (select product_id from recent_records)
{% if filter %}
AND {{ filter }}
{% endif %}
)
select *
from invalid_product_ids
{% endtest %}

View File

@ -1,6 +1,6 @@
{% docs blitz_symbol %}
The specific blitz product symbol, if it is a futures product it will have a -PERP suffix.
The specific Blitz product symbol, if it is a futures product it will have a -PERP suffix.
{% enddocs %}
@ -18,7 +18,7 @@ The wallet address of the trader, there can be multiple subaccounts associated w
{% docs blitz_subaccount %}
Independent blitz account of trader with its own margin, balance, positions, and trades. Any wallet can open an arbitrary number of these. Risk is not carried over from subaccount to subaccount.
Independent Blitz account of trader with its own margin, balance, positions, and trades. Any wallet can open an arbitrary number of these. Risk is not carried over from subaccount to subaccount.
{% enddocs %}
@ -128,13 +128,6 @@ The net change in the best bid and best ask prices in the order book, decimal ad
{% enddocs %}
{% docs blitz_mode %}
The type of liquidation, 0 being a LP position, 1 being a balance - ie a Borrow, and 2 being a perp position.
Only available in blitz V1, live until March 8th 2024.
{% enddocs %}
{% docs blitz_health_group %}
@ -160,21 +153,6 @@ To liquidate a position, there must be a payment (transfer) between the liquidat
{% enddocs %}
{% docs blitz_insurance_cover_unadj %}
USDC from the insurance fund pulled into the insolvent account and used to pay liquidators to take on the underwater positions.
Only available in blitz V1, live until March 8th 2024.
{% enddocs %}
{% docs blitz_insurance_cover %}
USDC from the insurance fund pulled into the insolvent account and used to pay liquidators to take on the underwater positions, decimal adjusted. All amounts and prices are adjusted 18 decimals points regardless of underlying asset contract.
Only available in blitz V1, live until March 8th 2024.
{% enddocs %}
{% docs blitz_book_address %}
@ -206,12 +184,6 @@ The name of the product
{% enddocs %}
{% docs blitz_version %}
The version of blitz with version 2 on or after March 8th 2024.
{% enddocs %}
{% docs blitz_token_address %}
The underlying asset token address deposited or withdrawn from the clearinghouse contract.
@ -233,20 +205,290 @@ Spread Liquidation: If there are perp and spot positions in different directions
If it is a spread liquidation this column will show the perp product_id, for both ids refer to the spread_product_ids array.
Only available in V2 blitz liquidations, which went live March 8th 2024.
{% enddocs %}
{% docs blitz_is_encode_spread %}
Indicates whether product_id encodes both a spot and perp product_id for spread_liquidation.
Only available in V2 blitz liquidations, which went live March 8th 2024.
{% enddocs %}
{% docs blitz_decoded_spread_product_ids %}
Array of product_ids that have been decoded from binary. Only available when is_encode_spread is true and the liquidation occurs on V2 blitz, which went live March 8th 2024.
Array of product_ids that have been decoded from binary. Only available when is_encode_spread is true.
{% enddocs %}
{% enddocs %}
{% docs blitz_first_trade_timestamp %}
The block timestamp of this subaccounts first trade.
{% enddocs %}
{% docs blitz_last_trade_timestamp %}
The block timestamp of this subaccounts most recent trade.
{% enddocs %}
{% docs blitz_account_age %}
The age of the account in days.
{% enddocs %}
{% docs blitz_trade_count %}
The total amount of trades executed by the account
{% enddocs %}
{% docs blitz_trade_count_rank %}
The rank against all accounts based on trade count volume.
{% enddocs %}
{% docs blitz_trade_count_24h %}
The total amount of trades made in the last 24 hours.
{% enddocs %}
{% docs blitz_trade_count_rank_24h %}
The rank against all accounts based on trade count volume in the last 24 hours.
{% enddocs %}
{% docs blitz_perp_trade_count %}
The total amount of perpetual trades executed by the account
{% enddocs %}
{% docs blitz_spot_trade_count %}
The total amount of spot trades executed by the account
{% enddocs %}
{% docs blitz_long_count %}
The total amount of buys/longs on the account.
{% enddocs %}
{% docs blitz_short_count %}
The total amount of sell/shorts on the account.
{% enddocs %}
{% docs blitz_total_usd_volume %}
The total USD denominated volume of the account.
{% enddocs %}
{% docs blitz_total_usd_volume_24h %}
The total USD denominated volume of the account in the last 24 hours.
{% enddocs %}
{% docs blitz_total_usd_volume_rank_24h %}
The rank against all accounts based on the total USD denominated volume of the account in the last 24 hours.
{% enddocs %}
{% docs blitz_total_usd_volume_rank %}
The rank against all accounts based on total usd volume on the account.
{% enddocs %}
{% docs blitz_avg_usd_trade_size %}
The average trade size in USD.
{% enddocs %}
{% docs blitz_total_fee_amount %}
The total amount of trading fees paid by the account.
{% enddocs %}
{% docs blitz_total_base_delta_amount %}
The total base delta amount of the account.
{% enddocs %}
{% docs blitz_total_quote_delta_amount %}
The total quote delta amount of the account.
{% enddocs %}
{% docs blitz_total_liquidation_amount %}
The total liquidation amount of the account.
{% enddocs %}
{% docs blitz_total_liquidation_count %}
The total count of liquidation accounts on the account.
{% enddocs %}
{% docs blitz_orderbook_side %}
Designates the bid or ask side of the orderbook price.
{% enddocs %}
{% docs blitz_orderbook_volume %}
The quantity for each bid/ask order at the given price level.
{% enddocs %}
{% docs blitz_orderbook_price %}
The price level for each bid/ask order.
{% enddocs %}
{% docs blitz_orderbook_round_price_0_01 %}
The price level for each bid/ask order, rounded to nearest cent.
{% enddocs %}
{% docs blitz_orderbook_round_price_0_1 %}
The price level for each bid/ask order, rounded to nearest ten cents.
{% enddocs %}
{% docs blitz_orderbook_round_price_1 %}
The price level for each bid/ask order, rounded to nearest dollar.
{% enddocs %}
{% docs blitz_orderbook_round_price_10 %}
The price level for each bid/ask order, rounded to nearest 10 dollars.
{% enddocs %}
{% docs blitz_orderbook_round_price_100 %}
The price level for each bid/ask order, rounded to nearest 100 dollars.
{% enddocs %}
{% docs blitz_hour %}
The hour in which the stats table data was pull and inserted into the table.
{% enddocs %}
{% docs blitz_distinct_sequencer_batches %}
The amount of sequencer transactions that included this product in the last hour.
{% enddocs %}
{% docs blitz_trader_count %}
The distinct traders in the last hour, based on a distinct count of wallet addresses.
{% enddocs %}
{% docs blitz_subaccount_count %}
The distinct traders in the last hour, based on a distinct count of subaccount.
{% enddocs %}
{% docs blitz_total_trade_count %}
The total number of trades on Blitz in the last hour.
{% enddocs %}
{% docs blitz_contract_price %}
The price of the contract when the data was inserted into the table.
{% enddocs %}
{% docs blitz_base_volume_24h %}
The 24 hour trading volume for the pair (unit in base).
{% enddocs %}
{% docs blitz_quote_volume_24h %}
The 24 hour trading volume for the pair (unit in quote).
{% enddocs %}
{% docs blitz_funding_rate %}
Current 24hr funding rate. Can compute hourly funding rate dividing by 24.
A funding rate is a mechanism used to ensure that the price of a perp contract tracks the underlying asset's price as closely as possible.
Positive funding rates reflect the perpetual trading at a premium to the underlying assets price.
{% enddocs %}
{% docs blitz_index_price %}
Last calculated index price for underlying of contract.
{% enddocs %}
{% docs blitz_last_price %}
Last transacted price of base currency based on given quote currency.
{% enddocs %}
{% docs blitz_mark_price %}
The calculated fair value of the contract, independent of the last traded price on the specific exchange.
{% enddocs %}
{% docs blitz_next_funding_rate %}
Timestamp of the next funding rate change, specific to hour the data was pulled from the API.
{% enddocs %}
{% docs blitz_open_interest %}
The open interest of the contract for the hour that the data was pulled. Open interest (OI) refers to the total number of outstanding derivative contracts (e.g., futures or options) that are currently held by market participants and have not yet been settled
{% enddocs %}
{% docs blitz_open_interest_usd %}
The open interest of the contract for the hour that the data was pulled, denominated in USD. Open interest (OI) refers to the total number of outstanding derivative contracts (e.g., futures or options) that are currently held by market participants and have not yet been settled
{% enddocs %}
{% docs blitz_quote_currency %}
Symbol of the target asset.
{% enddocs %}
{% docs blitz_stake_action %}
The staking action with the VRTX staking address
{% enddocs %}

View File

@ -28,4 +28,34 @@ Blitzs spot markets allow you to buy or sell listed crypto assets paired with
Blitzs on-chain clearinghouse operates as the hub combining perpetual and spot markets, collateral, and risk calculations into a single integrated system. The events in this table track when a wallet either deposits or withdraws from the clearinghouse contract.
{% enddocs %}
{% docs blitz_ez_account_stats %}
Subaccount level table showing aggregated total activity across the Blitz exchange.
{% enddocs %}
{% docs blitz_ez_market_stats %}
Orderbook level market stats based on a combination of on-chain data and data from Blitz's ticker V2 API which includes 24-hour pricing and volume information on each market pair available on Blitz.
{% enddocs %}
{% docs blitz_ez_market_depth %}
Liquidity data taken from Blitz's Orderbook API, showing amount of liquidity at each price level.
{% enddocs %}
{% docs blitz_ez_staking %}
All staking actions taken with the VRTX staking contract.
{% enddocs %}
{% docs blitz_ez_edge_trades %}
All edge trades paired with the associated trader/subaccount.
{% enddocs %}

View File

@ -48,6 +48,11 @@ There is more information on how to use dbt docs in the last section of this doc
- [ez_clearing_house_events](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_clearing_house_events)
- [ez_liquidations](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_liquidations)
- [ez_perp_trades](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_perp_trades)
- [ez_account_stats](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_account_stats)
- [ez_edge_trades](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_edge_trades)
- [ez_market_depth_stats](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_market_depth_stats)
- [ez_market_stats](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_market_stats)
- [ez_staking_actions](https://flipsidecrypto.github.io/blast-models/#!/model/model.blast_models.blitz__ez_staking_actions)
## **Helpful User-Defined Functions (UDFs)**

View File

@ -0,0 +1,57 @@
{{ config(
materialized = 'view',
persist_docs ={ "relation": true,
"columns": true },
meta ={
'database_tags':{
'table':{
'PROTOCOL': 'BLITZ',
'PURPOSE': 'CLOB, DEX, STATS'
}
}
}
) }}
SELECT
subaccount,
trader,
first_trade_timestamp,
last_trade_timestamp,
account_age,
trade_count,
DENSE_RANK() over (
ORDER BY
trade_count DESC
) AS trade_count_rank,
trade_count_24h,
DENSE_RANK() over (
ORDER BY
trade_count_24h DESC
) AS trade_count_rank_24h,
perp_trade_count,
spot_trade_count,
long_count,
short_count,
total_usd_volume,
DENSE_RANK() over (
ORDER BY
total_usd_volume DESC
) AS total_usd_volume_rank,
total_usd_volume_24h,
DENSE_RANK() over (
ORDER BY
total_usd_volume_24h DESC
) AS total_usd_volume_rank_24h,
avg_usd_trade_size,
total_fee_amount,
total_base_delta_amount,
total_quote_delta_amount,
total_liquidation_amount,
total_liquidation_amount_quote,
total_liquidation_count,
blitz_account_id as ez_account_stats_id,
inserted_timestamp,
modified_timestamp
FROM
{{ ref('silver__blitz_account_stats') }}
ORDER BY total_usd_volume_rank DESC

View File

@ -0,0 +1,60 @@
version: 2
models:
- name: blitz__ez_account_stats
description: '{{ doc("blitz_ez_account_stats") }}'
columns:
- name: SUBACCOUNT
description: '{{ doc("blitz_subaccount") }}'
- name: TRADER
description: '{{ doc("blitz_trader") }}'
- name: FIRST_TRADE_TIMESTAMP
description: '{{ doc("blitz_first_trade_timestamp") }}'
- name: LAST_TRADE_TIMESTAMP
description: '{{ doc("blitz_last_trade_timestamp") }}'
- name: ACCOUNT_AGE
description: '{{ doc("blitz_account_age") }}'
- name: TRADE_COUNT
description: '{{ doc("blitz_trade_count") }}'
- name: TRADE_COUNT_RANK
description: '{{ doc("blitz_trade_count_rank") }}'
- name: TRADE_COUNT_24H
description: '{{ doc("blitz_trade_count_24h") }}'
- name: TRADE_COUNT_RANK_24H
description: '{{ doc("blitz_trade_count_rank_24h") }}'
- name: PERP_TRADE_COUNT
description: '{{ doc("blitz_perp_trade_count") }}'
- name: SPOT_TRADE_COUNT
description: '{{ doc("blitz_spot_trade_count") }}'
- name: LONG_COUNT
description: '{{ doc("blitz_long_count") }}'
- name: SHORT_COUNT
description: '{{ doc("blitz_short_count") }}'
- name: TOTAL_USD_VOLUME
description: '{{ doc("blitz_total_usd_volume") }}'
- name: TOTAL_USD_VOLUME_RANK
description: '{{ doc("blitz_total_usd_volume_rank") }}'
- name: TOTAL_USD_VOLUME_24H
description: '{{ doc("blitz_total_usd_volume_24h") }}'
- name: TOTAL_USD_VOLUME_RANK_24H
description: '{{ doc("blitz_total_usd_volume_rank_24h") }}'
- name: AVG_USD_TRADE_SIZE
description: '{{ doc("blitz_avg_usd_trade_size") }}'
- name: TOTAL_FEE_AMOUNT
description: '{{ doc("blitz_total_fee_amount") }}'
- name: TOTAL_BASE_DELTA_AMOUNT
description: '{{ doc("blitz_total_base_delta_amount") }}'
- name: TOTAL_QUOTE_DELTA_AMOUNT
description: '{{ doc("blitz_total_quote_delta_amount") }}'
- name: TOTAL_LIQUIDATION_AMOUNT
description: '{{ doc("blitz_total_liquidation_amount") }}'
- name: TOTAL_LIQUIDATION_AMOUNT_QUOTE
description: '{{ doc("blitz_amount_quote_unadj") }}'
- name: TOTAL_LIQUIDATION_COUNT
description: '{{ doc("blitz_total_liquidation_count") }}'
- name: EZ_ACCOUNT_STATS_ID
description: '{{ doc("pk") }}'
- name: INSERTED_TIMESTAMP
description: '{{ doc("inserted_timestamp") }}'
- name: MODIFIED_TIMESTAMP
description: '{{ doc("modified_timestamp") }}'

View File

@ -0,0 +1,58 @@
{{ config(
materialized = 'view',
persist_docs ={ "relation": true,
"columns": true },
meta={
'database_tags':{
'table': {
'PROTOCOL': 'BLITZ',
'PURPOSE': 'CLOB, DEX'
}
}
}
) }}
SELECT
block_number,
block_timestamp,
tx_hash,
edge_event_index,
user_event_index,
edge_digest,
user_digest,
trader,
subaccount,
symbol,
edge_order_type,
user_order_type,
edge_trade_type,
user_trade_type,
edge_is_taker,
user_is_taker,
edge_price_amount_unadj,
user_price_amount_unadj,
edge_price_amount,
user_price_amount,
edge_amount_unadj,
user_amount_unadj,
edge_amount,
user_amount,
edge_amount_usd,
user_amount_usd,
edge_fee_amount_unadj,
user_fee_amount_unadj,
edge_fee_amount,
user_fee_amount,
edge_base_delta_amount_unadj,
user_base_delta_amount_unadj,
edge_base_delta_amount,
user_base_delta_amount,
edge_quote_delta_amount_unadj,
user_quote_delta_amount_unadj,
edge_quote_delta_amount,
user_quote_delta_amount,
blitz_edge_trade_id as ez_edge_trades_id,
inserted_timestamp,
modified_timestamp,
FROM
{{ ref('silver__blitz_edge_trades') }}

View File

@ -0,0 +1,88 @@
version: 2
models:
- name: blitz__ez_edge_trades
description: '{{ doc("blitz_ez_edge_trades") }}'
columns:
- name: BLOCK_NUMBER
description: '{{ doc("blast_block_number") }}'
- name: BLOCK_TIMESTAMP
description: '{{ doc("blast_block_timestamp") }}'
- name: TX_HASH
description: '{{ doc("blast_logs_tx_hash") }}'
- name: EDGE_EVENT_INDEX
description: '{{ doc("blast_event_index") }}'
- name: USER_EVENT_INDEX
description: '{{ doc("blast_event_index") }}'
- name: EDGE_DIGEST
description: '{{ doc("blitz_digest") }}'
- name: USER_DIGEST
description: '{{ doc("blitz_digest") }}'
- name: TRADER
description: '{{ doc("blitz_trader") }}'
- name: SUBACCOUNT
description: '{{ doc("blitz_subaccount") }}'
- name: SYMBOL
description: '{{ doc("blitz_symbol") }}'
- name: EDGE_ORDER_TYPE
description: '{{ doc("blitz_order_type") }}'
- name: USER_ORDER_TYPE
description: '{{ doc("blitz_order_type") }}'
- name: EDGE_TRADE_TYPE
description: '{{ doc("blitz_trade_type") }}'
- name: USER_TRADE_TYPE
description: '{{ doc("blitz_trade_type") }}'
- name: EDGE_IS_TAKER
description: '{{ doc("blitz_is_taker") }}'
- name: USER_IS_TAKER
description: '{{ doc("blitz_is_taker") }}'
- name: EDGE_PRICE_AMOUNT_UNADJ
description: '{{ doc("blitz_price_amount_unadj") }}'
- name: USER_PRICE_AMOUNT_UNADJ
description: '{{ doc("blitz_price_amount_unadj") }}'
- name: EDGE_PRICE_AMOUNT
description: '{{ doc("blitz_price_amount") }}'
- name: USER_PRICE_AMOUNT
description: '{{ doc("blitz_price_amount") }}'
- name: EDGE_AMOUNT_UNADJ
description: '{{ doc("blitz_amount_unadj") }}'
- name: USER_AMOUNT_UNADJ
description: '{{ doc("blitz_amount_unadj") }}'
- name: EDGE_AMOUNT
description: '{{ doc("blitz_amount") }}'
- name: USER_AMOUNT
description: '{{ doc("blitz_amount") }}'
- name: EDGE_AMOUNT_USD
description: '{{ doc("blitz_amount_usd") }}'
- name: USER_AMOUNT_USD
description: '{{ doc("blitz_amount_usd") }}'
- name: EDGE_FEE_AMOUNT_UNADJ
description: '{{ doc("blitz_fee_amount_unadj") }}'
- name: USER_FEE_AMOUNT_UNADJ
description: '{{ doc("blitz_fee_amount_unadj") }}'
- name: EDGE_FEE_AMOUNT
description: '{{ doc("blitz_fee_amount") }}'
- name: USER_FEE_AMOUNT
description: '{{ doc("blitz_fee_amount") }}'
- name: EDGE_BASE_DELTA_AMOUNT_UNADJ
description: '{{ doc("blitz_base_delta_amount_unadj") }}'
- name: USER_BASE_DELTA_AMOUNT_UNADJ
description: '{{ doc("blitz_base_delta_amount_unadj") }}'
- name: EDGE_BASE_DELTA_AMOUNT
description: '{{ doc("blitz_base_delta_amount") }}'
- name: USER_BASE_DELTA_AMOUNT
description: '{{ doc("blitz_base_delta_amount") }}'
- name: EDGE_QUOTE_DELTA_AMOUNT_UNADJ
description: '{{ doc("blitz_quote_delta_amount_unadj") }}'
- name: USER_QUOTE_DELTA_AMOUNT_UNADJ
description: '{{ doc("blitz_quote_delta_amount_unadj") }}'
- name: EDGE_QUOTE_DELTA_AMOUNT
description: '{{ doc("blitz_quote_delta_amount") }}'
- name: USER_QUOTE_DELTA_AMOUNT
description: '{{ doc("blitz_quote_delta_amount") }}'
- name: EZ_EDGE_TRADES_ID
description: '{{ doc("pk") }}'
- name: INSERTED_TIMESTAMP
description: '{{ doc("inserted_timestamp") }}'
- name: MODIFIED_TIMESTAMP
description: '{{ doc("modified_timestamp") }}'

View File

@ -0,0 +1,31 @@
{{ config(
materialized = 'view',
persist_docs ={ "relation": true,
"columns": true },
meta={
'database_tags':{
'table': {
'PROTOCOL': 'BLITZ',
'PURPOSE': 'CLOB, DEX, STATS'
}
}
}
) }}
SELECT
hour,
ticker_id,
product_id,
orderbook_side,
volume,
price,
round_price_0_01,
round_price_0_1,
round_price_1,
round_price_10,
round_price_100,
blitz_market_depth_id as ez_market_depth_stats_id,
inserted_timestamp,
modified_timestamp
FROM
{{ ref('silver__blitz_market_depth') }}

View File

@ -0,0 +1,34 @@
version: 2
models:
- name: blitz__ez_market_depth_stats
description: '{{ doc("blitz_ez_market_depth") }}'
columns:
- name: HOUR
description: '{{ doc("blitz_hour") }}'
- name: TICKER_ID
description: '{{ doc("blitz_ticker_id") }}'
- name: PRODUCT_ID
description: '{{ doc("blitz_product_id") }}'
- name: ORDERBOOK_SIDE
description: '{{ doc("blitz_orderbook_side") }}'
- name: VOLUME
description: '{{ doc("blitz_orderbook_volume") }}'
- name: PRICE
description: '{{ doc("blitz_orderbook_price") }}'
- name: ROUND_PRICE_0_01
description: '{{ doc("blitz_orderbook_round_price_0_01") }}'
- name: ROUND_PRICE_0_1
description: '{{ doc("blitz_orderbook_round_price_0_1") }}'
- name: ROUND_PRICE_1
description: '{{ doc("blitz_orderbook_round_price_1") }}'
- name: ROUND_PRICE_10
description: '{{ doc("blitz_orderbook_round_price_10") }}'
- name: ROUND_PRICE_100
description: '{{ doc("blitz_orderbook_round_price_100") }}'
- name: EZ_MARKET_DEPTH_STATS_ID
description: '{{ doc("pk") }}'
- name: INSERTED_TIMESTAMP
description: '{{ doc("inserted_timestamp") }}'
- name: MODIFIED_TIMESTAMP
description: '{{ doc("modified_timestamp") }}'

View File

@ -0,0 +1,45 @@
{{ config(
materialized = 'view',
persist_docs ={ "relation": true,
"columns": true },
meta={
'database_tags':{
'table': {
'PROTOCOL': 'BLITZ',
'PURPOSE': 'CLOB, DEX, STATS'
}
}
}
) }}
SELECT
hour,
ticker_id,
product_id,
symbol,
distinct_sequencer_batches,
distinct_trader_count,
distinct_subaccount_count,
trade_count,
amount_usd,
fee_amount,
base_delta_amount,
quote_delta_amount,
base_volume_24h,
quote_volume_24h,
funding_rate,
index_price,
last_price,
mark_price,
next_funding_rate_timestamp,
open_interest,
open_interest_usd,
price_change_percent_24h,
product_type,
quote_currency,
quote_volume,
blitz_market_stats_id AS ez_market_stats_id,
inserted_timestamp,
modified_timestamp
FROM
{{ ref('silver__blitz_market_stats') }}

View File

@ -0,0 +1,58 @@
version: 2
models:
- name: blitz__ez_market_stats
description: '{{ doc("blitz_ez_market_stats") }}'
columns:
- name: HOUR
description: '{{ doc("blitz_hour") }}'
- name: TICKER_ID
description: '{{ doc("blitz_ticker_id") }}'
- name: PRODUCT_ID
description: '{{ doc("blitz_product_id") }}'
- name: SYMBOL
description: '{{ doc("blitz_symbol") }}'
- name: DISTINCT_SEQUENCER_BATCHES
description: '{{ doc("blitz_distinct_sequencer_batches") }}'
- name: DISTINCT_TRADER_COUNT
description: '{{ doc("blitz_trader_count") }}'
- name: DISTINCT_SUBACCOUNT_COUNT
description: '{{ doc("blitz_subaccount_count") }}'
- name: TRADE_COUNT
description: '{{ doc("blitz_total_trade_count") }}'
- name: AMOUNT_USD
description: '{{ doc("blitz_amount_usd") }}'
- name: FEE_AMOUNT
description: '{{ doc("blitz_fee_amount") }}'
- name: BASE_DELTA_AMOUNT
description: '{{ doc("blitz_base_delta_amount") }}'
- name: QUOTE_DELTA_AMOUNT
description: '{{ doc("blitz_quote_delta_amount") }}'
- name: BASE_VOLUME_24H
description: '{{ doc("blitz_base_volume_24h") }}'
- name: QUOTE_VOLUME_24H
description: '{{ doc("blitz_quote_volume_24h") }}'
- name: FUNDING_RATE
description: '{{ doc("blitz_funding_rate") }}'
- name: INDEX_PRICE
description: '{{ doc("blitz_index_price") }}'
- name: LAST_PRICE
description: '{{ doc("blitz_last_price") }}'
- name: MARK_PRICE
description: '{{ doc("blitz_mark_price") }}'
- name: NEXT_FUNDING_RATE_TIMESTAMP
description: '{{ doc("blitz_next_funding_rate") }}'
- name: OPEN_INTEREST
description: '{{ doc("blitz_open_interest") }}'
- name: OPEN_INTEREST_USD
description: '{{ doc("blitz_open_interest_usd") }}'
- name: PRODUCT_TYPE
description: '{{ doc("blitz_product_type") }}'
- name: QUOTE_CURRENCY
description: '{{ doc("blitz_quote_currency") }}'
- name: EZ_MARKET_STATS_ID
description: '{{ doc("pk") }}'
- name: INSERTED_TIMESTAMP
description: '{{ doc("inserted_timestamp") }}'
- name: MODIFIED_TIMESTAMP
description: '{{ doc("modified_timestamp") }}'

View File

@ -0,0 +1,234 @@
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = 'subaccount',
tags = 'curated'
) }}
WITH
{% if is_incremental() %}
new_subaccount_actions AS (
SELECT
DISTINCT(subaccount)
FROM
{{ ref('silver__blitz_perps') }}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '12 hours'
FROM
{{ this }}
)
UNION
SELECT
DISTINCT(subaccount)
FROM
{{ ref('silver__blitz_spot') }}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '12 hours'
FROM
{{ this }}
)
UNION
SELECT
DISTINCT(subaccount)
FROM
{{ ref('silver__blitz_liquidations') }}
WHERE
_inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '12 hours'
FROM
{{ this }}
)
UNION
SELECT
DISTINCT(subaccount)
FROM
{{this}}
WHERE
total_usd_volume_24h > 0
),
{% endif %}
trades_union AS (
SELECT
subaccount,
trader,
digest,
'perp' AS product_type,
trade_type,
block_timestamp,
amount,
amount_usd,
fee_amount,
base_delta_amount,
quote_delta_amount,
_inserted_timestamp
FROM
{{ ref('silver__blitz_perps') }}
{% if is_incremental() %}
WHERE
subaccount IN (
SELECT
subaccount
FROM
new_subaccount_actions
)
{% endif %}
UNION ALL
SELECT
subaccount,
trader,
digest,
'spot' AS product_type,
trade_type,
block_timestamp,
amount,
amount_usd,
fee_amount,
base_delta_amount,
quote_delta_amount,
_inserted_timestamp
FROM
{{ ref('silver__blitz_spot') }}
{% if is_incremental() %}
WHERE
subaccount IN (
SELECT
subaccount
FROM
new_subaccount_actions
)
{% endif %}
),
liquidations AS (
SELECT
trader,
subaccount,
SUM(amount) AS total_liquidation_amount,
SUM(amount_quote) AS total_liquidation_amount_quote,
COUNT(*) AS liquidation_count
FROM
{{ ref('silver__blitz_liquidations') }}
{% if is_incremental() %}
WHERE
subaccount IN (
SELECT
subaccount
FROM
new_subaccount_actions
)
{% endif %}
GROUP BY
1,
2
),
FINAL AS (
SELECT
t.subaccount,
t.trader,
MIN(
t.block_timestamp
) AS first_trade_timestamp,
MAX(
t.block_timestamp
) AS last_trade_timestamp,
DATEDIFF(
'day',
first_trade_timestamp,
last_trade_timestamp
) AS account_age,
COUNT(DISTINCT(digest)) AS trade_count,
COUNT(DISTINCT(CASE
WHEN t.block_timestamp >= SYSDATE() - INTERVAL '24 HOURS'
THEN digest
END)) AS trade_count_24h,
SUM(
CASE
WHEN product_type = 'perp' THEN + 1
ELSE 0
END
) AS perp_trade_count,
SUM(
CASE
WHEN product_type = 'spot' THEN + 1
ELSE 0
END
) AS spot_trade_count,
SUM(
CASE
WHEN trade_type = 'buy/long' THEN + 1
ELSE 0
END
) AS long_count,
SUM(
CASE
WHEN trade_type = 'sell/short' THEN + 1
ELSE 0
END
) AS short_count,
SUM(amount_usd) AS total_usd_volume,
SUM(
CASE
WHEN t.block_timestamp >= SYSDATE() - INTERVAL '24 HOURS' THEN amount_usd
ELSE 0
END
) AS total_usd_volume_24h,
AVG(amount_usd) AS avg_usd_trade_size,
SUM(fee_amount) AS total_fee_amount,
SUM(base_delta_amount) AS total_base_delta_amount,
SUM(quote_delta_amount) AS total_quote_delta_amount,
MAX(
l.total_liquidation_amount
) AS total_liquidation_amount,
MAX(
l.total_liquidation_amount_quote
) AS total_liquidation_amount_quote,
MAX(liquidation_count) AS total_liquidation_count,
MAX(
t._inserted_timestamp
) AS _inserted_timestamp
FROM
trades_union t
LEFT JOIN liquidations l
ON t.subaccount = l.subaccount
GROUP BY
1,
2
)
SELECT
f.*,
{{ dbt_utils.generate_surrogate_key(
['f.subaccount']
) }} AS blitz_account_id,
COALESCE(
{% if is_incremental() %}
a.inserted_timestamp,
{% endif %}
SYSDATE(),
NULL
) AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
FINAL f
{% if is_incremental() %}
LEFT JOIN
{{this}} a
ON
a.subaccount = f.subaccount
{% endif %}

View File

@ -0,0 +1,112 @@
version: 2
models:
- name: silver__blitz_account_stats
columns:
- name: SUBACCOUNT
tests:
- not_null
- name: TRADER
tests:
- not_null
- name: FIRST_TRADE_TIMESTAMP
tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- TIMESTAMP_LTZ
- TIMESTAMP_NTZ
- name: LAST_TRADE_TIMESTAMP
tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- TIMESTAMP_LTZ
- TIMESTAMP_NTZ
- name: ACCOUNT_AGE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TRADE_COUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: PERP_TRADES
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: SPOT_TRADES
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: LONG_COUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: SHORT_COUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TOTAL_USD_VOLUME
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TOTAL_FEE_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TOTAL_BASE_DELTA_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TOTAL_QUOTE_DELTA_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TOTAL_LIQUIDATION_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TOTAL_LIQUIDATION_AMOUNT_QUOTE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TOTAL_LIQUIDATION_AMOUNT_COUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: MODIFIED_TIMESTAMP
tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
interval: 1
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- TIMESTAMP_LTZ
- TIMESTAMP_NTZ

View File

@ -114,6 +114,8 @@ FINAL AS (
'-'
) [0]
END AS health_group_symbol,
p.taker_fee,
p.maker_fee,
_inserted_timestamp,
_log_id
FROM

View File

@ -0,0 +1,165 @@
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = '_log_id',
cluster_by = ['block_timestamp::DATE'],
tags = ['curated','reorg']
) }}
WITH perp_trades AS (
SELECT
*
FROM
{{ ref('silver__blitz_perps') }}
p
WHERE
1 = 1
{% if is_incremental() %}
AND _inserted_timestamp >= (
SELECT
MAX(
_inserted_timestamp
) - INTERVAL '12 hours'
FROM
{{ this }}
)
{% endif %}
),
edge_trades AS (
SELECT
event_index - 1 AS trader_event_before,
event_index + 1 AS trader_event_after,*
FROM
perp_trades
WHERE
trader = '0x0000000000000000000000000000000000000000'
),
FINAL AS (
SELECT
e.block_number,
e.block_timestamp,
e.tx_hash,
e.event_index AS edge_event_index,
e.trader_event_before AS user_event_index,
e.digest AS edge_digest,
p.digest AS user_digest,
p.trader,
p.subaccount,
p.symbol,
e.order_type AS edge_order_type,
p.order_type AS user_order_type,
e.is_taker AS edge_is_taker,
p.is_taker AS user_is_taker,
e.trade_type AS edge_trade_type,
p.trade_type AS user_trade_type,
e.price_amount_unadj AS edge_price_amount_unadj,
p.price_amount_unadj AS user_price_amount_unadj,
e.price_amount AS edge_price_amount,
p.price_amount AS user_price_amount,
e.amount_unadj AS edge_amount_unadj,
p.amount_unadj AS user_amount_unadj,
e.amount AS edge_amount,
p.amount AS user_amount,
e.amount_usd AS edge_amount_usd,
p.amount_usd AS user_amount_usd,
e.fee_amount_unadj AS edge_fee_amount_unadj,
p.fee_amount_unadj AS user_fee_amount_unadj,
e.fee_amount AS edge_fee_amount,
p.fee_amount AS user_fee_amount,
e.base_delta_amount_unadj AS edge_base_delta_amount_unadj,
p.base_delta_amount_unadj AS user_base_delta_amount_unadj,
e.base_delta_amount AS edge_base_delta_amount,
p.base_delta_amount AS user_base_delta_amount,
e.quote_delta_amount_unadj AS edge_quote_delta_amount_unadj,
p.quote_delta_amount_unadj AS user_quote_delta_amount_unadj,
e.quote_delta_amount AS edge_quote_delta_amount,
p.quote_delta_amount AS user_quote_delta_amount,
e._log_id,
e._inserted_timestamp
FROM
edge_trades e
LEFT JOIN (
SELECT
*
FROM
perp_trades
WHERE
trader <> '0x0000000000000000000000000000000000000000'
) p
ON e.tx_hash = p.tx_hash
AND e.trader_event_before = p.event_index
AND e.product_id = p.product_id
WHERE
user_digest IS NOT NULL
UNION ALL
SELECT
e.block_number,
e.block_timestamp,
e.tx_hash,
e.event_index AS edge_event_index,
e.trader_event_after AS user_event_index,
e.digest AS edge_digest,
p.digest AS user_digest,
p.trader,
p.subaccount,
p.symbol,
e.order_type AS edge_order_type,
p.order_type AS user_order_type,
e.is_taker AS edge_is_taker,
p.is_taker AS user_is_taker,
e.trade_type AS edge_trade_type,
p.trade_type AS user_trade_type,
e.price_amount_unadj AS edge_price_amount_unadj,
p.price_amount_unadj AS user_price_amount_unadj,
e.price_amount AS edge_price_amount,
p.price_amount AS user_price_amount,
e.amount_unadj AS edge_amount_unadj,
p.amount_unadj AS user_amount_unadj,
e.amount AS edge_amount,
p.amount AS user_amount,
e.amount_usd AS edge_amount_usd,
p.amount_usd AS user_amount_usd,
e.fee_amount_unadj AS edge_fee_amount_unadj,
p.fee_amount_unadj AS user_fee_amount_unadj,
e.fee_amount AS edge_fee_amount,
p.fee_amount AS user_fee_amount,
e.base_delta_amount_unadj AS edge_base_delta_amount_unadj,
p.base_delta_amount_unadj AS user_base_delta_amount_unadj,
e.base_delta_amount AS edge_base_delta_amount,
p.base_delta_amount AS user_base_delta_amount,
e.quote_delta_amount_unadj AS edge_quote_delta_amount_unadj,
p.quote_delta_amount_unadj AS user_quote_delta_amount_unadj,
e.quote_delta_amount AS edge_quote_delta_amount,
p.quote_delta_amount AS user_quote_delta_amount,
e._log_id,
e._inserted_timestamp
FROM
edge_trades e
LEFT JOIN (
SELECT
*
FROM
perp_trades
WHERE
trader <> '0x0000000000000000000000000000000000000000'
) p
ON e.tx_hash = p.tx_hash
AND e.trader_event_after = p.event_index
AND e.product_id = p.product_id
WHERE
user_digest IS NOT NULL
)
SELECT
*,
{{ dbt_utils.generate_surrogate_key(
['tx_hash','edge_event_index']
) }} AS blitz_edge_trade_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
FINAL qualify(ROW_NUMBER() over(PARTITION BY _log_id
ORDER BY
_inserted_timestamp DESC)) = 1

View File

@ -0,0 +1,170 @@
version: 2
models:
- name: silver__blitz_edge_trades
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- _LOG_ID
columns:
- name: BLOCK_NUMBER
tests:
- not_null
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: BLOCK_TIMESTAMP
tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
interval: 1
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- TIMESTAMP_LTZ
- TIMESTAMP_NTZ
- name: TX_HASH
tests:
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
regex: 0[xX][0-9a-fA-F]+
- name: EDGE_EVENT_INDEX
tests:
- not_null
- name: USER_EVENT_INDEX
tests:
- not_null
- name: TRADER
tests:
- not_null
- dbt_expectations.expect_column_values_to_match_regex:
regex: 0[xX][0-9a-fA-F]+
- name: SYMBOL
tests:
- not_null
- name: EDGE_ORDER_TYPE
tests:
- not_null
- name: USER_ORDER_TYPE
tests:
- not_null
- name: EDGE_IS_TAKER
tests:
- not_null
- name: USER_IS_TAKER
tests:
- not_null
- name: EDGE_TRADE_TYPE
tests:
- not_null
- name: USER_TRADE_TYPE
tests:
- not_null
- name: EDGE_PRICE_AMOUNT_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_PRICE_AMOUNT_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_PRICE_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_PRICE_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_AMOUNT_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_AMOUNT_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_AMOUNT_USD
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_AMOUNT_USD
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_QUOTE_DELTA_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_QUOTE_DELTA_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_QUOTE_DELTA
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_QUOTE_DELTA
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_BASE_DELTA_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_BASE_DELTA_UNADJ
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: EDGE_BASE_DELTA
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: USER_BASE_DELTA
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT

View File

@ -0,0 +1,104 @@
{{ config(
materialized = 'incremental',
incremental_strategy = 'delete+insert',
unique_key = ['product_id','hour','price'],
cluster_by = ['hour::DATE'],
tags = 'curated'
) }}
WITH market_depth AS ({% for item in range(55) %}
SELECT
t.ticker_id, t.product_id, DATE_TRUNC('hour', TRY_TO_TIMESTAMP(t.timestamp)) AS HOUR, 'asks' AS orderbook_side, A.value [0] :: FLOAT AS price, A.value [1] :: FLOAT AS volume, SYSDATE() AS _inserted_timestamp
FROM
(
SELECT
response :ticker_id AS ticker_id, response :timestamp :: STRING AS TIMESTAMP, response :asks AS asks, response :bids AS bids, product_id
FROM
(
SELECT
PARSE_JSON(live.udf_api(CONCAT('https://gateway.blast-prod.vertexprotocol.com/v2/orderbook?ticker_id=', ticker_id, '&depth=1000000'))) :data AS response, product_id
FROM
(
SELECT
ROW_NUMBER() over (
ORDER BY
product_id) AS row_num, product_id, ticker_id
FROM
{{ ref('silver__blitz_dim_products') }}
WHERE
product_id > 0
ORDER BY
product_id)
WHERE
row_num = {{ item + 1 }})) t, LATERAL FLATTEN(input => t.asks) A
UNION ALL
SELECT
t.ticker_id, t.product_id, DATE_TRUNC('hour', TRY_TO_TIMESTAMP(t.timestamp)) AS HOUR, 'bids' AS orderbook_side, A.value [0] :: FLOAT AS price, A.value [1] :: FLOAT AS volume, SYSDATE() AS _inserted_timestamp
FROM
(
SELECT
response :ticker_id AS ticker_id, response :timestamp :: STRING AS TIMESTAMP, response :asks AS asks, response :bids AS bids, product_id
FROM
(
SELECT
PARSE_JSON(live.udf_api(CONCAT('https://gateway.blast-prod.vertexprotocol.com/v2/orderbook?ticker_id=', ticker_id, '&depth=1000000'))) :data AS response, product_id
FROM
(
SELECT
ROW_NUMBER() over (
ORDER BY
product_id) AS row_num, product_id, ticker_id
FROM
{{ ref('silver__blitz_dim_products') }}
WHERE
product_id > 0
ORDER BY
product_id)
WHERE
row_num = {{ item + 1 }})) t, LATERAL FLATTEN(input => t.bids) A {% if not loop.last %}
UNION ALL
{% endif %}
{% endfor %}),
FINAL AS (
SELECT
ticker_id :: STRING AS ticker_id,
product_id,
HOUR,
orderbook_side,
price,
ROUND(
price,
2
) AS round_price_0_01,
ROUND(
price,
1
) AS round_price_0_1,
ROUND(
price,
0
) AS round_price_1,
ROUND(
price,
-1
) AS round_price_10,
ROUND(
price,
-2
) AS round_price_100,
volume,
_inserted_timestamp
FROM
market_depth
)
SELECT
*,
{{ dbt_utils.generate_surrogate_key(
['product_id','hour','price']
) }} AS blitz_market_depth_id,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
'{{ invocation_id }}' AS _invocation_id
FROM
FINAL

View File

@ -0,0 +1,69 @@
version: 2
models:
- name: silver__blitz_market_depth
tests:
- blitz_missing_products:
filter: product_id <> 0 AND (product_type = 'perp' and taker_fee is NULL)
columns:
- name: TICKER_ID
tests:
- not_null
- name: PRODUCT_ID
tests:
- not_null
- name: HOUR
tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
interval: 1
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- TIMESTAMP_LTZ
- TIMESTAMP_NTZ
- name: ORDERBOOK_SIDE
tests:
- not_null
- name: PRICE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: ROUND_PRICE_0_01
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: ROUND_PRICE_0_1
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: ROUND_PRICE_1
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: ROUND_PRICE_10
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: ROUND_PRICE_100
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: VOLUME
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT

View File

@ -0,0 +1,174 @@
{{ config(
materialized = 'incremental',
full_refresh = false,
unique_key = ['ticker_id','hour'],
cluster_by = ['HOUR::DATE'],
tags = 'curated'
) }}
WITH api_pull AS (
SELECT
PARSE_JSON(
live.udf_api(
'https://archive.blast-prod.vertexprotocol.com/v2/contracts'
)
) :data AS response
),
market_stats AS (
SELECT
DATE_TRUNC('hour', SYSDATE()) AS HOUR,
f.value :base_currency :: STRING AS base_currency,
f.value :base_volume :: FLOAT AS base_volume,
f.value :contract_price :: FLOAT AS contract_price,
f.value :contract_price_currency :: STRING AS contract_price_currency,
f.value :funding_rate :: FLOAT AS funding_rate,
f.value :index_price :: FLOAT AS index_price,
f.value :last_price :: FLOAT AS last_price,
f.value :mark_price :: FLOAT AS mark_price,
TRY_TO_TIMESTAMP(
f.value :next_funding_rate_timestamp :: STRING
) AS next_funding_rate_timestamp,
f.value :open_interest :: FLOAT AS open_interest,
f.value :open_interest_usd :: FLOAT AS open_interest_usd,
f.value :price_change_percent_24h :: FLOAT AS price_change_percent_24h,
f.value :product_type :: STRING AS product_type,
f.value :quote_currency :: STRING AS quote_currency,
f.value :quote_volume :: FLOAT AS quote_volume,
f.key AS ticker_id,
SYSDATE() AS _inserted_timestamp
FROM
api_pull A,
LATERAL FLATTEN(
input => response
) AS f
),
trade_snapshot AS (
SELECT
DATE_TRUNC(
'hour',
block_timestamp
) AS HOUR,
CONCAT(
symbol,
'_USDC'
) AS ticker_id,
symbol,
product_id,
COUNT(DISTINCT(tx_hash)) AS distinct_sequencer_batches,
COUNT(DISTINCT(trader)) AS distinct_trader_count,
COUNT(DISTINCT(subaccount)) AS distinct_subaccount_count,
COUNT(DISTINCT(digest)) AS trade_count,
SUM(amount_usd) AS amount_usd,
SUM(fee_amount) AS fee_amount,
SUM(base_delta_amount) AS base_delta_amount,
SUM(quote_delta_amount) AS quote_delta_amount,
MAX(_inserted_timestamp) AS _inserted_timestamp
FROM
{{ ref('silver__blitz_perps') }}
p
WHERE
block_timestamp > SYSDATE() - INTERVAL '12 hour'
GROUP BY
1,
2,
3,
4
),
products AS (
SELECT
*
FROM
{{ ref('silver__blitz_dim_products') }}
),
FINAL AS (
SELECT
s.hour,
s.ticker_id,
p.symbol,
p.product_id,
t.distinct_sequencer_batches,
t.distinct_trader_count,
t.distinct_subaccount_count,
t.trade_count,
t.amount_usd,
t.fee_amount,
t.base_delta_amount,
t.quote_delta_amount,
s.base_volume AS base_volume_24h,
s.quote_volume AS quote_volume_24h,
s.contract_price,
s.contract_price_currency,
s.funding_rate,
s.index_price,
s.last_price,
s.mark_price,
s.next_funding_rate_timestamp,
s.open_interest,
s.open_interest_usd,
s.price_change_percent_24h,
s.product_type,
s.quote_currency,
s.quote_volume,
t._inserted_timestamp,
FROM
market_stats s
LEFT JOIN trade_snapshot t
ON t.ticker_id = s.ticker_id
AND s.hour = t.hour
LEFT JOIN products p
ON s.ticker_id = p.ticker_id
{% if is_incremental() %}
UNION ALL
SELECT
s.hour,
s.ticker_id,
p.symbol,
p.product_id,
t.distinct_sequencer_batches,
t.distinct_trader_count,
t.distinct_subaccount_count,
t.trade_count,
t.amount_usd,
t.fee_amount,
t.base_delta_amount,
t.quote_delta_amount,
s.base_volume_24h,
s.quote_volume_24h,
s.contract_price,
s.contract_price_currency,
s.funding_rate,
s.index_price,
s.last_price,
s.mark_price,
s.next_funding_rate_timestamp,
s.open_interest,
s.open_interest_usd,
s.price_change_percent_24h,
s.product_type,
s.quote_currency,
s.quote_volume,
t._inserted_timestamp
FROM
{{ this }}
s
LEFT JOIN trade_snapshot t
ON t.ticker_id = s.ticker_id
AND s.hour = t.hour
LEFT JOIN products p
ON s.ticker_id = p.ticker_id
{% endif %}
)
SELECT
*,
SYSDATE() AS inserted_timestamp,
SYSDATE() AS modified_timestamp,
{{ dbt_utils.generate_surrogate_key(
['ticker_id','hour']
) }} AS blitz_market_stats_id,
'{{ invocation_id }}' AS _invocation_id
FROM
FINAL qualify(ROW_NUMBER() over(PARTITION BY ticker_id, HOUR
ORDER BY
_inserted_timestamp DESC)) = 1

View File

@ -0,0 +1,144 @@
version: 2
models:
- name: silver__blitz_market_stats
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- HOUR
- TICKER_ID
- blitz_missing_products:
filter: product_type = 'perp' AND product_id <> 0
columns:
- name: HOUR
tests:
- not_null
- dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
interval: 1
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- TIMESTAMP_LTZ
- TIMESTAMP_NTZ
- name: TICKER_ID
tests:
- not_null
- name: SYMBOL
tests:
- not_null
- name: PRODUCT_ID
tests:
- not_null
- name: DISTINCT_SEQUENCE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: DISTINCT_TRADER_COUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: DISTINCT_SUBACCOUNT_COUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: TRADE_COUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: AMOUNT_USD
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: FEE_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: BASE_DELTA_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: QUOTE_DELTA_AMOUNT
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: BASE_VOLUME
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: CONTRACT_PRICE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: CONTRACT_PRICE_CURRENCY
tests:
- not_null
- name: FUNDING_RATE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: INDEX_PRICE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: LAST_PRICE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: MARK_PRICE
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: NEXT_FUNDING_RATE_TIMESTAMP
tests:
- not_null
- name: OPEN_INTEREST
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: OPEN_INTEREST_USD
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT
- name: PRODUCT_TYPE
tests:
- not_null
- name: QUOTE_CURRENCY
tests:
- not_null
- name: QUOTE_VOLUME
tests:
- dbt_expectations.expect_column_values_to_be_in_type_list:
column_type_list:
- NUMBER
- FLOAT

View File

@ -5,6 +5,8 @@ models:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- _LOG_ID
- blitz_product_level_recency:
filter: product_type = 'perp' AND product_id NOT IN (0,48)
columns:
- name: BLOCK_NUMBER
tests:

View File

@ -5,6 +5,8 @@ models:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- _LOG_ID
- blitz_product_level_recency:
filter: product_type = 'spot' AND product_id <> 0
columns:
- name: BLOCK_NUMBER
tests: