diff --git a/.cursor/rules/dbt-overview-standard.mdc b/.cursor/rules/dbt-overview-standard.mdc index 1eba5ed..9ff88c9 100644 --- a/.cursor/rules/dbt-overview-standard.mdc +++ b/.cursor/rules/dbt-overview-standard.mdc @@ -42,7 +42,7 @@ The quicklinks section must be organized by schema and use the relative link to ### Schema Organization Rules: 1. **Group by Schema**: Organize models by their schema (core, defi, nft, price, social, governance, etc.) -2. **Use Exact Schema Names**: Use the exact schema names as they appear in the database (e.g., `NEAR.CORE`, `NEAR.DEFI`, `NEAR.NFT`) +2. **Use Exact Schema Names**: Use the exact schema names as they appear in the database (e.g., `.CORE`, `.DEFI`, `.NFT`) 3. **Model Type Subgrouping**: Within each schema, subgroup by model type: - **Dimension Tables:** (dim_* models) - **Fact Tables:** (fact_* models) @@ -53,15 +53,10 @@ The quicklinks section must be organized by schema and use the relative link to ### Implementation Guidelines for Coding Agents: 1. **Scan Directory Structure**: Read `models/gold/` directory to identify all schema subdirectories 2. **Extract Model Names**: For each schema directory, list all `.sql` files and remove the `.sql` extension -3. **Determine Schema Mapping**: Map directory names to database schema names: - - `gold/core/` → `NEAR.CORE` - - `gold/defi/` → `NEAR.DEFI` - - `gold/nft/` → `NEAR.NFT` - - `gold/price/` → `NEAR.PRICE` - - `gold/social/` → `NEAR.SOCIAL` - - `gold/governance/` → `NEAR.GOV` - - `gold/stats/` → `NEAR.STATS` - - `gold/atlas/` → `NEAR.ATLAS` +3. **Determine Schema Mapping**: Map model names to database schema names: +dbt models in this project utilize a double underscore in the model name to denote schema vs table __.sql: + - `core__fact_blocks` → `.CORE.FACT_BLOCKS` + - `defi__ez_dex_swaps` → `.DEFI.EZ_DEX_SWAPS` 4. **Categorize Models**: Group models by prefix: - `dim_*` → Dimension Tables - `fact_*` → Fact Tables @@ -164,22 +159,10 @@ Every `__overview__.md` file MUST include structured `` XML tags for easy i ``` -Place these XML tags at the end of the documentation (BUT STILL BEFORE THE JINJA ENDDOCS TAG), +Place these XML tags at the end of the documentation (BUT STILL BEFORE THE JINJA ENDDOCS TAG). -## Automatic __overview__.md Updates -The `__overview__.md` file MUST be automatically updated whenever any gold model is created, modified, or deleted to maintain documentation consistency. - - -### Trigger Conditions: -The following actions MUST trigger an automatic update to `__overview__.md`: -1. **New Gold Model Created**: When a new `.sql` file is added to any `models/gold/*/` directory -2. **Gold Model Modified**: When any existing `.sql` file in `models/gold/*/` directories is changed -3. **Gold Model Deleted**: When any `.sql` file is removed from `models/gold/*/` directories -4. **Gold Model Renamed**: When any `.sql` file in `models/gold/*/` directories is renamed -5. **Schema Directory Changes**: When new schema directories are added to or removed from `models/gold/` - -### Update Process for Coding Agents: -When any of the above trigger conditions occur, the coding agent MUST: +## Update Process for Coding Agents: +To update the overview, the coding agent MUST: 1. **Scan Current Gold Models**: - Read the entire `models/gold/` directory structure @@ -187,53 +170,17 @@ When any of the above trigger conditions occur, the coding agent MUST: - Extract model names (remove `.sql` extension) 2. **Generate Updated Quicklinks Section**: - - Follow the Implementation Guidelines for Coding Agents (section above) + - Follow these implementation guidelines - Create a complete new quicklinks section with all current gold models - Maintain proper schema organization and model type grouping 3. **Update __overview__.md**: - - Preserve the XML tag section at the top - Replace the entire "Quick Links to Table Documentation" section with the newly generated content - - Maintain all other existing content in the file - Ensure proper markdown formatting and link structure + - Create or update the XML tag block 4. **Validation Check**: - Verify all gold models have corresponding links - Confirm links use correct dbt docs format - Check that schema names and model names are accurate - Ensure alphabetical ordering within categories - -### Implementation Commands: -When a gold model change is detected, the coding agent should execute these steps: - -```bash -# 1. Scan for all gold models -find models/gold/ -name "*.sql" -type f - -# 2. Generate updated quicklinks content -# (Follow the Implementation Guidelines for Coding Agents) - -# 3. Update __overview__.md file -# (Replace the Quick Links section while preserving other content) -``` - -### File Preservation Rules: -When updating `__overview__.md`, the coding agent MUST: -- **Preserve XML Tags**: Keep the `` XML section exactly as is -- **Preserve Introduction**: Maintain the project description and usage instructions -- **Preserve Other Sections**: Keep all sections after the Quick Links (Data Model Overview, Using dbt docs, etc.) -- **Update Only Quick Links**: Replace only the "Quick Links to Table Documentation" section -- **Maintain Formatting**: Preserve markdown formatting and structure - -### Error Handling: -If the update process encounters issues: -1. **Backup Original**: Create a backup of the original `__overview__.md` before making changes -2. **Partial Updates**: If full regeneration fails, attempt to add/remove only the specific model that changed -3. **Validation**: After any update, verify the file is still valid markdown and all links are functional -4. **Rollback**: If validation fails, restore from backup and report the issue - -### Integration with dbt Workflow: -This automatic update process should integrate with standard dbt workflows: -- **Pre-commit**: Update `__overview__.md` before committing gold model changes -- **Post-run**: Verify `__overview__.md` is current after dbt model runs -- **CI/CD**: Include `__overview__.md` validation in continuous integration checks diff --git a/.cursor/rules/review-dbt-documentation.mdc b/.cursor/rules/review-dbt-documentation.mdc index 51dae2b..5f39493 100644 --- a/.cursor/rules/review-dbt-documentation.mdc +++ b/.cursor/rules/review-dbt-documentation.mdc @@ -50,7 +50,8 @@ This document outlines the comprehensive process for reviewing and improving col **Lineage Analysis** - Map the complete data lineage from source to gold model - Identify key transformations and their purposes -- Understand relationships between related models +- Understand relationships between related models for the sole purpose of generating a robust description +- Do not include data lineage analysis in the table description ### Step 2: Column Description Review **Individual Column Analysis**