* Update GitHub Actions workflow to reduce thread count and add extensive API integration documentation - Changed thread count from 24 to 5 in GitHub Actions workflows for improved performance. - Added comprehensive README files for various API integrations including Alchemy, NBA All Day, API Layer, Binance, and more. - Introduced new UDFs and UDTFs for Groq and Slack API integrations, enhancing functionality and usability. - Implemented tests for new UDFs and UDTFs to ensure reliability and correctness. - Updated existing UDF definitions and added new tests for enhanced coverage and robustness. * Refactor Slack UDFs to use webhook secret names and improve error handling - Updated UDF definitions to replace WEBHOOK_URL and BOT_TOKEN with WEBHOOK_SECRET_NAME for enhanced security. - Improved error messages for required parameters in the SQL logic. - Standardized comments for clarity and consistency across UDFs. - Ensured proper handling of user context for accessing secrets in the vault. * update test * fix test * update slack * remove test * fix tests * fix * fix test args * fix * add .gitignore * Add Slack Notification Macros and Enhance UDFs - Introduced a new dbt macro for sending Slack notifications from GitHub Actions with AI-powered failure analysis. - Added comprehensive README documentation for the new macro, detailing setup, configuration options, and usage examples. - Implemented a SQL macro to handle Slack message formatting and sending, including support for AI analysis and threading. - Updated existing UDFs to utilize webhook secret names for improved security and added detailed comments for clarity. - Enhanced error handling and logging within the macros to ensure robust operation and easier debugging. * update slack alerts * update * remove groq * Enhance Slack Alert Macros with AI Analysis Features - Updated README documentation to clarify AI provider options and added new parameters for model selection and custom prompts. - Modified SQL macros to support the new `model_name` and `ai_prompt` parameters for improved AI analysis capabilities. - Adjusted UDF signatures and comments to reflect the changes in AI provider functionality and requirements. - Improved test cases to validate the new features and ensure robust performance of the Slack alert macros. * update slack_alert * change secret path to data_platform * add backward compatibility for udf_api_v2 * revert to Object return type * update type |
||
|---|---|---|
| .. | ||
| README.md | ||
| slack_alert.sql | ||
GitHub Actions Slack Notifications
This directory contains a fast dbt macro system for sending intelligent Slack notifications from GitHub Actions workflows with AI-powered failure analysis.
Features
- ⚡ Fast Execution: Pure SQL dbt macro (no Python overhead)
- 🤖 AI-Powered Analysis: Automatic failure analysis using Cortex or Claude AI
- 💬 Rich Slack Messages: Beautiful Block Kit formatted notifications with color-coded sidebars
- 🧵 Auto-Threading: Detailed job logs posted as threaded replies
- 🎨 Custom Bot Appearance: Custom names, emojis, and avatars
- 🔗 Dual Delivery Methods: Support for both webhooks and bot tokens
- 📊 Comprehensive Details: Job failures, logs, and actionable links
Quick Setup
The failed_gha_slack_alert macro is ready to use immediately - no deployment required!
Setup Options
Option 1: Bot Token Method (Recommended)
- Create a Slack bot with
chat:writepermissions - Get the channel ID from Slack (e.g.,
C1234567890- not channel name) - Store bot token in Livequery vault at
_FSC_SYS/SLACK/intelligence - Add this step to your GitHub Actions workflow:
- name: Notify Slack on Failure
if: failure()
run: |
dbt run-operation failed_gha_slack_alert --vars '{
"owner": "${{ github.repository_owner }}",
"repo": "${{ github.event.repository.name }}",
"run_id": "${{ github.run_id }}",
"slack_channel": "C1234567890"
}' --target dev
Option 2: Webhook Method (Simple Setup)
- Create a Slack webhook URL in your workspace
- Store webhook URL in Livequery vault at
_FSC_SYS/SLACK/alerts - Add this step to your GitHub Actions workflow:
- name: Notify Slack on Failure
if: failure()
run: |
dbt run-operation failed_gha_slack_alert --vars '{
"owner": "${{ github.repository_owner }}",
"repo": "${{ github.event.repository.name }}",
"run_id": "${{ github.run_id }}",
"webhook_secret_name": "alerts"
}' --target dev
Configuration Options
Core Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
owner |
string | ✅ | GitHub repository owner |
repo |
string | ✅ | GitHub repository name |
run_id |
string | ✅ | GitHub Actions run ID |
slack_channel |
string | ✅* | Slack channel ID (e.g., 'C1234567890') - required for bot token method |
webhook_secret_name |
string | ✅* | Webhook vault secret name - required for webhook method |
AI & Analysis
| Parameter | Type | Default | Description |
|---|---|---|---|
enable_ai_analysis |
boolean | true |
Enable AI failure analysis |
ai_provider |
string | 'cortex' |
AI provider: 'cortex' (Snowflake built-in AI) |
model_name |
string | 'mistral-large' |
Required for Cortex: 'mistral-large', 'mistral-7b', 'llama2-70b-chat', 'mixtral-8x7b' |
ai_prompt |
string | '' |
Custom AI analysis prompt (leave empty for default) |
Threading & Appearance
| Parameter | Type | Default | Description |
|---|---|---|---|
enable_auto_threading |
boolean | false |
Auto-post detailed job logs as thread replies |
username |
string | 'GitHub Actions Bot' |
Custom bot display name |
icon_emoji |
string | ':github:' |
Bot emoji (e.g., :robot_face:, :stellar:) |
icon_url |
string | none |
Bot avatar URL (overrides icon_emoji) |
bot_secret_name |
string | 'intelligence' |
Name of bot token secret in vault |
Usage Examples
Basic Notification
dbt run-operation failed_gha_slack_alert --vars '{
"owner": "FlipsideCrypto",
"repo": "streamline-snowflake",
"run_id": "16729602656",
"slack_channel": "C087GJQ1ZHQ"
}' --target dev
AI Analysis with Custom Bot
dbt run-operation failed_gha_slack_alert --vars '{
"owner": "FlipsideCrypto",
"repo": "streamline-snowflake",
"run_id": "16729602656",
"slack_channel": "C087GJQ1ZHQ",
"enable_ai_analysis": true,
"ai_provider": "cortex",
"model_name": "mistral-7b",
"username": "CI/CD Alert Bot",
"icon_emoji": ":robot_face:"
}' --target dev
Auto-Threading with Custom Prompt
dbt run-operation failed_gha_slack_alert --vars '{
"owner": "FlipsideCrypto",
"repo": "streamline-snowflake",
"run_id": "16729602656",
"slack_channel": "C087GJQ1ZHQ",
"enable_ai_analysis": true,
"ai_provider": "cortex",
"model_name": "mixtral-8x7b",
"ai_prompt": "Focus on dependency issues and provide quick fixes:",
"enable_auto_threading": true,
"username": "Pipeline Monitor",
"icon_emoji": ":stellar:"
}' --target dev
Webhook Method
dbt run-operation failed_gha_slack_alert --vars '{
"owner": "FlipsideCrypto",
"repo": "streamline-snowflake",
"run_id": "16729602656",
"webhook_secret_name": "prod-alerts",
"enable_ai_analysis": true,
"ai_provider": "cortex",
"model_name": "mistral-large",
"username": "Production Monitor",
"icon_emoji": ":package:"
}' --target dev
GitHub Actions Integration
- name: Notify Slack on Failure
if: failure()
run: |
dbt run-operation failed_gha_slack_alert --vars '{
"owner": "${{ github.repository_owner }}",
"repo": "${{ github.event.repository.name }}",
"run_id": "${{ github.run_id }}",
"slack_channel": "C087GJQ1ZHQ",
"enable_ai_analysis": true,
"ai_provider": "cortex",
"model_name": "mistral-large",
"enable_auto_threading": true,
"username": "GitHub Actions",
"icon_emoji": ":github:"
}' --target dev
Message Format
Failure Messages Include
- 🔴 Red Sidebar: Visual failure indicator
- Header: Repository name with failure indicator (❌)
- Basic Info: Run ID, failed job count, workflow name
- 🤖 AI Analysis: Intelligent failure analysis with common patterns, root causes, and action items
- 🔗 Action Button: Direct link to workflow run
- 🧵 Threading (if enabled): Individual job details and logs as thread replies
Success Messages Include
- 🟢 Green Sidebar: Visual success indicator
- Header: Repository name with success indicator (✅)
- Basic Info: Run ID, workflow name, success status
- 🔗 Action Button: Direct link to workflow run
AI Analysis
The macro supports Snowflake's Cortex AI for intelligent failure analysis:
Cortex (Default)
- Uses Snowflake's built-in Cortex AI
- Requires
model_nameparameter to specify which model to use - Available models:
'mistral-large','mistral-7b','llama2-70b-chat','mixtral-8x7b' - Automatically analyzes logs and provides insights
- Custom prompts supported via
ai_promptparameter
Enable AI analysis with:
"enable_ai_analysis": true,
"ai_provider": "cortex",
"model_name": "mistral-large", # Required!
"ai_prompt": "Focus on the most critical issues:" # Optional
Environment Variables & Vault Setup
Webhook Method
SLACK_WEBHOOK_URL: Your Slack webhook URL (GitHub secret)
Bot Token Method
- No environment variables required!
- Bot tokens are stored in Livequery vault at:
_FSC_SYS/SLACK/{bot_secret_name} - Channel ID provided as parameter in macro call
Vault Paths for Bot Tokens
Store your bot tokens in these vault locations:
prod/livequery/slack/intelligence(default)prod/livequery/alerts(custom)prod/livequery/<your bot's name>(custom)
** The _FSC/SYS/.. will not work anymore, because we are not able to access studio to store CREDENTIALS anymore. So the context + _FSC/SYS/... is deprecated. It's in the sql code for backward compatability.
How to Get Slack Channel IDs
- Right-click method: Right-click channel → Copy → Copy link (ID is in URL)
- API method: Use
conversations.listendpoint - App method: Channel IDs appear in URLs like
/C1234567890/
Security Notes
- Never hardcode secrets in your workflow files
- Use GitHub's encrypted secrets for webhook URLs
- Bot tokens automatically managed through Livequery vault system
- Channel IDs are not sensitive and can be stored in code
Troubleshooting
Common Issues
- No notification sent: Check webhook URL or channel ID parameter
- Invalid channel ID: Must use channel ID (C1234567890), not name (#channel)
- AI analysis missing: Ensure GitHub Actions integration is properly set up
- Message formatting issues: Verify JSON syntax in custom_message parameter
- Bot permissions: Ensure bot has
chat:writescope for target channel - Vault access: Verify bot token stored at correct vault path
Debug Mode
Add this step before the notification to debug issues:
- name: Debug Notification
run: |
echo "Owner: ${{ github.repository_owner }}"
echo "Repo: ${{ github.event.repository.name }}"
echo "Run ID: ${{ github.run_id }}"
echo "Channel: C1234567890" # Your actual channel ID
Channel ID Validation
Test if your channel ID is valid:
SELECT slack_utils.validate_channel('C1234567890') as is_valid;
-- Should return true for valid channel IDs
Integration with Livequery
This macro integrates with Livequery's marketplace UDFs:
slack_utils.post_webhook(): For webhook-based notificationsslack.post_message()&slack.post_reply(): For bot token messaging with threadinggithub_actions.tf_failure_analysis_with_ai(): For AI-powered failure analysis
UDF Function Signatures
-- Webhook (backward compatible)
slack_utils.post_webhook(webhook_secret_name, payload)
-- Bot messaging (new parameter-based)
slack.post_message(channel_id, payload, bot_secret_name)
slack.post_reply(channel_id, thread_ts, payload, bot_secret_name)
-- Or use 2-parameter versions (uses 'intelligence' bot token)
slack.post_message(channel_id, payload)
slack.post_reply(channel_id, thread_ts, payload)
Ensure these UDFs are deployed before using the notification macro.
Performance & Benefits
⚡ Lightning Fast Execution
- Pure SQL: No Python interpreter overhead
- Direct UDF calls: Leverages Livequery's optimized marketplace functions
- Single transaction: All operations in one dbt run-operation call
- Instant feedback: Real-time execution with immediate Slack delivery
🎯 Production Ready
- Reliable: Battle-tested with GitHub Actions workflows
- Scalable: Handles multiple failed jobs with threading
- Secure: Vault-based credential management
- Flexible: Supports both webhook and bot token methods
🤖 Intelligent Analysis
- AI-Powered: Cortex and Claude integration for failure analysis
- Actionable Insights: Common patterns, root causes, and prioritized action items
- Context-Aware: Includes job names, workflow details, and error logs
- Formatted for Slack: Optimized mrkdwn formatting for better readability
The failed_gha_slack_alert macro provides enterprise-grade Slack notifications with zero deployment overhead and lightning-fast performance.
Examples Repository
See our examples repository for complete workflow configurations and advanced usage patterns.