diff --git a/doc/admin/observability/alerts.md b/doc/admin/observability/alerts.md index ea00e72c27f..ce0ca709063 100644 --- a/doc/admin/observability/alerts.md +++ b/doc/admin/observability/alerts.md @@ -3803,7 +3803,6 @@ with your code hosts connections or networking issues affecting communication wi **Descriptions** - warning repo-updater: less than 250 remaining calls to GitHub graphql API before hitting the rate limit -- critical repo-updater: less than 250 remaining calls to GitHub graphql API before hitting the rate limit for 50m0s **Next steps** @@ -3813,8 +3812,7 @@ with your code hosts connections or networking issues affecting communication wi ```json "observability.silenceAlerts": [ - "warning_repo-updater_github_graphql_rate_limit_remaining", - "critical_repo-updater_github_graphql_rate_limit_remaining" + "warning_repo-updater_github_graphql_rate_limit_remaining" ] ``` @@ -3829,7 +3827,6 @@ with your code hosts connections or networking issues affecting communication wi **Descriptions** - warning repo-updater: less than 250 remaining calls to GitHub rest API before hitting the rate limit -- critical repo-updater: less than 250 remaining calls to GitHub rest API before hitting the rate limit for 50m0s **Next steps** @@ -3839,8 +3836,7 @@ with your code hosts connections or networking issues affecting communication wi ```json "observability.silenceAlerts": [ - "warning_repo-updater_github_rest_rate_limit_remaining", - "critical_repo-updater_github_rest_rate_limit_remaining" + "warning_repo-updater_github_rest_rate_limit_remaining" ] ``` @@ -3855,7 +3851,6 @@ with your code hosts connections or networking issues affecting communication wi **Descriptions** - warning repo-updater: less than 5 remaining calls to GitHub search API before hitting the rate limit -- critical repo-updater: less than 5 remaining calls to GitHub search API before hitting the rate limit for 50m0s **Next steps** @@ -3865,8 +3860,7 @@ with your code hosts connections or networking issues affecting communication wi ```json "observability.silenceAlerts": [ - "warning_repo-updater_github_search_rate_limit_remaining", - "critical_repo-updater_github_search_rate_limit_remaining" + "warning_repo-updater_github_search_rate_limit_remaining" ] ``` diff --git a/doc/admin/observability/dashboards.md b/doc/admin/observability/dashboards.md index d3269f4e7c0..9e1ec9e7a34 100644 --- a/doc/admin/observability/dashboards.md +++ b/doc/admin/observability/dashboards.md @@ -12047,7 +12047,7 @@ Query: `max(src_repoupdater_errored_sync_jobs_percentage)`

Remaining calls to GitHub graphql API before hitting the rate limit

-Refer to the [alerts reference](./alerts.md#repo-updater-github-graphql-rate-limit-remaining) for 2 alerts related to this panel. +Refer to the [alerts reference](./alerts.md#repo-updater-github-graphql-rate-limit-remaining) for 1 alert related to this panel. To see this panel, visit `/-/debug/grafana/d/repo-updater/repo-updater?viewPanel=100220` on your Sourcegraph instance. @@ -12066,7 +12066,7 @@ Query: `max by (name) (src_github_rate_limit_remaining_v2{resource="graphql"})`

Remaining calls to GitHub rest API before hitting the rate limit

-Refer to the [alerts reference](./alerts.md#repo-updater-github-rest-rate-limit-remaining) for 2 alerts related to this panel. +Refer to the [alerts reference](./alerts.md#repo-updater-github-rest-rate-limit-remaining) for 1 alert related to this panel. To see this panel, visit `/-/debug/grafana/d/repo-updater/repo-updater?viewPanel=100221` on your Sourcegraph instance. @@ -12085,7 +12085,7 @@ Query: `max by (name) (src_github_rate_limit_remaining_v2{resource="rest"})`

Remaining calls to GitHub search API before hitting the rate limit

-Refer to the [alerts reference](./alerts.md#repo-updater-github-search-rate-limit-remaining) for 2 alerts related to this panel. +Refer to the [alerts reference](./alerts.md#repo-updater-github-search-rate-limit-remaining) for 1 alert related to this panel. To see this panel, visit `/-/debug/grafana/d/repo-updater/repo-updater?viewPanel=100222` on your Sourcegraph instance. diff --git a/monitoring/definitions/repo_updater.go b/monitoring/definitions/repo_updater.go index ddb175141a6..8d91d618870 100644 --- a/monitoring/definitions/repo_updater.go +++ b/monitoring/definitions/repo_updater.go @@ -393,11 +393,8 @@ func RepoUpdater() *monitoring.Dashboard { Query: `max by (name) (src_github_rate_limit_remaining_v2{resource="graphql"})`, // 5% of initial limit of 5000 Warning: monitoring.Alert().LessOrEqual(250), - // Critical if most of a 60-minute reset window is spent below - // the threshold. - Critical: monitoring.Alert().LessOrEqual(250).For(50 * time.Minute), - Panel: monitoring.Panel().LegendFormat("{{name}}"), - Owner: monitoring.ObservableOwnerRepoManagement, + Panel: monitoring.Panel().LegendFormat("{{name}}"), + Owner: monitoring.ObservableOwnerRepoManagement, NextSteps: ` - Consider creating a new token for the indicated resource (the 'name' label for series below the threshold in the dashboard) under a dedicated machine user to reduce rate limit pressure. `, @@ -408,11 +405,8 @@ func RepoUpdater() *monitoring.Dashboard { Query: `max by (name) (src_github_rate_limit_remaining_v2{resource="rest"})`, // 5% of initial limit of 5000 Warning: monitoring.Alert().LessOrEqual(250), - // Critical if most of a 60-minute reset window is spent below - // the threshold. - Critical: monitoring.Alert().LessOrEqual(250).For(50 * time.Minute), - Panel: monitoring.Panel().LegendFormat("{{name}}"), - Owner: monitoring.ObservableOwnerRepoManagement, + Panel: monitoring.Panel().LegendFormat("{{name}}"), + Owner: monitoring.ObservableOwnerRepoManagement, NextSteps: ` - Consider creating a new token for the indicated resource (the 'name' label for series below the threshold in the dashboard) under a dedicated machine user to reduce rate limit pressure. `, @@ -422,11 +416,8 @@ func RepoUpdater() *monitoring.Dashboard { Description: "remaining calls to GitHub search API before hitting the rate limit", Query: `max by (name) (src_github_rate_limit_remaining_v2{resource="search"})`, Warning: monitoring.Alert().LessOrEqual(5), - // Critical if most of a 60-minute reset window is spent below - // the threshold. - Critical: monitoring.Alert().LessOrEqual(5).For(50 * time.Minute), - Panel: monitoring.Panel().LegendFormat("{{name}}"), - Owner: monitoring.ObservableOwnerRepoManagement, + Panel: monitoring.Panel().LegendFormat("{{name}}"), + Owner: monitoring.ObservableOwnerRepoManagement, NextSteps: ` - Consider creating a new token for the indicated resource (the 'name' label for series below the threshold in the dashboard) under a dedicated machine user to reduce rate limit pressure. `,