artin/sourcegraph
1
0
Fork 0
mirror of https://github.com/sourcegraph/sourcegraph.git synced 2026-02-06 17:31:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Remove blobstore latency alert (#59665)

Browse Source
This commit is contained in:
Petri-Johan Last 2024-01-18 05:12:37 +02:00 committed by GitHub
parent 7fe86c6137
commit 1bba959307
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 0 additions and 72 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
doc/admin/observability/alerts.md
View File

@ -229,40 +229,6 @@ Generated query for warning alert: `max((histogram_quantile(0.9, sum by (le) (ra
<br />
## frontend: blob_load_latency
<p class="subtitle">90th percentile blob load latency over 10m</p>
**Descriptions**
- <span class="badge badge-critical">critical</span> frontend: 5s+ 90th percentile blob load latency over 10m for 10m0s
**Next steps**
- When this alert fires, calls to the blob route are slow to return a response. The UI will likely experience delays loading files and code snippets. It is likely that the gitserver and/or frontend services are experiencing issues, leading to slower responses.
- Confirm that the Sourcegraph gitserver and frontend services have enough CPU/memory using the provisioning panels.
- Trace a request to see what the slowest part is: https://docs.sourcegraph.com/admin/observability/tracing
- Check that gitserver containers have enough CPU/memory and are not getting throttled.
- More help interpreting this metric is available in the [dashboards reference](./dashboards.md#frontend-blob-load-latency).
- **Silence this alert:** If you are aware of this alert and want to silence notifications for it, add the following to your site configuration and set a reminder to re-evaluate the alert:
```json
"observability.silenceAlerts": [
"critical_frontend_blob_load_latency"
]
```
<sub>*Managed by the [Sourcegraph Source team](https://handbook.sourcegraph.com/departments/engineering/teams/source).*</sub>
<details>
<summary>Technical details</summary>
Generated query for critical alert: `max((histogram_quantile(0.9, sum by (le) (rate(src_http_request_duration_seconds_bucket{route="blob"}[10m])))) >= 5)`
</details>
<br />
## frontend: 99th_percentile_search_codeintel_request_duration
<p class="subtitle">99th percentile code-intel successful search request duration over 5m</p>

21
doc/admin/observability/dashboards.md generated
View File

@ -147,27 +147,6 @@ Query: `histogram_quantile(0.9, sum by(le) (rate(src_http_request_duration_secon
<br />
#### frontend: blob_load_latency
<p class="subtitle">90th percentile blob load latency over 10m</p>
- The blob API route provides the files and code snippets that the UI displays.
Refer to the [alerts reference](./alerts.md#frontend-blob-load-latency) for 1 alert related to this panel.
To see this panel, visit `/-/debug/grafana/d/frontend/frontend?viewPanel=100021` on your Sourcegraph instance.
<sub>*Managed by the [Sourcegraph Source team](https://handbook.sourcegraph.com/departments/engineering/teams/source).*</sub>
<details>
<summary>Technical details</summary>
Query: `histogram_quantile(0.9, sum by(le) (rate(src_http_request_duration_seconds_bucket{route="blob"}[10m])))`
</details>
<br />
### Frontend: Search-based code intelligence at a glance
#### frontend: 99th_percentile_search_codeintel_request_duration

17
monitoring/definitions/frontend.go
View File

@ -162,23 +162,6 @@ func Frontend() *monitoring.Dashboard {
- Trace a request to see what the slowest part is: https://docs.sourcegraph.com/admin/observability/tracing
`,
},
{
Name: "blob_load_latency",
Description: "90th percentile blob load latency over 10m",
Query: `histogram_quantile(0.9, sum by(le) (rate(src_http_request_duration_seconds_bucket{route="blob"}[10m])))`,
Critical: monitoring.Alert().GreaterOrEqual(5).For(10 * time.Minute),
Panel: monitoring.Panel().LegendFormat("latency").Unit(monitoring.Seconds),
Owner: monitoring.ObservableOwnerSource,
Interpretation: `
- The blob API route provides the files and code snippets that the UI displays.
`,
NextSteps: `
- When this alert fires, calls to the blob route are slow to return a response. The UI will likely experience delays loading files and code snippets. It is likely that the gitserver and/or frontend services are experiencing issues, leading to slower responses.
- Confirm that the Sourcegraph gitserver and frontend services have enough CPU/memory using the provisioning panels.
- Trace a request to see what the slowest part is: https://docs.sourcegraph.com/admin/observability/tracing
- Check that gitserver containers have enough CPU/memory and are not getting throttled.
`,
},
},
},
},