artin/sourcegraph
1
0
Fork 0
mirror of https://github.com/sourcegraph/sourcegraph.git synced 2026-02-06 14:51:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

log: add log sampling (#38451)

Browse Source 
Enables log sampling by default for Sourcegraph components that use the new internal logging library - the first 100 identical log entries per second will always be output, but thereafter only every 100th identical message will be output. It can be configured for each service using the environment variables SRC_LOG_SAMPLING_INITIAL and SRC_LOG_SAMPLING_THEREAFTER, and if SRC_LOG_SAMPLING_INITIAL is set to 0 or -1 the sampling will be disabled entirely.
This commit is contained in:
Robert Lin 2022-07-08 08:27:09 -07:00 committed by GitHub
parent 6b4fa3c9d4
commit 1f7e8d1fcf
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 20 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CHANGELOG.md
View File

@ -23,11 +23,12 @@ All notable changes to Sourcegraph are documented in this file.
### Changed
- Updated minimum required veresion of `git` to 2.35.2 in `gitserver` and `server` Docker image. This addresses a few vulnerabilities disclosed in https://github.blog/2022-04-12-git-security-vulnerability-announced/.
- Updated minimum required veresion of `git` to 2.35.2 in `gitserver` and `server` Docker image. This addresses [a few vulnerabilities announced by GitHub](https://github.blog/2022-04-12-git-security-vulnerability-announced/).
- Search: Pasting a query with line breaks into the main search query input will now replace them with spaces instead of removing them. [#37674](https://github.com/sourcegraph/sourcegraph/pull/37674)
- Rewrite resource estimator using the latest metrics [#37869](https://github.com/sourcegraph/sourcegraph/pull/37869)
- Selecting a line multiple times in the file view will only add a single browser history entry [#38204](https://github.com/sourcegraph/sourcegraph/pull/38204)
- The panels on the homepage (recent searches, etc) are now turned off by default. They can be re-enabled by setting `experimentalFeatures.showEnterpriseHomePanels` to true. [#38431](https://github.com/sourcegraph/sourcegraph/pull/38431)
- Log sampling is now enabled by default for Sourcegraph components that use the [new internal logging library](https://github.com/sourcegraph/log) - the first 100 identical log entries per second will always be output, but thereafter only every 100th identical message will be output. It can be configured for each service using the environment variables `SRC_LOG_SAMPLING_INITIAL` and `SRC_LOG_SAMPLING_THEREAFTER`, and if `SRC_LOG_SAMPLING_INITIAL` is set to `0` or `-1` the sampling will be disabled entirely. [#38451](https://github.com/sourcegraph/sourcegraph/pull/38451)
### Fixed

12
doc/admin/observability/logs.md
View File

@ -53,3 +53,15 @@ We also include the following non-OpenTelemetry fields:
"Function": "string",
}
```
## Log sampling
Sourcegraph services that have migrated to the [new internal logging standard](../../dev/how-to/add_logging.md) have log sampling enabled by default.
The first 100 identical log entries per second will always be output, but thereafter only every 100th identical message will be output.
This behaviour can be configured for each service using the following environment variables:
* `SRC_LOG_SAMPLING_INITIAL`: the number of entries with identical messages to always output per second
* `SRC_LOG_SAMPLING_THEREAFTER`: the number of entries with identical messages to discard before emitting another one per second, after `SRC_LOG_SAMPLING_INITIAL`.
Setting `SRC_LOG_SAMPLING_INITIAL` to `0` or `-1` will disable log sampling entirely.

2
go.mod
View File

@ -143,7 +143,7 @@ require (
github.com/sourcegraph/go-lsp v0.0.0-20200429204803-219e11d77f5d
github.com/sourcegraph/go-rendezvous v0.0.0-20210910070954-ef39ade5591d
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838
github.com/sourcegraph/run v0.9.0
github.com/sourcegraph/scip v0.1.0
github.com/sourcegraph/sourcegraph/enterprise/dev/ci/images v0.0.0-20220203145655-4d2a39d3038a

2
go.sum
View File

@ -2172,6 +2172,8 @@ github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf h1:oAdWFqhStsWii
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf/go.mod h1:ppFaPm6kpcHnZGqQTFhUIAQRIEhdQDWP1PCv4/ON354=
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2 h1:zJfrKH3roMV69m7n06C5y2Xk00HextnxvlG0gnpQolI=
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838 h1:8wknDSCUVYbaRT63OD+Iyv+stwze5kE2FAzbA1HrNMA=
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
github.com/sourcegraph/oauth2 v0.0.0-20210825125341-77c1d99ece3c h1:HGa4iJr6MGKnB5qbU7tI511NdGuHUHnNCqP67G6KmfE=
github.com/sourcegraph/oauth2 v0.0.0-20210825125341-77c1d99ece3c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
github.com/sourcegraph/run v0.9.0 h1:mj4pwBqCB+5qEaTp+rhauh5ubYI8n/icOkeiLTCT9Xg=

2
lib/go.mod
View File

@ -21,7 +21,7 @@ require (
github.com/muesli/termenv v0.9.0
github.com/sourcegraph/go-diff v0.6.1
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838
github.com/stretchr/testify v1.7.2
github.com/xeipuuv/gojsonschema v1.2.0
go.uber.org/atomic v1.9.0

2
lib/go.sum
View File

@ -311,6 +311,8 @@ github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf h1:oAdWFqhStsWii
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf/go.mod h1:ppFaPm6kpcHnZGqQTFhUIAQRIEhdQDWP1PCv4/ON354=
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2 h1:zJfrKH3roMV69m7n06C5y2Xk00HextnxvlG0gnpQolI=
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838 h1:8wknDSCUVYbaRT63OD+Iyv+stwze5kE2FAzbA1HrNMA=
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
github.com/sourcegraph/yaml v1.0.1-0.20200714132230-56936252f152 h1:z/MpntplPaW6QW95pzcAR/72Z5TWDyDnSo0EOcyij9o=
github.com/sourcegraph/yaml v1.0.1-0.20200714132230-56936252f152/go.mod h1:GIjDIg/heH5DOkXY3YJ/wNhfHsQHoXGjl8G8amsYQ1I=
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=