sourcegraph/internal/encryption
Erik Seliger 434944e17d
encryption: Standardize envelope encryption (#56711)
AWS and mounted keys both implemented their own version of envelope encryption for secrets. Cloud KMS did not, causing larger payloads to fail encrypting.
This PR adds a standard code path for all the key backends to do envelope encryption. This fixes the size limit issue with KMS and makes the way we encrypt secrets consistent across backends.
Also as a nice side-effect, we now never have to send the actual secret content over the network to encrypt/decrypt it.

To not require any migration of existing data, I implemented fallback mechanisms for the old data formats, so that customers will not need to change anything to get the benefits of the new envelope encryption. Once a value is decrypted and encrypted again (for example, on update), the data is automatically migrated to the new format.
2023-10-02 23:01:04 +02:00
..
aeshelper encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
awskms encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
cache ci: re-enable race detection (#52776) 2023-06-05 20:41:47 +02:00
cloudkms encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
envelope encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
keyring [github app] Add GitHub App authenticators (#50963) 2023-04-24 09:05:50 +02:00
mounted encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
testing bazel: introduce build files for Go (#46770) 2023-01-23 14:00:01 +01:00
wrapper encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
BUILD.bazel ci: re-enable race detection (#52776) 2023-06-05 20:41:47 +02:00
encryptable_test.go encryption: Introduce Encryptable (#40282) 2022-08-15 09:15:51 -05:00
encryptable.go encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
helpers.go Tracing: final cleanups (#54694) 2023-07-13 10:16:11 +02:00
json_encryptable_test.go encryption: Introduce Encryptable (#40282) 2022-08-15 09:15:51 -05:00
json_encryptable.go encryption: Simplify EncryptJSON (#44764) 2022-11-23 14:23:34 +00:00
key.go rename encryption.Key.ID() to Version() & return structured version data (#19706) 2021-04-06 12:31:30 +01:00
noop.go encryption: Do not proactively fetch version (#40118) 2022-08-09 11:39:17 +00:00
README.md encryption: Standardize envelope encryption (#56711) 2023-10-02 23:01:04 +02:00
rsa_test.go Enable staticcheck (#19491) 2021-03-29 14:03:24 -06:00
rsa.go errors: Introduce internal package (#30558) 2022-02-07 15:03:45 +00:00
utils_test.go encryption: Introduce Encryptable (#40282) 2022-08-15 09:15:51 -05:00

Encryption

This package provides tools to encrypt & decrypt data via the encryption.Key interface. This interface is built to wrap any encryption backend, such as cloud provider APIs, stdlib encryption libraries, or testing stubs.

This package was originally designed in RFC 310.

How to use this package

Keys should be passed in/set during initialisation, ideally from main(). Accessing keyring.Default is an antipattern, and only provided for cases where injection is particularly difficult. You should also only pass the individual key(s) that you need, rather than the whole ring.

Data should be kept encrypted for as long as possible. Right now our implementations are decrypting data inside the database package, which is an antipattern. We made this choice in order to make progress as the code did not lend itself easily to moving the decryption out of the store. Ideally you keep the data encrypted until it is passed to whatever needs the zero visibility data.

How to add a new Key

If you need to encrypt some data, but we don't have a dedicated key for that data yet, you should probably add a new Key. In order to do this you need to do a few things:

  • Add config to encryption.keys in site config (schema/site.schema.json), adding a new field of type #/definitions/EncryptionKey.
  • Add a line to keyring.NewRing to call NewKey(ctx, keyConfig.YourKeyName) to initialise this key in the keyring.

How to add a new Key implementation

If you want to implement a different encryption backend, you'll need to add a new Key implementation, in order to do this you should:

  • Create your implementation in a subpackage of encryption, eg encryption/somebackend.
  • Add a new SomethingEncryptionKey schema in schema/site.schema.json, with the type field set to the name of your implementation.
  • Add the name of your key to the type enum on the EncryptionKey schema definition.
  • Add a reference to the new schema to the oneOf array on the EncryptionKey. This means we generate a schema.EncryptionKeys type with all of the key configs as fields, this is done by the !go: {"taggedUnionType": true} expression on EncryptionKey.
  • Then add a case to the switch statement in keyring.NewKey() to initialise your key if the config is provided.

Zero visibility data (encryption.Secret)

The plaintext returned by the Key.Decrypt() method is considered 'zero visibility data'. This means that no human should ever be able to see this data, and if someone does it should be considered compromised, and be replaced. In order to make accidental disclosure more difficult the encryption package returns data in an encryption.Secret wrapper type. This type wraps a value in a struct with an unexported field, implementing the Stringer & json.Marshaler interfaces & redacting the data. The only method that returns the plaintext is Secret.Secret(), this means our handling of secrets is more auditable, and reduces the chances of accidentally leaking the value in logs.

Keyring

The encryption/keyring package provides a way to configure encryption keys & retrieve them in a typesafe manner, it parses site config and sets the keys in a keyring.Ring struct, so users can either access the keyring.Default or inject the ring, and access specific keys safely, rather than needing to spread around the concern of correctly configuring a key.

Composition & extension

The encryption.Key interface was built to be simple, and intended to be extended through composition & embedding. For example key migrations using a Key implementation that wraps two other Keys, decrypting with one & encrypting with the other. You could also create an encryption.Key wrapper that implements its own versioning system, encrypting with a 'primary' Key, but being able to decrypt data with the previous keys.

Implementations

  • GCP Cloud KMS
  • AWS KMS
  • Mounted Key
  • No Op