Ensure that a default customer-managed encryption key (CMEK) is specified for all BigQuery data sets

GCP
GCP

BigQuery encrypts every dataset at rest under a key encryption key that Google owns and manages, so the customer controls neither that key nor its rotation or destruction. The control asks each dataset to carry a default customer-managed encryption key in Cloud KMS, which BigQuery then uses for tables created in that dataset.

What the benchmark asks for

BigQuery itself does not require a CMEK. It encrypts your data either way, and CMEK is what you adopt if you want to hold the key encryption key yourself. The benchmark is what makes it mandatory. Under this control a dataset passes only when a default CMEK is set, so a dataset with no default encryption configuration fails even though its tables are still encrypted. The dataset carries the key in EncryptionConfiguration.kmsKeyName, which the dataset resource returns as defaultEncryptionConfiguration.kmsKeyName, and any table created in that dataset later uses that key if no other one is provided.

Setting the default does not re-encrypt tables that already exist. The control is satisfied at the dataset level by the default key, but the tables created before the default was set keep whatever key they were created with.

How it works

Default BigQuery encryption is already envelope encryption. A data encryption key encrypts the table data, and that data encryption key is itself wrapped by a key encryption key. CMEK does not add a layer. It replaces the key encryption key with one in your Cloud KMS key ring, so the ability to disable or destroy the key moves to your IAM policy and your key schedule.

The key must be an AES-256 symmetric key acting as a key encryption key. Asymmetric keys and signing keys cannot be used.

Encryption and decryption calls are made by a per-project BigQuery service account, not by your own principal. That account is named bq-<project number>@bigquery-encryption.iam.gserviceaccount.com, built from the project number rather than the project ID. It is created lazily, so a new project does not have one until something asks for it, and granting it a role on the key before it exists fails.

Once a table is CMEK-encrypted, every query that reads it calls Cloud KMS to unwrap the data encryption key. That is a runtime dependency. The call is made under roles/cloudkms.cryptoKeyEncrypterDecrypter, which the BigQuery service account holds on the key. Lose the key and the data bound to it is unreadable permanently. Google holds no copy and cannot reconstruct either the key or the plaintext. That is the intended behavior of a customer-held key, and it is the main operational difference from the default.

The key you name is a key resource, not a key version. BigQuery binds a table to whichever version is primary at the moment the table is created, and there is no way to pin a version in the dataset configuration. Rotating the key changes the version used by tables created after the rotation, and leaves existing tables bound to the version they were created with. Those older versions must stay enabled for the data to remain readable.

Location is a hard constraint. The Cloud KMS key ring has to sit in a location matching the dataset's location. A dataset in the US multi-region needs a key ring in the us multi-region, a dataset in EU needs one in europe, and a regional dataset needs a key ring in that exact region. The global KMS location is not offered in the Google Cloud console when you configure CMEK, and can only be selected through bq or GoogleSQL.

What the control does not cover

Every data asset that lives in BigQuery managed storage supports CMEK. Data you query from somewhere else does not, because BigQuery is not the service encrypting it.

BigLake and other external tables read bytes that live in another service. If you query data in an external data source such as Cloud Storage that holds CMEK-encrypted data, the encryption of that data is managed by Cloud Storage. A BigLake table over a Cloud Storage bucket is the common case. The key protecting those objects is set and managed in Cloud Storage. Check the external sources separately.

BigQuery also does not support customer-supplied encryption keys, where the caller passes raw key material on each request. CMEK through Cloud KMS is the only customer-controlled option for BigQuery tables, so a design that assumes the key never reaches Google at all does not apply here.

Query results written to anonymous or temporary destinations, and copies of a table written into a dataset with no default key, are governed by the destination's configuration rather than the source's.

What it costs to comply

The ongoing cost is key management rather than key material. You own key rotation, you own the IAM policy on the key, and you own the decision to destroy a key version. Google cannot recover data encrypted with a key you destroy, and there is no support path that reverses it. Destroying a key version that a table is still bound to permanently loses that table.

Operationally, three things change:

Queries acquire a dependency on Cloud KMS availability and on the key's state. A disabled key takes every table bound to it offline until it is re-enabled. Each query that touches a CMEK table makes a Cloud KMS decrypt call.

Teams that create datasets need to know which key ring belongs to their region. The location rule means a single key cannot cover a US dataset and a europe-west1 dataset, so an organization with datasets in several locations runs several key rings and several keys.

The service account grant is per project. Every project that holds CMEK datasets needs its own bq-<project number> account granted the encrypter/decrypter role on the keys it uses, which is one more step in project provisioning.

CMEK does not change who may read a table. A principal with bigquery.dataViewer on a CMEK dataset queries it exactly as they query any other dataset. What CMEK changes is who holds the key encryption key, and so who decides when it rotates and when a version is destroyed.

How to check your environment

List the datasets in a project:

bq ls

Then show each one and look for the key:

bq show <data_set_object>

A dataset with a default CMEK prints a kmsKeyName. Verify that the field is present. A dataset with no default encryption configuration prints nothing for that field, and fails the control.

The same field is visible in the console under Dataset info, listed as the customer-managed key. For a sweep across many datasets at once, query the INFORMATION_SCHEMA views or read the dataset resource through the BigQuery API, which returns defaultEncryptionConfiguration on each dataset.

Tables carry their own encryption configuration independently of the dataset default, so a dataset-level pass does not tell you that every table in it is CMEK-encrypted. If you need that, check tables as well as datasets.

How to implement it

First, force the BigQuery service account into existence in the project that holds the datasets:

bq show --encryption_service_account --project_id=PROJECT_ID

That command returns the account name, and creating it is a side effect of asking for it. Until it is run, the account you need to grant a role to does not exist.

Grant that account the right to use the key:

gcloud kms keys add-iam-policy-binding \
    --project=KMS_PROJECT_ID \
    --member serviceAccount:bq-PROJECT_NUMBER@bigquery-encryption.iam.gserviceaccount.com \
    --role roles/cloudkms.cryptoKeyEncrypterDecrypter \
    --location=KMS_KEY_LOCATION \
    --keyring=KMS_KEY_RING \
    KMS_KEY

Note the project number in the member, not the project ID. The --project flag names the project that holds the key, which is often not the project that holds the dataset.

Set the default key on the dataset by writing EncryptionConfiguration.kmsKeyName through datasets.insert for a new dataset or datasets.patch for an existing one. Every table created afterwards without its own encryption configuration uses that key. To move an older table onto the key, rewrite it, for example with a copy or a CREATE TABLE ... AS SELECT into a CMEK destination, then swap the names.

Cloud KMS Autokey is the other route. It generates and assigns a key when the BigQuery resource is created, which removes the need to pre-provision keys and grant the service account permissions. A team that creates datasets no longer has to know the location rule or the service account naming pattern, which is where most of the ongoing cost of this control actually sits.

Before rotating, remember that older key versions must stay enabled while tables bound to them still exist. A rotation schedule that disables old versions on a timer will take tables offline. Either keep versions enabled, or rewrite the tables onto the new version as part of rotation.

Further reading

Found it. Fixed it. Done.

Start free. Understand your cloud in under an hour.