Ensure that all BigQuery tables are encrypted with customer-managed encryption key (CMEK)

GCP
GCP

BigQuery encrypts every table at rest by default, but the key that protects the table's data encryption key is held and rotated by Google, so a project owner cannot disable it, audit its use, or destroy it to render the data unreadable. The control requires BigQuery tables to be encrypted with a Cloud KMS key the customer owns, which replaces the Google-managed key encryption key with one under the project's control.

What the benchmark asks for

Each table's configuration carries a kmsKeyName field naming the Cloud KMS key that wraps its data encryption key. The control passes when that field is set on every table, in every dataset, in every project. A dataset can carry a default key, which applies to tables created in it afterwards, but a default key does not reach back and re-encrypt the tables already there. Those have to be rewritten one at a time.

The benchmark treats the per-table key as the unit of compliance, and that is the right unit to scan on. It is the wrong unit to manage on. Setting the dataset default is what stops tomorrow's tables from failing, and a page that checks only the tables already present will keep reporting the same finding after every new load job.

How it works

Default BigQuery encryption is envelope encryption. A data encryption key encrypts the table data, and a key encryption key encrypts that data encryption key. CMEK does not add a layer. It changes who owns the outer one. The data encryption key, the encryption of the table bytes, and the performance of a query are the same either way.

The Cloud KMS keys BigQuery accepts for this are AES-256 symmetric keys, and they work strictly as key encryption keys. The key never touches table data. It wraps and unwraps the data encryption key that does.

Two consequences follow from that design, and both surprise people.

The first is that you cannot pin a table to a key version. BigQuery resolves the key resource ID to whichever version is primary at the moment the table is created, and records that. Rotating the key afterwards does not re-wrap existing tables. It changes which version new tables get. If you rotate for a compliance interval, the interval applies to tables created after the rotation, not to the ones already on disk.

The second is that access to the key is granted to BigQuery, not to the table's readers. BigQuery uses a per-project service agent, bq-PROJECT_NUMBER@bigquery-encryption.iam.gserviceaccount.com, and that agent needs the Cloud KMS CryptoKey Encrypter/Decrypter role on the key. The table's own IAM has nothing to do with the encryption path. A user with bigquery.dataViewer on a CMEK table reads it normally, because BigQuery decrypts on their behalf using its own key access.

That service agent does not exist until something asks for it. It is created on demand, so a rollout script that grants the key binding before the agent exists fails on a principal that is not there. Running bq show --encryption_service_account creates it and returns its address, which is the step to put first in any automation.

The key ring location has to match the dataset's location class. A regional dataset needs a key ring in that exact region. A multi-regional dataset needs a multi-regional key ring in the matching region group. A mismatch is not a warning. It blocks the configuration. The Google Cloud console will also not offer the global Cloud KMS region when you set up CMEK for BigQuery, so a team that standardized on the console for key selection cannot pick a global key there even though bq and GoogleSQL accept one.

What the control does not cover

Customer-supplied encryption keys are not an option here. BigQuery and BigLake tables cannot use CSEK, so there is no path for supplying raw key material directly to either. CMEK through Cloud KMS is the only way to bring your own key to BigQuery-managed storage.

BigLake tables are the gap worth knowing about. A BigLake table reads data held somewhere else, typically Cloud Storage, and BigQuery's CMEK setting does not cover that external data. Encryption of the underlying objects is managed by Cloud Storage, under whatever key that bucket uses. A report that shows every BigQuery table with a kmsKeyName can therefore still sit on top of Google-managed Cloud Storage objects. Audit the bucket separately.

The control also does not protect against a reader who has been granted access. CMEK removes Google's ability to read the data without the key, and gives the project the ability to make the data unreadable by disabling or destroying the key. It does nothing about an over-broad roles/bigquery.dataViewer binding.

What it costs to comply

Every query against a CMEK-encrypted table triggers a Cloud KMS decryption call, and those calls are billed as Cloud KMS usage, so the control carries an ongoing charge that rises with query volume. The larger cost in practice is the labour of creating the keys, protecting them, and managing them for as long as the tables they wrap exist.

The key becomes a thing your project has to keep alive. Google does not hold a copy, which is the point of the control and also its risk. If the key version is destroyed or the key ring is deleted, the tables it wraps cannot be read, and nobody at Google can recover them.

Migrating existing tables costs a full rewrite of each one. The data is copied into a new table under the key, which consumes BigQuery slots and, for large tables, takes real time. Anything reading the table during that window needs to be told.

The case where the cost is not worth paying is a dataset with no sensitive content and a high query rate. A public reference dataset, an events table feeding a dashboard that refreshes every minute, or a staging dataset that is rebuilt daily from a source that is itself encrypted. Default encryption already protects that data at rest. What CMEK adds there is a key to manage and a way to lose the data, in return for control over the encryption that nobody is going to exercise.

Cloud KMS Autokey reduces the setup cost where it is available. It provisions key rings, keys, and the encryption service account at the moment a BigQuery resource is created, so there is no manual sequence of creating a key ring in the right location, creating a key, resolving the service agent, and granting the role. It does not change the recovery risk.

How to check your environment

List the datasets in a project:

bq ls

Then show a table and look for kmsKeyName in the output:

bq show <table_object>

A table with no kmsKeyName uses Google-managed keys and fails the control. In the Google Cloud console the same fact appears under Table info on the Details tab, as a Customer-managed key entry that is present or absent.

Check the dataset default as well as the tables, because that determines what the next table will get. A dataset with no default key and a hundred compliant tables is one load job away from a new finding.

Confirm the encryption service account exists before you audit at scale, since the command that reports it also creates it:

bq show --encryption_service_account

Where a table does carry a kmsKeyName, check the key still exists and is enabled in Cloud KMS. A table pointing at a disabled key is configured correctly and unreadable.

How to implement it

Create the key ring in a location that matches the dataset. Regional dataset, key ring in the same region. Multi-regional dataset, multi-regional key ring in the matching region group. A mismatch is rejected outright rather than flagged for you to correct later.

Resolve the BigQuery encryption service account for the project, which creates it if it is not already there:

bq show --encryption_service_account

Grant that service account the Cloud KMS CryptoKey Encrypter/Decrypter role on the key. Grant it on the key, not on the project, so the binding names exactly what BigQuery may wrap with.

Convert an existing table by copying it onto itself under the key:

bq cp --destination_kms_key <customer_managed_key> source_dataset.source_table destination_dataset.destination_table

Source and destination are the same when you are converting in place.

Then set the default key on the dataset, so tables created afterwards pick it up without anyone remembering to pass a flag. A query that writes a result table, a load job from Cloud Storage, and a scheduled transfer all create tables, and none of them will set a key unless the dataset default does it for them.

Where Autokey is available, it does the key ring, key, and service account wiring at resource creation time, which removes most of the above from a new project's setup.

Plan rotation knowing what it does. A new primary version applies to tables created after it. Existing tables stay wrapped by the version that was primary when they were created, so meeting a rotation requirement across a whole dataset means rewriting the tables, by the same copy that converted them in the first place.

Further reading

  • Customer-managed Cloud KMS keys. The full BigQuery CMEK reference: envelope encryption, the encryption service account, location rules, the behavior of key versions, the limits on BigLake and external data, and the Autokey path.

Found it. Fixed it. Done.

Start free. Understand your cloud in under an hour.