Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally
  to expedite investigation and resolution of this issue.

Terraform Version

Terraform v1.7.2
on linux_amd64

• provider registry.terraform.io/hashicorp/archive v2.4.2
• provider registry.terraform.io/hashicorp/google v5.16.0
• provider registry.terraform.io/hashicorp/googleworkspace v0.7.0
• provider registry.terraform.io/hashicorp/kubernetes v2.25.2
• provider registry.terraform.io/hashicorp/random v3.6.0
• provider registry.terraform.io/hashicorp/template v2.2.0

Affected Resource(s)

data source google_secret_manager_secret_version

Terraform Configuration

data "google_secret_manager_secret_version" "test" {
  secret = "test"
}

data "template_file" "test" {
  template =  base64encode(data.google_secret_manager_secret_version.test.secret_data)
}

Debug Output

No response

Expected Behavior

The binary content of the secret version uploaded on gcp secretmanager is preserved

eg;

Actual Behavior

The binary content of the secret version data is altered.

Steps to reproduce

1. Create a gcp secret version by uploading a binary file, for instance a zip archive
2. Get the secret data in a google_secret_manager_secret_version datasource
3. Obtain the binary data from terraform, for instance using the template provider like in the example above
4. terraform apply
5. terraform state show template_file.text; and compare the binary content

Important Factoids

I can get the binary data from a secret version using gcloud:

$  gcloud secrets versions access 4 --secret 'test'     --format='get(payload.data)' | tr '_-' '/+' 
UEsDBAoAAAAAAPNkVFgwTbbeFgAAABYAAAAKABwAdG1wL2JpbmFyeVVUCQADeo/UZYqP1GV1eAsAAQToAwAABOgDAABBCkIKMQoyCuKCrArOqQrCsMKwwrAKUEsBAh4DCgAAAAAA82RUWDBNtt4WAAAAFgAAAAoAGAAAAAAAAQAAAKSBAAAAAHRtcC9iaW5hcnlVVAUAA3qP1GV1eAsAAQToAwAABOgDAABQSwUGAAAAAAEAAQBQAAAAWgAAAAAA

The data is decodable into a zip file:

$  gcloud secrets versions access 4 --secret 'test'     --format='get(payload.data)' |   tr '_-' '/+'   | base64 -d > /tmp/binarycontent.zip
$ unzip -l /tmp/binarycontent.zip
Archive:  /tmp/binarycontent.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
       22  2024-02-20 12:39   tmp/binary
---------                     -------
       22                     1 file

I can also get the text content of the secret using gcloud, and encode it using base64:

$  gcloud secrets versions access 4 --secret 'test' | base64 -w0 ;echo
UEsDBAoAAAAAAO+/vWRUWDBN77+977+9FgAAABYAAAAKABwAdG1wL2JpbmFyeVVUCQADeu+/ve+/vWXvv73vv73vv71ldXgLAAEE77+9AwAABO+/vQMAAEEKQgoxCjIK4oKsCs6pCsKwwrDCsApQSwECHgMKAAAAAADvv71kVFgwTe+/ve+/vRYAAAAWAAAACgAYAAAAAAABAAAA77+977+9AAAAAHRtcC9iaW5hcnlVVAUAA3rvv73vv71ldXgLAAEE77+9AwAABO+/vQMAAFBLBQYAAAAAAQABAFAAAABaAAAAAAA=

In this case, the binary content is altered, and I cannot recreate a readable zip archive from this.
However, it is the exact same content that terraform provides in its secret_data field:

$  terraform state show template_file.test
# template_file.test:
data "template_file" "test" {
    id       = "55dd8970cd7d253ac9f2508db666748867714058cd38b8089e749eaa81d7ac6a"
    rendered = "UEsDBAoAAAAAAO+/vWRUWDBN77+977+9FgAAABYAAAAKABwAdG1wL2JpbmFyeVVUCQADeu+/ve+/vWXvv73vv73vv71ldXgLAAEE77+9AwAABO+/vQMAAEEKQgoxCjIK4oKsCs6pCsKwwrDCsApQSwECHgMKAAAAAADvv71kVFgwTe+/ve+/vRYAAAAWAAAACgAYAAAAAAABAAAA77+977+9AAAAAHRtcC9iaW5hcnlVVAUAA3rvv73vv71ldXgLAAEE77+9AwAABO+/vQMAAFBLBQYAAAAAAQABAFAAAABaAAAAAAA="
    template = (sensitive value)
}

So it seems that terraform interprets the secret data as a string, resulting in the corruption of binary data.

The gcloud secret version acess documentation mentions:

    To get the raw bytes, have Google Cloud CLI print the response as
    base64-encoded and decode:

        $ gcloud secrets versions access 123 --secret=my-secret \
            --format='get(payload.data)' | tr '_-' '/+' | base64 -d

References

No response

b/326473689