project_service_identity module returns null email

[serpro69]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.8.5
on linux_amd64

• provider registry.terraform.io/hashicorp/external v2.3.3
• provider registry.terraform.io/hashicorp/google v5.36.0
• provider registry.terraform.io/hashicorp/google-beta v5.36.0
• provider registry.terraform.io/hashicorp/null v3.2.2
• provider registry.terraform.io/hashicorp/random v3.6.2
• provider registry.terraform.io/hashicorp/time v0.11.2

Affected Resource(s)

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/project_service_identity

Terraform Configuration

locals {
  service = {
    cloudkms = {
      project_id = var.kms_project_id,
      name       = "cloudkms.googleapis.com"
    },
    storage = {
      project_id = var.gcs_project_id,
      name       = "storage.googleapis.com"
    }
  }
}

resource "google_project_service" "main" {
  for_each = local.service

  project = each.value.project_id
  service = each.value.name

  depends_on = [
    module.project
  ]
}

data "google_project_service" "storage" {
  project    = local.service.storage.project_id
  service    = local.service.storage.name
  depends_on = [google_project_service.main]
}

data "google_project_service" "kms" {
  project    = local.service.cloudkms.project_id
  service    = local.service.cloudkms.name
  depends_on = [google_project_service.main]
}

resource "google_project_service_identity" "storage" {
  provider = google-beta

  project = data.google_project.gcs.project_id
  service = local.service.storage.name

  depends_on = [
    data.google_project_service.storage
  ]
}

resource "google_project_iam_member" "storage" {
  for_each = toset(["cloudkms.admin", "cloudkms.cryptoOperator"])

  project = data.google_project.gcs.project_id
  role    = "roles/${each.value}"
  member  = "serviceAccount:${google_project_service_identity.storage.email}"

  depends_on = [
    google_project_service.main,
    google_project_service_identity.storage
  ]
}

Debug Output

No response

Expected Behavior

email should not be null

Actual Behavior

email variable is null
all other variables (id, service, and project) contain values, so it's only the email that's returning null for some reason

│ Error: Invalid template interpolation value
│
│   on tfstate/iam.tf line 7, in resource "google_project_iam_member" "storage":
│    7:   member  = "serviceAccount:${google_project_service_identity.storage.email}"
│     ├────────────────
│     │ google_project_service_identity.storage.email is null
│
│ The expression result is null. Cannot include a null value in a string template.
╵
╷
│ Error: Invalid template interpolation value
│
│   on tfstate/iam.tf line 7, in resource "google_project_iam_member" "storage":
│    7:   member  = "serviceAccount:${google_project_service_identity.storage.email}"
│     ├────────────────
│     │ google_project_service_identity.storage.email is null
│
│ The expression result is null. Cannot include a null value in a string template.

Steps to reproduce

1. terraform apply

Important Factoids

No response

References

No response

b/351534769

[noisy-murmure]

We have the same problem. Especially with Big Query and Cloud Storage.

[ggtisc]

Confirmed issue!

After running a terraform apply it returns the next error message:

│ Error: Invalid template interpolation value
│
│   on tfstate/iam.tf line 7, in resource "google_project_iam_member" "storage":
│    7:   member  = "serviceAccount:${google_project_service_identity.storage.email}"
│     ├────────────────
│     │ google_project_service_identity.storage.email is null
│
│ The expression result is null. Cannot include a null value in a string template.

[benhxy]

Watching

[AlvaroGG0]

Any updates?

[SarahFrench]

Stopping by to give some context on this bug. This issue has been forwarded to the service team to work on but I can flesh out some details to help with resolving an issue.

When creating a service identity there are 3 potential outcomes, shown here by the result of running the same gcloud command for different GCP services. They are:

1. Service identity created and an email is returned
2. Service identity created, no email returned
3. API returns an error as they don't support service identities

$ gcloud beta services identity create --service=containerscanning.googleapis.com                    
Service identity created: service-XXXXXXXXXX@gcp-sa-containerscanning.iam.gserviceaccount.com

$ gcloud beta services identity create --service=logging.googleapis.com          
Service identity created

$ gcloud beta services identity create --service=stackdriverprovisioning.googleapis.com
ERROR: (gcloud.beta.services.identity.create) INVALID_ARGUMENT: Service stackdriverprovisioning.googleapis.com has not been configured for service identities.

This issue mentions the Storage API, and I can see it's an example of the second scenario:

$ gcloud beta services identity create --service=storage.googleapis.com  
Service identity created

---

In the past the google_project_service_identity resource would return an explicit error when the response didn't contain an email address. This was reported in a bug report here #10367 and the resolution was to stop the resource returning an error in scenario 2, so errors now only get surfaced if scenario 3 occurs and the service identity cannot be created (API error is returned).

As things currently stand, to address this GH issue there would need to be changes in the upstream APIs to ensure that an email address is returned when making a service identity for the Storage API, though ideally all APIs would behave that way.

[SarahFrench]

Marking as upstream and not a bug because the solution is a change in the upstream Google Cloud API

[benhxy]

The "not returning email" in storage/logging cases is caused by the service owning multiple service agents. When GenerateServiceIdentity creates multiple service agents, it returns none of them. This might need a breaking API change. We are reviewing this internally.