Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

google_bigquery_dataset_access unable to handle service account emails with mixed/upper cases #19547

Open
kamal-1 opened this issue Sep 20, 2024 · 5 comments · May be fixed by GoogleCloudPlatform/magic-modules#11875

Comments

@kamal-1
Copy link

kamal-1 commented Sep 20, 2024

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.
  • If an issue is assigned to a user, that user is claiming responsibility for the issue.
  • Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.9.6
on darwin_arm64

  • provider registry.terraform.io/hashicorp/google v6.3.0

Affected Resource(s)

google_bigquery_dataset_access

Terraform Configuration

resource "google_bigquery_dataset_access" "bigquery_dataset_access" {
  for_each      = var.access_requests
  project       = each.value.project_id
  dataset_id    = each.value.dataset_id
  role          = each.value.role
  user_by_email = each.value.user_by_email
}

Debug Output

https://gist.github.com/kamal-1/49cd54c47b1733296b918c08f71d0bb7

Expected Behavior

Terraform apply should succeed after the access is granted to the service account.

Actual Behavior

Terraform apply completes with an error. Note that the access is granted but it fails while validation for service accounts having upper case or mixed case email addresses.

│ Error: Provider produced inconsistent result after apply

│ When applying changes to google_bigquery_dataset_access.bigquery_dataset_access["project-READER-ENTERPRISE-REPORTS-@project.iam.gserviceaccount.com"], provider
│ "provider["registry.terraform.io/hashicorp/google"]" produced an unexpected new value: Root object was present, but now absent.

│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

2024-09-19T21:44:24.224-0700 [DEBUG] provider.terraform-provider-google_v6.3.0_x5: 2024/09/19 21:44:24 [DEBUG] Skipping item with role= "OWNER", looking for "READER")
2024-09-19T21:44:24.224-0700 [DEBUG] provider.terraform-provider-google_v6.3.0_x5: 2024/09/19 21:44:24 [DEBUG] Skipping item with userByEmail= , looking for "[email protected]")
2024-09-19T21:44:24.224-0700 [DEBUG] provider.terraform-provider-google_v6.3.0_x5: 2024/09/19 21:44:24 [DEBUG] Skipping item with userByEmail= "[email protected]", looking for "[email protected]")

Steps to reproduce

  1. Create a service account with upper case email address
  2. Use google_bigquery_dataset_access to assign a BigQuery role to the upper case service account email.
  3. Run terraform apply. Fails with error Error: Provider produced inconsistent result after apply

Important Factoids

  • Using application default credentials to perform terraform plan and terraform apply
  • It works fine if the service account email is all lower case. Issue happens only for upper/mixed case service accounts.

References

There have been similar issues in the past. We were hoping that 17994 being the latest one, would have fixed the issue but even the latest version 6.3.0 has the issue.

#17994
#11985

b/369584826

@kamal-1 kamal-1 added the bug label Sep 20, 2024
@github-actions github-actions bot added forward/review In review; remove label to forward service/bigquery labels Sep 20, 2024
@kamal-1
Copy link
Author

kamal-1 commented Sep 23, 2024

I recently discovered that it is not possible to create service accounts with all caps/mixed case via Google Cloud Console. However, it is possible to do that via gcloud CLI.

gcloud iam service-accounts create TEST-SERVICE-ACCOUNT --description "Creating service account with all caps" --display-name TEST-SERVICE-ACCOUNT

Output: Created service account [TEST-SERVICE-ACCOUNT]

@ggtisc ggtisc self-assigned this Sep 24, 2024
@ggtisc
Copy link
Collaborator

ggtisc commented Sep 25, 2024

According to terraform registry isn't possible to create a google_service_account with uppercase email address.

Apparently it is possible to create it through CLI but I don't have permission to confirm this.

@ggtisc ggtisc removed their assignment Sep 25, 2024
@ggtisc ggtisc removed the forward/review In review; remove label to forward label Sep 25, 2024
@kamal-1
Copy link
Author

kamal-1 commented Sep 25, 2024

According to terraform registry isn't possible to create a google_service_account with uppercase email address.

Apparently it is possible to create it through CLI but I don't have permission to confirm this.

My comment here confirms this understanding. CLI allows the service accounts to be created with upper case.

#19547 (comment)

@kamal-1
Copy link
Author

kamal-1 commented Sep 30, 2024

I tried an alternative of using google_bigquery_dataset_iam_member resource compared to google_bigquery_dataset_access resource.

google_bigquery_dataset_iam_member is able to handle upper/mixed case service accounts absolutely fine.

google_bigquery_dataset_iam_member.editor: Creating...
google_bigquery_dataset_iam_member.editor: Creation complete after 6s []

Can the same be implemented for google_bigquery_dataset_access? If not, are there any drawbacks or using google_bigquery_dataset_iam_member?

@wj-chen
Copy link

wj-chen commented Sep 30, 2024

I opened GoogleCloudPlatform/magic-modules#11875 to fix google_bigquery_dataset_access.

For a comparison of all the resources that handle dataset permissions, see the Notes in https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigquery_dataset_iam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants