Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add acceptance tests for how provider handles request_reason argument #11835

Conversation

SarahFrench
Copy link
Collaborator

@SarahFrench SarahFrench commented Sep 26, 2024

This PR:

  • test how the provider behaves when provider configuration arguments come from different sources (config vs ENVs)
  • schema-level validation that's in place, e.g. handling of empty strings
    • See note below
  • use cases: how does this argument impact the providers behaviour in plan/apply
    • This cannot be tested as the X-Goog-Request-Reason header is for auditing purposes in GCP

Writing these tests made me realise that empty string validation doesn't cover request_reason, so I opened: hashicorp/terraform-provider-google#19643 . The acceptance tests are written to define the current behaviour and my expectation is that the PF schema will be update in future to add validation, make those tests fail, and the tests will be updated.

Release Note Template for Downstream PRs (will be copied)


@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 293 insertions(+), 1 deletion(-))
google-beta provider: Diff ( 4 files changed, 293 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 4105
Passed tests: 3688
Skipped tests: 416
Affected tests: 1

Click here to see the affected service packages

All service packages are affected

#### Non-exercised tests

Tests were added that are skipped in VCR:

  • TestAccFwProvider_request_reason_configPrecedenceOverEnvironmentVariables
  • TestAccFwProvider_request_reason_emptyStringValidation
  • TestAccFwProvider_request_reason_precedenceOrderEnvironmentVariables

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccComputeRegionPerInstanceConfig_removeInstanceOnDestroy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeRegionPerInstanceConfig_removeInstanceOnDestroy[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

@SarahFrench
Copy link
Collaborator Author

Tests running in TeamCity:

@SarahFrench SarahFrench marked this pull request as ready for review September 26, 2024 21:28
shuyama1
shuyama1 previously approved these changes Sep 26, 2024
melinath
melinath previously approved these changes Sep 26, 2024
@shuyama1 shuyama1 dismissed their stale review September 26, 2024 21:44

Approved the PR for testing branch settings earlier. Sorry for the noise

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 293 insertions(+), 1 deletion(-))
google-beta provider: Diff ( 4 files changed, 293 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 4103
Passed tests: 3690
Skipped tests: 413
Affected tests: 0

Click here to see the affected service packages

All service packages are affected

$\textcolor{green}{\textsf{All tests passed!}}$

View the build log

Copy link
Contributor

@NickElliot NickElliot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

took a moment to follow how the tests were operating, but LGTM!

@SarahFrench
Copy link
Collaborator Author

Thanks for the review! Yeah, I probably should have left a note about the 2 data sources used in these acceptance tests. You've read through it now but they basically show what values the SDK and PF sides of the provider have been configured with. Basically non-user facing versions of google_client_config that expose all values impacted by user-supplied arguments. It makes tests a lot easier versus trying to test things indirectly, though indirect testing is still needed for the 'usage' test cases in particular.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 294 insertions(+), 1 deletion(-))
google-beta provider: Diff ( 4 files changed, 294 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 4116
Passed tests: 3701
Skipped tests: 413
Affected tests: 2

Click here to see the affected service packages

All service packages are affected

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataprocCluster_withAutoscalingPolicy
  • TestAccVertexAIFeatureOnlineStoreFeatureview_vertexAiFeatureonlinestoreFeatureviewCrossProjectExample

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccVertexAIFeatureOnlineStoreFeatureview_vertexAiFeatureonlinestoreFeatureviewCrossProjectExample[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataprocCluster_withAutoscalingPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

@SarahFrench
Copy link
Collaborator Author

Going ahead with merge as the VCR test failure above, after I addressed merge conflicts, isn't related to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants