Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(baseline): Allow force creating fresh baseline #2124

Open
wants to merge 43 commits into
base: master
Choose a base branch
from

Conversation

richardwerkman
Copy link
Member

@richardwerkman richardwerkman commented Aug 5, 2022

resolves #1560
resolves #2823
resolves #2847

  • Add command for baseline
  • Add sub command for baseline recreate
  • Remove the --target option and replace with an extra argument for baseline command

The new commands now work as follows:
dotnet stryker with-baseline main
dotnet stryker with-baseline main recreate
or
dotnet stryker baseline main
dotnet stryker baseline main recreate

@richardwerkman richardwerkman reopened this Aug 5, 2022
@richardwerkman richardwerkman marked this pull request as draft August 5, 2022 16:07
@rouke-broersma rouke-broersma changed the title feat(Create baseline): Add baseline create command feat(baseline): Allow force creating fresh baseline Aug 5, 2022
disouzam added a commit to disouzam/blast-furnace-mass-balance that referenced this pull request Dec 27, 2022
…references

-  Full baseline is never generated because since filter marks mutations as ignored due to not being unchanged compared to target #1803
    - stryker-mutator/stryker-net#1803 (comment)

-  feat(baseline): Allow force creating fresh baseline #2124
    - stryker-mutator/stryker-net#2124
danihengeveld and others added 5 commits October 13, 2023 15:12
* chore(deps): update dependency buildalyzer to v5.0.1 (#2665)

* chore(deps): update dependency buildalyzer to v5.0.1

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore: fix page links for site

* fix: Don't fail the test run if restoring test assemblies fails (#2664)

* chore(deps): update xunit-dotnet monorepo to v2.5.1 (#2673)

* chore(deps): update xunit-dotnet monorepo to v2.5.1

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.4.0 (#2677)

* chore(deps): update dependency yamldotnet to v13.4.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore: F# range math (#2679)

* F# range math

* Windows x Linux

* Up

* chore: Passing Stryker options to F# project components (#2680)

Co-authored-by: Rouke Broersma <[email protected]>

* fix: Use solution file when it's available (#2690)

* Use solution file when it's available

* fix unit test

* sonarqube

* chore(compilation): Improve Error and Trace Logging (#2688)

Add logs

Co-authored-by: Rouke Broersma <[email protected]>

* fix: Remove usings from MutantControl (#2694)

chore: remove usings from MutantControl

Co-authored-by: Rouke Broersma <[email protected]>

* chore: Code quality improvements (#2696)

* Use string builder in loop

* Use find & handle other platform newlines

* Use find

* Fix duplicate message

* Add more range tests

* Improve range code

* docs: add SAS configuration (#2676) (#2701)

docs: add SAS configuration

* chore(deps): update dependency launchdarkly.eventsource to v5.1.0 (#2702)

* chore(deps): update dependency launchdarkly.eventsource to v5.1.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.5.0 (#2703)

* chore(deps): update dependency yamldotnet to v13.5.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.5.1 (#2704)

* chore(deps): update dependency yamldotnet to v13.5.1

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.7.0 (#2705)

* chore(deps): update dependency yamldotnet to v13.7.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update xunit-dotnet monorepo (#2711)

* chore(deps): update xunit-dotnet monorepo

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore: Breaking down FilePattern (#2681)

* Simplifying DiffIgnoreChangesInput

* Update FilePattern.cs

* Up

* up

* Rename the class

* up

---------

Co-authored-by: Rouke Broersma <[email protected]>

* chore: Splitting some C# related extensions in a separate file (#2697)

Splitting some C# related extensions in a separate file

Co-authored-by: Rouke Broersma <[email protected]>

* fix: tests for `OpenReportEnabledInput` help text

* fix: typo in re(c)reate

* add: tests for baseline target input.

change: target input default to 'master' to stay consistent with since target input.

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Renovatebot <[email protected]>
Co-authored-by: Rouke Broersma <[email protected]>
Co-authored-by: Pent Ploompuu <[email protected]>
Co-authored-by: Petr <[email protected]>
Co-authored-by: Rouke Broersma <[email protected]>
Co-authored-by: Jack Steel <[email protected]>
Co-authored-by: Cyrille DUPUYDAUBY <[email protected]>
Co-authored-by: Max S <[email protected]>
* chore(deps): update dependency buildalyzer to v5.0.1 (#2665)

* chore(deps): update dependency buildalyzer to v5.0.1

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore: fix page links for site

* fix: Don't fail the test run if restoring test assemblies fails (#2664)

* chore(deps): update xunit-dotnet monorepo to v2.5.1 (#2673)

* chore(deps): update xunit-dotnet monorepo to v2.5.1

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.4.0 (#2677)

* chore(deps): update dependency yamldotnet to v13.4.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore: F# range math (#2679)

* F# range math

* Windows x Linux

* Up

* chore: Passing Stryker options to F# project components (#2680)

Co-authored-by: Rouke Broersma <[email protected]>

* fix: Use solution file when it's available (#2690)

* Use solution file when it's available

* fix unit test

* sonarqube

* chore(compilation): Improve Error and Trace Logging (#2688)

Add logs

Co-authored-by: Rouke Broersma <[email protected]>

* fix: Remove usings from MutantControl (#2694)

chore: remove usings from MutantControl

Co-authored-by: Rouke Broersma <[email protected]>

* chore: Code quality improvements (#2696)

* Use string builder in loop

* Use find & handle other platform newlines

* Use find

* Fix duplicate message

* Add more range tests

* Improve range code

* docs: add SAS configuration (#2676) (#2701)

docs: add SAS configuration

* chore(deps): update dependency launchdarkly.eventsource to v5.1.0 (#2702)

* chore(deps): update dependency launchdarkly.eventsource to v5.1.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.5.0 (#2703)

* chore(deps): update dependency yamldotnet to v13.5.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.5.1 (#2704)

* chore(deps): update dependency yamldotnet to v13.5.1

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update dependency yamldotnet to v13.7.0 (#2705)

* chore(deps): update dependency yamldotnet to v13.7.0

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore(deps): update xunit-dotnet monorepo (#2711)

* chore(deps): update xunit-dotnet monorepo

* Update package lock files

---------

Co-authored-by: Renovatebot <[email protected]>

* chore: Breaking down FilePattern (#2681)

* Simplifying DiffIgnoreChangesInput

* Update FilePattern.cs

* Up

* up

* Rename the class

* up

---------

Co-authored-by: Rouke Broersma <[email protected]>

* chore: Splitting some C# related extensions in a separate file (#2697)

Splitting some C# related extensions in a separate file

Co-authored-by: Rouke Broersma <[email protected]>

* fix: tests for `OpenReportEnabledInput` help text

* fix: typo in re(c)reate

* add: tests for baseline target input.

change: target input default to 'master' to stay consistent with since target input.

* fix: cli info text test

* also check if WithBaselineInput is true

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Renovatebot <[email protected]>
Co-authored-by: Rouke Broersma <[email protected]>
Co-authored-by: Pent Ploompuu <[email protected]>
Co-authored-by: Petr <[email protected]>
Co-authored-by: Rouke Broersma <[email protected]>
Co-authored-by: Jack Steel <[email protected]>
Co-authored-by: Cyrille DUPUYDAUBY <[email protected]>
Co-authored-by: Max S <[email protected]>
@sonarcloud
Copy link

sonarcloud bot commented Oct 13, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

93.3% 93.3% Coverage
0.0% 0.0% Duplication

danihengeveld and others added 2 commits October 14, 2023 14:49
* fix: tests for `OpenReportEnabledInput` help text

* fix: typo in re(c)reate

* add: tests for baseline target input.

change: target input default to 'master' to stay consistent with since target input.

* fix: cli info text test

* also check if WithBaselineInput is true

* re-add all options to baseline and recreate commands
/// <summary>
/// When true, a new baseline should be created.
/// </summary>
public bool BaselineRecreateEnabled { get; init; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public bool BaselineRecreateEnabled { get; init; }
public bool RecreateBaseline { get; init; }

Copy link

sonarcloud bot commented Mar 15, 2024

@richardwerkman richardwerkman marked this pull request as ready for review March 15, 2024 10:25

Enabling `with-baseline` saves the mutation report to a storage location such as the filesystem. The mutation report is loaded at the start of the next mutation run. Any changed source code or unit test results in a reset of the mutants affected by the change. For unchanged mutants the previous result is reused. This feature expands on the [since](#since-flag-committish) feature by providing you with a full report after a partial mutation testrun.
Enabling `baseline`, alias `with-baseline`, saves the mutation report to a storage location such as the filesystem. The mutation report is loaded at the start of the next mutation run. Any changed source code or unit test results in a reset of the mutants affected by the change. For unchanged mutants the previous result is reused. This feature expands on the [since](#since-flag-committish) feature by providing you with a full report after a partial mutation testrun. The target of the baseline is a `committish` passed as an extra argument. The target determines which committish is used to establish or locate a baseline.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does not still need to be a committish? We are trying to move away from git for baseline.

@@ -492,33 +492,26 @@ Use [globbing syntax](https://en.wikipedia.org/wiki/Glob_(programming)) for wild

## Baseline

### `baseline` &lt;`command`&gt;
### `with-baseline` &lt;`committish`&gt;
Copy link
Member

@rouke-broersma rouke-broersma Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### `with-baseline` &lt;`committish`&gt;
### `baseline` &lt;`target-version`&gt;


When [with-baseline](#with-baseline-flag-committish) is enabled and Stryker cannot find an existing report for the current branch the fallback version is used. When Stryker is still unable to find a baseline we will do a complete instead of partial testrun. The complete testrun will then be saved as the new baseline for the next mutation testrun.
When [with-baseline](#with-baseline-committish) is enabled and Stryker cannot find an existing report for the current branch the fallback version is used. When Stryker is still unable to find a baseline we will do a complete instead of partial testrun. The complete testrun will then be saved as the new baseline for the next mutation testrun.
Copy link
Member

@rouke-broersma rouke-broersma Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When [with-baseline](#with-baseline-committish) is enabled and Stryker cannot find an existing report for the current branch the fallback version is used. When Stryker is still unable to find a baseline we will do a complete instead of partial testrun. The complete testrun will then be saved as the new baseline for the next mutation testrun.
When [baseline](#baseline-target-version) is enabled and Stryker cannot find an existing report for the current version the fallback version is used. When Stryker is still unable to find a baseline with either the current version or the fallback version a complete testrun is performed. The complete testrun will then be saved as the new baseline for the next mutation testrun.

Comment on lines 497 to 499
Default: `false`
Command line: `with-baseline feat-2`
Config file: `"baseline": { }`
Copy link
Member

@rouke-broersma rouke-broersma Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Default: `false`
Command line: `with-baseline feat-2`
Config file: `"baseline": { }`
Default: `false`
Command: `baseline feat-2`
Command Alias: `with-baseline feat-2`
Config file: `"baseline": { }`

RegisterCliInputs(baselineCmd);

var committishArg = baselineCmd.Argument("committish", "The committish to compare with the current HEAD. This can be a branch, tag or commit id.");
committishArg.IsRequired(true);
Copy link
Member

@rouke-broersma rouke-broersma Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we not using a default target version anymore?

Copy link

sonarcloud bot commented Jun 21, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants