Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different scores at each run in Angular 17 project #4878

Open
rdechiara opened this issue Jun 4, 2024 · 1 comment
Open

Different scores at each run in Angular 17 project #4878

rdechiara opened this issue Jun 4, 2024 · 1 comment
Labels
🐛 Bug Something isn't working

Comments

@rdechiara
Copy link

Summary

Almost each time I run Stryker against my Angular v17 project I get different score without changing the code. I have noticed this issue because after getting a low score for a component I added the needed tests and the score went up. When I run the tests again, without any modification, the score dropped by about 20%.
Stryker config

// @ts-check
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
const config = {
  _comment:
    "This config was generated using 'stryker init'. Please see the guide for more information: https://stryker-mutator.io/docs/stryker-js/guides/angular",
  mutate: [
    "src/**/*.component.ts",
    "src/**/*.service.ts",
    "src/**/*.interceptor.ts",
    "!src/**/*.spec.ts",
    "!src/test.ts",
    "!src/environments/*.ts",
  ],
  testRunner: "karma",
  karma: {
    configFile: "karma.conf.js",
    projectType: "angular-cli",
    config: {
      browsers: ["ChromeHeadless"],
    },
  },
  reporters: ["progress", "clear-text", "html"],
  concurrency: 4,
  concurrency_comment:
    "Recommended to use about half of your available cores when running stryker with angular",
  coverageAnalysis: "perTest",
};
export default config;

Test runner config

Stryker environment

+-- @stryker-mutator/[email protected]
+-- @stryker-mutator/[email protected]

Test runner environment

npx stryker run

Didn't configure karma so it has all defaults.

Your Environment

software version(s)
node 18.19.0
npm npm 10.8
Operating System Windows 11

Add stryker.log

@rdechiara rdechiara added the 🐛 Bug Something isn't working label Jun 4, 2024
@nicojs
Copy link
Member

nicojs commented Aug 6, 2024

Could you provide (screenshots of the) HTML report? I'm curious to see where the differences are coming from.

StrykerJS should be deterministic, but some variables can still change the score. For example, computer use during mutation testing might cause some test runner processes to starve, resulting in timeout mutants (false positive).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants