Skip to content

Commit

Permalink
wip: fixes and new convenience options
Browse files Browse the repository at this point in the history
  • Loading branch information
noomorph committed Sep 29, 2023
1 parent d554ff0 commit 617b232
Show file tree
Hide file tree
Showing 20 changed files with 466 additions and 85 deletions.
28 changes: 17 additions & 11 deletions docs/docs/config/01-grouping/01-by-suite.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ This is perhaps the most common way to group test results, and it makes the most

![Grouping by suite](../../../img/screenshots/config-01-grouping-01.jpg)

The suite hierarchy consists of up to four levels: **parent suite**, **suite**, **subsuite**, and **test case**.
The suite hierarchy consists of up to four levels: **parent suite**, **suite**, **sub-suite** and **test case**.

:::info
:::info Glossary

<dl>
<dt><strong>Parent Suite</strong></dt>
Expand All @@ -42,10 +42,10 @@ Below we'll explore a few examples of how to configure the grouping by suite.

## Default preset

By default, `jest-allure2-reporter` provides 3 levels of grouping: **suite**, **subsuite**, and **test case**:
By default, `jest-allure2-reporter` provides 3 levels of grouping: **suite**, **sub-suite**, and **test case**:

1. The **suite** level is based on the _test file path_.
1. The **subsuite** level is based on the _top-level describe block_.
1. The **sub-suite** level is based on the _top-level describe block_.
1. The **test case** level is based on the _test name_ (including the inner describe block names).

<Tabs groupId="configTab">
Expand Down Expand Up @@ -136,13 +136,19 @@ module.exports = {
testEnvironment: 'jest-allure2-reporter/environment-node',
reporters: [
'default',
['jest-allure2-reporter', /** @type {import('jest-allure2-reporter').Options}*/
['jest-allure2-reporter',
/** @type {import('jest-allure2-reporter').ReporterOptions}*/
{
labels: {
parentSuite: ({ file }) => file.pathSegments[0],
suite: ({ file }) => file.pathSegments.slice(1, -1).join(' '),
subsuite: ({ file }) => file.pathSegments.slice(-1)[0],
test: ({ test }) => test.fullName,
testCase: {
name: ({ testCase }) => [
...testCase.ancestorTitles,
testCase.title
].join(' » '),
labels: {
parentSuite: ({ filePath }) => file.pathSegments[0],
suite: ({ filePath }) => file.pathSegments[1],
subSuite: ({ filePath }) => file.pathSegments.slice(2).join('/'),
},
},
},
],
Expand Down Expand Up @@ -200,7 +206,7 @@ module.exports = {
labels: {
parentSuite: ({ file }) => file.path,
suite: ({ test }) => test.ancestorTitles[0],
subsuite: ({ test }) => test.ancestorTitles.slice(1).join(' ') || undefined,
subSuite: ({ test }) => test.ancestorTitles.slice(1).join(' ') || undefined,
test: ({ test }) => test.title,
},
}],
Expand Down
125 changes: 109 additions & 16 deletions docs/docs/config/01-grouping/02-by-story.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,22 +14,46 @@ anywhere. Please use GitHub docs for the latest stable version.

:::


This grouping option is a concept that comes from the [Behavior-Driven Development](https://en.wikipedia.org/wiki/Behavior-driven_development) (BDD) methodology.
Unlike the [suite-based grouping](01-by-suite.mdx), which is based on the technical structure of your test suite, the story-based grouping
helps you to focus on the business value of your tests and view them from the end-user perspective.

![Grouping by story](../../../img/screenshots/config-01-grouping-05.jpg)

This grouping option comes from the [Behavior-Driven Development](https://en.wikipedia.org/wiki/Behavior-driven_development) (BDD) methodology and
allows users to group test results based on the **epic**, **feature** and **story** to which each test case belongs, where:
The story-oriented hierarchy has 4 mandatory levels: **epic**, **feature**, **story** and **test case**.

:::info Glossary

* **epic** is a high-level business goal.
* **feature** is a functionality that delivers business value.
* **story** is a user story that describes a feature from the end-user perspective.
<dl>
<dt><strong>Epic</strong></dt>
<dd>High-level business goal</dd>

This grouping is not enabled by default. Moreover, you need to decide how exactly you want to enable it: via [configuration](#configuration-api) or [annotations](#annotations-api).
<dt><strong>Feature</strong></dt>
<dd>Functionality that delivers business value</dd>

<dt><strong>Story</strong></dt>
<dd>User story that describes a feature from the end-user perspective</dd>

<dt><strong>Test Case</strong></dt>
<dd>Atomic, lowest-level unit. In Jest, it is a single <code>it</code> or <code>test</code> function.</dd>
</dl>

:::

Before you start using this grouping option, you need to decide how exactly you want to implement it:

* via [annotations](#annotations-api) – a more granular approach, which allows you to control the grouping on a per-test basis;
* via [configuration](#configuration-api) – a quick option to enable it all at once, based on general rules;
* via _mixing these approaches_ – a compromise between the two, where the configuration serves as a fallback for missing annotations.

## Annotations API

The [annotation-based approach](../../api/08-labels.mdx) gives you a fine-grained control over the names of your Epic, Feature and Story labels, but it requires you to add annotations to your test cases.
The [annotation-based approach](../../api/08-labels.mdx) gives you a fine-grained control over the names of your epic, feature and story labels, but it requires you to add annotations to _every and each test case_ (sic!) which can be tedious.

In the previous example, it would make sense to group both client and server tests under the same features like **Login screen** and **Forgot password screen**, whereas the epic would be **Authentication**.
Let's take the same project as in the previous article, where there are two parts: client and server.
Both them deal with the same functionality – authentication and restoring forgotten passwords.
Hence, it would make sense to group both client and server tests under the same epic named **Authentication**, and continue grouping them by features and stories regardless of the application layer.

<Tabs groupId="configTab">
<TabItem value="demo" label="Report">
Expand Down Expand Up @@ -78,18 +102,47 @@ describe('Login controller', () => {
</TabItem>
</Tabs>

As mentioned before, the annotation-based approach requires you to annotate literally **every test case** with all
the three labels (epic, feature and story), otherwise the report will be stubbornly displaying a flat structure in **Behaviors** section.

:::tip

To relax the requirement to annotate all your test cases, you can add a fallback via configuration, e.g.:

```js title="jest.config.js"
/** @type {import('@jest/types').Config.InitialOptions} */
module.exports = {
testEnvironment: 'jest-allure2-reporter/environment-node',
reporters: [
'default',
['jest-allure2-reporter', /** @type {import('jest-allure2-reporter').Options}*/ {
labels: {
epic: ({ value }) => value ?? 'Uncategorized',
feature: ({ value }) => value ?? 'Untitled feature',
story: ({ value }) => value ?? 'Untitled story',
},
}],
],
};
```
:::
## Configuration API
The **configuration-based approach** allows you to group test cases based on the available attributes like file path, ancestor describe blocks, test name and so on.
The **configuration-based approach** allows you to group test cases based on the available attributes like the test file path, the ancestor describe blocks and any
other contextually available information.
It's a good option if you don't want to add annotations to your test cases by hand, but it's less flexible than the annotation-based approach. Still, it might be useful if your grouping by suite focuses mostly [on the file structure](#file-oriented-example), and you want to add "a fresh perspective" by grouping tests by describe blocks and test names, for example.
It is much faster to implement than if you were to annotate every test case by hand, but it is also less flexible.
Still, there are many cases where it can be useful, especially if you have a large test suite and you want to add some structure to it.
For example, if your grouping by suite focuses mostly [on the file structure](01-by-suite.mdx#file-oriented-example),
the story-based grouping may add "a fresh perspective" by grouping tests by describe blocks and test names, for example.
Here's a simple example where we map:
Let's explore a simple example, where we'll map:
* **epic** to the top-level describe block
* **feature** to the second-level describe block
* **story** to the remaining describe blocks and test name itself
* **feature** to the middle-level describe blocks
* **story** to the lowest-level describe block
<Tabs groupId="configTab">
<TabItem value="demo" label="Report">
Expand Down Expand Up @@ -130,9 +183,9 @@ module.exports = {
'default',
['jest-allure2-reporter', /** @type {import('jest-allure2-reporter').Options}*/ {
labels: {
epic: ({ test }) => test.ancestorTitles[0],
feature: ({ test }) => test.ancestorTitles.slice(1).join(' ') || undefined,
story: ({ test }) => test.title,
epic: ({ testCase }) => testCase.ancestorTitles.at(0) ?? '(uncategorized)',
feature: ({ testCase }) => testCase.ancestorTitles.slice(1, -1).join(' > ') ?? '(uncategorized)',
story: ({ testCase }) => testCase.ancestorTitles.at(-1) ?? '(uncategorized)',
},
}],
],
Expand All @@ -142,3 +195,43 @@ module.exports = {
</TabItem>
</Tabs>
## Many-to-many mapping
It is worth mentioning that Allure allows you to map a test case to multiple epics, features and stories, but
you should use this feature with caution, as it may lead to a very complex report structure.
<Tabs groupId="approach">
<TabItem value="jsdoc" label="JSDoc">
```js title="login.test.js"
it('should validate e-mail', () => {
/**
* @epic Authentication
* @feature Login screen
* @story Validation
*
* @epic Security
* @feature XSS prevention
* @story Login form
*/

// ...
});
```
</TabItem>
<TabItem value="dsl" label="Function">
```js title="login.test.js"
$Epic('Authentication');
$Feature('Login screen');
$Story('Validation');
$Epic('Security');
$Feature('XSS prevention');
$Story('Login form');
it('should validate e-mail', () => {
// ...
});
```
</TabItem>
</Tabs>
39 changes: 37 additions & 2 deletions docs/docs/config/01-grouping/03-by-package.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ It strictly follows `com.example.package.ClassName` naming convention, where:
* `com.example.package.ClassName` is a **test class**,
* `shouldAssertAndDoSomething` is a **test method**.

It doesn't work well with JavaScript, and, that's why you can use only two grouping levels: **package** and **test method**.
It doesn't map well to JavaScript, hence for the most time you'll be able to utilize
only two grouping levels: **package** and **test method**. [^1]

A couple of feasible options are:

Expand Down Expand Up @@ -64,7 +65,7 @@ module.exports = {
'default',
['jest-allure2-reporter', /** @type {import('jest-allure2-reporter').Options}*/ {
labels: {
package: ({ package }) => package.name,
package: ({ manifest }) => manifest.name,
// NOTE: `testClass` won't work due to the aforementioned issue
testClass: ({ file }) => file.path,
testMethod: ({ test }) => test.fullName,
Expand All @@ -77,3 +78,37 @@ module.exports = {
</TabItem>
</Tabs>

### Achieving three levels

:::info Disclaimer

The example below is simplified and does not handle edge cases like folder names with spaces, and other non-alphanumeric characters.

:::

So, especially curious souls may try this hacky configuration to get all three levels,
but it's an open question whether it's worth the effort:

```js title="jest.config.js"
/** @type {import('@jest/types').Config.InitialOptions} */
module.exports = {
testEnvironment: 'jest-allure2-reporter/environment-node',
reporters: [
'default',
['jest-allure2-reporter',
/** @type {import('jest-allure2-reporter').Options}*/
{
labels: {
package: ({ filePath }) => filePath.slice(0, -1).join('.'),
testClass: ({ filePath }) => filePath.join('.').replace(/\.test\.[jt]s$/, ''),
testMethod: ({ testCase }) => testCase.fullName,
},
}],
],
};
```

This example is a proof of concept to help you understand better how this grouping strategy was supposed to work in the first place.
It demonstrates that if you map file paths like `src/components/MyComponent.test.js` to pseudo-classes like `src.components.MyComponent`,
the generated report will recognize these labels and group tests accordingly.

34 changes: 24 additions & 10 deletions packages/e2e/configs/default.js
Original file line number Diff line number Diff line change
@@ -1,18 +1,32 @@
// eslint-disable-next-line node/no-extraneous-require,@typescript-eslint/no-var-requires,import/no-extraneous-dependencies
const _ = require('lodash');
const PRESET = process.env.ALLURE_PRESET ?? 'default';

/** @type {import('jest-allure2-reporter').ReporterOptions} */
const jestAllure2ReporterOptions = {
resultsDir: `allure-results/${PRESET}`,
testCase: {
name: ({ testCase }) =>
[...testCase.ancestorTitles, testCase.title].join(' » '),
labels: {
parentSuite: ({ filePath }) => filePath[0],
suite: ({ filePath }) => filePath.slice(1, 2).join('/'),
subSuite: ({ filePath }) => filePath.slice(2).join('/'),
epic: ({ value }) => value ?? 'Uncategorized',
story: ({ value }) => value ?? 'Untitled story',
feature: ({ value }) => value ?? 'Untitled feature',
package: ({ filePath }) => filePath.slice(0, -1).join('.'),
testClass: ({ filePath }) => filePath.join('.').replace(/\.test\.[jt]s$/, ''),
testMethod: ({ testCase }) => testCase.fullName,
},
},
};

/** @type {import('@jest/types').Config.InitialOptions} */
module.exports = {
// eslint-disable-next-line node/no-unpublished-require,import/no-extraneous-dependencies
...require('@wix/jest-config-jest-allure2-reporter'),

rootDir: './src/programmatic/grouping',
testEnvironment: 'jest-allure2-reporter/dist/environment/node',
reporters: [
'default',
[
'jest-allure2-reporter',
/** @type {Partial<import('jest-allure2-reporter').ReporterOptions>} */ {
resultsDir: `allure-results/${PRESET}`,
},
],
],
reporters: ['default', ['jest-allure2-reporter', jestAllure2ReporterOptions]],
};
5 changes: 4 additions & 1 deletion packages/e2e/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
},
"prettier": "@wix/prettier-config-jest-allure2-reporter",
"eslintConfig": {
"extends": "@wix/jest-allure2-reporter"
"extends": "@wix/jest-allure2-reporter",
"env": {
"jest": true
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import { $Epic, $Feature, $Story, $Tag } from 'jest-allure2-reporter';

$Tag('client');
$Epic('Authentication');
$Feature('Restore account');
describe('Forgot password screen', () => {
$Story('Happy path');
describe('Rendering', () => {
it('should render the forgot password form', () => {
// ...
});
});

describe('Form Submission', () => {
$Story('Validation');
it('should show error on invalid e-mail format', () => {
// ...
});

$Story('Happy path');
it('should send forgot password request on valid e-mail', () => {
// ...
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import { $Epic, $Feature, $Story, $Tag } from 'jest-allure2-reporter';

$Tag('client');
$Epic('Authentication');
$Feature('Login');
describe('Login screen', () => {
$Story('Happy path');
describe('Rendering', () => {
it('should render the login form by default', () => {
// ...
});
});

$Story('Validation');
describe('Form Submission', () => {
it('should show error on invalid e-mail format', () => {
// ...
});

it('should show error on short or invalid password format', () => {
// ...
});
});

describe('Navigation', () => {
$Feature('Restore account');
$Story('Happy path');
it('should navigate to forgot password screen on "forgot password" click', () => {
// ...
});
});
});
Loading

0 comments on commit 617b232

Please sign in to comment.