Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration test improvements #3330

Merged
merged 11 commits into from
Sep 26, 2024
Merged

Conversation

stefanceriu
Copy link
Member

  • use the home screen search field to find the special integration tests room
  • send 2 messages in that room following both the plain and the rich message composers
  • redirect logs to /Users/Shared
  • configure all rust logs to tracing
  • check that the previously sent messages do not appear in the logs
  • allow more time for a photo to be selected
  • allow more time for pickers to load
  • fix problems with the segmented control on the members screen

@stefanceriu stefanceriu added the pr-misc for other changes label Sep 25, 2024
@stefanceriu stefanceriu requested a review from a team as a code owner September 25, 2024 14:50
@stefanceriu stefanceriu requested review from pixlwave and removed request for a team September 25, 2024 14:50
Copy link

github-actions bot commented Sep 25, 2024

Warnings
⚠️ Some of the commits are missing ticket numbers. Please consider squashing all commits that don't have a tracking number.
⚠️ You seem to have made changes to views. Please consider adding screenshots.

Generated by 🚫 Danger Swift against 6ea9dcc

Copy link

codecov bot commented Sep 25, 2024

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
832 4 828 0
View the full list of 3 ❄️ flaky tests
UserSessionFlowCoordinatorTests testStackUnwinding()

Flake rate in main: 4.76% (Passed 20 times, Failed 1 times)

Stack Traces | 0.756s run time
XCTAssertTrue failed (.../UnitTests/Sources/UserSessionFlowCoordinatorTests.swift:117)
PreviewTests test_userProfileScreen()

Flake rate in main: 19.15% (Passed 38 times, Failed 9 times)

Stack Traces | 3.73s run time
failed - Snapshot "Other User" does not match reference.

@−
"file:.../__Snapshots__/PreviewTests/test_userProfileScreen-iPhone-16-pseudo.Other-User.png"
@+
"file:.../tmp/PreviewTests/test_userProfileScreen-iPhone-16-pseudo.Other-User.png"

To configure output for a custom diff tool, use 'withSnapshotTesting'. For example:

    withSnapshotTesting(diffTool: .ksdiff) {
      // ...
    }

The percentage of pixels that match 0.9015503 is less than required 1.0
The lowest perceptual color precision 0.31062502 is less than required 0.98 (.../PreviewTests/Sources/PreviewTests.swift:66)
PreviewTests test_identityConfirmationScreen()

Flake rate in main: 13.04% (Passed 40 times, Failed 6 times)

Stack Traces | 10s run time
failed - Snapshot does not match reference.

@−
"file:.../__Snapshots__/PreviewTests/test_identityConfirmationScreen-iPhone-16-en-GB.1.png"
@+
"file:.../tmp/PreviewTests/test_identityConfirmationScreen-iPhone-16-en-GB.1.png"

To configure output for a custom diff tool, use 'withSnapshotTesting'. For example:

    withSnapshotTesting(diffTool: .ksdiff) {
      // ...
    }

The percentage of pixels that match 0.94485474 is less than required 1.0
The lowest perceptual color precision 0.39937502 is less than required 0.98 (.../PreviewTests/Sources/PreviewTests.swift:66)

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

Copy link
Member

@pixlwave pixlwave left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

ElementX/Sources/Other/Logging/RustTracing.swift Outdated Show resolved Hide resolved
Copy link
Member

@bmarty bmarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding privacy check!

IntegrationTests/Sources/UserFlowTests.swift Show resolved Hide resolved
.github/workflows/integration-tests.yml Show resolved Hide resolved
@stefanceriu
Copy link
Member Author

Going to merge this now. I will follow it up with a PR that disables custom tracing configurations (as we don't control 3rd party crates and don't want to take that risk) and a few other changes.

Copy link

sonarcloud bot commented Sep 26, 2024

@stefanceriu stefanceriu merged commit dc6d7f2 into develop Sep 26, 2024
7 checks passed
@stefanceriu stefanceriu deleted the stefan/integrationTestImprovements branch September 26, 2024 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-misc for other changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants