Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential benchmark flaws #49

Open
LJNeon opened this issue Jun 14, 2021 · 0 comments
Open

Potential benchmark flaws #49

LJNeon opened this issue Jun 14, 2021 · 0 comments

Comments

@LJNeon
Copy link

LJNeon commented Jun 14, 2021

I've noticed two potential flaws with the benckmarks used. I ran all benchmarks on the LTS version of node (v14.17.0) and used import instead of require. I also included the colorette package from #44.

  1. The load time of all libraries is essentially the same, but the library that's placed first will be significantly slower. An example where kleur was moved to the top is below.
# Load Times
  kleur         x 65μs
  kleur/colors  x 4μs
  chalk         x 4μs
  ansi-colors   x 4μs
  colorette     x 9μs

Also, it appears that console.time() has significant overhead. When I used process.hrtime.bigint() instead, I got much faster results:

# Load Times
  kleur         x 5.4μs
  kleur/colors  x 0.7μs
  chalk         x 0.9μs
  ansi-colors   x 0.8μs
  colorette     x 1μs
  1. When you import the entire module, both kleurs/colors and colorette perform much worse in the first test.
# All Colors
  ansi-colors   x 166,182 ops/sec ±0.96% (90 runs sampled)
  chalk         x 710,172 ops/sec ±2.34% (83 runs sampled)
  kleur         x 857,874 ops/sec ±2.88% (85 runs sampled)
  kleur/colors  x 251,574 ops/sec ±0.77% (85 runs sampled)
  colorette     x 234,007 ops/sec ±0.80% (89 runs sampled)

I believe this is due to the import * as colors from "lib"; syntax, but that's just a guess. I have no time nor expertise to look further into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant