Cythonize rainflow counters

[johannes-mueller]

Our current rainflow solutions perform quite poorly as they are written in plain python. In order to meet demand for more performance the sometimes is raised, we should cythonize the modules to see if we can speed them up.

[maierbn]

As first step, we measure runtime of rainflow, meanstress conversion and damage calculation (as used in ViLoG). The scripts is process_benchmark.py, written by @wos2rng.

[maierbn]

Processing rainflow took 0.0748 seconds
Computation of meanstress conversion took 0.1074 seconds
Computation of damage took 0.0068 seconds

on @maierbn 's laptop

[johannes-mueller]

I have written an implementation of the four point rainflow counter in Cython and in Rust. I tested it for a dataset of 1M datapoints with 100k turning points.

Cython doubles the speed, the Rust implementation is 50x faster.

By commenting the lines here you can choose between the two for testing purposes.

In order to install and build it a simple pip install -e . is sufficient provided that a C compiler and a Rust compiler are installed on the system.

@maierbn I put a modified benchmark_rainflow.py next to your benchmark file, which also profiles using pyinstrument (you need to do pip install pyinstrument).

You need to fiddle around with the commented code to first get the load.txt file generated. See the comments in benchmark_rainflow.py, you will get the idea.

Finally some profiling results:

Plain python:

Processing rainflow took 0.8280 seconds

  _     ._   __/__   _ _  _  _ _/_   Recorded: 15:48:14  Samples:  811
 /_//_/// /_\ / //_// / //_'/ //     Duration: 0.828     CPU time: 0.828
/   _/                      v4.6.2

Program: demos/performance/benchmark_rainflow.py

0.828 <module>  benchmark_rainflow.py:1
└─ 0.826 process_rainflow  wrapper.py:6
   └─ 0.820 FourPointDetector.process  pylife/stress/rainflow/fourpoint.py:90
      ├─ 0.752 [self]  pylife/stress/rainflow/fourpoint.py
      ├─ 0.021 len  <built-in>
      ├─ 0.019 list.append  <built-in>
      ├─ 0.017 FourPointDetector._new_turns  pylife/stress/rainflow/general.py:280
      │  └─ 0.015 find_turns  pylife/stress/rainflow/general.py:26
      │     └─ 0.008 clean_nans  pylife/stress/rainflow/general.py:61
      │        └─ 0.008 any  <built-in>
      └─ 0.009 list.pop  <built-in>

Cython

Processing rainflow took 0.4176 seconds

  _     ._   __/__   _ _  _  _ _/_   Recorded: 15:34:08  Samples:  8
 /_//_/// /_\ / //_// / //_'/ //     Duration: 0.418     CPU time: 0.418
/   _/                      v4.6.2

Program: demos/performance/benchmark_rainflow.py

0.418 <module>  benchmark_rainflow.py:1
└─ 0.418 process_rainflow  wrapper.py:6
   └─ 0.416 FourPointDetector.process  pylife/stress/rainflow/fourpoint.py:93
      ├─ 0.406 [self]  pylife/stress/rainflow/fourpoint.py
      └─ 0.009 FourPointDetector._new_turns  pylife/stress/rainflow/general.py:280
         └─ 0.008 find_turns  pylife/stress/rainflow/general.py:26
            └─ 0.004 [self]  pylife/stress/rainflow/general.py

Rust

Processing rainflow took 0.0149 seconds

  _     ._   __/__   _ _  _  _ _/_   Recorded: 15:34:21  Samples:  7
 /_//_/// /_\ / //_// / //_'/ //     Duration: 0.015     CPU time: 0.015
/   _/                      v4.6.2

Program: demos/performance/benchmark_rainflow.py

0.014 <module>  benchmark_rainflow.py:1
└─ 0.014 process_rainflow  wrapper.py:6
   └─ 0.014 FourPointDetector.process  pylife/stress/rainflow/fourpoint.py:93
      ├─ 0.008 FourPointDetector._new_turns  pylife/stress/rainflow/general.py:280
      │  ├─ 0.007 find_turns  pylife/stress/rainflow/general.py:26
      │  │  ├─ 0.005 [self]  pylife/stress/rainflow/general.py
      │  │  └─ 0.002 diff  numpy/lib/function_base.py:1324
      │  └─ 0.001 [self]  pylife/stress/rainflow/general.py
      ├─ 0.004 fourpoint_loop  <built-in>
      └─ 0.001 ndarray.astype  <built-in>