kmeans of scikit-learn crash when using julia MKL first on AMD archlinux

[songjhaha]

we create a julia-wrapped python package kmeans as in https://github.com/Suzhou-Tongyuan/jnumpy/tree/main/demo/kmeans, and a segfault occurs when testing on a 64-bit archlinux using the AMD architecture:

The following code does not work:

import jnumpy as np
np.init_jl()
data = np.random.rand(100, 8)
from kmeans import jl_kmeans
jl_kmeans(data, 3)

from sklearn.cluster import KMeans
clf = KMeans(n_clusters=3)
clf.fit(data)

While the following code works:

from sklearn.cluster._kmeans import k_means

import jnumpy as np
np.init_jl()
data = np.random.rand(100, 8)
from kmeans import jl_kmeans
jl_kmeans(data, 3)

from sklearn.cluster import KMeans
clf = KMeans(n_clusters=3)
clf.fit(data)

We have addressed the issue. Removing using MKL from fast_kmeans.jl works the code, or importing the cython extension sklearn.cluster._kmeans_common at first in Python also works.

[songjhaha]

error message:

  signal (11): Segmentation fault
  in expression starting at none:0
  __kmp_env_blk_init at /home/songjhaha/.julia/artifacts/947793e42b663bacd09f00d96aa96a47095f3b1c/lib/libiomp5.so (unknown line)
  _Z20__kmp_env_initializePKc at /home/songjhaha/.julia/artifacts/947793e42b663bacd09f00d96aa96a47095f3b1c/lib/libiomp5.so (unknown line)
  __kmp_middle_initialize at /home/songjhaha/.julia/artifacts/947793e42b663bacd09f00d96aa96a47095f3b1c/lib/libiomp5.so (unknown line)
  omp_get_max_threads at /home/songjhaha/.julia/artifacts/947793e42b663bacd09f00d96aa96a47095f3b1c/lib/libiomp5.so (unknown line)
  __pyx_f_7sklearn_5utils_15_openmp_helpers__openmp_effective_n_threads at /project/sklearn/utils/_openmp_helpers.c:1482 [inlined]
  __pyx_pf_7sklearn_5utils_15_openmp_helpers_2_openmp_effective_n_threads at /project/sklearn/utils/_openmp_helpers.c:1702 [inlined]
...

versions of julia, scikit-learn and MKL.jl:
julia: 1.7.3
scikit-learn: 1.0.2
MKL.jl: 0.5.0