Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testpr: use rattler-build. #8

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

testpr: use rattler-build. #8

wants to merge 4 commits into from

Conversation

oursland
Copy link
Contributor

No description provided.

@oursland oursland marked this pull request as ready for review September 15, 2024 08:08
@oursland oursland force-pushed the testpr-rattler-build branch 4 times, most recently from d1ae4fe to 5e48334 Compare September 15, 2024 08:50
@oursland oursland force-pushed the testpr-rattler-build branch 5 times, most recently from 0a03ed6 to 885995c Compare September 15, 2024 19:58
@oursland oursland mentioned this pull request Sep 15, 2024
Open
@oursland oursland force-pushed the testpr-rattler-build branch 4 times, most recently from e17da02 to a83609a Compare September 15, 2024 20:18
@oursland oursland force-pushed the testpr-rattler-build branch 2 times, most recently from eade083 to a471e36 Compare September 15, 2024 20:27
@Tobias-Fischer
Copy link
Contributor

Hi @wolfv @baszalmstra - some exciting stuff happening here :).

There are some issues in switching from boa to pixi/rattler-build. It seems like the created package for some reason contains the whole environment, and .git folder etc. - how do we avoid this? The CI runner is also running out of space super quickly, any idea why?

@oursland
Copy link
Contributor Author

oursland commented Sep 15, 2024 •
edited
Loading

I believe this is because the current uploaded packages are non-existent. linux_64 correctly identifies the packages to be built are already built and skips them, completing successfully. macos_arm64 finds many packages, but tries to build the others, and the other platforms have no uploaded artifacts.

@Tobias-Fischer
Copy link
Contributor

I am happy with the list of package that need to be built, but not happy with the contents of the packages :)

@traversaro
Copy link
Member

There are some issues in switching from boa to pixi/rattler-build. It seems like the created package for some reason contains the whole environment, and .git folder etc. - how do we avoid this? The CI runner is also running out of space super quickly, any idea why?

I experienced this as well! The issue is prefix-dev/rattler-build#981 . Quoting @wolfv :

You can easily work around this by building outside the recipe folder or by setting a different output directory explicitly.

@traversaro traversaro mentioned this pull request Sep 17, 2024
Open
@traversaro
Copy link
Member

traversaro commented Sep 17, 2024 •
edited
Loading

I attempted a fix in oursland#2, not sure if it works.

oursland added a commit to oursland/ros-jazzy that referenced this pull request Sep 20, 2024
@oursland
WIP: rclpy interaction crashes.
8466bcd 
Current processes:

rebuild:
  reset ; rm -rf output/bld && pixi run generate-recipes && time pixi run -v rattler-build build --recipe recipes/ros-jazzy-rosidl-generator-py --recipe recipes/ros-jazzy-rclpy --recipe recipes/ros-jazzy-rcl-interfaces -m conda_build_config.yaml -c robostack-jazzy -c conda-forge --keep-build

test:
  mamba create -n ros-jazzy -c conda-forge -c file:///Users/jso/code/ros-jazzy/output python=3.11\* ros-jazzy-rclpy
  mamba run --live-stream -n ros-jazzy lldb python -- -c 'import rclpy; rclpy.init(); node = rclpy.create_node("test"); rclpy.spin(node)'

  Then run with the 'r' command:

The second line will run a simple python script that exhibits the crash.  The log looks like this:

  (lldb) target create "python"
  Current executable set to '/Users/jso/code/FreeCAD/FreeCAD/.conda/ros-jazzy/bin/python' (arm64).
  (lldb) settings set -- target.run-args  "-c" "import rclpy; rclpy.init(); node = rclpy.create_node(\"test\"); rclpy.spin(node)"
  (lldb) r
  Process 78922 launched: '/Users/jso/code/FreeCAD/FreeCAD/.conda/ros-jazzy/bin/python' (arm64)
  Process 78922 stopped
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x60)
      frame #0: 0x00000001027d6314 libpython3.11.dylib`set_attribute_error_context + 60
  libpython3.11.dylib`set_attribute_error_context:
  ->  0x1027d6314 <+60>: ldr    x0, [x8, #0x60]
      0x1027d6318 <+64>: bl     0x1028a0b0c    ; PyErr_GivenExceptionMatches
      0x1027d631c <+68>: cbz    w0, 0x1027d6384 ; <+172>
      0x1027d6320 <+72>: ldr    x8, [x21, #0x358]
  Target 0: (python) stopped.
  (lldb) bt
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x60)
    * frame #0: 0x00000001027d6314 libpython3.11.dylib`set_attribute_error_context + 60
      frame #1: 0x00000001027d68f0 libpython3.11.dylib`PyObject_GetAttr + 192
      frame #2: 0x00000001027d67e0 libpython3.11.dylib`PyObject_GetAttrString + 108
      frame RoboStack#3: 0x0000000101a87b04 libbuiltin_interfaces__rosidl_generator_py.dylib`builtin_interfaces__msg__time__convert_from_py + 60
      frame RoboStack#4: 0x000000010219c0a8 librcl_interfaces__rosidl_generator_py.dylib`rcl_interfaces__msg__parameter_event__convert_from_py + 356
      frame RoboStack#5: 0x0000000100e48250 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol2190 + 360
      frame RoboStack#6: 0x0000000100e2b844 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol1795 + 76
      frame RoboStack#7: 0x0000000100e2c1a0 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol1802 + 208
      frame RoboStack#8: 0x0000000100dc17c0 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol708 + 4556
      frame RoboStack#9: 0x00000001000b7f4c python`cfunction_call + 124
      frame RoboStack#10: 0x00000001000606f0 python`_PyObject_MakeTpCall + 332
      frame RoboStack#11: 0x0000000100162e3c python`_PyEval_EvalFrameDefault + 46484
      frame #12: 0x00000001001674c8 python`_PyEval_Vector + 184
      frame #13: 0x00000001000608b8 python`_PyObject_FastCallDictTstate + 156
      frame #14: 0x00000001000617f0 python`_PyObject_Call_Prepend + 176
      frame #15: 0x00000001000dc888 python`slot_tp_init + 196
      frame #16: 0x00000001000d4de4 python`type_call + 464
      frame #17: 0x00000001000606f0 python`_PyObject_MakeTpCall + 332
      frame #18: 0x0000000100162e3c python`_PyEval_EvalFrameDefault + 46484
      frame #19: 0x00000001001674c8 python`_PyEval_Vector + 184
      frame #20: 0x00000001000608b8 python`_PyObject_FastCallDictTstate + 156
      frame #21: 0x00000001000617f0 python`_PyObject_Call_Prepend + 176
      frame #22: 0x00000001000dc888 python`slot_tp_init + 196
      frame #23: 0x00000001000d4de4 python`type_call + 464
      frame #24: 0x00000001000606f0 python`_PyObject_MakeTpCall + 332
      frame #25: 0x0000000100162e3c python`_PyEval_EvalFrameDefault + 46484
      frame #26: 0x00000001001568f4 python`PyEval_EvalCode + 220
      frame #27: 0x00000001001bc970 python`run_mod + 144
      frame #28: 0x00000001001c04a4 python`PyRun_SimpleStringFlags + 272
      frame #29: 0x00000001001e1b3c python`Py_RunMain + 1396
      frame #30: 0x00000001001e3050 python`pymain_main + 1252
      frame #31: 0x0000000100003398 python`main + 56
      frame #32: 0x00000001921f0274 dyld`start + 2840
@Tobias-Fischer
Copy link
Contributor

@wolfv - I might be blind but where do I see which package is being built? I wonder if it still builds in the correct order or ignores dependencies.

@wolfv
Copy link
Member

wolfv commented Sep 21, 2024

@Tobias-Fischer I see some builds in osx-64. They should be in the right order ... :)

@Tobias-Fischer
Copy link
Contributor

Is it possible to see the order; and/or see the package that is being built currently @wolfv?

@wolfv
Copy link
Member

wolfv commented Sep 21, 2024

It prints these things in the right order. I think one problem is that we should associate the source with each output. Otherwise each output will refetch all sources again. I thought I had already changed that in vinca but apparently not. The other option would be to generate the multiple recipes and use the --recipe-dir option.

Screenshot 2024-09-21 at 15 00 37 Screenshot 2024-09-21 at 15 00 12

@oursland
Copy link
Contributor Author

I prefer the multiple recipes option for testpr, but believe that any issues preventing proper operation of the single, large recipe also be fixed.

I'll make the changes to this PR for multiple recipes.

@wolfv
Copy link
Member

wolfv commented Sep 21, 2024

I think the issue was that we skipped recipe generation for packages already on robostack-jazzy but didn't include the channel when building with rattler-build. Let's see if it's fixed now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@oursland @Tobias-Fischer @traversaro @wolfv