-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bootstrappable Builds #8929
base: master
Are you sure you want to change the base?
Bootstrappable Builds #8929
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# Workaround for: https://bugs.launchpad.net/ubuntu/+source/guix/+bug/2064115 | ||
|
||
abi <abi/4.0>, | ||
include <tunables/global> | ||
|
||
profile guix /usr/bin/guix flags=(unconfined) { | ||
userns, | ||
|
||
include if exists <local/guix> | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
name: ci/gh-actions/guix | ||
|
||
on: | ||
push: | ||
paths: | ||
- 'contrib/depends/**' | ||
- 'contrib/guix/**' | ||
- '!contrib/**.md' | ||
- '.github/workflows/guix.yml' | ||
pull_request: | ||
paths: | ||
- 'contrib/depends/**' | ||
- 'contrib/guix/**' | ||
- '!contrib/**.md' | ||
- '.github/workflows/guix.yml' | ||
|
||
jobs: | ||
cache-sources: | ||
runs-on: ubuntu-24.04 | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
- name: depends sources cache | ||
id: cache | ||
uses: actions/cache@v4 | ||
with: | ||
path: contrib/depends/sources | ||
key: sources-${{ hashFiles('contrib/depends/packages/*') }} | ||
- name: download depends sources | ||
if: steps.cache.outputs.cache-hit != 'true' | ||
run: make -C contrib/depends download | ||
|
||
build-guix: | ||
runs-on: ubuntu-24.04 | ||
needs: [cache-sources] | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
toolchain: | ||
- target: "x86_64-linux-gnu" | ||
- target: "aarch64-linux-gnu" | ||
- target: "arm-linux-gnueabihf" | ||
- target: "riscv64-linux-gnu" | ||
- target: "i686-linux-gnu" | ||
- target: "x86_64-w64-mingw32" | ||
- target: "i686-w64-mingw32" | ||
- target: "x86_64-unknown-freebsd" | ||
- target: "x86_64-apple-darwin" | ||
- target: "aarch64-apple-darwin" | ||
- target: "aarch64-linux-android" | ||
- target: "arm-linux-androideabi" | ||
|
||
name: ${{ matrix.toolchain.target }} | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
submodules: recursive | ||
- name: remove bundled packages | ||
run: sudo rm -rf /usr/local | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quick question. Do we need this if we are building with GUIX? Since the premise of GUIX is sandboxed builds. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's just to free up disk space. See monero-project/monero-gui#4223 |
||
- name: depends sources cache | ||
uses: actions/cache/restore@v4 | ||
with: | ||
path: contrib/depends/sources | ||
key: sources-${{ hashFiles('contrib/depends/packages/*') }} | ||
- name: install dependencies | ||
run: sudo apt update; sudo apt -y install guix git ca-certificates apparmor-utils | ||
- name: fix apparmor | ||
run: sudo cp .github/workflows/guix /etc/apparmor.d/guix; sudo /etc/init.d/apparmor reload; sudo aa-enforce guix || true | ||
- name: purge apparmor | ||
run: sudo apt purge apparmor | ||
- name: build | ||
run: SUBSTITUTE_URLS='http://bordeaux.guix.gnu.org' HOSTS="${{ matrix.toolchain.target }}" ./contrib/guix/guix-build | ||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: ${{ matrix.toolchain.target }} | ||
path: | | ||
guix/guix-build-*/output/${{ matrix.toolchain.target }}/* | ||
guix/guix-build-*/logs/${{ matrix.toolchain.target }}/* | ||
bundle-logs: | ||
runs-on: ubuntu-24.04 | ||
needs: [build-guix] | ||
steps: | ||
- uses: actions/download-artifact@v4 | ||
with: | ||
merge-multiple: true | ||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: "logs" | ||
path: '**/logs/**' |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -25,10 +25,8 @@ miniupnpcstrings.h | |
version/ | ||
ClangBuildAnalyzerSession.txt | ||
|
||
# gitian | ||
contrib/gitian/builder/ | ||
contrib/gitian/docker/ | ||
contrib/gitian/sigs/ | ||
# guix | ||
/guix | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Finished builds are available in
|
||
|
||
# Created by https://www.gitignore.io | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -85,6 +85,7 @@ include builders/$(build_os).mk | |
include builders/default.mk | ||
include packages/packages.mk | ||
|
||
ifeq ($(GUIX_ENVIRONMENT),) | ||
build_id_string:=$(BUILD_ID_SALT) | ||
build_id_string+=$(shell $(build_CC) --version 2>/dev/null) | ||
build_id_string+=$(shell $(build_AR) --version 2>/dev/null) | ||
|
@@ -98,6 +99,10 @@ $(host_arch)_$(host_os)_id_string+=$(shell $(host_AR) --version 2>/dev/null) | |
$(host_arch)_$(host_os)_id_string+=$(shell $(host_CXX) --version 2>/dev/null) | ||
$(host_arch)_$(host_os)_id_string+=$(shell $(host_RANLIB) --version 2>/dev/null) | ||
$(host_arch)_$(host_os)_id_string+=$(shell $(host_STRIP) --version 2>/dev/null) | ||
else | ||
build_id_string:=$(realpath $(GUIX_ENVIRONMENT)) | ||
$(host_arch)_$(host_os)_id_string:=$(realpath $(GUIX_ENVIRONMENT)) | ||
Comment on lines
+103
to
+104
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any change to |
||
endif | ||
|
||
packages += $($(host_arch)_$(host_os)_packages) $($(host_os)_packages) | ||
native_packages += $($(host_arch)_$(host_os)_native_packages) $($(host_os)_native_packages) | ||
|
@@ -111,7 +116,7 @@ $(host_arch)_$(host_os)_native_toolchain?=$($(host_os)_native_toolchain) | |
include funcs.mk | ||
|
||
toolchain_path=$($($(host_arch)_$(host_os)_native_toolchain)_prefixbin) | ||
final_build_id_long+=$(shell $(build_SHA256SUM) toolchain.cmake.in) | ||
final_build_id_long+=:[sha256sum]:$(shell $(build_SHA256SUM) toolchain.cmake.in) | ||
final_build_id+=$(shell echo -n "$(final_build_id_long)" | $(build_SHA256SUM) | cut -c-$(HASH_LENGTH)) | ||
$(host_prefix)/.stamp_$(final_build_id): $(native_packages) $(packages) | ||
$(AT)rm -rf $(@D) | ||
|
@@ -124,8 +129,8 @@ $(host_prefix)/.stamp_$(final_build_id): $(native_packages) $(packages) | |
$(host_prefix)/share/toolchain.cmake : toolchain.cmake.in $(host_prefix)/.stamp_$(final_build_id) | ||
$(AT)@mkdir -p $(@D) | ||
$(AT)sed -e 's|@HOST@|$(host)|' \ | ||
-e 's|@CC@|$(toolchain_path)$(host_CC)|' \ | ||
-e 's|@CXX@|$(toolchain_path)$(host_CXX)|' \ | ||
-e 's|@CC@|$(host_CC)|' \ | ||
-e 's|@CXX@|$(host_CXX)|' \ | ||
-e 's|@AR@|$(toolchain_path)$(host_AR)|' \ | ||
-e 's|@RANLIB@|$(toolchain_path)$(host_RANLIB)|' \ | ||
-e 's|@NM@|$(toolchain_path)$(host_NM)|' \ | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,18 +1,25 @@ | ||
define int_vars | ||
#Set defaults for vars which may be overridden per-package | ||
$(1)_cc=$($($(1)_type)_CC) | ||
$(1)_cxx=$($($(1)_type)_CXX) | ||
$(1)_objc=$($($(1)_type)_OBJC) | ||
$(1)_objcxx=$($($(1)_type)_OBJCXX) | ||
$(1)_ar=$($($(1)_type)_AR) | ||
$(1)_ranlib=$($($(1)_type)_RANLIB) | ||
$(1)_libtool=$($($(1)_type)_LIBTOOL) | ||
$(1)_nm=$($($(1)_type)_NM) | ||
$(1)_cflags=$($($(1)_type)_CFLAGS) $($($(1)_type)_$(release_type)_CFLAGS) | ||
$(1)_cxxflags=$($($(1)_type)_CXXFLAGS) $($($(1)_type)_$(release_type)_CXXFLAGS) | ||
$(1)_arflags=$($($(1)_type)_ARFLAGS) $($($(1)_type)_$(release_type)_ARFLAGS) | ||
$(1)_ldflags=$($($(1)_type)_LDFLAGS) $($($(1)_type)_$(release_type)_LDFLAGS) -L$($($(1)_type)_prefix)/lib | ||
$(1)_cppflags=$($($(1)_type)_CPPFLAGS) $($($(1)_type)_$(release_type)_CPPFLAGS) -I$($($(1)_type)_prefix)/include | ||
$(1)_cc=$$($$($(1)_type)_CC) | ||
$(1)_cxx=$$($$($(1)_type)_CXX) | ||
$(1)_objc=$$($$($(1)_type)_OBJC) | ||
$(1)_objcxx=$$($$($(1)_type)_OBJCXX) | ||
$(1)_ar=$$($$($(1)_type)_AR) | ||
$(1)_ranlib=$$($$($(1)_type)_RANLIB) | ||
$(1)_libtool=$$($$($(1)_type)_LIBTOOL) | ||
$(1)_nm=$$($$($(1)_type)_NM) | ||
$(1)_cflags=$$($$($(1)_type)_CFLAGS) \ | ||
$$($$($(1)_type)_$$(release_type)_CFLAGS) | ||
$(1)_cxxflags=$$($$($(1)_type)_CXXFLAGS) \ | ||
$$($$($(1)_type)_$$(release_type)_CXXFLAGS) | ||
$(1)_arflags=$$($$($(1)_type)_ARFLAGS) \ | ||
$$($$($(1)_type)_$(release_type)_ARFLAGS) | ||
$(1)_ldflags=$$($$($(1)_type)_LDFLAGS) \ | ||
$$($$($(1)_type)_$$(release_type)_LDFLAGS) \ | ||
-L$$($($(1)_type)_prefix)/lib | ||
$(1)_cppflags=$$($$($(1)_type)_CPPFLAGS) \ | ||
$$($$($(1)_type)_$$(release_type)_CPPFLAGS) \ | ||
-I$$($$($(1)_type)_prefix)/include | ||
Comment on lines
+3
to
+22
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Delay expansion of package variables. |
||
$(1)_recipe_hash:= | ||
endef | ||
|
||
|
@@ -37,6 +44,7 @@ endef | |
|
||
define int_get_build_recipe_hash | ||
$(eval $(1)_all_file_checksums:=$(shell $(build_SHA256SUM) $(meta_depends) packages/$(1).mk $(addprefix $(PATCHES_PATH)/$(1)/,$($(1)_patches)) | cut -d" " -f1)) | ||
final_build_id_long+=:[$(1)_all_file_checksums]$(foreach checksum,$($(1)_all_file_checksums),$(shell echo ":$(checksum)")): | ||
$(eval $(1)_recipe_hash:=$(shell echo -n "$($(1)_all_file_checksums)" | $(build_SHA256SUM) | cut -d" " -f1)) | ||
endef | ||
|
||
|
@@ -46,7 +54,7 @@ $(eval $(1)_all_dependencies:=$(call int_get_all_dependencies,$(1),$($($(1)_type | |
$(foreach dep,$($(1)_all_dependencies),$(eval $(1)_build_id_deps+=$(dep)-$($(dep)_version)-$($(dep)_recipe_hash))) | ||
$(eval $(1)_build_id_long:=$(1)-$($(1)_version)-$($(1)_recipe_hash)-$(release_type) $($(1)_build_id_deps) $($($(1)_type)_id_string)) | ||
$(eval $(1)_build_id:=$(shell echo -n "$($(1)_build_id_long)" | $(build_SHA256SUM) | cut -c-$(HASH_LENGTH))) | ||
final_build_id_long+=$($(package)_build_id_long) | ||
final_build_id_long+=:[recipe]:$(1)-$($(1)_version)-$($(1)_recipe_hash)-$(release_type):[deps]$(foreach dep,$($(1)_build_id_deps),$(shell echo ":$(dep)")):[$($(1)_type)_id]:$($($(1)_type)_id_string): | ||
Comment on lines
-49
to
+57
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Increase verbosity of |
||
|
||
#compute package-specific paths | ||
$(1)_build_subdir?=. | ||
|
@@ -267,4 +275,4 @@ $(foreach package,$(all_packages),$(eval $(call int_config_attach_build_config,$ | |
$(foreach package,$(all_packages),$(eval $(call int_add_cmds,$(package)))) | ||
|
||
#special exception: if a toolchain package exists, all non-native packages depend on it | ||
$(foreach package,$(packages),$(eval $($(package)_unpacked): |$($($(host_arch)_$(host_os)_native_toolchain)_cached) )) | ||
$(foreach package,$(packages),$(eval $($(package)_extracted): |$($($(host_arch)_$(host_os)_native_toolchain)_cached) )) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,23 +1,65 @@ | ||
OSX_MIN_VERSION=10.13 | ||
OSX_SDK_VERSION=11.0 | ||
XCODE_VERSION=12.2 | ||
XCODE_BUILD_ID=12B45b | ||
LD64_VERSION=609 | ||
ifeq (aarch64, $(host_arch)) | ||
CC_target=arm64-apple-$(host_os) | ||
else | ||
CC_target=$(host) | ||
endif | ||
darwin_CC=clang -target $(CC_target) -mmacosx-version-min=$(OSX_MIN_VERSION) --sysroot $(host_prefix)/native/SDK/ -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -mlinker-version=$(LD64_VERSION) -B$(host_prefix)/native/bin/$(host)- | ||
darwin_CXX=clang++ -target $(CC_target) -mmacosx-version-min=$(OSX_MIN_VERSION) --sysroot $(host_prefix)/native/SDK/ -iwithsysroot/usr/include/c++/v1 -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks -mlinker-version=$(LD64_VERSION) -stdlib=libc++ -B$(host_prefix)/native/bin/$(host)- | ||
|
||
OSX_SDK=$(host_prefix)/native/SDK | ||
|
||
darwin_native_toolchain=darwin_sdk native_cctools | ||
|
||
clang_prog=$(shell $(SHELL) $(.SHELLFLAGS) "command -v clang") | ||
clangxx_prog=$(shell $(SHELL) $(.SHELLFLAGS) "command -v clang++") | ||
|
||
# Flag explanations: | ||
# | ||
# -mlinker-version | ||
# | ||
# Ensures that modern linker features are enabled. See here for more | ||
# details: https://github.com/bitcoin/bitcoin/pull/19407. | ||
# | ||
# -B$(build_prefix)/bin | ||
# | ||
# Explicitly point to our binaries (e.g. cctools) so that they are | ||
# ensured to be found and preferred over other possibilities. | ||
# | ||
# -isysroot$(OSX_SDK) -nostdlibinc | ||
# | ||
# Disable default include paths built into the compiler as well as | ||
# those normally included for libc and libc++. The only path that | ||
# remains implicitly is the clang resource dir. | ||
# | ||
# -iwithsysroot / -iframeworkwithsysroot | ||
# | ||
# Adds the desired paths from the SDK | ||
# | ||
|
||
darwin_CC=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \ | ||
-u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \ | ||
-u LIBRARY_PATH \ | ||
$(clang_prog) --target=$(host) -mmacosx-version-min=$(OSX_MIN_VERSION) \ | ||
-B$(build_prefix)/bin -mlinker-version=$(LD64_VERSION) \ | ||
-isysroot$(OSX_SDK) \ | ||
-isysroot$(OSX_SDK) -nostdlibinc \ | ||
-iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks | ||
|
||
darwin_CXX=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \ | ||
-u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \ | ||
-u LIBRARY_PATH \ | ||
$(clangxx_prog) --target=$(host) -mmacosx-version-min=$(OSX_MIN_VERSION) \ | ||
-B$(build_prefix)/bin -mlinker-version=$(LD64_VERSION) \ | ||
-isysroot$(OSX_SDK) -nostdlibinc \ | ||
-iwithsysroot/usr/include/c++/v1 \ | ||
-iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks | ||
|
||
darwin_CFLAGS=-pipe | ||
darwin_CXXFLAGS=$(darwin_CFLAGS) | ||
darwin_ARFLAGS=cr | ||
|
||
darwin_release_CFLAGS=-O1 | ||
darwin_release_CFLAGS=-O2 | ||
darwin_release_CXXFLAGS=$(darwin_release_CFLAGS) | ||
|
||
darwin_debug_CFLAGS=-O1 | ||
darwin_debug_CXXFLAGS=$(darwin_debug_CFLAGS) | ||
|
||
darwin_native_toolchain=native_cctools darwin_sdk | ||
|
||
darwin_cmake_system=Darwin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we switch to debian (12)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, that might fix the AppArmor issue. Good suggestion, will try.Nvm, GitHub only has Linux runners for Ubuntu.