diff --git a/RelocatableSoftware/HSFReloc/README.md b/RelocatableSoftware/HSFReloc/README.md index 26d4132..81b656e 100644 --- a/RelocatableSoftware/HSFReloc/README.md +++ b/RelocatableSoftware/HSFReloc/README.md @@ -4,12 +4,13 @@ Example program/library code to demonstrate package relocatability concepts. The following software is required to build the base set of programs: - [CMake](https://www.cmake.org) version 3.3 or newer -- C++11 compatible compiler (GNU, Clang, Intel, MSVC) +- C++11 compatible compiler (GNU, Clang, Intel) +- macOS or Linux OS (Windows not yet supported) The following software is optional: -- [Qt5](https://www.qt.io) for the [qt](qt) subproject -- [Poco](http://pocoproject.org) for the [poco](poco) subproject +- [Qt5](https://www.qt.io) for the [qt](programs/hsfreloc-qt.cpp) example application +- [Poco](http://pocoproject.org) for the [poco](hsfreloc-poco.cpp) example application To build the software, simply create a build directory and run `cmake` inside it, pointing it to the top level source directory (i.e. the directory holding this @@ -25,7 +26,7 @@ $ cmake --build . ``` If CMake has issues finding any dependencies, ensure their root install -prefixes are listed in `CMAKE_PREFIX_PATH` and passing this list to +prefixes are listed in `CMAKE_PREFIX_PATH` and pass this list to `cmake` on the command line or setting it in your environment. Build Outputs @@ -38,8 +39,79 @@ If you configure the project for an IDE like Xcode, the `BuildProducts` directory will contain an extra level of directories to support each build type (`Release`, `Debug` etc) + +Compiled Languages +================== +The [programs](programs) directory focuses on C/C++ implementations, but PRs are +welcome to demonstrate application/library self-location in other languages. + +The `hsfreloc` program +---------------------- + +This basic C/C++ example demonstrates the use of the [`binreloc`](programs/binreloc) +library for simple application self-location. It also comes with a basic resource +file "resource.txt" to show how these can be located. Simply running the application +will print its location and the contents of the resource file: + +``` console +$ ./BuildProducts/bin/hsfreloc +[application in]: /AbsPathToWhereYouRanCMake/./BuildProducts/bin +[resource]: 'hello from builtin hsfreloc resource file! +' +$ +``` + +Relocatability can be tested by copying the `BuildProducts` directory to any other location +you like on the local machine. Rerun `hsfreloc` and it should print its new location and +the resource contents. You can prove that it's not using build time paths by removing +the original build directory. + + +The `hsfreloc-poco` program +--------------------------- + +Demonstrates the self-location interfaces supplied by the [Poco](http://pocoproject.org) libraries. + + +The `hsfreloc-qt` program +------------------------- + +Demonstrates the self-location interfaces supplied by the [Qt5](https://www.qt.io) libraries. + + Scripting Languages =================== The directories under [scripting](scripting) contain (very) basic -examples of relocatable Python and Ruby programs. +examples of relocatable Python and Ruby programs. If your language du jour +is not in here, feel free to add a PR showing how it implements self-location. + + +Dealing with Packaging Issues +============================= + +When implementing new or patching existing HEP packages, the above techniques +should ensure relocatablity. Packaging involves integrating many pieces of software, +many of which may not be fully relocatable. This section aims to provide demonstrations +of the typical cases and fixes where possible (and "this is broken" where not...). +Examples are: + +1. A program/library links to others - use of `patchelf` and `install_name_tool`, RPATHS, + RUNPATHS, and linker tricks. +2. A program/library hard codes a path into binary. For example + + ``` c + /* bad.cc */ + #define RESOURCEPATH /usr/share/badpackage + ``` + + Likely to be the most awkward case, but conda/spack may have the tools to repoint these + paths. +3. A package has text based files that hard code paths, e.g. + + ``` sh + #!/some/build/time/path/to/python + ``` + + Simple search-and-replace should work here. + diff --git a/RelocatableSoftware/HSFReloc/programs/CMakeLists.txt b/RelocatableSoftware/HSFReloc/programs/CMakeLists.txt index ac5a298..5f9691a 100644 --- a/RelocatableSoftware/HSFReloc/programs/CMakeLists.txt +++ b/RelocatableSoftware/HSFReloc/programs/CMakeLists.txt @@ -9,7 +9,7 @@ set(MANGLE_BINRELOC "hsf_reloc_binreloc${MANGLE_BINRELOC}") configure_file(binreloc/hsf_binreloc.h.in binreloc/hsf_binreloc.h @ONLY) # - Configure "front end" to binreloc, adding known *relative path* -# from exe dire to resource dir +# from exe dir to resource dir file(RELATIVE_PATH HSF_BINDIR_TO_RESOURCEDIR "${CMAKE_INSTALL_FULL_BINDIR}" "${CMAKE_INSTALL_FULL_DATADIR}/${PROJECT_NAME}/resources" @@ -43,7 +43,7 @@ else() endif() #----------------------------------------------------------------------- -# - Optional Qt5 deom for its builtin self-location +# - Optional Qt5 demo for its builtin self-location # - Only need core lib find_package(Qt5Core QUIET) diff --git a/RelocatableSoftware/README.md b/RelocatableSoftware/README.md index 7fb2caf..fe67fee 100644 --- a/RelocatableSoftware/README.md +++ b/RelocatableSoftware/README.md @@ -285,7 +285,7 @@ their own interpreters, whose paths may also end up hard coded into scripts: print("hello world") ``` -The resulting stack is then not relocatable as the interpreter path will not exist after relocation. +The resulting stack is then not relocatable as the interpreter path will not exist after relocation. Rather than hard coding system or custom interpreter paths, script authors should prefer the use of the [`env`](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/env.html) program as the shebang, e.g. @@ -298,12 +298,12 @@ print("hello world") Use of `env` makes the program relocatble, but defers location of the interpreter to the `PATH` environment variable, and consequently the configuration management system for the software stack. Whilst package authors should prefer -usage of the `env` pattern, software stack managers can also consider rewriting the shebang line during install +usage of the `env` pattern, software stack managers can also consider rewriting the shebang line during install and on any relocation to the absolute path of the required interpreter. As it is plain text, simple regular expression replacement can be used, but the chosen packaging system must support this, and care must be taken if the resultant stack is to be deployed over network file systems (and hence unknown mount points). -**TODO?** Binaries *also* have an interpreter (on Linux, `ld-linux.so`, On macOS, `dyld`). These are also hardcoded, +**TODO?** Binaries *also* have an interpreter (on Linux, `ld-linux.so`, On macOS, `dyld`). These are also hardcoded, though can be changed with, e.g., `patchelf` for ELF binaries. @@ -370,7 +370,7 @@ Development Tools CMake ----- To support use of a Project by a CMake based client project, scripts for -use with CMake's [`find_package`](http://www.cmake.org/cmake/help/v3.2/command/find_package.html) command in "config" +use with CMake's [`find_package`](http://www.cmake.org/cmake/help/v3.2/command/find_package.html) command in "config" mode should be provided. A `FindPACKAGENAME.cmake` should *not* be implemented, including the use of CMake commands like `find_path`, `find_library` as these are intended to locate packages not supplying any CMake support files. CMake "ProjectConfig.cmake" files are installed alongside the project and can self-locate the project's headers/libraries/executables @@ -467,7 +467,7 @@ additional arguments. For libraries using environment variables, it may be possible to wrap these with a small facade library. This would do nothing more that self-locate, set the needed environment variables and expose the rest -of the library symbols. However, this has implications for usability +of the library symbols. However, this has implications for usability and runtime manipulation of the environment by clients. When absolute paths are hardcoded into binaries, then only intrusive @@ -497,8 +497,8 @@ Relocation and Packaging ======================== **TODO** Topics involving relocatability when it comes to the packaging level. Define "packaging" as the things we need to do/write to allow `spack|brew|whatever install mypackage` to work, whether building `mypackage` from source locally -or downloading and unpacking a binary. The "package manager" needs to include tooling to manage relocation, and this -may include things like changing RPATHs in binaries, to shebangs in interpreted programs. Other topics like deployment +or downloading and unpacking a binary. The "package manager" needs to include tooling to manage relocation, and this +may include things like changing RPATHs in binaries, to shebangs in interpreted programs. Other topics like deployment to CVMFS etc. Take simple example of two packages and a "typical" versioned tree plus "views"? e.g.