From 89a5547dc00752f18fe622f9361a85e8dd8996c4 Mon Sep 17 00:00:00 2001 From: Joel Stanley Date: Tue, 20 Dec 2016 11:37:46 -0600 Subject: [PATCH] =?UTF-8?q?Add=20script=20for=20circumventing=20MacOS=20SI?= =?UTF-8?q?P=20and=20using=20nix-built=20grpc;=20upda=E2=80=A6=20(#85)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add script for circumventing MacOS SIP and using nix-built grpc; update README.md * Guard destructive /usr/local and brew updates with a user prompt --- README.md | 115 ++++++++++++++++++++++------------ bin/install-macos-nix-grpc.sh | 38 +++++++++++ 2 files changed, 112 insertions(+), 41 deletions(-) create mode 100755 bin/install-macos-nix-grpc.sh diff --git a/README.md b/README.md index 8f20529..68d314e 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,15 @@ Building and testing -------------------- -You will need to have GRPC installed already. See the "Installing GRPC" section -below for more information. +If you want to use `stack` on MacOS, you will need to have GRPC installed +already -- see the [Installing GRPC for `stack`](#stackgrpc) section below. Any +example build and test recipes below which use `stack` assume you have already +performed the steps described in that section. -`nix-build release.nix -A grpc-haskell` will build and test the whole thing and -put the completed package into the nix store. - -`nix-shell` can be used to give you a development environment where you can use -the `cabal` and `stack test` toolchains for development and testing iteration. +Without `stack`, `nix-build release.nix -A grpc-haskell` will build and test the +whole thing and put the completed package into the nix store. `nix-shell` can be +used to give you a development environment where you can use the `cabal` and +`stack` toolchains for development and testing: ```bash $ nix-shell release-nix -A grpc-haskell.env @@ -20,55 +21,87 @@ $ nix-shell release-nix -A grpc-haskell.env [nix-shell]$ stack build --fast && stack test --fast ``` -Note that, for `stack`, the `nix-shell` environment only needed to run the +Note that, for `stack`, the `nix-shell` environment is only needed to run the tests, because it uses some custom python tooling (for grpc interop testing). You should still be able to `stack build` without using the -`nix-shell` environment at all. See the section below on installing GRPC for use -by `stack`. +`nix-shell` environment at all. NB: You can also instruct `stack` to run the tests inside the `nix-shell` environment directly, via `stack --nix test --fast`. However, this will -frequently rebuild the custom ghc that is used by the `nix` tooling so is not -recommended for development iterations. +frequently rebuild the custom ghc that is used in `release.nix`, so is not +recommended during develop-debug cycles (use the `cabal` path for that, or +iterate within a `nix-shell`). -Finally, since `stack` does not use `nix` for any Haskell dependencies, -repository references for dependent packages such as `protobuf-wire` must be -updated both in `nix/.nix` AND in the `stack.yaml`. +Finally, since `stack` does not use `nix` for any Haskell package dependencies, +be sure to update repository references for dependent packages such as +`protobuf-wire` in both `nix/.nix` AND in `stack.yaml`. -Installing GRPC for `stack` ---------------------------- +Installing GRPC for `stack` (MacOS) +----------------------------------------------------------- -If you want to use the `stack` tooling, you will need a working installation of -the GRPC C core libraries. +If you want to use `stack` in a relatively natural and painless manner, you will +need a working installation of the GRPC C core libraries. -On MacOS, because -of [this](https://github.com/commercialhaskell/stack/issues/1161), dependencies -on the nix-built `grpc` don't work properly, so the library needs to be -installed somewhere the linker will pick it up without `DYLD_LIBRARY_PATH` set. +On MacOS, because of +issues [related](https://github.com/commercialhaskell/stack/issues/1161) to +System Integrity Protection, dependencies on the nix-built `grpc` don't seem to +work properly in the stack toolflow when `DYLD_LIBRARY_PATH` refers to the +`nix`-built `grpc` library in the nix store, so the library needs to be +installed somewhere that the loader can pick it up without `DYLD_LIBRARY_PATH` +set (e.g., in `/usr/local/lib`). -We suggest that use `brew` to do this: +There are basically two methods to accomplish this: -``` -brew tap grpc/grpc -brew edit grpc -... -brew install grpc -``` +1. Run `bin/install-macos-nix-grpc.sh`. -Make sure you select a release version that is reasonably close to our grpc -dependency, e.g.: + This script will build the same version of `grpc` used in `release.nix`, which + is what is used in the end-to-end system and in our CI testing flows. It then + pretends to be an impoverished version of `brew` and installs symlinks from + the nix store into `/usr/local/include/grpc` and + `/usr/local/lib/libgrpc.dylib`. It should be run manually whenever the `grpc` + dependency is updated. Note that it is intentionally destructive to any + existing `brew` installs of `grpc`. -``` -url "https://github.com/grpc/grpc/archive/release-0_15_0.tar.gz" -sha256 "d02235dff278869e94cb0dcb31cfea935693c6f87bd73f43d44147185e6becdd" -``` + Is it an ugly hack? Yes, but it's better than having either a rootless system + to circumvent SIP or building atop a version of `grpc` which may differ from + CI and production environments (which may be the case with `brew`-installed + grpc). -or + After running this script, you should be able to use `stack` normally, in + combination with a `nix-shell` environment for running the tests: -``` -url "https://github.com/grpc/grpc/archive/v1.0.1.tar.gz" -sha256 "efad782944da13d362aab9b81f001b7b8b1458794751de818e9848c47acd4b32" -``` + ```bash + $ bin/install-macos-nix-grpc.sh + $ stack build --fast + $ stack --nix test --fast + ``` + +1. Use `brew` + + If you don't want to hack the global pathing yourself using the above script, + you can rely on homebrew to do this for you instead. However, you will need to + specify the version of the `grpc` release that you want to use. + + ```bash + $ brew tap grpc/grpc + $ brew edit grpc + $ brew install grpc + ``` + + Make sure you select a release version that is reasonably close to our grpc + dependency, e.g.: + + ``` + url "https://github.com/grpc/grpc/archive/release-0_15_0.tar.gz" + sha256 "d02235dff278869e94cb0dcb31cfea935693c6f87bd73f43d44147185e6becdd" + ``` + + or + + ``` + url "https://github.com/grpc/grpc/archive/v1.0.1.tar.gz" + sha256 "efad782944da13d362aab9b81f001b7b8b1458794751de818e9848c47acd4b32" + ``` Using the Library ----------------- diff --git a/bin/install-macos-nix-grpc.sh b/bin/install-macos-nix-grpc.sh new file mode 100755 index 0000000..2a6fa79 --- /dev/null +++ b/bin/install-macos-nix-grpc.sh @@ -0,0 +1,38 @@ +#!/bin/bash +set -e + +function purge_grpc { + echo "Purging old grpc references in /usr/local..." + if brew list grpc; then + echo "Removing brew-installed grpc..." + brew uninstall grpc + else + echo "No brew-installed grpc detected." + fi + rm -rf /usr/local/include/grpc + rm -f /usr/local/lib/libgrpc.dylib +} + +read -p "This script nukes brew-installed grpc libs and destructively updates /usr/local. Cool? [yN] " -n 1 -r +echo +if [[ ! $REPLY =~ ^[Yy]$ ]] +then + echo "Okay, aborting." + exit 1 +fi + +purge_grpc + +echo "Building grpc from release.nix..." +grpc=$(nix-build release.nix -A grpc) +echo "Nix store path for grpc is ${grpc}." + +echo "Creating symlinks into /usr/local/include and /usr/local/lib..." +ln -sf "${grpc}/include/grpc" /usr/local/include/grpc +ln -sf "${grpc}/lib/libgrpc.dylib" /usr/local/lib/libgrpc.dylib + +echo "Creating the following symlinks:" +ls -ld /usr/local/include/grpc +ls -l /usr/local/lib/libgrpc.dylib + +echo "All done."