Add script for circumventing MacOS SIP and using nix-built grpc; upda… (#85)
* Add script for circumventing MacOS SIP and using nix-built grpc; update README.md * Guard destructive /usr/local and brew updates with a user prompt
This commit is contained in:
Joel Stanley
GitHub Enterprise
parent
8a6a7c3715
commit
89a5547dc0
115
README.md
115
README.md
|
|
@ -1,14 +1,15 @@
|
|||
Building and testing
|
||||
--------------------
|
||||
|
||||
You will need to have GRPC installed already. See the "Installing GRPC" section
|
||||
below for more information.
|
||||
If you want to use `stack` on MacOS, you will need to have GRPC installed
|
||||
already -- see the [Installing GRPC for `stack`](#stackgrpc) section below. Any
|
||||
example build and test recipes below which use `stack` assume you have already
|
||||
performed the steps described in that section.
|
||||
|
||||
`nix-build release.nix -A grpc-haskell` will build and test the whole thing and
|
||||
put the completed package into the nix store.
|
||||
|
||||
`nix-shell` can be used to give you a development environment where you can use
|
||||
the `cabal` and `stack test` toolchains for development and testing iteration.
|
||||
Without `stack`, `nix-build release.nix -A grpc-haskell` will build and test the
|
||||
whole thing and put the completed package into the nix store. `nix-shell` can be
|
||||
used to give you a development environment where you can use the `cabal` and
|
||||
`stack` toolchains for development and testing:
|
||||
|
||||
```bash
|
||||
$ nix-shell release-nix -A grpc-haskell.env
|
||||
|
|
@ -20,55 +21,87 @@ $ nix-shell release-nix -A grpc-haskell.env
|
|||
[nix-shell]$ stack build --fast && stack test --fast
|
||||
```
|
||||
|
||||
Note that, for `stack`, the `nix-shell` environment only needed to run the
|
||||
Note that, for `stack`, the `nix-shell` environment is only needed to run the
|
||||
tests, because it uses some custom python tooling (for grpc interop
|
||||
testing). You should still be able to `stack build` without using the
|
||||
`nix-shell` environment at all. See the section below on installing GRPC for use
|
||||
by `stack`.
|
||||
`nix-shell` environment at all.
|
||||
|
||||
NB: You can also instruct `stack` to run the tests inside the `nix-shell`
|
||||
environment directly, via `stack --nix test --fast`. However, this will
|
||||
frequently rebuild the custom ghc that is used by the `nix` tooling so is not
|
||||
recommended for development iterations.
|
||||
frequently rebuild the custom ghc that is used in `release.nix`, so is not
|
||||
recommended during develop-debug cycles (use the `cabal` path for that, or
|
||||
iterate within a `nix-shell`).
|
||||
|
||||
Finally, since `stack` does not use `nix` for any Haskell dependencies,
|
||||
repository references for dependent packages such as `protobuf-wire` must be
|
||||
updated both in `nix/<pkg>.nix` AND in the `stack.yaml`.
|
||||
Finally, since `stack` does not use `nix` for any Haskell package dependencies,
|
||||
be sure to update repository references for dependent packages such as
|
||||
`protobuf-wire` in both `nix/<pkg>.nix` AND in `stack.yaml`.
|
||||
|
||||
Installing GRPC for `stack`
|
||||
---------------------------
|
||||
<a name="stackgrpc"></a>Installing GRPC for `stack` (MacOS)
|
||||
-----------------------------------------------------------
|
||||
|
||||
If you want to use the `stack` tooling, you will need a working installation of
|
||||
the GRPC C core libraries.
|
||||
If you want to use `stack` in a relatively natural and painless manner, you will
|
||||
need a working installation of the GRPC C core libraries.
|
||||
|
||||
On MacOS, because
|
||||
of [this](https://github.com/commercialhaskell/stack/issues/1161), dependencies
|
||||
on the nix-built `grpc` don't work properly, so the library needs to be
|
||||
installed somewhere the linker will pick it up without `DYLD_LIBRARY_PATH` set.
|
||||
On MacOS, because of
|
||||
issues [related](https://github.com/commercialhaskell/stack/issues/1161) to
|
||||
System Integrity Protection, dependencies on the nix-built `grpc` don't seem to
|
||||
work properly in the stack toolflow when `DYLD_LIBRARY_PATH` refers to the
|
||||
`nix`-built `grpc` library in the nix store, so the library needs to be
|
||||
installed somewhere that the loader can pick it up without `DYLD_LIBRARY_PATH`
|
||||
set (e.g., in `/usr/local/lib`).
|
||||
|
||||
We suggest that use `brew` to do this:
|
||||
There are basically two methods to accomplish this:
|
||||
|
||||
```
|
||||
brew tap grpc/grpc
|
||||
brew edit grpc
|
||||
...
|
||||
brew install grpc
|
||||
```
|
||||
1. Run `bin/install-macos-nix-grpc.sh`.
|
||||
|
||||
Make sure you select a release version that is reasonably close to our grpc
|
||||
dependency, e.g.:
|
||||
This script will build the same version of `grpc` used in `release.nix`, which
|
||||
is what is used in the end-to-end system and in our CI testing flows. It then
|
||||
pretends to be an impoverished version of `brew` and installs symlinks from
|
||||
the nix store into `/usr/local/include/grpc` and
|
||||
`/usr/local/lib/libgrpc.dylib`. It should be run manually whenever the `grpc`
|
||||
dependency is updated. Note that it is intentionally destructive to any
|
||||
existing `brew` installs of `grpc`.
|
||||
|
||||
```
|
||||
url "https://github.com/grpc/grpc/archive/release-0_15_0.tar.gz"
|
||||
sha256 "d02235dff278869e94cb0dcb31cfea935693c6f87bd73f43d44147185e6becdd"
|
||||
```
|
||||
Is it an ugly hack? Yes, but it's better than having either a rootless system
|
||||
to circumvent SIP or building atop a version of `grpc` which may differ from
|
||||
CI and production environments (which may be the case with `brew`-installed
|
||||
grpc).
|
||||
|
||||
or
|
||||
After running this script, you should be able to use `stack` normally, in
|
||||
combination with a `nix-shell` environment for running the tests:
|
||||
|
||||
```
|
||||
url "https://github.com/grpc/grpc/archive/v1.0.1.tar.gz"
|
||||
sha256 "efad782944da13d362aab9b81f001b7b8b1458794751de818e9848c47acd4b32"
|
||||
```
|
||||
```bash
|
||||
$ bin/install-macos-nix-grpc.sh
|
||||
$ stack build --fast
|
||||
$ stack --nix test --fast
|
||||
```
|
||||
|
||||
1. Use `brew`
|
||||
|
||||
If you don't want to hack the global pathing yourself using the above script,
|
||||
you can rely on homebrew to do this for you instead. However, you will need to
|
||||
specify the version of the `grpc` release that you want to use.
|
||||
|
||||
```bash
|
||||
$ brew tap grpc/grpc
|
||||
$ brew edit grpc
|
||||
$ brew install grpc
|
||||
```
|
||||
|
||||
Make sure you select a release version that is reasonably close to our grpc
|
||||
dependency, e.g.:
|
||||
|
||||
```
|
||||
url "https://github.com/grpc/grpc/archive/release-0_15_0.tar.gz"
|
||||
sha256 "d02235dff278869e94cb0dcb31cfea935693c6f87bd73f43d44147185e6becdd"
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```
|
||||
url "https://github.com/grpc/grpc/archive/v1.0.1.tar.gz"
|
||||
sha256 "efad782944da13d362aab9b81f001b7b8b1458794751de818e9848c47acd4b32"
|
||||
```
|
||||
|
||||
Using the Library
|
||||
-----------------
|
||||
|
|
|
|||
|
|
@ -0,0 +1,38 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
function purge_grpc {
|
||||
echo "Purging old grpc references in /usr/local..."
|
||||
if brew list grpc; then
|
||||
echo "Removing brew-installed grpc..."
|
||||
brew uninstall grpc
|
||||
else
|
||||
echo "No brew-installed grpc detected."
|
||||
fi
|
||||
rm -rf /usr/local/include/grpc
|
||||
rm -f /usr/local/lib/libgrpc.dylib
|
||||
}
|
||||
|
||||
read -p "This script nukes brew-installed grpc libs and destructively updates /usr/local. Cool? [yN] " -n 1 -r
|
||||
echo
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]
|
||||
then
|
||||
echo "Okay, aborting."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
purge_grpc
|
||||
|
||||
echo "Building grpc from release.nix..."
|
||||
grpc=$(nix-build release.nix -A grpc)
|
||||
echo "Nix store path for grpc is ${grpc}."
|
||||
|
||||
echo "Creating symlinks into /usr/local/include and /usr/local/lib..."
|
||||
ln -sf "${grpc}/include/grpc" /usr/local/include/grpc
|
||||
ln -sf "${grpc}/lib/libgrpc.dylib" /usr/local/lib/libgrpc.dylib
|
||||
|
||||
echo "Creating the following symlinks:"
|
||||
ls -ld /usr/local/include/grpc
|
||||
ls -l /usr/local/lib/libgrpc.dylib
|
||||
|
||||
echo "All done."
|
||||
Loading…
Reference in New Issue