diff options
Diffstat (limited to '')
| -rw-r--r-- | _posts/2017-01-07-building-boost.md | 140 | ||||
| -rw-r--r-- | _posts/2020-05-06-docker-bind-mounts.md | 162 | ||||
| -rw-r--r-- | _posts/2020-05-20-makefile-escaping.md | 158 |
3 files changed, 303 insertions, 157 deletions
diff --git a/_posts/2017-01-07-building-boost.md b/_posts/2017-01-07-building-boost.md index fa0bd3d..a0f095a 100644 --- a/_posts/2017-01-07-building-boost.md +++ b/_posts/2017-01-07-building-boost.md @@ -27,8 +27,6 @@ auxiliary files, etc. Windows' `cd`, for example, simply prints the current working directory; Cygwin's `pwd` serves the same purpose. `cat` is used to display files. -Windows' command prompts are denoted with `>`s at the beginning of each line; -Cygwin's — with `$`s. Visual Studio ------------- @@ -42,41 +40,43 @@ file names (the "gd" suffix). ### x86 -``` -> cd +{% capture out1 %} D:\workspace\third-party\boost_1_61_0\msvc +{% endcapture %} -> bootstrap -... - -> b2 --stagedir=stage\x86 ^ +{% capture cmd3 %} +b2 --stagedir=stage\x86 ^ runtime-link=static ^ --with-filesystem ^ --with-program_options ^ ... -... -``` +{% endcapture %} + +{% include shell.html cmd='cd' out=out1 %} +{% include shell.html cmd='bootstrap' %} +{% include shell.html cmd=cmd3 %} ### x64 The only important difference is that you have to pass `address-model=64` to `b2` (notice also the different "staging" directory). -``` -> cd +{% capture out1 %} D:\workspace\third-party\boost_1_61_0\msvc +{% endcapture %} -> bootstrap -... - -> b2 --stagedir=stage\x64 ^ +{% capture cmd3 %} +b2 --stagedir=stage\x64 ^ runtime-link=static ^ address-model=64 ^ --with-filesystem ^ --with-program_options ^ ... -... -``` +{% endcapture %} + +{% include shell.html cmd='cd' out=out1 %} +{% include shell.html cmd='bootstrap' %} +{% include shell.html cmd=cmd3 %} Cygwin + MinGW-w64 ------------------ @@ -100,17 +100,16 @@ on the target architecture. ### x86 -``` -$ pwd +{% capture out1 %} /cygdrive/d/workspace/third-party/boost_1_61_0/mingw +{% endcapture %} -$ ./bootstrap.sh -... - -$ cat user-config-x86.jam +{% capture out3 %} using gcc : : i686-w64-mingw32-g++ ; +{% endcapture %} -$ ./b2 toolset=gcc-mingw \ +{% capture cmd4 %} +./b2 toolset=gcc-mingw \ target-os=windows \ link=static \ variant=debug \ @@ -119,8 +118,12 @@ $ ./b2 toolset=gcc-mingw \ --with-filesystem \ --with-program_options \ ... -... -``` +{% endcapture %} + +{% include shell.html cmd='pwd' out=out1 %} +{% include shell.html cmd='./bootstrap.sh' %} +{% include shell.html cmd='cat user-config-x86.jam' out=out3 %} +{% include shell.html cmd=cmd4 %} The "user" configuration file above stopped working at some point; not sure as to who's to blame, Cygwin or Boost. @@ -140,17 +143,16 @@ instead of `i686-w64-mingw32-g++`. Again, as in the example for Visual Studio, a different "staging" directory needs to be specified using the `--stagedir` parameter. -``` -$ cd +{% capture out1 %} /cygdrive/d/workspace/third-party/boost_1_61_0/mingw +{% endcapture %} -$ ./bootstrap.sh -... - -$ cat user-config-x64.jam +{% capture out3 %} using gcc : : x86_64-w64-mingw32-g++ ; +{% endcapture %} -$ ./b2 toolset=gcc-mingw \ +{% capture cmd4 %} +./b2 toolset=gcc-mingw \ address-model=64 \ target-os=windows \ link=static \ @@ -160,8 +162,12 @@ $ ./b2 toolset=gcc-mingw \ --with-filesystem \ --with-program_options \ ... -... -``` +{% endcapture %} + +{% include shell.html cmd='pwd' out=out1 %} +{% include shell.html cmd='./bootstrap.sh' %} +{% include shell.html cmd='cat user-config-x64.jam' out=out3 %} +{% include shell.html cmd=cmd4 %} The "user" configuration file above stopped working at some point; not sure as to who's to blame, Cygwin or Boost. @@ -180,31 +186,39 @@ You may want to adjust the paths. #### x86 -``` -> cd -D:\workspace\build\test_project\msvc\x64 +{% capture out1 %} +D:\workspace\build\test_project\msvc\x86 +{% endcapture %} -> cmake -G "Visual Studio 14 2015" ^ - -D BOOST_ROOT=D:\workspace\third-party\boost_1_61_0\msvc ^ +{% capture cmd2 %} +cmake -G "Visual Studio 14 2015" ^ + -D BOOST_ROOT=D:\workspace\third-party\boost_1_61_0\msvc ^ -D BOOST_LIBRARYDIR=D:\workspace\third-party\boost_1_61_0\msvc\stage\x86\lib ^ - -D Boost_USE_STATIC_LIBS=ON ^ + -D Boost_USE_STATIC_LIBS=ON ^ -D Boost_USE_STATIC_RUNTIME=ON ^ ... -``` +{% endcapture %} + +{% include shell.html cmd='cd' out=out1 %} +{% include shell.html cmd=cmd2 %} #### x64 -``` -> cd -D:\workspace\build\test_project\msvc\x86 +{% capture out1 %} +D:\workspace\build\test_project\msvc\x64 +{% endcapture %} -> cmake -G "Visual Studio 14 2015 Win64" ^ - -D BOOST_ROOT=D:\workspace\third-party\boost_1_61_0\msvc ^ +{% capture cmd2 %} +cmake -G "Visual Studio 14 2015 Win64" ^ + -D BOOST_ROOT=D:\workspace\third-party\boost_1_61_0\msvc ^ -D BOOST_LIBRARYDIR=D:\workspace\third-party\boost_1_61_0\msvc\stage\x64\lib ^ - -D Boost_USE_STATIC_LIBS=ON ^ - -D Boost_USE_STATIC_RUNTIME=ON ^ + -D Boost_USE_STATIC_LIBS=ON ^ + -D Boost_USE_STATIC_RUNTIME=ON ^ ... -``` +{% endcapture %} + +{% include shell.html cmd='cd' out=out1 %} +{% include shell.html cmd=cmd2 %} ### Cygwin & MinGW-w64 @@ -215,11 +229,12 @@ You may also want to adjust the paths. #### x86 -``` -$ pwd +{% capture out1 %} /cygdrive/d/workspace/build/test_project/mingw/x86/debug +{% endcapture %} -$ cmake -G "Unix Makefiles" \ +{% capture cmd2 %} +cmake -G "Unix Makefiles" \ -D CMAKE_BUILD_TYPE=Debug \ -D CMAKE_C_COMPILER=i686-w64-mingw32-gcc \ -D CMAKE_CXX_COMPILER=i686-w64-mingw32-g++ \ @@ -227,15 +242,19 @@ $ cmake -G "Unix Makefiles" \ -D BOOST_LIBRARYDIR=/cygdrive/d/workspace/third-party/boost_1_61_0/mingw/stage/x86/debug/lib \ -D Boost_USE_STATIC_LIBS=ON \ ... -``` +{% endcapture %} + +{% include shell.html cmd='pwd' out=out1 %} +{% include shell.html cmd=cmd2 %} #### x64 -``` -$ pwd +{% capture out1 %} /cygdrive/d/workspace/build/test_project/mingw/x64/debug +{% endcapture %} -$ cmake -G "Unix Makefiles" \ +{% capture cmd2 %} +cmake -G "Unix Makefiles" \ -D CMAKE_BUILD_TYPE=Debug \ -D CMAKE_C_COMPILER=x86_64-w64-mingw32-gcc \ -D CMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++ \ @@ -243,4 +262,7 @@ $ cmake -G "Unix Makefiles" \ -D BOOST_LIBRARYDIR=/cygdrive/d/workspace/third-party/boost_1_61_0/mingw/stage/x64/debug/lib \ -D Boost_USE_STATIC_LIBS=ON \ ... -``` +{% endcapture %} + +{% include shell.html cmd='pwd' out=out1 %} +{% include shell.html cmd=cmd2 %} diff --git a/_posts/2020-05-06-docker-bind-mounts.md b/_posts/2020-05-06-docker-bind-mounts.md index 5f85597..4141bf7 100644 --- a/_posts/2020-05-06-docker-bind-mounts.md +++ b/_posts/2020-05-06-docker-bind-mounts.md @@ -14,13 +14,21 @@ after being mounted in the container. Case in point: - docker run -it --rm -v "$( pwd ):/data" alpine touch /data/test.txt +{% capture cmd1 %} +docker run -it --rm -v "$( pwd ):/data" alpine touch /data/test.txt +{% endcapture %} + +{% include shell.html cmd=cmd1 %} would create file ./test.txt owned by root:root. You can fix that by using the `--user` parameter: - docker run -it --rm -v "$( pwd ):/data" --user "$( id -u ):$( id -g )" alpine touch /data/test.txt +{% capture cmd1 %} +docker run -it --rm -v "$( pwd ):/data" --user "$( id -u ):$( id -g )" alpine touch /data/test.txt +{% endcapture %} + +{% include shell.html cmd=cmd1 %} That would create file ./test.txt owned by the current user (if the current working directory is writable by the current user, of course). @@ -40,36 +48,54 @@ How do you run the service as regular user though? It's tempting to use the `USER` directive in the Dockerfile, but that can be overridden by `--user`: - $ cat Dockerfile - FROM alpine - - RUN addgroup --gid 9099 test-group && \ - adduser \ - --disabled-password \ - --gecos '' \ - --home /home/test-user \ - --ingroup test-group \ - --uid 9099 \ - test-user - - RUN touch /root.txt - USER test-user:test-group - RUN touch /home/test-user/test-user.txt - - CMD id && stat -c '%U %G' /root.txt && stat -c '%U %G' /home/test-user/test-user.txt - - $ docker build -t id . - ... - - $ docker run -it --rm id - uid=9099(test-user) gid=9099(test-group) - root root - test-user test-group - - $ docker run -it --rm --user root id - uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video) - root root - test-user test-group +{% capture cmd1 %} +cat Dockerfile +{% endcapture %} +{% capture out1 %} +FROM alpine + +RUN addgroup --gid 9099 test-group && \ + adduser \ + --disabled-password \ + --gecos '' \ + --home /home/test-user \ + --ingroup test-group \ + --uid 9099 \ + test-user + +RUN touch /root.txt +USER test-user:test-group +RUN touch /home/test-user/test-user.txt + +CMD id && stat -c '%U %G' /root.txt && stat -c '%U %G' /home/test-user/test-user.txt +{% endcapture %} + +{% capture cmd2 %} +docker build -t id . +{% endcapture %} + +{% capture cmd3 %} +docker run -it --rm id +{% endcapture %} +{% capture out3 %} +uid=9099(test-user) gid=9099(test-group) +root root +test-user test-group +{% endcapture %} + +{% capture cmd4 %} +docker run -it --rm --user root id +{% endcapture %} +{% capture out4 %} +uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video) +root root +test-user test-group +{% endcapture %} + +{% include shell.html cmd=cmd1 out=out1 %} +{% include shell.html cmd=cmd2 %} +{% include shell.html cmd=cmd3 out=out3 %} +{% include shell.html cmd=cmd4 out=out4 %} I suppose that's the reason why many popular images override ENTRYPOINT, using a custom script (and `gosu`, which is basically `sudo`, I think) to forcefully @@ -89,16 +115,32 @@ For example, let's try to map ./data to /data inside a Redis container (this assumes ./data doesn't exist and you're running as regular user with UID 1000; press Ctrl+C to stop Redis): - $ mkdir data - - $ stat -c '%u' data - 1000 - - $ docker run -it --rm --name redis -v "$( pwd )/data:/data" redis:6.0 - ... - - $ stat -c '%u' data - 999 +{% capture cmd1 %} +mkdir data +{% endcapture %} + +{% capture cmd2 %} +stat -c '%u' data +{% endcapture %} +{% capture out2 %} +1000 +{% endcapture %} + +{% capture cmd3 %} +docker run -it --rm --name redis -v "$( pwd )/data:/data" redis:6.0 +{% endcapture %} + +{% capture cmd4 %} +stat -c '%u' data +{% endcapture %} +{% capture out4 %} +999 +{% endcapture %} + +{% include shell.html cmd=cmd1 %} +{% include shell.html cmd=cmd2 out=out2 %} +{% include shell.html cmd=cmd3 %} +{% include shell.html cmd=cmd4 out=out4 %} As you can see, ./data changed its owner from user with UID 1000 (the host user) to user with UID 999 (the `redis` user inside the container). @@ -111,16 +153,32 @@ explicitly accommodates for it by _not_ changing its owner if the container is run as anybody other than root. For example: - $ mkdir data - - $ stat -c '%u' data - 1000 - - $ docker run -it --rm --name redis -v "$( pwd )/data:/data" --user "$( id -u ):$( id -g )" redis:6.0 - ... - - $ stat -c '%u' data - 1000 +{% capture cmd1 %} +mkdir data +{% endcapture %} + +{% capture cmd2 %} +stat -c '%u' data +{% endcapture %} +{% capture out2 %} +1000 +{% endcapture %} + +{% capture cmd3 %} +docker run -it --rm --name redis -v "$( pwd )/data:/data" --user "$( id -u ):$( id -g )" redis:6.0 +{% endcapture %} + +{% capture cmd4 %} +stat -c '%u' data +{% endcapture %} +{% capture out4 %} +1000 +{% endcapture %} + +{% include shell.html cmd=cmd1 %} +{% include shell.html cmd=cmd2 out=out2 %} +{% include shell.html cmd=cmd3 %} +{% include shell.html cmd=cmd4 out=out4 %} Sometimes `--user` is not enough though. That specified user is almost certainly missing from container's /etc/passwd, diff --git a/_posts/2020-05-20-makefile-escaping.md b/_posts/2020-05-20-makefile-escaping.md index 90acf85..f3d301e 100644 --- a/_posts/2020-05-20-makefile-escaping.md +++ b/_posts/2020-05-20-makefile-escaping.md @@ -84,8 +84,7 @@ scripts. This is to prevent a single argument from being expanded into multiple arguments by the shell. -``` -$ cat Makefile +{% capture out1 %} # Prologue goes here... test_var := Same line? @@ -96,15 +95,19 @@ test: @printf '%s\n' '$(test_var)' @printf '%s\n' $$test_var @printf '%s\n' "$$test_var" +{% endcapture %} -$ make test +{% capture out2 %} Same line? Same line? Same line? Same line? -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd='make test' out=out2 %} This is quite often sufficient to write valid recipes. @@ -119,28 +122,30 @@ What if `test_var` included a single quote `'`? In that case, even the quoted `printf` invocation would break because of the mismatch. -``` -$ cat Makefile +{% capture out1 %} # Prologue goes here... test_var := Includes ' quote test: printf '%s\n' '$(test_var)' +{% endcapture %} -$ make test +{% capture out2 %} printf '%s\n' 'Includes ' quote' bash: -c: line 0: unexpected EOF while looking for matching `'' make: *** [Makefile:11: test] Error 2 -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd='make test' out=out2 %} One solution is to take advantage of how `bash` parses command arguments, and replace every quote `'` by `'\''`. This works because `bash` merges a string like `'Includes '\'' quote'` into `Includes ' quote`. -``` -$ cat Makefile +{% capture out1 %} # Prologue goes here... escape = $(subst ','\'',$(1)) @@ -149,11 +154,15 @@ test_var := Includes ' quote test: printf '%s\n' '$(call escape,$(test_var))' +{% endcapture %} -$ make test +{% capture out2 %} printf '%s\n' 'Includes '\'' quote' Includes ' quote -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd='make test' out=out2 %} Surprisingly, this works even in much more complicated cases. You can have a recipe that executes a command that takes a whole other command @@ -161,8 +170,7 @@ You can have a recipe that executes a command that takes a whole other command I guess the most common use case is doing something like `ssh 'rm -rf $(junk_dir)'`, but I'll use nested `bash` calls instead for simplicity. -``` -$ cat Makefile +{% capture out1 %} # Prologue goes here... escape = $(subst ','\'',$(1)) @@ -176,15 +184,19 @@ test: printf '%s\n' '$(call escape,$(test_var))' bash -c '$(call escape,$(echo_test_var))' bash -c '$(call escape,$(bash_test_var))' +{% endcapture %} -$ make test +{% capture out2 %} printf '%s\n' 'Includes '\'' quote' Includes ' quote bash -c 'printf '\''%s\n'\'' '\''Includes '\''\'\'''\'' quote'\''' Includes ' quote bash -c 'bash -c '\''printf '\''\'\'''\''%s\n'\''\'\'''\'' '\''\'\'''\''Includes '\''\'\'''\''\'\''\'\'''\'''\''\'\'''\'' quote'\''\'\'''\'''\''' Includes ' quote -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd='make test' out=out2 %} That's somewhat insane, but it works. @@ -196,8 +208,7 @@ outside world in a Makefile (the other being environment variables). This little `escape` function we've defined is actually sufficient to deal with the output of the `shell` function safely. -``` -$ cat Makefile +{% capture out1 %} # Prologue goes here... escape = $(subst ','\'',$(1)) @@ -211,19 +222,42 @@ composite_var := Composite value - $(simple_var) - $(cwd) test: @printf '%s\n' '$(call escape,$(cwd))' @printf '%s\n' '$(call escape,$(composite_var))' - -$ ( mkdir -p -- "Includes ' quote" && cd -- "Includes ' quote" && make -f ../Makefile test ; ) +{% endcapture %} + +{% capture cmd2 %} +mkdir "Includes ' quote" && \ + cd "Includes ' quote" && \ + make -f ../Makefile test +{% endcapture %} +{% capture out2 %} Includes ' quote Composite value - Simple value - Includes ' quote - -$ ( mkdir -p -- 'Maybe a comment #' && cd -- 'Maybe a comment #' && make -f ../Makefile test ; ) +{% endcapture %} + +{% capture cmd3 %} +mkdir 'Maybe a comment #' && \ + cd 'Maybe a comment #' && \ + make -f ../Makefile test +{% endcapture %} +{% capture out3 %} Maybe a comment # Composite value - Simple value - Maybe a comment # - -$ ( mkdir -p -- 'Variable ${reference}' && cd -- 'Variable ${reference}' && make -f ../Makefile test ; ) +{% endcapture %} + +{% capture cmd4 %} +mkdir 'Variable ${reference}' && \ + cd 'Variable ${reference}' && \ + make -f ../Makefile test +{% endcapture %} +{% capture out4 %} Variable ${reference} Composite value - Simple value - Variable ${reference} -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd=cmd2 out=out2 %} +{% include shell.html cmd=cmd3 out=out3 %} +{% include shell.html cmd=cmd4 out=out4 %} Environment variables --------------------- @@ -246,9 +280,7 @@ signs `$`. expanded recursively either when defined (for `:=` assignments) or when used (in all other cases, including `?=`). - -``` -$ cat Makefile +{% capture out1 %} # Prologue goes here... escape = $(subst ','\'',$(1)) @@ -260,30 +292,40 @@ export test_var test: @printf '%s\n' '$(call escape,$(test_var))' @printf '%s\n' "$$test_var" +{% endcapture %} -$ test_var='Variable ${reference}' make test +{% capture cmd2 %} +test_var='Variable ${reference}' make test +{% endcapture %} +{% capture out2 %} Makefile:15: warning: undefined variable 'reference' Variable Variable ${reference} -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd=cmd2 out=out2 %} Here, `$(test_var)` is expanded recursively, substituting an empty string for the `${reference}` part. One attempt to solve this is to escape the dollar sign in the variable value, but that breaks the `"$$test_var"` case: -``` -$ test_var='Variable $${reference}' make test +{% capture cmd1 %} +test_var='Variable $${reference}' make test +{% endcapture %} +{% capture out1 %} Variable ${reference} Variable $${reference} -``` +{% endcapture %} + +{% include shell.html cmd=cmd1 out=out1 %} A working solution would be to use the `escape` function on the unexpanded variable value. Turns out, you can do just that using the `value` function in `make`. -``` -$ cat Makefile +{% capture out1 %} # Prologue goes here... escape = $(subst ','\'',$(1)) @@ -296,24 +338,35 @@ export test_var test: @printf '%s\n' '$(call escape,$(test_var))' @printf '%s\n' "$$test_var" +{% endcapture %} -$ test_var="Quote '"' and variable ${reference}' make test +{% capture cmd2 %} +test_var="Quote '"' and variable ${reference}' make test +{% endcapture %} +{% capture out2 %} Quote ' and variable ${reference} Quote ' and variable ${reference} -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd=cmd2 out=out2 %} This doesn't quite work though when [overriding variables] on the command line. For example, this doesn't work: [overriding variables]: https://www.gnu.org/software/make/manual/html_node/Overriding.html#Overriding -``` -$ make test test_var='Variable ${reference}' +{% capture cmd1 %} +make test test_var='Variable ${reference}' +{% endcapture %} +{% capture out1 %} Makefile:16: warning: undefined variable 'reference' make: warning: undefined variable 'reference' Variable Variable -``` +{% endcapture %} + +{% include shell.html cmd=cmd1 out=out1 %} This is because `make` ignores all assignments to `test_var` if it's overridden on the command line (including `test_var := $(value test_var)`). @@ -371,8 +424,7 @@ wouldn't work. You can even safely use other variable as the default value of `test_var`, and it works: -``` -> cat Makefile +{% capture out1 %} # Prologue goes here... escape = $(subst ','\'',$(1)) @@ -401,16 +453,30 @@ composite_var := Composite value - $(simple_var) - $(test_var) test: @printf '%s\n' '$(call escape,$(test_var))' @printf '%s\n' '$(call escape,$(composite_var))' +{% endcapture %} -> make test +{% capture out2 %} New simple value in test_var Composite value - New simple value - New simple value in test_var +{% endcapture %} -> make test test_var='Variable ${reference}' +{% capture cmd3 %} +make test test_var='Variable ${reference}' +{% endcapture %} +{% capture out3 %} Variable ${reference} Composite value - New simple value - Variable ${reference} +{% endcapture %} -> test_var='Variable ${reference}' make test +{% capture cmd4 %} +test_var='Variable ${reference}' make test +{% endcapture %} +{% capture out4 %} Variable ${reference} Composite value - New simple value - Variable ${reference} -``` +{% endcapture %} + +{% include shell.html cmd='cat Makefile' out=out1 %} +{% include shell.html cmd='make test' out=out2 %} +{% include shell.html cmd=cmd3 out=out3 %} +{% include shell.html cmd=cmd4 out=out4 %} |