Newline (\n) handled differently by zsh vs bash - how to fix?

If I run the script below using zsh and bash, I'm getting different results. Interestingly, this is caused by the \n newlines in the input string.

Using zsh, I get the correct result:

rene@MININT-G1P7G69 Desktop % zsh bash_vs_zsh.sh
Hashed signature: AgTW6P8HlAlfOikDPMa/Q92tX2a0GSdDLVeeZE219BQ=

With bash, I'm getting an incorrect hash:

rene@MININT-G1P7G69 Desktop % bash bash_vs_zsh.sh
Hashed signature: R2FaEYqGsb9QtCQTJEvySoqs0VEgtCyWEWg1R5jRzEo=

If I remove the \n instances from the input, the results are equal.

signature="get\ndbs\n\nmon, 27 apr 2020 16:11:57 gmt\n\n"
masterKey="t7ejJOwk0HEgkeYCm9v3n8vNwVaW27uriUmTTc3JcBtwqHBfwqO1tAJqKOBpeivurzPl1DxsNFUehEQN5lzGRw=="
hashedSignature=$(echo -n $signature | openssl dgst -sha256 -mac hmac -macopt hexkey:$(echo -n $masterKey | base64 --decode | hexdump -v -e '/1 "%02x"') -binary | base64)
echo "Hashed signature: $hashedSignature"

How can I make bash behave as expected?

1 Answer

1. In Bash and in Zsh "\n" becomes just \n after quote removal, not a newline character. The difference is in how echo builtin prints it later. In Zsh echo with or without -e interprets \n (and other sequences) like echo -e in Bash; but in Bash echo without -e doesn't. You have at least two possibilities:

   • Define signature as you did and let echo -en interpret \n substrings later.

   • Define signature so it actually contains newlines in the first place. To get newlines at this stage you can use ANSI-C quoting:

     signature=$'get\ndbs\n\nmon, 27 apr 2020 16:11:57 gmt\n\n'

     But then you need to make echo not interpret sequences just in case the string you want to pass should contain one or more sequences literally. The idea is to have all the interpretation in one place.

   In general printf is better than echo. I would choose ANSI-C quoting while defining the variable, then printf that prints literally.
2. In Bash you need to explicitly double-quote variables and command substitutions, unless you know what you're doing. Zsh treats double-quoted or unquoted variables like Bash treats double-quoted ones, so to replicate the behavior of Zsh you do want to quote in Bash for sure.

The following snippet outputs the same string (the one you called "correct") in Bash and in Zsh:

signature=$'get\ndbs\n\nmon, 27 apr 2020 16:11:57 gmt\n\n'
masterKey="t7ejJOwk0HEgkeYCm9v3n8vNwVaW27uriUmTTc3JcBtwqHBfwqO1tAJqKOBpeivurzPl1DxsNFUehEQN5lzGRw=="
hashedSignature="$(printf '%s' "$signature" | openssl dgst -sha256 -mac hmac -macopt hexkey:"$(printf '%s' "$masterKey" | base64 --decode | hexdump -v -e '/1 "%02x"')" -binary | base64)"
echo "Hashed signature: $hashedSignature"