Git clean/smudge filters for ansible vault secrets

I am trying to setup clean/smudge filter in git to have automatic encrypting and decrypting of files containing secrets thru ansible-vault command.

Peculiarity of the ansible-vault command is that it is not idempotent (it creates a different binary each time it is invoked on the same data).

I started with the implementation suggested in this blog page. Unfortunately it did not work correctly, as whenever smudge is called (be it a git checkout , or just git status), the secret files looks as modified for git, even if it is not.

So I wondered if git would be comparing the binary he has in the index with the clean filtered current file, and I tried to build on those script like follows:

#!/bin/sh -x
# clean filter, it is invoked with %f

if [ ! -r "$HOME/.vault_password" ]; then
exit 1
fi

tmp=mktemp
cat > $tmp

# get the plain text from the binary in the index
tmphead=mktemp
git show HEAD:$1 > $tmphead
contenthead=echo "embedded" | ansible-vault view $tmphead --vault-password-file=$HOME/.vault_password
export PAGER=cat
echo -n "$contenthead" | tee $tmphead

# if current and index plain text version differ
if [ "md5sum $tmp | cut -d' ' -f1" != "md5sum $tmphead | cut -d' ' -f1" ]; then
tmpcrypt=mktemp
cp $tmp $tmpcrypt
# generate a new crypted blob
echo "embedded" | ansible-vault encrypt $tmpcrypt --vault-password-file=$HOME/.vault_password > /dev/null 2>&1
cat "$tmpcrypt"
else
# just return the HEAD version
cat "$tmphead"
fi

rm $tmp $tmphead $tmpcrypt


The difference here is that it tries to compare the current and HEAD versions of the plain text (unencrypted) secret files, and only in case they differ output a new binary blob encrypted with ansible-vault.

Unfortunately, after this change git continues to think the secret file is always modified. Even after git adding the file again, so that the git blob is computed, git thinks the file is different and let the change go into the commit. Note that git diff return empty changes,

The problem here is caused by the random salt in ansible-vault encryption. You can hack VaultEditor class to pass the salt to it from an argument in ansible-vault. The random salt is generated in lib/ansible/parsing/vault/__init__.py on this line. It is called from lib/ansible/cli/vault.py where you can easily add the argument for fixed salt. If you do change it, please submit an upstream patch to Ansible, I'd love to use it.

This problem is further discussed here on hacker news. And there are other implementations with tools that take fixed salt, namely gitcrypt, transcrypt. Here is also a link to one more implementation using ansible-vault called ansible-vault-tools, but this one has the same salt issue as far as I know.

StackExchange DevOps Q#106, answer score: 9