refs: implement reference transaction hook

The low-level reference transactions used to update references are
currently completely opaque to the user. While certainly desirable in
most usecases, there are some which might want to hook into the
transaction to observe all queued reference updates as well as observing
the abortion or commit of a prepared transaction.

One such usecase would be to have a set of replicas of a given Git
repository, where we perform Git operations on all of the repositories
at once and expect the outcome to be the same in all of them. While
there exist hooks already for a certain subset of Git commands that
could be used to implement a voting mechanism for this, many others
currently don't have any mechanism for this.

The above scenario is the motivation for the new "reference-transaction"
hook that reaches directly into Git's reference transaction mechanism.
The hook receives as parameter the current state the transaction was
moved to ("prepared", "committed" or "aborted") and gets via its
standard input all queued reference updates. While the exit code gets
ignored in the "committed" and "aborted" states, a non-zero exit code in
the "prepared" state will cause the transaction to be aborted
prematurely.

Given the usecase described above, a voting mechanism can now be
implemented via this hook: as soon as it gets called, it will take all
of stdin and use it to cast a vote to a central service. When all
replicas of the repository agree, the hook will exit with zero,
otherwise it will abort the transaction by returning non-zero. The most
important upside is that this will catch _all_ commands writing
references at once, allowing to implement strong consistency for
reference updates via a single mechanism.

In order to test the impact on the case where we don't have any
"reference-transaction" hook installed in the repository, this commit
introduce two new performance tests for git-update-refs(1). Run against
an empty repository, it produces the following results:

  Test                         origin/master     HEAD
  --------------------------------------------------------------------
  1400.2: update-ref           2.70(2.10+0.71)   2.71(2.10+0.73) +0.4%
  1400.3: update-ref --stdin   0.21(0.09+0.11)   0.21(0.07+0.14) +0.0%

The performance test p1400.2 creates, updates and deletes a branch a
thousand times, thus averaging runtime of git-update-refs over 3000
invocations. p1400.3 instead calls `git-update-refs --stdin` three times
and queues a thousand creations, updates and deletes respectively.

As expected, p1400.3 consistently shows no noticeable impact, as for
each batch of updates there's a single call to access(3P) for the
negative hook lookup. On the other hand, for p1400.2, one can see an
impact caused by this patchset. But doing five runs of the performance
tests where each one was run with GIT_PERF_REPEAT_COUNT=10, the overhead
ranged from -1.5% to +1.1%. These inconsistent performance numbers can
be explained by the overhead of spawning 3000 processes. This shows that
the overhead of assembling the hook path and executing access(3P) once
to check if it's there is mostly outweighed by the operating system's
overhead.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pull/763/merge
Patrick Steinhardt 2 years ago committed by Junio C Hamano
parent 101b3204f3
commit 6754159767
  1. 29
      Documentation/githooks.txt
  2. 76
      refs.c
  3. 32
      t/perf/p1400-update-ref.sh
  4. 109
      t/t1416-ref-transaction-hooks.sh

@ -404,6 +404,35 @@ Both standard output and standard error output are forwarded to
`git send-pack` on the other end, so you can simply `echo` messages
for the user.
ref-transaction
~~~~~~~~~~~~~~~
This hook is invoked by any Git command that performs reference
updates. It executes whenever a reference transaction is prepared,
committed or aborted and may thus get called multiple times.
The hook takes exactly one argument, which is the current state the
given reference transaction is in:
- "prepared": All reference updates have been queued to the
transaction and references were locked on disk.
- "committed": The reference transaction was committed and all
references now have their respective new value.
- "aborted": The reference transaction was aborted, no changes
were performed and the locks have been released.
For each reference update that was added to the transaction, the hook
receives on standard input a line of the format:
<old-value> SP <new-value> SP <ref-name> LF
The exit status of the hook is ignored for any state except for the
"prepared" state. In the "prepared" state, a non-zero exit status will
cause the transaction to be aborted. The hook will not be called with
"aborted" state in that case.
push-to-checkout
~~~~~~~~~~~~~~~~

@ -9,6 +9,7 @@
#include "iterator.h"
#include "refs.h"
#include "refs/refs-internal.h"
#include "run-command.h"
#include "object-store.h"
#include "object.h"
#include "tag.h"
@ -16,6 +17,7 @@
#include "worktree.h"
#include "argv-array.h"
#include "repository.h"
#include "sigchain.h"
/*
* List of all available backends
@ -1986,10 +1988,65 @@ int ref_update_reject_duplicates(struct string_list *refnames,
return 0;
}
static const char hook_not_found;
static const char *hook;
static int run_transaction_hook(struct ref_transaction *transaction,
const char *state)
{
struct child_process proc = CHILD_PROCESS_INIT;
struct strbuf buf = STRBUF_INIT;
int ret = 0, i;
if (hook == &hook_not_found)
return ret;
if (!hook)
hook = find_hook("reference-transaction");
if (!hook) {
hook = &hook_not_found;
return ret;
}
argv_array_pushl(&proc.args, hook, state, NULL);
proc.in = -1;
proc.stdout_to_stderr = 1;
proc.trace2_hook_name = "reference-transaction";
ret = start_command(&proc);
if (ret)
return ret;
sigchain_push(SIGPIPE, SIG_IGN);
for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];
strbuf_reset(&buf);
strbuf_addf(&buf, "%s %s %s\n",
oid_to_hex(&update->old_oid),
oid_to_hex(&update->new_oid),
update->refname);
if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
if (errno != EPIPE)
ret = -1;
break;
}
}
close(proc.in);
sigchain_pop(SIGPIPE);
strbuf_release(&buf);
ret |= finish_command(&proc);
return ret;
}
int ref_transaction_prepare(struct ref_transaction *transaction,
struct strbuf *err)
{
struct ref_store *refs = transaction->ref_store;
int ret;
switch (transaction->state) {
case REF_TRANSACTION_OPEN:
@ -2012,7 +2069,17 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
return -1;
}
return refs->be->transaction_prepare(refs, transaction, err);
ret = refs->be->transaction_prepare(refs, transaction, err);
if (ret)
return ret;
ret = run_transaction_hook(transaction, "prepared");
if (ret) {
ref_transaction_abort(transaction, err);
die(_("ref updates aborted by hook"));
}
return 0;
}
int ref_transaction_abort(struct ref_transaction *transaction,
@ -2036,6 +2103,8 @@ int ref_transaction_abort(struct ref_transaction *transaction,
break;
}
run_transaction_hook(transaction, "aborted");
ref_transaction_free(transaction);
return ret;
}
@ -2064,7 +2133,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,
break;
}
return refs->be->transaction_finish(refs, transaction, err);
ret = refs->be->transaction_finish(refs, transaction, err);
if (!ret)
run_transaction_hook(transaction, "committed");
return ret;
}
int refs_verify_refname_available(struct ref_store *refs,

@ -0,0 +1,32 @@
#!/bin/sh
test_description="Tests performance of update-ref"
. ./perf-lib.sh
test_perf_fresh_repo
test_expect_success "setup" '
test_commit PRE &&
test_commit POST &&
printf "create refs/heads/%d PRE\n" $(test_seq 1000) >create &&
printf "update refs/heads/%d POST PRE\n" $(test_seq 1000) >update &&
printf "delete refs/heads/%d POST\n" $(test_seq 1000) >delete
'
test_perf "update-ref" '
for i in $(test_seq 1000)
do
git update-ref refs/heads/branch PRE &&
git update-ref refs/heads/branch POST PRE &&
git update-ref -d refs/heads/branch
done
'
test_perf "update-ref --stdin" '
git update-ref --stdin <create &&
git update-ref --stdin <update &&
git update-ref --stdin <delete
'
test_done

@ -0,0 +1,109 @@
#!/bin/sh
test_description='reference transaction hooks'
. ./test-lib.sh
test_expect_success setup '
mkdir -p .git/hooks &&
test_commit PRE &&
test_commit POST &&
POST_OID=$(git rev-parse POST)
'
test_expect_success 'hook allows updating ref if successful' '
test_when_finished "rm .git/hooks/reference-transaction" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
echo "$*" >>actual
EOF
cat >expect <<-EOF &&
prepared
committed
EOF
git update-ref HEAD POST &&
test_cmp expect actual
'
test_expect_success 'hook aborts updating ref in prepared state' '
test_when_finished "rm .git/hooks/reference-transaction" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = prepared
then
exit 1
fi
EOF
test_must_fail git update-ref HEAD POST 2>err &&
test_i18ngrep "ref updates aborted by hook" err
'
test_expect_success 'hook gets all queued updates in prepared state' '
test_when_finished "rm .git/hooks/reference-transaction actual" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = prepared
then
while read -r line
do
printf "%s\n" "$line"
done >actual
fi
EOF
cat >expect <<-EOF &&
$ZERO_OID $POST_OID HEAD
$ZERO_OID $POST_OID refs/heads/master
EOF
git update-ref HEAD POST <<-EOF &&
update HEAD $ZERO_OID $POST_OID
update refs/heads/master $ZERO_OID $POST_OID
EOF
test_cmp expect actual
'
test_expect_success 'hook gets all queued updates in committed state' '
test_when_finished "rm .git/hooks/reference-transaction actual" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = committed
then
while read -r line
do
printf "%s\n" "$line"
done >actual
fi
EOF
cat >expect <<-EOF &&
$ZERO_OID $POST_OID HEAD
$ZERO_OID $POST_OID refs/heads/master
EOF
git update-ref HEAD POST &&
test_cmp expect actual
'
test_expect_success 'hook gets all queued updates in aborted state' '
test_when_finished "rm .git/hooks/reference-transaction actual" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = aborted
then
while read -r line
do
printf "%s\n" "$line"
done >actual
fi
EOF
cat >expect <<-EOF &&
$ZERO_OID $POST_OID HEAD
$ZERO_OID $POST_OID refs/heads/master
EOF
git update-ref --stdin <<-EOF &&
start
update HEAD POST $ZERO_OID
update refs/heads/master POST $ZERO_OID
abort
EOF
test_cmp expect actual
'
test_done
Loading…
Cancel
Save