mirrors-c
/
git
mirror of https://github.com/git/git.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity Software Blog

Merge branch 'nd/shallow-clone' 

Fetching from a shallow-cloned repository used to be forbidden,
primarily because the codepaths involved were not carefully vetted
and we did not bother supporting such usage. This attempts to allow
object transfer out of a shallow-cloned repository in a controlled
way (i.e. the receiver become a shallow repository with truncated
history).

* nd/shallow-clone: (31 commits)
  t5537: fix incorrect expectation in test case 10
  shallow: remove unused code
  send-pack.c: mark a file-local function static
  git-clone.txt: remove shallow clone limitations
  prune: clean .git/shallow after pruning objects
  clone: use git protocol for cloning shallow repo locally
  send-pack: support pushing from a shallow clone via http
  receive-pack: support pushing to a shallow clone via http
  smart-http: support shallow fetch/clone
  remote-curl: pass ref SHA-1 to fetch-pack as well
  send-pack: support pushing to a shallow clone
  receive-pack: allow pushes that update .git/shallow
  connected.c: add new variant that runs with --shallow-file
  add GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
  receive/send-pack: support pushing from a shallow clone
  receive-pack: reorder some code in unpack()
  fetch: add --update-shallow to accept refs that update .git/shallow
  upload-pack: make sure deepening preserves shallow roots
  fetch: support fetching from a shallow repository
  clone: support remote shallow repository
  ...
This commit is contained in:
Junio C Hamano 2014-01-17 12:21:14 -08:00
parent d8cf714c0e 3b32a7ca90
commit 92251b1b5b
36 changed files with 1535 additions and 168 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
Documentation/config.txt
View File

@ -2030,6 +2030,10 @@ receive.updateserverinfo::
If set to true, git-receive-pack will run git-update-server-info
after receiving data from git-push and updating refs.
receive.shallowupdate::
If set to true, .git/shallow can be updated when new refs
require new shallow roots. Otherwise those refs are rejected.
remote.pushdefault::
The remote to push to by default. Overrides
`branch.<name>.remote` for all branches, and is overridden by

14
Documentation/fetch-options.txt
View File

@ -14,8 +14,18 @@
branch history. Tags for the deepened commits are not fetched.
--unshallow::
Convert a shallow repository to a complete one, removing all
the limitations imposed by shallow repositories.
If the source repository is complete, convert a shallow
repository to a complete one, removing all the limitations
imposed by shallow repositories.
+
If the source repository is shallow, fetch as much as possible so that
the current repository has the same history as the source repository.
--update-shallow::
By default when fetching from a shallow repository,
`git fetch` refuses refs that require updating
.git/shallow. This option updates .git/shallow and accept such
refs.
ifndef::git-pull[]
--dry-run::

7
Documentation/git-clone.txt
View File

@ -181,12 +181,7 @@ objects from the source repository into a pack in the cloned repository.
--depth <depth>::
Create a 'shallow' clone with a history truncated to the
specified number of revisions. A shallow repository has a
number of limitations (you cannot clone or fetch from
it, nor push from nor into it), but is adequate if you
are only interested in the recent history of a large project
with a long history, and would want to send in fixes
as patches.
specified number of revisions.
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,

2
Documentation/git-prune.txt
View File

@ -24,6 +24,8 @@ objects unreachable from any of these head objects from the object database.
In addition, it
prunes the unpacked objects that are also found in packs by
running 'git prune-packed'.
It also removes entries from .git/shallow that are not reachable by
any ref.
Note that unreachable, packed objects will remain. If this is
not desired, see linkgit:git-repack[1].

7
Documentation/gitremote-helpers.txt
View File

@ -437,6 +437,13 @@ set by Git if the remote helper has the 'option' capability.
'option check-connectivity' \{'true'|'false'\}::
Request the helper to check connectivity of a clone.
'option cloning \{'true'|'false'\}::
Notify the helper this is a clone request (i.e. the current
repository is guaranteed empty).
'option update-shallow \{'true'|'false'\}::
Allow to extend .git/shallow if the new refs require it.
SEE ALSO
--------
linkgit:git-remote[1]

7
Documentation/technical/pack-protocol.txt
View File

@ -161,6 +161,7 @@ MUST peel the ref if it's an annotated tag.
----
advertised-refs = (no-refs / list-of-refs)
*shallow
flush-pkt
no-refs = PKT-LINE(zero-id SP "capabilities^{}"
@ -174,6 +175,8 @@ MUST peel the ref if it's an annotated tag.
other-tip = obj-id SP refname LF
other-peeled = obj-id SP refname "^{}" LF
shallow = PKT-LINE("shallow" SP obj-id)
capability-list = capability *(SP capability)
capability = 1*(LC_ALPHA / DIGIT / "-" / "_")
LC_ALPHA = %x61-7A
@ -461,7 +464,9 @@ contain all the objects that the server will need to complete the new
references.
----
update-request = command-list [pack-file]
update-request = *shallow command-list [pack-file]
shallow = PKT-LINE("shallow" SP obj-id)
command-list = PKT-LINE(command NUL capability-list LF)
*PKT-LINE(command LF)

18
builtin/clone.c
View File

@ -252,6 +252,12 @@ static int add_one_reference(struct string_list_item *item, void *cb_data)
die(_("reference repository '%s' is not a local repository."),
item->string);
if (!access(mkpath("%s/shallow", ref_git), F_OK))
die(_("reference repository '%s' is shallow"), item->string);
if (!access(mkpath("%s/info/grafts", ref_git), F_OK))
die(_("reference repository '%s' is grafted"), item->string);
strbuf_addf(&alternate, "%s/objects", ref_git);
add_to_alternates_file(alternate.buf);
strbuf_release(&alternate);
@ -791,8 +797,15 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
else
repo = repo_name;
is_local = option_local != 0 && path && !is_bundle;
if (is_local && option_depth)
warning(_("--depth is ignored in local clones; use file:// instead."));
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, ignoring --local"));
is_local = 0;
}
}
if (option_local > 0 && !is_local)
warning(_("--local is ignored"));
@ -887,6 +900,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
remote = remote_get(option_origin);
transport = transport_get(remote, remote->url[0]);
transport->cloning = 1;
if (!transport->get_refs_list || (!is_local && !transport->fetch))
die(_("Don't know how to clone %s"), transport->url);

23
builtin/fetch-pack.c
View File

@ -3,6 +3,7 @@
#include "fetch-pack.h"
#include "remote.h"
#include "connect.h"
#include "sha1-array.h"
static const char fetch_pack_usage[] =
"git fetch-pack [--all] [--stdin] [--quiet|-q] [--keep|-k] [--thin] "
@ -13,6 +14,13 @@ static void add_sought_entry_mem(struct ref ***sought, int *nr, int *alloc,
const char *name, int namelen)
{
struct ref *ref = xcalloc(1, sizeof(*ref) + namelen + 1);
unsigned char sha1[20];
if (namelen > 41 && name[40] == ' ' && !get_sha1_hex(name, sha1)) {
hashcpy(ref->old_sha1, sha1);
name += 41;
namelen -= 41;
}
memcpy(ref->name, name, namelen);
ref->name[namelen] = '\0';
@ -39,6 +47,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix)
char **pack_lockfile_ptr = NULL;
struct child_process *conn;
struct fetch_pack_args args;
struct sha1_array shallow = SHA1_ARRAY_INIT;
packet_trace_identity("fetch-pack");
@ -110,6 +119,14 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix)
args.check_self_contained_and_connected = 1;
continue;
}
if (!strcmp("--cloning", arg)) {
args.cloning = 1;
continue;
}
if (!strcmp("--update-shallow", arg)) {
args.update_shallow = 1;
continue;
}
usage(fetch_pack_usage);
}
@ -158,10 +175,10 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix)
if (!conn)
return args.diag_url ? 0 : 1;
}
get_remote_heads(fd[0], NULL, 0, &ref, 0, NULL);
get_remote_heads(fd[0], NULL, 0, &ref, 0, NULL, &shallow);
ref = fetch_pack(&args, fd, conn, ref, dest,
sought, nr_sought, pack_lockfile_ptr);
ref = fetch_pack(&args, fd, conn, ref, dest, sought, nr_sought,
&shallow, pack_lockfile_ptr);
if (pack_lockfile) {
printf("lock %s\n", pack_lockfile);
fflush(stdout);

15
builtin/fetch.c
View File

@ -36,7 +36,7 @@ static int prune = -1; /* unspecified */
static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity;
static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
static int tags = TAGS_DEFAULT, unshallow;
static int tags = TAGS_DEFAULT, unshallow, update_shallow;
static const char *depth;
static const char *upload_pack;
static struct strbuf default_rla = STRBUF_INIT;
@ -105,6 +105,8 @@ static struct option builtin_fetch_options[] = {
{ OPTION_STRING, 0, "recurse-submodules-default",
&recurse_submodules_default, NULL,
N_("default mode for recursion"), PARSE_OPT_HIDDEN },
OPT_BOOL(0, "update-shallow", &update_shallow,
N_("accept refs that update .git/shallow")),
OPT_END()
};
@ -524,6 +526,8 @@ static int iterate_ref_map(void *cb_data, unsigned char sha1[20])
struct ref **rm = cb_data;
struct ref *ref = *rm;
while (ref && ref->status == REF_STATUS_REJECT_SHALLOW)
ref = ref->next;
if (!ref)
return -1; /* end of the list */
*rm = ref->next;
@ -570,6 +574,13 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
struct ref *ref = NULL;
const char *merge_status_marker = "";
if (rm->status == REF_STATUS_REJECT_SHALLOW) {
if (want_status == FETCH_HEAD_MERGE)
warning(_("reject %s because shallow roots are not allowed to be updated"),
rm->peer_ref ? rm->peer_ref->name : rm->name);
continue;
}
commit = lookup_commit_reference_gently(rm->old_sha1, 1);
if (!commit)
rm->fetch_head_status = FETCH_HEAD_NOT_FOR_MERGE;
@ -798,6 +809,8 @@ static struct transport *prepare_transport(struct remote *remote)
set_option(transport, TRANS_OPT_KEEP, "yes");
if (depth)
set_option(transport, TRANS_OPT_DEPTH, depth);
if (update_shallow)
set_option(transport, TRANS_OPT_UPDATE_SHALLOW, "yes");
return transport;
}

1
builtin/gc.c
View File

@ -16,6 +16,7 @@
#include "run-command.h"
#include "sigchain.h"
#include "argv-array.h"
#include "commit.h"
#define FAILED_RUN "failed to run %s"

4
builtin/prune.c
View File

@ -180,5 +180,9 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
s = mkpathdup("%s/pack", get_object_directory());
remove_temporary_files(s);
free(s);
if (is_repository_shallow())
prune_shallow(show_only);
return 0;
}

313
builtin/receive-pack.c
View File

@ -13,6 +13,7 @@
#include "string-list.h"
#include "sha1-array.h"
#include "connected.h"
#include "argv-array.h"
#include "version.h"
static const char receive_pack_usage[] = "git receive-pack <git-dir>";
@ -43,6 +44,8 @@ static int fix_thin = 1;
static const char *head_name;
static void *head_name_to_free;
static int sent_capabilities;
static int shallow_update;
static const char *alt_shallow_file;
static enum deny_action parse_deny_action(const char *var, const char *value)
{
@ -121,6 +124,11 @@ static int receive_pack_config(const char *var, const char *value, void *cb)
return 0;
}
if (strcmp(var, "receive.shallowupdate") == 0) {
shallow_update = git_config_bool(var, value);
return 0;
}
return git_default_config(var, value, cb);
}
@ -178,6 +186,8 @@ static void write_head_info(void)
if (!sent_capabilities)
show_ref("capabilities^{}", null_sha1);
advertise_shallow_grafts(1);
/* EOF */
packet_flush(1);
}
@ -187,6 +197,7 @@ struct command {
const char *error_string;
unsigned int skip_update:1,
did_not_exist:1;
int index;
unsigned char old_sha1[20];
unsigned char new_sha1[20];
char ref_name[FLEX_ARRAY]; /* more */
@ -418,7 +429,46 @@ static void refuse_unconfigured_deny_delete_current(void)
rp_error("%s", refuse_unconfigured_deny_delete_current_msg[i]);
}
static const char *update(struct command *cmd)
static int command_singleton_iterator(void *cb_data, unsigned char sha1[20]);
static int update_shallow_ref(struct command *cmd, struct shallow_info *si)
{
static struct lock_file shallow_lock;
struct sha1_array extra = SHA1_ARRAY_INIT;
const char *alt_file;
uint32_t mask = 1 << (cmd->index % 32);
int i;
trace_printf_key("GIT_TRACE_SHALLOW",
"shallow: update_shallow_ref %s\n", cmd->ref_name);
for (i = 0; i < si->shallow->nr; i++)
if (si->used_shallow[i] &&
(si->used_shallow[i][cmd->index / 32] & mask) &&
!delayed_reachability_test(si, i))
sha1_array_append(&extra, si->shallow->sha1[i]);
setup_alternate_shallow(&shallow_lock, &alt_file, &extra);
if (check_shallow_connected(command_singleton_iterator,
0, cmd, alt_file)) {
rollback_lock_file(&shallow_lock);
sha1_array_clear(&extra);
return -1;
}
commit_lock_file(&shallow_lock);
/*
* Make sure setup_alternate_shallow() for the next ref does
* not lose these new roots..
*/
for (i = 0; i < extra.nr; i++)
register_shallow(extra.sha1[i]);
si->shallow_ref[cmd->index] = 0;
sha1_array_clear(&extra);
return 0;
}
static const char *update(struct command *cmd, struct shallow_info *si)
{
const char *name = cmd->ref_name;
struct strbuf namespaced_name_buf = STRBUF_INIT;
@ -526,6 +576,10 @@ static const char *update(struct command *cmd)
return NULL; /* good */
}
else {
if (shallow_update && si->shallow_ref[cmd->index] &&
update_shallow_ref(cmd, si))
return "shallow error";
lock = lock_any_ref_for_update(namespaced_name, old_sha1,
0, NULL);
if (!lock) {
@ -666,12 +720,16 @@ static int command_singleton_iterator(void *cb_data, unsigned char sha1[20])
return 0;
}
static void set_connectivity_errors(struct command *commands)
static void set_connectivity_errors(struct command *commands,
struct shallow_info *si)
{
struct command *cmd;
for (cmd = commands; cmd; cmd = cmd->next) {
struct command *singleton = cmd;
if (shallow_update && si->shallow_ref[cmd->index])
/* to be checked in update_shallow_ref() */
continue;
if (!check_everything_connected(command_singleton_iterator,
0, &singleton))
continue;
@ -679,18 +737,26 @@ static void set_connectivity_errors(struct command *commands)
}
}
struct iterate_data {
struct command *cmds;
struct shallow_info *si;
};
static int iterate_receive_command_list(void *cb_data, unsigned char sha1[20])
{
struct command **cmd_list = cb_data;
struct iterate_data *data = cb_data;
struct command **cmd_list = &data->cmds;
struct command *cmd = *cmd_list;
while (cmd) {
if (!is_null_sha1(cmd->new_sha1)) {
for (; cmd; cmd = cmd->next) {
if (shallow_update && data->si->shallow_ref[cmd->index])
/* to be checked in update_shallow_ref() */
continue;
if (!is_null_sha1(cmd->new_sha1) && !cmd->skip_update) {
hashcpy(sha1, cmd->new_sha1);
*cmd_list = cmd->next;
return 0;
}
cmd = cmd->next;
}
*cmd_list = NULL;
return -1; /* end of list */
@ -710,10 +776,14 @@ static void reject_updates_to_hidden(struct command *commands)
}
}
static void execute_commands(struct command *commands, const char *unpacker_error)
static void execute_commands(struct command *commands,
const char *unpacker_error,
struct shallow_info *si)
{
int checked_connectivity;
struct command *cmd;
unsigned char sha1[20];
struct iterate_data data;
if (unpacker_error) {
for (cmd = commands; cmd; cmd = cmd->next)
@ -721,10 +791,10 @@ static void execute_commands(struct command *commands, const char *unpacker_erro
return;
}
cmd = commands;
if (check_everything_connected(iterate_receive_command_list,
0, &cmd))
set_connectivity_errors(commands);
data.cmds = commands;
data.si = si;
if (check_everything_connected(iterate_receive_command_list, 0, &data))
set_connectivity_errors(commands, si);
reject_updates_to_hidden(commands);
@ -741,6 +811,7 @@ static void execute_commands(struct command *commands, const char *unpacker_erro
free(head_name_to_free);
head_name = head_name_to_free = resolve_refdup("HEAD", sha1, 0, NULL);
checked_connectivity = 1;
for (cmd = commands; cmd; cmd = cmd->next) {
if (cmd->error_string)
continue;
@ -748,11 +819,26 @@ static void execute_commands(struct command *commands, const char *unpacker_erro
if (cmd->skip_update)
continue;
cmd->error_string = update(cmd);
cmd->error_string = update(cmd, si);
if (shallow_update && !cmd->error_string &&
si->shallow_ref[cmd->index]) {
error("BUG: connectivity check has not been run on ref %s",
cmd->ref_name);
checked_connectivity = 0;
}
}
if (shallow_update) {
if (!checked_connectivity)
error("BUG: run 'git fsck' for safety.\n"
"If there are errors, try to remove "
"the reported refs above");
if (alt_shallow_file && *alt_shallow_file)
unlink(alt_shallow_file);
}
}
static struct command *read_head_info(void)
static struct command *read_head_info(struct sha1_array *shallow)
{
struct command *commands = NULL;
struct command **p = &commands;
@ -766,6 +852,14 @@ static struct command *read_head_info(void)
line = packet_read_line(0, &len);
if (!line)
break;
if (len == 48 && starts_with(line, "shallow ")) {
if (get_sha1_hex(line + 8, old_sha1))
die("protocol error: expected shallow sha, got '%s'", line + 8);
sha1_array_append(shallow, old_sha1);
continue;
}
if (len < 83 ||
line[40] != ' ' ||
line[81] != ' ' ||
@ -817,11 +911,14 @@ static const char *parse_pack_header(struct pack_header *hdr)
static const char *pack_lockfile;
static const char *unpack(int err_fd)
static const char *unpack(int err_fd, struct shallow_info *si)
{
struct pack_header hdr;
struct argv_array av = ARGV_ARRAY_INIT;
const char *hdr_err;
int status;
char hdr_arg[38];
struct child_process child;
int fsck_objects = (receive_fsck_objects >= 0
? receive_fsck_objects
: transfer_fsck_objects >= 0
@ -838,72 +935,63 @@ static const char *unpack(int err_fd)
"--pack_header=%"PRIu32",%"PRIu32,
ntohl(hdr.hdr_version), ntohl(hdr.hdr_entries));
if (si->nr_ours || si->nr_theirs) {
alt_shallow_file = setup_temporary_shallow(si->shallow);
argv_array_pushl(&av, "--shallow-file", alt_shallow_file, NULL);
}
memset(&child, 0, sizeof(child));
if (ntohl(hdr.hdr_entries) < unpack_limit) {
int code, i = 0;
struct child_process child;
const char *unpacker[5];
unpacker[i++] = "unpack-objects";
argv_array_pushl(&av, "unpack-objects", hdr_arg, NULL);
if (quiet)
unpacker[i++] = "-q";
argv_array_push(&av, "-q");
if (fsck_objects)
unpacker[i++] = "--strict";
unpacker[i++] = hdr_arg;
unpacker[i++] = NULL;
memset(&child, 0, sizeof(child));
child.argv = unpacker;
argv_array_push(&av, "--strict");
child.argv = av.argv;
child.no_stdout = 1;
child.err = err_fd;
child.git_cmd = 1;
code = run_command(&child);
if (!code)
return NULL;
return "unpack-objects abnormal exit";
status = run_command(&child);
if (status)
return "unpack-objects abnormal exit";
} else {
const char *keeper[7];
int s, status, i = 0;
int s;
char keep_arg[256];
struct child_process ip;
s = sprintf(keep_arg, "--keep=receive-pack %"PRIuMAX" on ", (uintmax_t) getpid());
if (gethostname(keep_arg + s, sizeof(keep_arg) - s))
strcpy(keep_arg + s, "localhost");
keeper[i++] = "index-pack";
keeper[i++] = "--stdin";
argv_array_pushl(&av, "index-pack",
"--stdin", hdr_arg, keep_arg, NULL);
if (fsck_objects)
keeper[i++] = "--strict";
argv_array_push(&av, "--strict");
if (fix_thin)
keeper[i++] = "--fix-thin";
keeper[i++] = hdr_arg;
keeper[i++] = keep_arg;
keeper[i++] = NULL;
memset(&ip, 0, sizeof(ip));
ip.argv = keeper;
ip.out = -1;
ip.err = err_fd;
ip.git_cmd = 1;
status = start_command(&ip);
if (status) {
argv_array_push(&av, "--fix-thin");
child.argv = av.argv;
child.out = -1;
child.err = err_fd;
child.git_cmd = 1;
status = start_command(&child);
if (status)
return "index-pack fork failed";
}
pack_lockfile = index_pack_lockfile(ip.out);
close(ip.out);
status = finish_command(&ip);
if (!status) {
reprepare_packed_git();
return NULL;
}
return "index-pack abnormal exit";
pack_lockfile = index_pack_lockfile(child.out);
close(child.out);
status = finish_command(&child);
if (status)
return "index-pack abnormal exit";
reprepare_packed_git();
}
return NULL;
}
static const char *unpack_with_sideband(void)
static const char *unpack_with_sideband(struct shallow_info *si)
{
struct async muxer;
const char *ret;
if (!use_sideband)
return unpack(0);
return unpack(0, si);
memset(&muxer, 0, sizeof(muxer));
muxer.proc = copy_to_sideband;
@ -911,12 +999,101 @@ static const char *unpack_with_sideband(void)
if (start_async(&muxer))
return NULL;
ret = unpack(muxer.in);
ret = unpack(muxer.in, si);
finish_async(&muxer);
return ret;
}
static void prepare_shallow_update(struct command *commands,
struct shallow_info *si)
{
int i, j, k, bitmap_size = (si->ref->nr + 31) / 32;
si->used_shallow = xmalloc(sizeof(*si->used_shallow) *
si->shallow->nr);
assign_shallow_commits_to_refs(si, si->used_shallow, NULL);
si->need_reachability_test =
xcalloc(si->shallow->nr, sizeof(*si->need_reachability_test));
si->reachable =
xcalloc(si->shallow->nr, sizeof(*si->reachable));
si->shallow_ref = xcalloc(si->ref->nr, sizeof(*si->shallow_ref));
for (i = 0; i < si->nr_ours; i++)
si->need_reachability_test[si->ours[i]] = 1;
for (i = 0; i < si->shallow->nr; i++) {
if (!si->used_shallow[i])
continue;
for (j = 0; j < bitmap_size; j++) {
if (!si->used_shallow[i][j])
continue;
si->need_reachability_test[i]++;
for (k = 0; k < 32; k++)
if (si->used_shallow[i][j] & (1 << k))
si->shallow_ref[j * 32 + k]++;
}
/*
* true for those associated with some refs and belong
* in "ours" list aka "step 7 not done yet"
*/
si->need_reachability_test[i] =
si->need_reachability_test[i] > 1;
}
/*
* keep hooks happy by forcing a temporary shallow file via
* env variable because we can't add --shallow-file to every
* command. check_everything_connected() will be done with
* true .git/shallow though.
*/
setenv(GIT_SHALLOW_FILE_ENVIRONMENT, alt_shallow_file, 1);
}
static void update_shallow_info(struct command *commands,
struct shallow_info *si,
struct sha1_array *ref)
{
struct command *cmd;
int *ref_status;
remove_nonexistent_theirs_shallow(si);
if (!si->nr_ours && !si->nr_theirs) {
shallow_update = 0;
return;
}
for (cmd = commands; cmd; cmd = cmd->next) {
if (is_null_sha1(cmd->new_sha1))
continue;
sha1_array_append(ref, cmd->new_sha1);
cmd->index = ref->nr - 1;
}
si->ref = ref;
if (shallow_update) {
prepare_shallow_update(commands, si);
return;
}
ref_status = xmalloc(sizeof(*ref_status) * ref->nr);
assign_shallow_commits_to_refs(si, NULL, ref_status);
for (cmd = commands; cmd; cmd = cmd->next) {
if (is_null_sha1(cmd->new_sha1))
continue;
if (ref_status[cmd->index]) {
cmd->error_string = "shallow update not allowed";
cmd->skip_update = 1;
}
}
if (alt_shallow_file && *alt_shallow_file) {
unlink(alt_shallow_file);
alt_shallow_file = NULL;
}
free(ref_status);
}
static void report(struct command *commands, const char *unpack_status)
{
struct command *cmd;
@ -958,6 +1135,9 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
int i;
char *dir = NULL;
struct command *commands;
struct sha1_array shallow = SHA1_ARRAY_INIT;
struct sha1_array ref = SHA1_ARRAY_INIT;
struct shallow_info si;
packet_trace_identity("receive-pack");
@ -998,9 +1178,6 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
if (!enter_repo(dir, 0))
die("'%s' does not appear to be a git repository", dir);
if (is_repository_shallow())
die("attempt to push into a shallow repository");
git_config(receive_pack_config, NULL);
if (0 <= transfer_unpack_limit)
@ -1014,12 +1191,17 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
if (advertise_refs)
return 0;
if ((commands = read_head_info()) != NULL) {
if ((commands = read_head_info(&shallow)) != NULL) {
const char *unpack_status = NULL;
if (!delete_only(commands))
unpack_status = unpack_with_sideband();
execute_commands(commands, unpack_status);
prepare_shallow_info(&si, &shallow);
if (!si.nr_ours && !si.nr_theirs)
shallow_update = 0;
if (!delete_only(commands)) {
unpack_status = unpack_with_sideband(&si);
update_shallow_info(commands, &si, &ref);
}
execute_commands(commands, unpack_status, &si);
if (pack_lockfile)
unlink_or_warn(pack_lockfile);
if (report_status)
@ -1035,8 +1217,11 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
}
if (auto_update_server_info)
update_server_info(0);
clear_shallow_info(&si);
}
if (use_sideband)
packet_flush(1);
sha1_array_clear(&shallow);
sha1_array_clear(&ref);
return 0;
}

9
builtin/send-pack.c
View File

@ -10,6 +10,7 @@
#include "quote.h"
#include "transport.h"
#include "version.h"
#include "sha1-array.h"
static const char send_pack_usage[] =
"git send-pack [--all | --mirror] [--dry-run] [--force] [--receive-pack=<git-receive-pack>] [--verbose] [--thin] [<host>:]<directory> [<ref>...]\n"
@ -99,7 +100,8 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
const char *dest = NULL;
int fd[2];
struct child_process *conn;
struct extra_have_objects extra_have;
struct sha1_array extra_have = SHA1_ARRAY_INIT;
struct sha1_array shallow = SHA1_ARRAY_INIT;
struct ref *remote_refs, *local_refs;
int ret;
int helper_status = 0;
@ -228,9 +230,8 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)
args.verbose ? CONNECT_VERBOSE : 0);
}
memset(&extra_have, 0, sizeof(extra_have));
get_remote_heads(fd[0], NULL, 0, &remote_refs, REF_NORMAL, &extra_have);
get_remote_heads(fd[0], NULL, 0, &remote_refs, REF_NORMAL,
&extra_have, &shallow);
transport_verify_remote_names(nr_refspecs, refspecs);

3
cache.h
View File

@ -354,6 +354,7 @@ static inline enum object_type object_type(unsigned int mode)
#define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"
#define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
#define GRAFT_ENVIRONMENT "GIT_GRAFT_FILE"
#define GIT_SHALLOW_FILE_ENVIRONMENT "GIT_SHALLOW_FILE"
#define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
#define CONFIG_ENVIRONMENT "GIT_CONFIG"
#define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
@ -1243,6 +1244,8 @@ __attribute__((format (printf, 2, 3)))
extern void trace_argv_printf(const char **argv, const char *format, ...);
extern void trace_repo_setup(const char *prefix);
extern int trace_want(const char *key);
__attribute__((format (printf, 2, 3)))
extern void trace_printf_key(const char *key, const char *fmt, ...);
extern void trace_strbuf(const char *key, const struct strbuf *buf);
void packet_trace_identity(const char *prog);

37
commit.h
View File

@ -194,6 +194,8 @@ extern struct commit_list *get_octopus_merge_bases(struct commit_list *in);
/* largest positive number a signed 32-bit integer can contain */
#define INFINITE_DEPTH 0x7fffffff
struct sha1_array;
struct ref;
extern int register_shallow(const unsigned char *sha1);
extern int unregister_shallow(const unsigned char *sha1);
extern int for_each_commit_graft(each_commit_graft_fn, void *);
@ -201,11 +203,38 @@ extern int is_repository_shallow(void);
extern struct commit_list *get_shallow_commits(struct object_array *heads,
int depth, int shallow_flag, int not_shallow_flag);
extern void check_shallow_file_for_update(void);
extern void set_alternate_shallow_file(const char *path);
extern int write_shallow_commits(struct strbuf *out, int use_pack_protocol);
extern void set_alternate_shallow_file(const char *path, int override);
extern int write_shallow_commits(struct strbuf *out, int use_pack_protocol,
const struct sha1_array *extra);
extern void setup_alternate_shallow(struct lock_file *shallow_lock,
const char **alternate_shallow_file);
extern char *setup_temporary_shallow(void);
const char **alternate_shallow_file,
const struct sha1_array *extra);
extern char *setup_temporary_shallow(const struct sha1_array *extra);
extern void advertise_shallow_grafts(int);
struct shallow_info {
struct sha1_array *shallow;
int *ours, nr_ours;
int *theirs, nr_theirs;
struct sha1_array *ref;
/* for receive-pack */
uint32_t **used_shallow;
int *need_reachability_test;
int *reachable;
int *shallow_ref;
struct commit **commits;
int nr_commits;
};
extern void prepare_shallow_info(struct shallow_info *, struct sha1_array *);
extern void clear_shallow_info(struct shallow_info *);
extern void remove_nonexistent_theirs_shallow(struct shallow_info *);
extern void assign_shallow_commits_to_refs(struct shallow_info *info,
uint32_t **used,
int *ref_status);
extern int delayed_reachability_test(struct shallow_info *si, int c);
extern void prune_shallow(int show_only);
int is_descendant_of(struct commit *, struct commit_list *);
int in_merge_bases(struct commit *, struct commit *);

22
connect.c
View File

@ -8,6 +8,7 @@
#include "connect.h"
#include "url.h"
#include "string-list.h"
#include "sha1-array.h"
static char *server_capabilities;
static const char *parse_feature_value(const char *, const char *, int *);
@ -45,13 +46,6 @@ int check_ref_type(const struct ref *ref, int flags)
return check_ref(ref->name, strlen(ref->name), flags);
}
static void add_extra_have(struct extra_have_objects *extra, unsigned char *sha1)
{
ALLOC_GROW(extra->array, extra->nr + 1, extra->alloc);
hashcpy(&(extra->array[extra->nr][0]), sha1);
extra->nr++;
}
static void die_initial_contact(int got_at_least_one_head)
{
if (got_at_least_one_head)
@ -122,7 +116,8 @@ static void annotate_refs_with_symref_info(struct ref *ref)
*/
struct ref **get_remote_heads(int in, char *src_buf, size_t src_len,
struct ref **list, unsigned int flags,
struct extra_have_objects *extra_have)
struct sha1_array *extra_have,
struct sha1_array *shallow_points)
{
struct ref **orig_list = list;
int got_at_least_one_head = 0;
@ -148,6 +143,15 @@ struct ref **get_remote_heads(int in, char *src_buf, size_t src_len,
if (len > 4 && starts_with(buffer, "ERR "))
die("remote error: %s", buffer + 4);
if (len == 48 && starts_with(buffer, "shallow ")) {
if (get_sha1_hex(buffer + 8, old_sha1))
die("protocol error: expected shallow sha-1, got '%s'", buffer + 8);
if (!shallow_points)
die("repository on the other end cannot be shallow");
sha1_array_append(shallow_points, old_sha1);
continue;
}
if (len < 42 || get_sha1_hex(buffer, old_sha1) || buffer[40] != ' ')
die("protocol error: expected sha/ref, got '%s'", buffer);
name = buffer + 41;
@ -160,7 +164,7 @@ struct ref **get_remote_heads(int in, char *src_buf, size_t src_len,
if (extra_have &&
name_len == 5 && !memcmp(".have", name, 5)) {
add_extra_have(extra_have, old_sha1);
sha1_array_append(extra_have, old_sha1);
continue;
}

42
connected.c
View File

@ -19,17 +19,17 @@ int check_everything_connected(sha1_iterate_fn fn, int quiet, void *cb_data)
*
* Returns 0 if everything is connected, non-zero otherwise.
*/
int check_everything_connected_with_transport(sha1_iterate_fn fn,
int quiet,
void *cb_data,
struct transport *transport)
static int check_everything_connected_real(sha1_iterate_fn fn,
int quiet,
void *cb_data,
struct transport *transport,
const char *shallow_file)
{
struct child_process rev_list;
const char *argv[] = {"rev-list", "--objects",
"--stdin", "--not", "--all", NULL, NULL};
const char *argv[9];
char commit[41];
unsigned char sha1[20];
int err = 0;
int err = 0, ac = 0;
struct packed_git *new_pack = NULL;
if (fn(cb_data, sha1))
@ -47,8 +47,18 @@ int check_everything_connected_with_transport(sha1_iterate_fn fn,
strbuf_release(&idx_file);
}
if (shallow_file) {
argv[ac++] = "--shallow-file";
argv[ac++] = shallow_file;
}
argv[ac++] = "rev-list";
argv[ac++] = "--objects";
argv[ac++] = "--stdin";
argv[ac++] = "--not";
argv[ac++] = "--all";
if (quiet)
argv[5] = "--quiet";
argv[ac++] = "--quiet";
argv[ac] = NULL;
memset(&rev_list, 0, sizeof(rev_list));
rev_list.argv = argv;
@ -92,3 +102,19 @@ int check_everything_connected_with_transport(sha1_iterate_fn fn,
sigchain_pop(SIGPIPE);
return finish_command(&rev_list) || err;
}
int check_everything_connected_with_transport(sha1_iterate_fn fn,
int quiet,
void *cb_data,
struct transport *transport)
{
return check_everything_connected_real(fn, quiet, cb_data,
transport, NULL);
}
int check_shallow_connected(sha1_iterate_fn fn, int quiet, void *cb_data,
const char *shallow_file)
{
return check_everything_connected_real(fn, quiet, cb_data,
NULL, shallow_file);
}

2
connected.h
View File

@ -18,6 +18,8 @@ typedef int (*sha1_iterate_fn)(void *, unsigned char [20]);
* Return 0 if Ok, non zero otherwise (i.e. some missing objects)
*/
extern int check_everything_connected(sha1_iterate_fn, int quiet, void *cb_data);
extern int check_shallow_connected(sha1_iterate_fn, int quiet, void *cb_data,
const char *shallow_file);
extern int check_everything_connected_with_transport(sha1_iterate_fn, int quiet,
void *cb_data,
struct transport *transport);

6
environment.c
View File

@ -10,6 +10,7 @@
#include "cache.h"
#include "refs.h"
#include "fmt-merge-msg.h"
#include "commit.h"
int trust_executable_bit = 1;
int trust_ctime = 1;
@ -97,6 +98,7 @@ const char * const local_repo_env[] = {
INDEX_ENVIRONMENT,
NO_REPLACE_OBJECTS_ENVIRONMENT,
GIT_PREFIX_ENVIRONMENT,
GIT_SHALLOW_FILE_ENVIRONMENT,
NULL
};
@ -124,6 +126,7 @@ static char *expand_namespace(const char *raw_namespace)
static void setup_git_env(void)
{
const char *gitfile;
const char *shallow_file;
git_dir = getenv(GIT_DIR_ENVIRONMENT);
if (!git_dir)
@ -147,6 +150,9 @@ static void setup_git_env(void)
read_replace_refs = 0;
namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
namespace_len = strlen(namespace);
shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
if (shallow_file)
set_alternate_shallow_file(shallow_file, 0);
}
int is_bare_repository(void)

131
fetch-pack.c
View File

@ -13,6 +13,7 @@
#include "transport.h"
#include "version.h"
#include "prio-queue.h"
#include "sha1-array.h"
static int transfer_unpack_limit = -1;
static int fetch_unpack_limit = -1;
@ -309,7 +310,7 @@ static int find_common(struct fetch_pack_args *args,
}
if (is_repository_shallow())
write_shallow_commits(&req_buf, 1);
write_shallow_commits(&req_buf, 1, NULL);
if (args->depth > 0)
packet_buf_write(&req_buf, "deepen %d", args->depth);
packet_buf_flush(&req_buf);
@ -772,6 +773,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,
int fd[2],
const struct ref *orig_ref,
struct ref **sought, int nr_sought,
struct shallow_info *si,
char **pack_lockfile)
{
struct ref *ref = copy_ref_list(orig_ref);
@ -848,7 +850,10 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,
if (args->stateless_rpc)
packet_flush(fd[1]);
if (args->depth > 0)
setup_alternate_shallow(&shallow_lock, &alternate_shallow_file);
setup_alternate_shallow(&shallow_lock, &alternate_shallow_file,
NULL);
else if (si->nr_ours || si->nr_theirs)
alternate_shallow_file = setup_temporary_shallow(si->shallow);
else
alternate_shallow_file = NULL;
if (get_pack(args, fd, pack_lockfile))
@ -922,14 +927,121 @@ static int remove_duplicates_in_refs(struct ref **ref, int nr)
return dst;
}
static void update_shallow(struct fetch_pack_args *args,
struct ref **sought, int nr_sought,
struct shallow_info *si)
{
struct sha1_array ref = SHA1_ARRAY_INIT;
int *status;
int i;
if (args->depth > 0 && alternate_shallow_file) {
if (*alternate_shallow_file == '\0') { /* --unshallow */
unlink_or_warn(git_path("shallow"));
rollback_lock_file(&shallow_lock);
} else
commit_lock_file(&shallow_lock);
return;
}
if (!si->shallow || !si->shallow->nr)
return;
if (alternate_shallow_file) {
/*
* The temporary shallow file is only useful for
* index-pack and unpack-objects because it may
* contain more roots than we want. Delete it.
*/
if (*alternate_shallow_file)
unlink(alternate_shallow_file);
free((char *)alternate_shallow_file);
}
if (args->cloning) {
/*
* remote is shallow, but this is a clone, there are
* no objects in repo to worry about. Accept any
* shallow points that exist in the pack (iow in repo
* after get_pack() and reprepare_packed_git())
*/
struct sha1_array extra = SHA1_ARRAY_INIT;
unsigned char (*sha1)[20] = si->shallow->sha1;
for (i = 0; i < si->shallow->nr; i++)
if (has_sha1_file(sha1[i]))
sha1_array_append(&extra, sha1[i]);
if (extra.nr) {
setup_alternate_shallow(&shallow_lock,
&alternate_shallow_file,
&extra);
commit_lock_file(&shallow_lock);
}
sha1_array_clear(&extra);
return;
}
if (!si->nr_ours && !si->nr_theirs)
return;
remove_nonexistent_theirs_shallow(si);
if (!si->nr_ours && !si->nr_theirs)
return;
for (i = 0; i < nr_sought; i++)
sha1_array_append(&ref, sought[i]->old_sha1);
si->ref = &ref;
if (args->update_shallow) {
/*
* remote is also shallow, .git/shallow may be updated
* so all refs can be accepted. Make sure we only add
* shallow roots that are actually reachable from new
* refs.
*/
struct sha1_array extra = SHA1_ARRAY_INIT;
unsigned char (*sha1)[20] = si->shallow->sha1;
assign_shallow_commits_to_refs(si, NULL, NULL);
if (!si->nr_ours && !si->nr_theirs) {
sha1_array_clear(&ref);
return;
}
for (i = 0; i < si->nr_ours; i++)
sha1_array_append(&extra, sha1[si->ours[i]]);
for (i = 0; i < si->nr_theirs; i++)
sha1_array_append(&extra, sha1[si->theirs[i]]);
setup_alternate_shallow(&shallow_lock,
&alternate_shallow_file,
&extra);
commit_lock_file(&shallow_lock);
sha1_array_clear(&extra);
sha1_array_clear(&ref);
return;
}
/*
* remote is also shallow, check what ref is safe to update
* without updating .git/shallow
*/
status = xcalloc(nr_sought, sizeof(*status));
assign_shallow_commits_to_refs(si, NULL, status);
if (si->nr_ours || si->nr_theirs) {
for (i = 0; i < nr_sought; i++)
if (status[i])
sought[i]->status = REF_STATUS_REJECT_SHALLOW;
}
free(status);
sha1_array_clear(&ref);
}
struct ref *fetch_pack(struct fetch_pack_args *args,
int fd[], struct child_process *conn,
const struct ref *ref,
const char *dest,
struct ref **sought, int nr_sought,
struct sha1_array *shallow,
char **pack_lockfile)
{
struct ref *ref_cpy;
struct shallow_info si;
fetch_pack_setup();
if (nr_sought)
@ -939,16 +1051,11 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
packet_flush(fd[1]);
die("no matching remote head");
}
ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought, pack_lockfile);
if (args->depth > 0 && alternate_shallow_file) {
if (*alternate_shallow_file == '\0') { /* --unshallow */
unlink_or_warn(git_path("shallow"));
rollback_lock_file(&shallow_lock);
} else
commit_lock_file(&shallow_lock);
}
prepare_shallow_info(&si, shallow);
ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
&si, pack_lockfile);
reprepare_packed_git();
update_shallow(args, sought, nr_sought, &si);
clear_shallow_info(&si);
return ref_cpy;
}