From 96931572a72f753e66bb788eb854358806d9c185 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:08 +0000 Subject: [PATCH 01/33] t9700: accommodate for Windows paths Ever since fe53bbc9beb (Git.pm: Always set Repository to absolute path if autodetecting, 2009-05-07), the t9700 test _must_ fail on Windows because of that age-old Unix paths vs Windows paths problem. The underlying root cause is that Git cannot run with a regular Win32 variant of Perl, the assumption that every path is a Unix path is just too strong in Git's Perl code. As a consequence, Git for Windows is basically stuck with using the MSYS2 variant of Perl which uses a POSIX emulation layer (which is a friendly fork of Cygwin) _and_ a best-effort Unix <-> Windows paths conversion whenever crossing the boundary between MSYS2 and regular Win32 processes. It is best effort only, though, using heuristics to automagically convert correctly in most cases, but not in all cases. In the context of this here patch, this means that asking `git.exe` for the absolute path of the `.git/` directory will return a Win32 path because `git.exe` is a regular Win32 executable that has no idea about Unix-ish paths. But above-mentioned commit introduced a test that wants to verify that this path is identical to the one that the Git Perl module reports (which refuses to use Win32 paths and uses Unix-ish paths instead). Obviously, this must fail because no heuristics can kick in at that layer. This test failure has not even been caught when Git introduced Windows support in its CI definition in 2e90484eb4a (ci: add a Windows job to the Azure Pipelines definition, 2019-01-29), as all tests relying on Perl had to be disabled even from the start (because the CI runs would otherwise have resulted in prohibitively long runtimes, not because Windows is super slow per se, but because Git's test suite keeps insisting on using technology that requires a POSIX emulation layer, which _is_ super slow on Windows). To work around this failure, let's use the `cygpath` utility to convert the absolute `gitdir` path into the form that the Perl code expects. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t9700/test.pl | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/t/t9700/test.pl b/t/t9700/test.pl index 58a9b328d558f3..570b0c5680fc73 100755 --- a/t/t9700/test.pl +++ b/t/t9700/test.pl @@ -117,7 +117,12 @@ sub adjust_dirsep { unlink $tmpfile; # paths -is($r->repo_path, $abs_repo_dir . "/.git", "repo_path"); +my $abs_git_dir = $abs_repo_dir . "/.git"; +if ($^O eq 'msys' or $^O eq 'cygwin') { + $abs_git_dir = `cygpath -am "$abs_repo_dir/.git"`; + $abs_git_dir =~ s/\r?\n?$//; +} +is($r->repo_path, $abs_git_dir, "repo_path"); is($r->wc_path, $abs_repo_dir . "/", "wc_path"); is($r->wc_subdir, "", "wc_subdir initial"); $r->wc_chdir("directory1"); @@ -127,7 +132,7 @@ sub adjust_dirsep { # Object generation in sub directory chdir("directory2"); my $r2 = Git->repository(); -is($r2->repo_path, $abs_repo_dir . "/.git", "repo_path (2)"); +is($r2->repo_path, $abs_git_dir, "repo_path (2)"); is($r2->wc_path, $abs_repo_dir . "/", "wc_path (2)"); is($r2->wc_subdir, "directory2/", "wc_subdir initial (2)"); From 33ccd0f6ff191cb483c62b520ed4e3ec5e30c192 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:09 +0000 Subject: [PATCH 02/33] apply: symbolic links lack a "trustable executable bit" When 0482c32c334b (apply: ignore working tree filemode when !core.filemode, 2023-12-26) fixed `git apply` to stop warning about executable files, it inadvertently changed the code flow also for symbolic links and directories. Let's narrow the scope of the special `!trust_executable_git` code path to apply only to regular files. This is needed to let t4115.5(symlink escape when creating new files) pass on Windows when symbolic link support is enabled in the MSYS2 runtime. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- apply.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/apply.c b/apply.c index a2ceb3fb40d3b5..de5750354ad2f8 100644 --- a/apply.c +++ b/apply.c @@ -3779,7 +3779,7 @@ static int check_preimage(struct apply_state *state, if (*ce && !(*ce)->ce_mode) BUG("ce_mode == 0 for path '%s'", old_name); - if (trust_executable_bit) + if (trust_executable_bit || !S_ISREG(st->st_mode)) st_mode = ce_mode_from_stat(*ce, st->st_mode); else if (*ce) st_mode = (*ce)->ce_mode; From 78d52c8464f16fc2dd38d6f27e555298ba923a87 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:10 +0000 Subject: [PATCH 03/33] mingw: special-case `open(symlink, O_CREAT | O_EXCL)` The `_wopen()` function would gladly follow a symbolic link to a non-existent file and create it when given above-mentioned flags. Git expects the `open()` call to fail, though. So let's add yet another work-around to pretend that Windows behaves according to POSIX, see: https://pubs.opengroup.org/onlinepubs/007904875/functions/open.html#:~:text=If%20O_CREAT%20and%20O_EXCL%20are,set%2C%20the%20result%20is%20undefined. This is required to let t4115.8(--reject removes .rej symlink if it exists) pass on Windows when enabling the MSYS2 runtime's symbolic link support. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- compat/mingw.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/compat/mingw.c b/compat/mingw.c index 90ba5cea9d3ace..ba1b7b6dd1e6a7 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -629,6 +629,7 @@ int mingw_open (const char *filename, int oflags, ...) int fd, create = (oflags & (O_CREAT | O_EXCL)) == (O_CREAT | O_EXCL); wchar_t wfilename[MAX_PATH]; open_fn_t open_fn; + WIN32_FILE_ATTRIBUTE_DATA fdata; DECLARE_PROC_ADDR(ntdll.dll, NTSTATUS, NTAPI, RtlGetLastNtStatus, void); @@ -653,6 +654,19 @@ int mingw_open (const char *filename, int oflags, ...) else if (xutftowcs_path(wfilename, filename) < 0) return -1; + /* + * When `symlink` exists and is a symbolic link pointing to a + * non-existing file, `_wopen(symlink, O_CREAT | O_EXCL)` would + * create that file. Not what we want: Linux would say `EEXIST` + * in that instance, which is therefore what Git expects. + */ + if (create && + GetFileAttributesExW(wfilename, GetFileExInfoStandard, &fdata) && + (fdata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)) { + errno = EEXIST; + return -1; + } + fd = open_fn(wfilename, oflags, mode); /* From c8fb79c262116ab18e6d8e1961de60488abc4c74 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:11 +0000 Subject: [PATCH 04/33] t0001: handle `diff --no-index` gracefully The test case 're-init to move gitdir symlink' wants to compare the contents of `newdir/.git`, which is a symbolic link pointing to a file. However, `git diff --no-index`, which is used by `test_cmp` on Windows, does not resolve symlinks; It shows the symlink _target_ instead (with a file mode of 120000). That is totally unexpected by the test case, which as a consequence fails, meaning that it's a bug in the test case itself. Co-authored-by: Junio C Hamano Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t0001-init.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/t0001-init.sh b/t/t0001-init.sh index 618da080dc9ea9..e4d32bb4d259f6 100755 --- a/t/t0001-init.sh +++ b/t/t0001-init.sh @@ -425,7 +425,11 @@ test_expect_success SYMLINKS 're-init to move gitdir symlink' ' git init --separate-git-dir ../realgitdir ) && echo "gitdir: $(pwd)/realgitdir" >expected && - test_cmp expected newdir/.git && + case "$GIT_TEST_CMP" in + # `git diff --no-index` does not resolve symlinks + *--no-index*) cmp expected newdir/.git;; + *) test_cmp expected newdir/.git;; + esac && test_cmp expected newdir/here && test_path_is_dir realgitdir/refs ' From d2de5d909964974885d0cc81488eaf33e0fab6e3 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:12 +0000 Subject: [PATCH 05/33] t0301: another fix for Windows compatibility Just like 0fdcfa2f9f5 (t0301: fixes for windows compatibility, 2021-09-14) explained, we should not call `mkdir -m` in the test suite because that would fail on Windows. There was one forgotten instance of this which was hidden by a `SYMLINK` prerequisite. Currently, this prevents this test case from being executed on Windows, but with the upcoming support for symbolic links, it would become a problem. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t0301-credential-cache.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/t/t0301-credential-cache.sh b/t/t0301-credential-cache.sh index dc30289f7539ee..6f7cfd9e33f633 100755 --- a/t/t0301-credential-cache.sh +++ b/t/t0301-credential-cache.sh @@ -123,7 +123,8 @@ test_expect_success SYMLINKS 'use user socket if user directory is a symlink to rmdir \"\$HOME/dir/\" && rm \"\$HOME/.git-credential-cache\" " && - mkdir -p -m 700 "$HOME/dir/" && + mkdir -p "$HOME/dir/" && + chmod 700 "$HOME/dir/" && ln -s "$HOME/dir" "$HOME/.git-credential-cache" && check approve cache <<-\EOF && protocol=https From 14b033f692899dcb289f043c98936c298c7ada7d Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 5 Dec 2025 15:02:21 +0000 Subject: [PATCH 06/33] t0600: fix incomplete prerequisite for a test case The 'symref transaction supports symlinks' test case is guarded by the `SYMLINK` prerequisite because `core.prefersymlinkrefs = true` requires symbolic links to be supported. However, the `preferSymlinkRefs` feature is not supported on Windows, therefore this test case needs the `MINGW` prerequisite, too. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t0600-reffiles-backend.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh index b11126ed478129..74bfa2e9ba060d 100755 --- a/t/t0600-reffiles-backend.sh +++ b/t/t0600-reffiles-backend.sh @@ -467,7 +467,7 @@ test_expect_success POSIXPERM 'git reflog expire honors core.sharedRepository' ' esac ' -test_expect_success SYMLINKS 'symref transaction supports symlinks' ' +test_expect_success SYMLINKS,!MINGW 'symref transaction supports symlinks' ' test_when_finished "git symbolic-ref -d TEST_SYMREF_HEAD" && git update-ref refs/heads/new @ && test_config core.prefersymlinkrefs true && From c670ae34a3fef348a32cf4adf6cd9145699b0edd Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:14 +0000 Subject: [PATCH 07/33] t1006: accommodate for symlink support in MSYS2 The MSYS2 runtime (which inherits this trait from the Cygwin runtime, and which is used by Git for Windows' Bash to emulate POSIX functionality on Windows, the same Bash that is also used to run Git's test suite on Windows) has a mode where it can create native symbolic links on Windows. Naturally, this is a bit of a strange feature, given that Cygwin goes out of its way to support Unix-like paths even if no Win32 program understands those, and the symbolic links have to use Win32 paths instead (which Win32 programs understand very well). As a consequence, the symbolic link targets get normalized before the links are created. This results in certain quirks that Git's test suite is ill equipped to accommodate (because Git's test suite expects to be able to use Unix-like paths even on Windows). The test script t1006-cat-file.sh contains two prime examples, two test cases that need to skip a couple assertions because they are simply wrong in the context of Git for Windows. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t1006-cat-file.sh | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 1f61b666a7d382..0eee3bb8781b30 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -1048,18 +1048,28 @@ test_expect_success 'git cat-file --batch-check --follow-symlinks works for out- echo .. >>expect && echo HEAD:dir/subdir/out-of-repo-link-dir | git cat-file --batch-check --follow-symlinks >actual && test_cmp expect actual && - echo symlink 3 >expect && - echo ../ >>expect && + if test_have_prereq MINGW,SYMLINKS + then + test_write_lines "symlink 2" .. + else + test_write_lines "symlink 3" ../ + fi >expect && echo HEAD:dir/subdir/out-of-repo-link-dir-trailing | git cat-file --batch-check --follow-symlinks >actual && test_cmp expect actual ' test_expect_success 'git cat-file --batch-check --follow-symlinks works for symlinks with internal ..' ' - echo HEAD: | git cat-file --batch-check >expect && - echo HEAD:up-down | git cat-file --batch-check --follow-symlinks >actual && - test_cmp expect actual && - echo HEAD:up-down-trailing | git cat-file --batch-check --follow-symlinks >actual && - test_cmp expect actual && + if test_have_prereq !MINGW + then + # The `up-down` and `up-down-trailing` symlinks are normalized + # in MSYS in `winsymlinks` mode and are therefore in a + # different shape than Git expects them. + echo HEAD: | git cat-file --batch-check >expect && + echo HEAD:up-down | git cat-file --batch-check --follow-symlinks >actual && + test_cmp expect actual && + echo HEAD:up-down-trailing | git cat-file --batch-check --follow-symlinks >actual && + test_cmp expect actual + fi && echo HEAD:up-down-file | git cat-file --batch-check --follow-symlinks >actual && test_cmp found actual && echo symlink 7 >expect && From d265cdeb353beb5a3b9d6e96e43407a12d737904 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:15 +0000 Subject: [PATCH 08/33] t1305: skip symlink tests that do not apply to Windows In Git for Windows, the gitdir is canonicalized so that even when the gitdir is specified via a symbolic link, the `gitdir:` conditional include will only match the real directory path. Unfortunately, t1305 codifies a different behavior in two test cases, which are hereby skipped on Windows. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t1305-config-include.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/t1305-config-include.sh b/t/t1305-config-include.sh index 8ff2b0c232b485..6e51f892f320bb 100755 --- a/t/t1305-config-include.sh +++ b/t/t1305-config-include.sh @@ -286,7 +286,7 @@ test_expect_success SYMLINKS 'conditional include, relative path with symlinks' ) ' -test_expect_success SYMLINKS 'conditional include, gitdir matching symlink' ' +test_expect_success SYMLINKS,!MINGW 'conditional include, gitdir matching symlink' ' ln -s foo bar && ( cd bar && @@ -298,7 +298,7 @@ test_expect_success SYMLINKS 'conditional include, gitdir matching symlink' ' ) ' -test_expect_success SYMLINKS 'conditional include, gitdir matching symlink, icase' ' +test_expect_success SYMLINKS,!MINGW 'conditional include, gitdir matching symlink, icase' ' ( cd bar && echo "[includeIf \"gitdir/i:BAR/\"]path=bar8" >>.git/config && From 02ea262e1594748cbba62ec494482fcd990e6673 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:16 +0000 Subject: [PATCH 09/33] t6423: introduce Windows-specific handling for symlinking to /dev/null The device `/dev/null` does not exist on Windows, it's called `NUL` there. Calling `ln -s /dev/null my-symlink` in a symlink-enabled MSYS2 Bash will therefore literally link to a file or directory called `null` that is supposed to be in the current drive's top-level `dev` directory. Which typically does not exist. The test, however, really wants the created symbolic link to point to the NUL device. Let's instead use the `mklink` utility on Windows to perform that job, and keep using `ln -s /dev/null ` on non-Windows platforms. While at it, add the missing `SYMLINKS` prereq because this test _still_ would not pass on Windows before support for symbolic links is upstreamed from Git for Windows. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t6423-merge-rename-directories.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/t/t6423-merge-rename-directories.sh b/t/t6423-merge-rename-directories.sh index 533ac85dc83409..53535a8ebfc393 100755 --- a/t/t6423-merge-rename-directories.sh +++ b/t/t6423-merge-rename-directories.sh @@ -5158,13 +5158,18 @@ test_setup_12m () { git switch B && git rm dir/subdir/file && mkdir dir && - ln -s /dev/null dir/subdir && + if test_have_prereq MINGW + then + cmd //c 'mklink dir\subdir NUL' + else + ln -s /dev/null dir/subdir + fi && git add . && git commit -m "B" ) } -test_expect_success '12m: Change parent of renamed-dir to symlink on other side' ' +test_expect_success SYMLINKS '12m: Change parent of renamed-dir to symlink on other side' ' test_setup_12m && ( cd 12m && From a4c7170aa3a2166dac7ef7d65bae87bb397840ae Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 11:46:17 +0000 Subject: [PATCH 10/33] t7800: work around the MSYS path conversion on Windows Git's test suite's relies on Unix shell scripting, which is understandable, of course, given Git's firm roots (and indeed, ongoing focus) on Linux. This fact, combined with Unix shell scripting's natural habitat -- which is, naturally... *drumroll*... Unix -- often has unintended side effects, where developers expect the test suite to run in a Unix environment, which is an incorrect assumption. One instance of this problem can be observed in the 'difftool --dir-diff handles modified symlinks' test case in `t7800-difftool.sh`, which assumes that all absolute paths start with a forward slash. That assumption is incorrect in general, e.g. on Windows, where absolute paths have many shapes and forms, none of which starts with a forward slash. The only saving grace is that this test case is currently not run on Windows because of the `SYMLINK` prerequisite. However, I am currently working towards upstreaming symbolic link support from Git for Windows to upstream Git, which will put a crack into that saving grace. Let's change that test case so that it does not rely on absolute paths (which are passed to the "external command" `ls` as parameters and are therefore part of its output, and which the test case wants to filter out before verifying that the output is as expected) starting with a forward slash. Let's instead rely on the much more reliable fact that `ls` will output the path in a line that ends in a colon, and simply filter out those lines by matching said colon instead. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t7800-difftool.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh index 9b74db55634b16..bf0f67378dbb23 100755 --- a/t/t7800-difftool.sh +++ b/t/t7800-difftool.sh @@ -752,11 +752,11 @@ test_expect_success SYMLINKS 'difftool --dir-diff handles modified symlinks' ' c EOF git difftool --symlinks --dir-diff --extcmd ls >output && - grep -v ^/ output >actual && + grep -v ":\$" output >actual && test_cmp expect actual && git difftool --no-symlinks --dir-diff --extcmd ls >output && - grep -v ^/ output >actual && + grep -v ":\$" output >actual && test_cmp expect actual && # The left side contains symlink "c" that points to "b" @@ -786,11 +786,11 @@ test_expect_success SYMLINKS 'difftool --dir-diff handles modified symlinks' ' EOF git difftool --symlinks --dir-diff --extcmd ls >output && - grep -v ^/ output >actual && + grep -v ":\$" output >actual && test_cmp expect actual && git difftool --no-symlinks --dir-diff --extcmd ls >output && - grep -v ^/ output >actual && + grep -v ":\$" output >actual && test_cmp expect actual ' From df27101f012ba945d287b8b84b6b4a3710a8b067 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 15:33:45 +0000 Subject: [PATCH 11/33] mingw: do resolve symlinks in `getcwd()` As pointed out in https://github.com/git-for-windows/git/issues/1676, the `git rev-parse --is-inside-work-tree` command currently fails when the current directory's path contains symbolic links. The underlying reason for this bug is that `getcwd()` is supposed to resolve symbolic links, but our `mingw_getcwd()` implementation did not. We do have all the building blocks for that, though: the `GetFinalPathByHandleW()` function will resolve symbolic links. However, we only called that function if `GetLongPathNameW()` failed, for historical reasons: the latter function was supported for a long time, but the former API function was introduced only with Windows Vista, and we used to support also Windows XP. With that support having been dropped, we are free to call the symbolic link-resolving function right away. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- compat/mingw.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index f09b49ff21ddab..cf4f3c92e7a889 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -1239,18 +1239,16 @@ char *mingw_getcwd(char *pointer, int len) { wchar_t cwd[MAX_PATH], wpointer[MAX_PATH]; DWORD ret = GetCurrentDirectoryW(ARRAY_SIZE(cwd), cwd); + HANDLE hnd; if (!ret || ret >= ARRAY_SIZE(cwd)) { errno = ret ? ENAMETOOLONG : err_win_to_posix(GetLastError()); return NULL; } - ret = GetLongPathNameW(cwd, wpointer, ARRAY_SIZE(wpointer)); - if (!ret && GetLastError() == ERROR_ACCESS_DENIED) { - HANDLE hnd = CreateFileW(cwd, 0, - FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, - OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); - if (hnd == INVALID_HANDLE_VALUE) - return NULL; + hnd = CreateFileW(cwd, 0, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, + OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); + if (hnd != INVALID_HANDLE_VALUE) { ret = GetFinalPathNameByHandleW(hnd, wpointer, ARRAY_SIZE(wpointer), 0); CloseHandle(hnd); if (!ret || ret >= ARRAY_SIZE(wpointer)) @@ -1259,13 +1257,11 @@ char *mingw_getcwd(char *pointer, int len) return NULL; return pointer; } - if (!ret || ret >= ARRAY_SIZE(wpointer)) - return NULL; - if (GetFileAttributesW(wpointer) == INVALID_FILE_ATTRIBUTES) { + if (GetFileAttributesW(cwd) == INVALID_FILE_ATTRIBUTES) { errno = ENOENT; return NULL; } - if (xwcstoutf(pointer, wpointer, len) < 0) + if (xwcstoutf(pointer, cwd, len) < 0) return NULL; convert_slashes(pointer); return pointer; From 722e65dc9bd71384984406b493652a24f4ac61ce Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 16 Dec 2025 15:33:46 +0000 Subject: [PATCH 12/33] init: do parse _all_ core.* settings early In Git for Windows, `has_symlinks` is set to 0 by default. Therefore, we need to parse the config setting `core.symlinks` to know if it has been set to `true`. In `git init`, we must do that before copying the templates because they might contain symbolic links. Even if the support for symbolic links on Windows has not made it to upstream Git yet, we really should make sure that all the `core.*` settings are parsed before proceeding, as they might very well change the behavior of `git init` in a way the user intended. This fixes https://github.com/git-for-windows/git/issues/3414 Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- environment.c | 4 ++-- environment.h | 2 ++ setup.c | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/environment.c b/environment.c index a770b5921d9546..b65b85a01f18cf 100644 --- a/environment.c +++ b/environment.c @@ -324,8 +324,8 @@ static enum fsync_component parse_fsync_components(const char *var, const char * return (current & ~negative) | positive; } -static int git_default_core_config(const char *var, const char *value, - const struct config_context *ctx, void *cb) +int git_default_core_config(const char *var, const char *value, + const struct config_context *ctx, void *cb) { /* This needs a better name */ if (!strcmp(var, "core.filemode")) { diff --git a/environment.h b/environment.h index 51898c99cd1e45..e61f843fdbb637 100644 --- a/environment.h +++ b/environment.h @@ -106,6 +106,8 @@ const char *strip_namespace(const char *namespaced_ref); int git_default_config(const char *, const char *, const struct config_context *, void *); +int git_default_core_config(const char *var, const char *value, + const struct config_context *ctx, void *cb); /* * TODO: All the below state either explicitly or implicitly relies on diff --git a/setup.c b/setup.c index 3a6a048620dd7d..b723f8b33931bd 100644 --- a/setup.c +++ b/setup.c @@ -2693,7 +2693,7 @@ int init_db(const char *git_dir, const char *real_git_dir, * have set up the repository format such that we can evaluate * includeIf conditions correctly in the case of re-initialization. */ - repo_config(the_repository, platform_core_config, NULL); + repo_config(the_repository, git_default_core_config, NULL); safe_create_dir(the_repository, git_dir, 0); From f6b8e4a4c74f40a7f7fbde4cf024267f0734f85c Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 16 Dec 2025 15:33:47 +0000 Subject: [PATCH 13/33] strbuf_readlink(): avoid calling `readlink()` twice in corner-cases The `strbuf_readlink()` function calls `readlink()`` twice if the hint argument specifies the exact size of the link target (e.g. by passing stat.st_size as returned by `lstat()`). This is necessary because `readlink(..., hint) == hint` could mean that the buffer was too small. Use `hint + 1` as buffer size to prevent this. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- strbuf.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/strbuf.c b/strbuf.c index 6c3851a7f84d72..44a8f6a554ee43 100644 --- a/strbuf.c +++ b/strbuf.c @@ -578,12 +578,12 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint) while (hint < STRBUF_MAXLINK) { ssize_t len; - strbuf_grow(sb, hint); - len = readlink(path, sb->buf, hint); + strbuf_grow(sb, hint + 1); + len = readlink(path, sb->buf, hint + 1); if (len < 0) { if (errno != ERANGE) break; - } else if (len < hint) { + } else if (len <= hint) { strbuf_setlen(sb, len); return 0; } From 9ce11d9149f834bc150fa002762ab1142d8ec2d6 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 16 Dec 2025 15:33:48 +0000 Subject: [PATCH 14/33] strbuf_readlink(): support link targets that exceed PATH_MAX The `strbuf_readlink()` function refuses to read link targets that exceed PATH_MAX (even if a sufficient size was specified by the caller). As some platforms (*cough* Windows *cough*) support longer paths, remove this restriction (similar to `strbuf_getcwd()`). Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- strbuf.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/strbuf.c b/strbuf.c index 44a8f6a554ee43..fa4e30f112bb09 100644 --- a/strbuf.c +++ b/strbuf.c @@ -566,8 +566,6 @@ ssize_t strbuf_write(struct strbuf *sb, FILE *f) return sb->len ? fwrite(sb->buf, 1, sb->len, f) : 0; } -#define STRBUF_MAXLINK (2*PATH_MAX) - int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint) { size_t oldalloc = sb->alloc; @@ -575,7 +573,7 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint) if (hint < 32) hint = 32; - while (hint < STRBUF_MAXLINK) { + for (;;) { ssize_t len; strbuf_grow(sb, hint + 1); From 6f6fe02f5fe587ec9788f8a5a34281949d7b2ca1 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 16 Dec 2025 15:33:49 +0000 Subject: [PATCH 15/33] trim_last_path_component(): avoid hard-coding the directory separator Currently, this function hard-codes the directory separator as the forward slash. However, on Windows the backslash character is valid, too. And we want to call this function in the upcoming support for symlinks on Windows with the symlink targets (which naturally use the canonical directory separator on Windows, which is _not_ the forward slash). Prepare that function to be useful also in that context. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- lockfile.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lockfile.c b/lockfile.c index 1d5ed016828746..67082a9caaeb18 100644 --- a/lockfile.c +++ b/lockfile.c @@ -19,14 +19,14 @@ static void trim_last_path_component(struct strbuf *path) int i = path->len; /* back up past trailing slashes, if any */ - while (i && path->buf[i - 1] == '/') + while (i && is_dir_sep(path->buf[i - 1])) i--; /* * then go backwards until a slash, or the beginning of the * string */ - while (i && path->buf[i - 1] != '/') + while (i && !is_dir_sep(path->buf[i - 1])) i--; strbuf_setlen(path, i); From dae450dd0e995b26d0140cf3753d103f5470f44d Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 12 May 2015 11:09:01 +0200 Subject: [PATCH 16/33] mingw: don't call `GetFileAttributes()` twice in `mingw_lstat()` The Win32 API function `GetFileAttributes()` cannot handle paths with trailing dir separators. The current `mingw_stat()`/`mingw_lstat()` implementation calls `GetFileAttributes()` twice if the path has trailing slashes (first with the original path that was passed as function parameter, and and a second time with a path copy with trailing '/' removed). With the conversion to wide Unicode, we get the length of the path for free, and also have a (wide char) buffer that can be modified. This makes it easy to avoid that extraneous Win32 API call. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 48 ++++++++++++------------------------------------ 1 file changed, 12 insertions(+), 36 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index cf4f3c92e7a889..f5a0fe33254b0a 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -928,9 +928,18 @@ static int do_lstat(int follow, const char *file_name, struct stat *buf) { WIN32_FILE_ATTRIBUTE_DATA fdata; wchar_t wfilename[MAX_PATH]; - if (xutftowcs_path(wfilename, file_name) < 0) + int wlen = xutftowcs_path(wfilename, file_name); + if (wlen < 0) return -1; + /* strip trailing '/', or GetFileAttributes will fail */ + while (wlen && is_dir_sep(wfilename[wlen - 1])) + wfilename[--wlen] = 0; + if (!wlen) { + errno = ENOENT; + return -1; + } + if (GetFileAttributesExW(wfilename, GetFileExInfoStandard, &fdata)) { buf->st_ino = 0; buf->st_gid = 0; @@ -990,39 +999,6 @@ static int do_lstat(int follow, const char *file_name, struct stat *buf) return -1; } -/* We provide our own lstat/fstat functions, since the provided - * lstat/fstat functions are so slow. These stat functions are - * tailored for Git's usage (read: fast), and are not meant to be - * complete. Note that Git stat()s are redirected to mingw_lstat() - * too, since Windows doesn't really handle symlinks that well. - */ -static int do_stat_internal(int follow, const char *file_name, struct stat *buf) -{ - size_t namelen; - char alt_name[PATH_MAX]; - - if (!do_lstat(follow, file_name, buf)) - return 0; - - /* if file_name ended in a '/', Windows returned ENOENT; - * try again without trailing slashes - */ - if (errno != ENOENT) - return -1; - - namelen = strlen(file_name); - if (namelen && file_name[namelen-1] != '/') - return -1; - while (namelen && file_name[namelen-1] == '/') - --namelen; - if (!namelen || namelen >= PATH_MAX) - return -1; - - memcpy(alt_name, file_name, namelen); - alt_name[namelen] = 0; - return do_lstat(follow, alt_name, buf); -} - static int get_file_info_by_handle(HANDLE hnd, struct stat *buf) { BY_HANDLE_FILE_INFORMATION fdata; @@ -1048,11 +1024,11 @@ static int get_file_info_by_handle(HANDLE hnd, struct stat *buf) int mingw_lstat(const char *file_name, struct stat *buf) { - return do_stat_internal(0, file_name, buf); + return do_lstat(0, file_name, buf); } int mingw_stat(const char *file_name, struct stat *buf) { - return do_stat_internal(1, file_name, buf); + return do_lstat(1, file_name, buf); } int mingw_fstat(int fd, struct stat *buf) From c36848eda76742f7a7c203868a077b790301344f Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sat, 16 May 2015 01:18:14 +0200 Subject: [PATCH 17/33] mingw: implement `stat()` with symlink support With respect to symlinks, the current `mingw_stat()` implementation is almost identical to `mingw_lstat()`: except for the file type (`st_mode & S_IFMT`), it returns information about the link rather than the target. Implement `mingw_stat()` by opening the file handle requesting minimal permissions, and then calling `GetFileInformationByHandle()` on it. This way, all links are resolved by the Windows file system layer. If symlinks are disabled, use `mingw_lstat()` as before, but fail with `ELOOP` if a symlink would have to be resolved. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/compat/mingw.c b/compat/mingw.c index f5a0fe33254b0a..59afd696863450 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -1026,9 +1026,26 @@ int mingw_lstat(const char *file_name, struct stat *buf) { return do_lstat(0, file_name, buf); } + int mingw_stat(const char *file_name, struct stat *buf) { - return do_lstat(1, file_name, buf); + wchar_t wfile_name[MAX_PATH]; + HANDLE hnd; + int result; + + /* open the file and let Windows resolve the links */ + if (xutftowcs_path(wfile_name, file_name) < 0) + return -1; + hnd = CreateFileW(wfile_name, 0, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, + OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); + if (hnd == INVALID_HANDLE_VALUE) { + errno = err_win_to_posix(GetLastError()); + return -1; + } + result = get_file_info_by_handle(hnd, buf); + CloseHandle(hnd); + return result; } int mingw_fstat(int fd, struct stat *buf) From aa0ca80bbb00ec53f359c79719b95bcc141ac85c Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 12 May 2015 00:58:39 +0200 Subject: [PATCH 18/33] mingw: drop the separate `do_lstat()` function With the new `mingw_stat()` implementation, `do_lstat()` is only called from `mingw_lstat()` (with the function parameter `follow == 0`). Remove the extra function and the old `mingw_stat()`-specific (`follow == 1`) logic. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index 59afd696863450..ec6c2801d3cbf4 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -917,14 +917,7 @@ static int has_valid_directory_prefix(wchar_t *wfilename) return 1; } -/* We keep the do_lstat code in a separate function to avoid recursion. - * When a path ends with a slash, the stat will fail with ENOENT. In - * this case, we strip the trailing slashes and stat again. - * - * If follow is true then act like stat() and report on the link - * target. Otherwise report on the link itself. - */ -static int do_lstat(int follow, const char *file_name, struct stat *buf) +int mingw_lstat(const char *file_name, struct stat *buf) { WIN32_FILE_ATTRIBUTE_DATA fdata; wchar_t wfilename[MAX_PATH]; @@ -958,13 +951,7 @@ static int do_lstat(int follow, const char *file_name, struct stat *buf) if (handle != INVALID_HANDLE_VALUE) { if ((findbuf.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) && (findbuf.dwReserved0 == IO_REPARSE_TAG_SYMLINK)) { - if (follow) { - char buffer[MAXIMUM_REPARSE_DATA_BUFFER_SIZE]; - buf->st_size = readlink(file_name, buffer, MAXIMUM_REPARSE_DATA_BUFFER_SIZE); - } else { - buf->st_mode = S_IFLNK; - } - buf->st_mode |= S_IREAD; + buf->st_mode = S_IFLNK | S_IREAD; if (!(findbuf.dwFileAttributes & FILE_ATTRIBUTE_READONLY)) buf->st_mode |= S_IWRITE; } @@ -1022,11 +1009,6 @@ static int get_file_info_by_handle(HANDLE hnd, struct stat *buf) return 0; } -int mingw_lstat(const char *file_name, struct stat *buf) -{ - return do_lstat(0, file_name, buf); -} - int mingw_stat(const char *file_name, struct stat *buf) { wchar_t wfile_name[MAX_PATH]; From 886044373be1bd4d2e7ecf758c42474e9efbe067 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sun, 24 May 2015 00:17:56 +0200 Subject: [PATCH 19/33] mingw: let `mingw_lstat()` error early upon problems with reparse points When obtaining lstat information for reparse points, we need to call `FindFirstFile()` in addition to `GetFileInformationEx()` to obtain the type of the reparse point (symlink, mount point etc.). However, currently there is no error handling whatsoever if `FindFirstFile()` fails. Call `FindFirstFile()` before modifying the `stat *buf` output parameter and error out if the call fails. Note: The `FindFirstFile()` return value includes all the data that we get from `GetFileAttributesEx()`, so we could replace `GetFileAttributesEx()` with `FindFirstFile()`. We don't do that because `GetFileAttributesEx()` is about twice as fast for single files. I.e. we only pay the extra cost of calling `FindFirstFile()` in the rare case that we encounter a reparse point. Please also note that the indentation the remaining reparse point code changed, and hence the best way to look at this diff is with `--color-moved -w`. That code was _not_ moved because a subsequent commit will move it to an altogether different function, anyway. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index ec6c2801d3cbf4..23a926c7d14994 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -920,6 +920,7 @@ static int has_valid_directory_prefix(wchar_t *wfilename) int mingw_lstat(const char *file_name, struct stat *buf) { WIN32_FILE_ATTRIBUTE_DATA fdata; + WIN32_FIND_DATAW findbuf = { 0 }; wchar_t wfilename[MAX_PATH]; int wlen = xutftowcs_path(wfilename, file_name); if (wlen < 0) @@ -934,6 +935,13 @@ int mingw_lstat(const char *file_name, struct stat *buf) } if (GetFileAttributesExW(wfilename, GetFileExInfoStandard, &fdata)) { + /* for reparse points, use FindFirstFile to get the reparse tag */ + if (fdata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) { + HANDLE handle = FindFirstFileW(wfilename, &findbuf); + if (handle == INVALID_HANDLE_VALUE) + goto error; + FindClose(handle); + } buf->st_ino = 0; buf->st_gid = 0; buf->st_uid = 0; @@ -946,20 +954,16 @@ int mingw_lstat(const char *file_name, struct stat *buf) filetime_to_timespec(&(fdata.ftLastWriteTime), &(buf->st_mtim)); filetime_to_timespec(&(fdata.ftCreationTime), &(buf->st_ctim)); if (fdata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) { - WIN32_FIND_DATAW findbuf; - HANDLE handle = FindFirstFileW(wfilename, &findbuf); - if (handle != INVALID_HANDLE_VALUE) { - if ((findbuf.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) && - (findbuf.dwReserved0 == IO_REPARSE_TAG_SYMLINK)) { - buf->st_mode = S_IFLNK | S_IREAD; - if (!(findbuf.dwFileAttributes & FILE_ATTRIBUTE_READONLY)) - buf->st_mode |= S_IWRITE; - } - FindClose(handle); + if ((findbuf.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) && + (findbuf.dwReserved0 == IO_REPARSE_TAG_SYMLINK)) { + buf->st_mode = S_IFLNK | S_IREAD; + if (!(findbuf.dwFileAttributes & FILE_ATTRIBUTE_READONLY)) + buf->st_mode |= S_IWRITE; } } return 0; } +error: switch (GetLastError()) { case ERROR_ACCESS_DENIED: case ERROR_SHARING_VIOLATION: From db1d156aa01826c8371e1ebdfeafa65744b66010 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 10 Jan 2017 23:21:56 +0100 Subject: [PATCH 20/33] mingw: teach dirent about symlinks Move the `S_IFLNK` detection to `file_attr_to_st_mode()`. Implement `DT_LNK` detection in dirent.c's `readdir()` function. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 13 +++---------- compat/win32.h | 6 ++++-- compat/win32/dirent.c | 5 ++++- 3 files changed, 11 insertions(+), 13 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index 23a926c7d14994..a3a48db581df87 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -946,21 +946,14 @@ int mingw_lstat(const char *file_name, struct stat *buf) buf->st_gid = 0; buf->st_uid = 0; buf->st_nlink = 1; - buf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes); + buf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes, + findbuf.dwReserved0); buf->st_size = fdata.nFileSizeLow | (((off_t)fdata.nFileSizeHigh)<<32); buf->st_dev = buf->st_rdev = 0; /* not used by Git */ filetime_to_timespec(&(fdata.ftLastAccessTime), &(buf->st_atim)); filetime_to_timespec(&(fdata.ftLastWriteTime), &(buf->st_mtim)); filetime_to_timespec(&(fdata.ftCreationTime), &(buf->st_ctim)); - if (fdata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) { - if ((findbuf.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) && - (findbuf.dwReserved0 == IO_REPARSE_TAG_SYMLINK)) { - buf->st_mode = S_IFLNK | S_IREAD; - if (!(findbuf.dwFileAttributes & FILE_ATTRIBUTE_READONLY)) - buf->st_mode |= S_IWRITE; - } - } return 0; } error: @@ -1003,7 +996,7 @@ static int get_file_info_by_handle(HANDLE hnd, struct stat *buf) buf->st_gid = 0; buf->st_uid = 0; buf->st_nlink = 1; - buf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes); + buf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes, 0); buf->st_size = fdata.nFileSizeLow | (((off_t)fdata.nFileSizeHigh)<<32); buf->st_dev = buf->st_rdev = 0; /* not used by Git */ diff --git a/compat/win32.h b/compat/win32.h index a97e880757b6f1..671bcc81f93351 100644 --- a/compat/win32.h +++ b/compat/win32.h @@ -6,10 +6,12 @@ #include #endif -static inline int file_attr_to_st_mode (DWORD attr) +static inline int file_attr_to_st_mode (DWORD attr, DWORD tag) { int fMode = S_IREAD; - if (attr & FILE_ATTRIBUTE_DIRECTORY) + if ((attr & FILE_ATTRIBUTE_REPARSE_POINT) && tag == IO_REPARSE_TAG_SYMLINK) + fMode |= S_IFLNK; + else if (attr & FILE_ATTRIBUTE_DIRECTORY) fMode |= S_IFDIR; else fMode |= S_IFREG; diff --git a/compat/win32/dirent.c b/compat/win32/dirent.c index 52420ec7d4dad7..24ee9b814d6adf 100644 --- a/compat/win32/dirent.c +++ b/compat/win32/dirent.c @@ -12,7 +12,10 @@ static inline void finddata2dirent(struct dirent *ent, WIN32_FIND_DATAW *fdata) xwcstoutf(ent->d_name, fdata->cFileName, sizeof(ent->d_name)); /* Set file type, based on WIN32_FIND_DATA */ - if (fdata->dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) + if ((fdata->dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) + && fdata->dwReserved0 == IO_REPARSE_TAG_SYMLINK) + ent->d_type = DT_LNK; + else if (fdata->dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) ent->d_type = DT_DIR; else ent->d_type = DT_REG; From 4c49a3d9bf57415ea31ac357452b646c75b64d82 Mon Sep 17 00:00:00 2001 From: Bill Zissimopoulos Date: Thu, 28 May 2020 16:35:57 -0700 Subject: [PATCH 21/33] mingw: compute the correct size for symlinks in `mingw_lstat()` POSIX specifies that upon successful return from `lstat()`: "the value of the st_size member shall be set to the length of the pathname contained in the symbolic link not including any terminating null byte". Git typically doesn't trust the `stat.st_size` member of symlinks (e.g. see `strbuf_readlink()`). Therefore, it is tempting to save on the extra overhead of opening and reading the reparse point merely to calculate the exact size of the link target. This is, in fact, what Git for Windows did, from May 2015 to May 2020. At least almost: some functions take shortcuts if `st_size` is 0 (e.g. `diff_populate_filespec()`), hence Git for Windows hard-coded the length of all symlinks to MAX_PATH. This did cause problems, though, specifically in Git repositories that were also accessed by Git for Cygwin or Git for WSL. For example, doing `git reset --hard` using Git for Windows would update the size of symlinks in the index to be MAX_PATH; at a later time Git for Cygwin or Git for WSL would find that symlinks have changed size during `git status` and update the index. And then Git for Windows would think that the index needs to be updated. Even if the symlinks did not, in fact, change. To avoid that, the correct size must be determined. Signed-off-by: Bill Zissimopoulos Signed-off-by: Johannes Schindelin --- compat/mingw.c | 114 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 104 insertions(+), 10 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index a3a48db581df87..c7571951dc7bd9 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -21,6 +21,7 @@ #define SECURITY_WIN32 #include #include +#include #include #define STATUS_DELETE_PENDING ((NTSTATUS) 0xC0000056) @@ -917,10 +918,102 @@ static int has_valid_directory_prefix(wchar_t *wfilename) return 1; } +#ifndef _WINNT_H +/* + * The REPARSE_DATA_BUFFER structure is defined in the Windows DDK (in + * ntifs.h) and in MSYS1's winnt.h (which defines _WINNT_H). So define + * it ourselves if we are on MSYS2 (whose winnt.h defines _WINNT_). + */ +typedef struct _REPARSE_DATA_BUFFER { + DWORD ReparseTag; + WORD ReparseDataLength; + WORD Reserved; +#ifndef _MSC_VER + _ANONYMOUS_UNION +#endif + union { + struct { + WORD SubstituteNameOffset; + WORD SubstituteNameLength; + WORD PrintNameOffset; + WORD PrintNameLength; + ULONG Flags; + WCHAR PathBuffer[1]; + } SymbolicLinkReparseBuffer; + struct { + WORD SubstituteNameOffset; + WORD SubstituteNameLength; + WORD PrintNameOffset; + WORD PrintNameLength; + WCHAR PathBuffer[1]; + } MountPointReparseBuffer; + struct { + BYTE DataBuffer[1]; + } GenericReparseBuffer; + } DUMMYUNIONNAME; +} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER; +#endif + +static int read_reparse_point(const WCHAR *wpath, BOOL fail_on_unknown_tag, + char *tmpbuf, int *plen, DWORD *ptag) +{ + HANDLE handle; + WCHAR *wbuf; + REPARSE_DATA_BUFFER *b = alloca(MAXIMUM_REPARSE_DATA_BUFFER_SIZE); + DWORD dummy; + + /* read reparse point data */ + handle = CreateFileW(wpath, 0, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, + OPEN_EXISTING, + FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OPEN_REPARSE_POINT, NULL); + if (handle == INVALID_HANDLE_VALUE) { + errno = err_win_to_posix(GetLastError()); + return -1; + } + if (!DeviceIoControl(handle, FSCTL_GET_REPARSE_POINT, NULL, 0, b, + MAXIMUM_REPARSE_DATA_BUFFER_SIZE, &dummy, NULL)) { + errno = err_win_to_posix(GetLastError()); + CloseHandle(handle); + return -1; + } + CloseHandle(handle); + + /* get target path for symlinks or mount points (aka 'junctions') */ + switch ((*ptag = b->ReparseTag)) { + case IO_REPARSE_TAG_SYMLINK: + wbuf = (WCHAR*) (((char*) b->SymbolicLinkReparseBuffer.PathBuffer) + + b->SymbolicLinkReparseBuffer.SubstituteNameOffset); + *(WCHAR*) (((char*) wbuf) + + b->SymbolicLinkReparseBuffer.SubstituteNameLength) = 0; + break; + case IO_REPARSE_TAG_MOUNT_POINT: + wbuf = (WCHAR*) (((char*) b->MountPointReparseBuffer.PathBuffer) + + b->MountPointReparseBuffer.SubstituteNameOffset); + *(WCHAR*) (((char*) wbuf) + + b->MountPointReparseBuffer.SubstituteNameLength) = 0; + break; + default: + if (fail_on_unknown_tag) { + errno = EINVAL; + return -1; + } else { + *plen = MAX_PATH; + return 0; + } + } + + if ((*plen = + xwcstoutf(tmpbuf, normalize_ntpath(wbuf), MAX_PATH)) < 0) + return -1; + return 0; +} + int mingw_lstat(const char *file_name, struct stat *buf) { WIN32_FILE_ATTRIBUTE_DATA fdata; - WIN32_FIND_DATAW findbuf = { 0 }; + DWORD reparse_tag = 0; + int link_len = 0; wchar_t wfilename[MAX_PATH]; int wlen = xutftowcs_path(wfilename, file_name); if (wlen < 0) @@ -935,28 +1028,29 @@ int mingw_lstat(const char *file_name, struct stat *buf) } if (GetFileAttributesExW(wfilename, GetFileExInfoStandard, &fdata)) { - /* for reparse points, use FindFirstFile to get the reparse tag */ + /* for reparse points, get the link tag and length */ if (fdata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) { - HANDLE handle = FindFirstFileW(wfilename, &findbuf); - if (handle == INVALID_HANDLE_VALUE) - goto error; - FindClose(handle); + char tmpbuf[MAX_PATH]; + + if (read_reparse_point(wfilename, FALSE, tmpbuf, + &link_len, &reparse_tag) < 0) + return -1; } buf->st_ino = 0; buf->st_gid = 0; buf->st_uid = 0; buf->st_nlink = 1; buf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes, - findbuf.dwReserved0); - buf->st_size = fdata.nFileSizeLow | - (((off_t)fdata.nFileSizeHigh)<<32); + reparse_tag); + buf->st_size = S_ISLNK(buf->st_mode) ? link_len : + fdata.nFileSizeLow | (((off_t) fdata.nFileSizeHigh) << 32); buf->st_dev = buf->st_rdev = 0; /* not used by Git */ filetime_to_timespec(&(fdata.ftLastAccessTime), &(buf->st_atim)); filetime_to_timespec(&(fdata.ftLastWriteTime), &(buf->st_mtim)); filetime_to_timespec(&(fdata.ftCreationTime), &(buf->st_ctim)); return 0; } -error: + switch (GetLastError()) { case ERROR_ACCESS_DENIED: case ERROR_SHARING_VIOLATION: From ad74d540f2f6bd19f248606e29ae45d226c264f9 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 19 May 2015 21:48:55 +0200 Subject: [PATCH 22/33] mingw: factor out the retry logic In several places, Git's Windows-specific code follows the pattern where it tries to perform an operation, and retries several times when that operation fails, sleeping an increasing amount of time, before finally giving up and asking the user whether to rety (after, say, closing an editor that held a handle to a file, preventing the operation from succeeding). This logic is a bit hard to use, and inconsistent: `mingw_unlink()` and `mingw_rmdir()` duplicate the code to retry, and both of them do so incompletely. They also do not restore `errno` if the user answers 'no'. Introduce a `retry_ask_yes_no()` helper function that handles retry with small delay, asking the user, and restoring `errno`. Note that in `mingw_unlink()`, we include the `_wchmod()` call in the retry loop (which may fail if the file is locked exclusively). In `mingw_rmdir()`, we include special error handling in the retry loop. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 104 ++++++++++++++++++++++--------------------------- 1 file changed, 46 insertions(+), 58 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index c7571951dc7bd9..26e64c6a5a2997 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -28,8 +28,6 @@ #define HCAST(type, handle) ((type)(intptr_t)handle) -static const int delay[] = { 0, 1, 10, 20, 40 }; - void open_in_gdb(void) { static struct child_process cp = CHILD_PROCESS_INIT; @@ -205,15 +203,12 @@ static int read_yes_no_answer(void) return -1; } -static int ask_yes_no_if_possible(const char *format, ...) +static int ask_yes_no_if_possible(const char *format, va_list args) { char question[4096]; const char *retry_hook; - va_list args; - va_start(args, format); vsnprintf(question, sizeof(question), format, args); - va_end(args); retry_hook = mingw_getenv("GIT_ASK_YESNO"); if (retry_hook) { @@ -238,6 +233,31 @@ static int ask_yes_no_if_possible(const char *format, ...) } } +static int retry_ask_yes_no(int *tries, const char *format, ...) +{ + static const int delay[] = { 0, 1, 10, 20, 40 }; + va_list args; + int result, saved_errno = errno; + + if ((*tries) < ARRAY_SIZE(delay)) { + /* + * We assume that some other process had the file open at the wrong + * moment and retry. In order to give the other process a higher + * chance to complete its operation, we give up our time slice now. + * If we have to retry again, we do sleep a bit. + */ + Sleep(delay[*tries]); + (*tries)++; + return 1; + } + + va_start(args, format); + result = ask_yes_no_if_possible(format, args); + va_end(args); + errno = saved_errno; + return result; +} + /* Windows only */ enum hide_dotfiles_type { HIDE_DOTFILES_FALSE = 0, @@ -298,7 +318,7 @@ static wchar_t *normalize_ntpath(wchar_t *wbuf) int mingw_unlink(const char *pathname, int handle_in_use_error) { - int ret, tries = 0; + int tries = 0; wchar_t wpathname[MAX_PATH]; if (xutftowcs_path(wpathname, pathname) < 0) return -1; @@ -306,29 +326,19 @@ int mingw_unlink(const char *pathname, int handle_in_use_error) if (DeleteFileW(wpathname)) return 0; - /* read-only files cannot be removed */ - _wchmod(wpathname, 0666); - while ((ret = _wunlink(wpathname)) == -1 && tries < ARRAY_SIZE(delay)) { + do { + /* read-only files cannot be removed */ + _wchmod(wpathname, 0666); + if (!_wunlink(wpathname)) + return 0; if (!is_file_in_use_error(GetLastError())) break; if (!handle_in_use_error) - return ret; + return -1; - /* - * We assume that some other process had the source or - * destination file open at the wrong moment and retry. - * In order to give the other process a higher chance to - * complete its operation, we give up our time slice now. - * If we have to retry again, we do sleep a bit. - */ - Sleep(delay[tries]); - tries++; - } - while (ret == -1 && is_file_in_use_error(GetLastError()) && - ask_yes_no_if_possible("Unlink of file '%s' failed. " - "Should I try again?", pathname)) - ret = _wunlink(wpathname); - return ret; + } while (retry_ask_yes_no(&tries, "Unlink of file '%s' failed. " + "Should I try again?", pathname)); + return -1; } static int is_dir_empty(const wchar_t *wpath) @@ -355,7 +365,7 @@ static int is_dir_empty(const wchar_t *wpath) int mingw_rmdir(const char *pathname) { - int ret, tries = 0; + int tries = 0; wchar_t wpathname[MAX_PATH]; struct stat st; @@ -381,7 +391,11 @@ int mingw_rmdir(const char *pathname) if (xutftowcs_path(wpathname, pathname) < 0) return -1; - while ((ret = _wrmdir(wpathname)) == -1 && tries < ARRAY_SIZE(delay)) { + do { + if (!_wrmdir(wpathname)) { + invalidate_lstat_cache(); + return 0; + } if (!is_file_in_use_error(GetLastError())) errno = err_win_to_posix(GetLastError()); if (errno != EACCES) @@ -390,23 +404,9 @@ int mingw_rmdir(const char *pathname) errno = ENOTEMPTY; break; } - /* - * We assume that some other process had the source or - * destination file open at the wrong moment and retry. - * In order to give the other process a higher chance to - * complete its operation, we give up our time slice now. - * If we have to retry again, we do sleep a bit. - */ - Sleep(delay[tries]); - tries++; - } - while (ret == -1 && errno == EACCES && is_file_in_use_error(GetLastError()) && - ask_yes_no_if_possible("Deletion of directory '%s' failed. " - "Should I try again?", pathname)) - ret = _wrmdir(wpathname); - if (!ret) - invalidate_lstat_cache(); - return ret; + } while (retry_ask_yes_no(&tries, "Deletion of directory '%s' failed. " + "Should I try again?", pathname)); + return -1; } static inline int needs_hiding(const char *path) @@ -2384,20 +2384,8 @@ int mingw_rename(const char *pold, const char *pnew) SetFileAttributesW(wpnew, attrs); } } - if (tries < ARRAY_SIZE(delay) && gle == ERROR_ACCESS_DENIED) { - /* - * We assume that some other process had the source or - * destination file open at the wrong moment and retry. - * In order to give the other process a higher chance to - * complete its operation, we give up our time slice now. - * If we have to retry again, we do sleep a bit. - */ - Sleep(delay[tries]); - tries++; - goto repeat; - } if (gle == ERROR_ACCESS_DENIED && - ask_yes_no_if_possible("Rename from '%s' to '%s' failed. " + retry_ask_yes_no(&tries, "Rename from '%s' to '%s' failed. " "Should I try again?", pold, pnew)) goto repeat; From 25313cea76e059d01a0607d0d5291b62011f4526 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sun, 24 May 2015 01:55:05 +0200 Subject: [PATCH 23/33] mingw: change default of `core.symlinks` to false Symlinks on Windows don't work the same way as on Unix systems. For example, there are different types of symlinks for directories and files, and unless using a recent-ish Windows version in Developer Mode, creating symlinks requires administrative privileges. By default, disable symlink support on Windows. That is, users explicitly have to enable it with `git config [--system|--global] core.symlinks true`; For convenience, `git init` (and `git clone`) will perform a test whether the current setup allows creating symlinks and will configure that setting in the repository config. The test suite ignores system / global config files. Allow testing *with* symlink support by checking if native symlinks are enabled in MSYS2 (via setting the special environment variable `MSYS=winsymlinks:nativestrict` to ask the MSYS2 runtime to enable creating symlinks). Note: This assumes that Git's test suite is run in MSYS2's Bash, which is true for the time being (an experiment to switch to BusyBox-w32 failed due to the experimental nature of BusyBox-w32). Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/compat/mingw.c b/compat/mingw.c index 26e64c6a5a2997..0fe00a5b703d07 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -2862,6 +2862,15 @@ static void setup_windows_environment(void) if (!tmp && (tmp = getenv("USERPROFILE"))) setenv("HOME", tmp, 1); } + + /* + * Change 'core.symlinks' default to false, unless native symlinks are + * enabled in MSys2 (via 'MSYS=winsymlinks:nativestrict'). Thus we can + * run the test suite (which doesn't obey config files) with or without + * symlink support. + */ + if (!(tmp = getenv("MSYS")) || !strstr(tmp, "winsymlinks:nativestrict")) + has_symlinks = 0; } static void get_current_user_sid(PSID *sid, HANDLE *linked_token) From b698f4a002a29e1342760fa8cc5d31a3e4a857f6 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sat, 16 May 2015 00:32:03 +0200 Subject: [PATCH 24/33] mingw: add symlink-specific error codes The Win32 API calls do not set `errno`; Instead, error codes for failed operations must be obtained via the `GetLastError()` function. Git would not know what to do with those error values, though, which is why Git's Windows compatibility layer translates them to `errno` values. Let's handle a couple of symlink-related error codes that will become relevant with the upcoming support for symlinks on Windows. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/compat/mingw.c b/compat/mingw.c index 0fe00a5b703d07..0e8807196fcbed 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -102,6 +102,7 @@ int err_win_to_posix(DWORD winerr) case ERROR_INVALID_PARAMETER: error = EINVAL; break; case ERROR_INVALID_PASSWORD: error = EPERM; break; case ERROR_INVALID_PRIMARY_GROUP: error = EINVAL; break; + case ERROR_INVALID_REPARSE_DATA: error = EINVAL; break; case ERROR_INVALID_SIGNAL_NUMBER: error = EINVAL; break; case ERROR_INVALID_TARGET_HANDLE: error = EIO; break; case ERROR_INVALID_WORKSTATION: error = EACCES; break; @@ -116,6 +117,7 @@ int err_win_to_posix(DWORD winerr) case ERROR_NEGATIVE_SEEK: error = ESPIPE; break; case ERROR_NOACCESS: error = EFAULT; break; case ERROR_NONE_MAPPED: error = EINVAL; break; + case ERROR_NOT_A_REPARSE_POINT: error = EINVAL; break; case ERROR_NOT_ENOUGH_MEMORY: error = ENOMEM; break; case ERROR_NOT_READY: error = EAGAIN; break; case ERROR_NOT_SAME_DEVICE: error = EXDEV; break; @@ -136,6 +138,9 @@ int err_win_to_posix(DWORD winerr) case ERROR_PIPE_NOT_CONNECTED: error = EPIPE; break; case ERROR_PRIVILEGE_NOT_HELD: error = EACCES; break; case ERROR_READ_FAULT: error = EIO; break; + case ERROR_REPARSE_ATTRIBUTE_CONFLICT: error = EINVAL; break; + case ERROR_REPARSE_TAG_INVALID: error = EINVAL; break; + case ERROR_REPARSE_TAG_MISMATCH: error = EINVAL; break; case ERROR_SEEK: error = EIO; break; case ERROR_SEEK_ON_DEVICE: error = ESPIPE; break; case ERROR_SHARING_BUFFER_EXCEEDED: error = ENFILE; break; From 282aba42e8aeed336318614523ff8b36d3131e5f Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sun, 24 May 2015 01:06:10 +0200 Subject: [PATCH 25/33] mingw: handle symlinks to directories in `mingw_unlink()` The `_wunlink()` and `DeleteFileW()` functions refuse to delete symlinks to directories on Windows; The error code woutl be `ERROR_ACCESS_DENIED` in that case. Take that error code as an indicator that we need to try `_wrmdir()` as well. In the best case, it will remove a symlink. In the worst case, it will fail with the same error code again. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/compat/mingw.c b/compat/mingw.c index 0e8807196fcbed..b1cc30d0f13922 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -338,9 +338,16 @@ int mingw_unlink(const char *pathname, int handle_in_use_error) return 0; if (!is_file_in_use_error(GetLastError())) break; + /* + * _wunlink() / DeleteFileW() for directory symlinks fails with + * ERROR_ACCESS_DENIED (EACCES), so try _wrmdir() as well. This is the + * same error we get if a file is in use (already checked above). + */ + if (!_wrmdir(wpathname)) + return 0; + if (!handle_in_use_error) return -1; - } while (retry_ask_yes_no(&tries, "Unlink of file '%s' failed. " "Should I try again?", pathname)); return -1; From 5cb3b10500339fec403b84e30721e4f6a68e42f6 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Tue, 19 May 2015 22:42:48 +0200 Subject: [PATCH 26/33] mingw: support renaming symlinks Older MSVCRT's `_wrename()` function cannot rename symlinks over existing files: it returns success without doing anything. Newer MSVCR*.dll versions probably do not share this problem: according to CRT sources, they just call `MoveFileEx()` with the `MOVEFILE_COPY_ALLOWED` flag. Avoid the `_wrename()` call, and go with directly calling `MoveFileEx()`, with proper error handling of course. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 38 ++++++++++++++++---------------------- 1 file changed, 16 insertions(+), 22 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index b1cc30d0f13922..55f0bb478e6127 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -2275,7 +2275,7 @@ int mingw_accept(int sockfd1, struct sockaddr *sa, socklen_t *sz) int mingw_rename(const char *pold, const char *pnew) { static int supports_file_rename_info_ex = 1; - DWORD attrs, gle; + DWORD attrs = INVALID_FILE_ATTRIBUTES, gle; int tries = 0; wchar_t wpold[MAX_PATH], wpnew[MAX_PATH]; int wpnew_len; @@ -2286,15 +2286,6 @@ int mingw_rename(const char *pold, const char *pnew) if (wpnew_len < 0) return -1; - /* - * Try native rename() first to get errno right. - * It is based on MoveFile(), which cannot overwrite existing files. - */ - if (!_wrename(wpold, wpnew)) - return 0; - if (errno != EEXIST) - return -1; - repeat: if (supports_file_rename_info_ex) { /* @@ -2370,13 +2361,22 @@ int mingw_rename(const char *pold, const char *pnew) * to retry. */ } else { - if (MoveFileExW(wpold, wpnew, MOVEFILE_REPLACE_EXISTING)) + if (MoveFileExW(wpold, wpnew, + MOVEFILE_REPLACE_EXISTING | MOVEFILE_COPY_ALLOWED)) return 0; gle = GetLastError(); } - /* TODO: translate more errors */ - if (gle == ERROR_ACCESS_DENIED && + /* revert file attributes on failure */ + if (attrs != INVALID_FILE_ATTRIBUTES) + SetFileAttributesW(wpnew, attrs); + + if (!is_file_in_use_error(gle)) { + errno = err_win_to_posix(gle); + return -1; + } + + if (attrs == INVALID_FILE_ATTRIBUTES && (attrs = GetFileAttributesW(wpnew)) != INVALID_FILE_ATTRIBUTES) { if (attrs & FILE_ATTRIBUTE_DIRECTORY) { DWORD attrsold = GetFileAttributesW(wpold); @@ -2388,16 +2388,10 @@ int mingw_rename(const char *pold, const char *pnew) return -1; } if ((attrs & FILE_ATTRIBUTE_READONLY) && - SetFileAttributesW(wpnew, attrs & ~FILE_ATTRIBUTE_READONLY)) { - if (MoveFileExW(wpold, wpnew, MOVEFILE_REPLACE_EXISTING)) - return 0; - gle = GetLastError(); - /* revert file attributes on failure */ - SetFileAttributesW(wpnew, attrs); - } + SetFileAttributesW(wpnew, attrs & ~FILE_ATTRIBUTE_READONLY)) + goto repeat; } - if (gle == ERROR_ACCESS_DENIED && - retry_ask_yes_no(&tries, "Rename from '%s' to '%s' failed. " + if (retry_ask_yes_no(&tries, "Rename from '%s' to '%s' failed. " "Should I try again?", pold, pnew)) goto repeat; From 49920839bb2f9e21d1cf91e54024ecd986fe32a9 Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sun, 24 May 2015 01:17:31 +0200 Subject: [PATCH 27/33] mingw: allow `mingw_chdir()` to change to symlink-resolved directories If symlinks are enabled, resolve all symlinks when changing directories, as required by POSIX. Note: Git's `real_path()` function bases its link resolution algorithm on this property of `chdir()`. Unfortunately, the current directory on Windows is limited to only MAX_PATH (260) characters. Therefore using symlinks and long paths in combination may be problematic. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/compat/mingw.c b/compat/mingw.c index 55f0bb478e6127..5d2a8c247ce622 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -866,9 +866,27 @@ int mingw_access(const char *filename, int mode) int mingw_chdir(const char *dirname) { wchar_t wdirname[MAX_PATH]; + if (xutftowcs_path(wdirname, dirname) < 0) return -1; - return _wchdir(wdirname); + + if (has_symlinks) { + HANDLE hnd = CreateFileW(wdirname, 0, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, + OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); + if (hnd == INVALID_HANDLE_VALUE) { + errno = err_win_to_posix(GetLastError()); + return -1; + } + if (!GetFinalPathNameByHandleW(hnd, wdirname, ARRAY_SIZE(wdirname), 0)) { + errno = err_win_to_posix(GetLastError()); + CloseHandle(hnd); + return -1; + } + CloseHandle(hnd); + } + + return _wchdir(normalize_ntpath(wdirname)); } int mingw_chmod(const char *filename, int mode) From 8fef8220f4d60b48358882fbc961c4604eab3abf Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sun, 24 May 2015 01:24:41 +0200 Subject: [PATCH 28/33] mingw: implement `readlink()` Implement `readlink()` by reading NTFS reparse points via the `read_reparse_point()` function that was introduced earlier to determine the length of symlink targets. Works for symlinks and directory junctions. If symlinks are disabled, fail with `ENOSYS`. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw-posix.h | 3 +-- compat/mingw.c | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/compat/mingw-posix.h b/compat/mingw-posix.h index 0939feff27ffec..896aa976b1949b 100644 --- a/compat/mingw-posix.h +++ b/compat/mingw-posix.h @@ -121,8 +121,6 @@ struct utsname { * trivial stubs */ -static inline int readlink(const char *path UNUSED, char *buf UNUSED, size_t bufsiz UNUSED) -{ errno = ENOSYS; return -1; } static inline int symlink(const char *oldpath UNUSED, const char *newpath UNUSED) { errno = ENOSYS; return -1; } static inline int fchmod(int fildes UNUSED, mode_t mode UNUSED) @@ -197,6 +195,7 @@ int setitimer(int type, struct itimerval *in, struct itimerval *out); int sigaction(int sig, struct sigaction *in, struct sigaction *out); int link(const char *oldpath, const char *newpath); int uname(struct utsname *buf); +int readlink(const char *path, char *buf, size_t bufsiz); /* * replacements of existing functions diff --git a/compat/mingw.c b/compat/mingw.c index 5d2a8c247ce622..b407a2ac07f9d4 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -2698,6 +2698,30 @@ int link(const char *oldpath, const char *newpath) return 0; } +int readlink(const char *path, char *buf, size_t bufsiz) +{ + WCHAR wpath[MAX_PATH]; + char tmpbuf[MAX_PATH]; + int len; + DWORD tag; + + if (xutftowcs_path(wpath, path) < 0) + return -1; + + if (read_reparse_point(wpath, TRUE, tmpbuf, &len, &tag) < 0) + return -1; + + /* + * Adapt to strange readlink() API: Copy up to bufsiz *bytes*, potentially + * cutting off a UTF-8 sequence. Insufficient bufsize is *not* a failure + * condition. There is no conversion function that produces invalid UTF-8, + * so convert to a (hopefully large enough) temporary buffer, then memcpy + * the requested number of bytes (including '\0' for robustness). + */ + memcpy(buf, tmpbuf, min(bufsiz, len + 1)); + return min(bufsiz, len); +} + pid_t waitpid(pid_t pid, int *status, int options) { HANDLE h = OpenProcess(SYNCHRONIZE | PROCESS_QUERY_INFORMATION, From 1dd5f9d6cdad0f9a1fb8139274c99df008dd03ad Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sun, 24 May 2015 01:32:03 +0200 Subject: [PATCH 29/33] mingw: implement basic `symlink()` functionality (file symlinks only) Implement `symlink()`. This implementation always creates _file_ symlinks (remember: Windows discerns between symlinks pointing to directories and those pointing to files). Support for directory symlinks will be added in a subseqeuent commit. This implementation fails with `ENOSYS` if symlinks are disabled or unsupported. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw-posix.h | 3 +-- compat/mingw.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/compat/mingw-posix.h b/compat/mingw-posix.h index 896aa976b1949b..2d989fd762474e 100644 --- a/compat/mingw-posix.h +++ b/compat/mingw-posix.h @@ -121,8 +121,6 @@ struct utsname { * trivial stubs */ -static inline int symlink(const char *oldpath UNUSED, const char *newpath UNUSED) -{ errno = ENOSYS; return -1; } static inline int fchmod(int fildes UNUSED, mode_t mode UNUSED) { errno = ENOSYS; return -1; } #ifndef __MINGW64_VERSION_MAJOR @@ -195,6 +193,7 @@ int setitimer(int type, struct itimerval *in, struct itimerval *out); int sigaction(int sig, struct sigaction *in, struct sigaction *out); int link(const char *oldpath, const char *newpath); int uname(struct utsname *buf); +int symlink(const char *target, const char *link); int readlink(const char *path, char *buf, size_t bufsiz); /* diff --git a/compat/mingw.c b/compat/mingw.c index b407a2ac07f9d4..8d366794c44e55 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -2698,6 +2698,34 @@ int link(const char *oldpath, const char *newpath) return 0; } +int symlink(const char *target, const char *link) +{ + wchar_t wtarget[MAX_PATH], wlink[MAX_PATH]; + int len; + + /* fail if symlinks are disabled or API is not supported (WinXP) */ + if (!has_symlinks) { + errno = ENOSYS; + return -1; + } + + if ((len = xutftowcs_path(wtarget, target)) < 0 + || xutftowcs_path(wlink, link) < 0) + return -1; + + /* convert target dir separators to backslashes */ + while (len--) + if (wtarget[len] == '/') + wtarget[len] = '\\'; + + /* create file symlink */ + if (!CreateSymbolicLinkW(wlink, wtarget, 0)) { + errno = err_win_to_posix(GetLastError()); + return -1; + } + return 0; +} + int readlink(const char *path, char *buf, size_t bufsiz) { WCHAR wpath[MAX_PATH]; From 7b6dbc73f74e2fb6539034ab54f8d1a098d398da Mon Sep 17 00:00:00 2001 From: Karsten Blees Date: Sun, 24 May 2015 01:48:35 +0200 Subject: [PATCH 30/33] mingw: add support for symlinks to directories Symlinks on Windows have a flag that indicates whether the target is a file or a directory. Symlinks of wrong type simply don't work. This even affects core Win32 APIs (e.g. `DeleteFile()` refuses to delete directory symlinks). However, `CreateFile()` with FILE_FLAG_BACKUP_SEMANTICS does work. Check the target type by first creating a tentative file symlink, opening it, and checking the type of the resulting handle. If it is a directory, recreate the symlink with the directory flag set. It is possible to create symlinks before the target exists (or in case of symlinks to symlinks: before the target type is known). If this happens, create a tentative file symlink and postpone the directory decision: keep a list of phantom symlinks to be processed whenever a new directory is created in `mingw_mkdir()`. Limitations: This algorithm may fail if a link target changes from file to directory or vice versa, or if the target directory is created in another process. It's the best Git can do, though. Signed-off-by: Karsten Blees Signed-off-by: Johannes Schindelin --- compat/mingw.c | 164 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 164 insertions(+) diff --git a/compat/mingw.c b/compat/mingw.c index 8d366794c44e55..59a32e454ed7d1 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -296,6 +296,131 @@ int mingw_core_config(const char *var, const char *value, return 0; } +static inline int is_wdir_sep(wchar_t wchar) +{ + return wchar == L'/' || wchar == L'\\'; +} + +static const wchar_t *make_relative_to(const wchar_t *path, + const wchar_t *relative_to, wchar_t *out, + size_t size) +{ + size_t i = wcslen(relative_to), len; + + /* Is `path` already absolute? */ + if (is_wdir_sep(path[0]) || + (iswalpha(path[0]) && path[1] == L':' && is_wdir_sep(path[2]))) + return path; + + while (i > 0 && !is_wdir_sep(relative_to[i - 1])) + i--; + + /* Is `relative_to` in the current directory? */ + if (!i) + return path; + + len = wcslen(path); + if (i + len + 1 > size) { + error("Could not make '%ls' relative to '%ls' (too large)", + path, relative_to); + return NULL; + } + + memcpy(out, relative_to, i * sizeof(wchar_t)); + wcscpy(out + i, path); + return out; +} + +enum phantom_symlink_result { + PHANTOM_SYMLINK_RETRY, + PHANTOM_SYMLINK_DONE, + PHANTOM_SYMLINK_DIRECTORY +}; + +/* + * Changes a file symlink to a directory symlink if the target exists and is a + * directory. + */ +static enum phantom_symlink_result +process_phantom_symlink(const wchar_t *wtarget, const wchar_t *wlink) +{ + HANDLE hnd; + BY_HANDLE_FILE_INFORMATION fdata; + wchar_t relative[MAX_PATH]; + const wchar_t *rel; + + /* check that wlink is still a file symlink */ + if ((GetFileAttributesW(wlink) + & (FILE_ATTRIBUTE_REPARSE_POINT | FILE_ATTRIBUTE_DIRECTORY)) + != FILE_ATTRIBUTE_REPARSE_POINT) + return PHANTOM_SYMLINK_DONE; + + /* make it relative, if necessary */ + rel = make_relative_to(wtarget, wlink, relative, ARRAY_SIZE(relative)); + if (!rel) + return PHANTOM_SYMLINK_DONE; + + /* let Windows resolve the link by opening it */ + hnd = CreateFileW(rel, 0, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, + OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); + if (hnd == INVALID_HANDLE_VALUE) { + errno = err_win_to_posix(GetLastError()); + return PHANTOM_SYMLINK_RETRY; + } + + if (!GetFileInformationByHandle(hnd, &fdata)) { + errno = err_win_to_posix(GetLastError()); + CloseHandle(hnd); + return PHANTOM_SYMLINK_RETRY; + } + CloseHandle(hnd); + + /* if target exists and is a file, we're done */ + if (!(fdata.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)) + return PHANTOM_SYMLINK_DONE; + + /* otherwise recreate the symlink with directory flag */ + if (DeleteFileW(wlink) && CreateSymbolicLinkW(wlink, wtarget, 1)) + return PHANTOM_SYMLINK_DIRECTORY; + + errno = err_win_to_posix(GetLastError()); + return PHANTOM_SYMLINK_RETRY; +} + +/* keep track of newly created symlinks to non-existing targets */ +struct phantom_symlink_info { + struct phantom_symlink_info *next; + wchar_t *wlink; + wchar_t *wtarget; +}; + +static struct phantom_symlink_info *phantom_symlinks = NULL; +static CRITICAL_SECTION phantom_symlinks_cs; + +static void process_phantom_symlinks(void) +{ + struct phantom_symlink_info *current, **psi; + EnterCriticalSection(&phantom_symlinks_cs); + /* process phantom symlinks list */ + psi = &phantom_symlinks; + while ((current = *psi)) { + enum phantom_symlink_result result = process_phantom_symlink( + current->wtarget, current->wlink); + if (result == PHANTOM_SYMLINK_RETRY) { + psi = ¤t->next; + } else { + /* symlink was processed, remove from list */ + *psi = current->next; + free(current); + /* if symlink was a directory, start over */ + if (result == PHANTOM_SYMLINK_DIRECTORY) + psi = &phantom_symlinks; + } + } + LeaveCriticalSection(&phantom_symlinks_cs); +} + /* Normalizes NT paths as returned by some low-level APIs. */ static wchar_t *normalize_ntpath(wchar_t *wbuf) { @@ -479,6 +604,8 @@ int mingw_mkdir(const char *path, int mode UNUSED) if (xutftowcs_path(wpath, path) < 0) return -1; ret = _wmkdir(wpath); + if (!ret) + process_phantom_symlinks(); if (!ret && needs_hiding(path)) return set_hidden_flag(wpath, 1); return ret; @@ -2723,6 +2850,42 @@ int symlink(const char *target, const char *link) errno = err_win_to_posix(GetLastError()); return -1; } + + /* convert to directory symlink if target exists */ + switch (process_phantom_symlink(wtarget, wlink)) { + case PHANTOM_SYMLINK_RETRY: { + /* if target doesn't exist, add to phantom symlinks list */ + wchar_t wfullpath[MAX_PATH]; + struct phantom_symlink_info *psi; + + /* convert to absolute path to be independent of cwd */ + len = GetFullPathNameW(wlink, MAX_PATH, wfullpath, NULL); + if (!len || len >= MAX_PATH) { + errno = err_win_to_posix(GetLastError()); + return -1; + } + + /* over-allocate and fill phantom_symlink_info structure */ + psi = xmalloc(sizeof(struct phantom_symlink_info) + + sizeof(wchar_t) * (len + wcslen(wtarget) + 2)); + psi->wlink = (wchar_t *)(psi + 1); + wcscpy(psi->wlink, wfullpath); + psi->wtarget = psi->wlink + len + 1; + wcscpy(psi->wtarget, wtarget); + + EnterCriticalSection(&phantom_symlinks_cs); + psi->next = phantom_symlinks; + phantom_symlinks = psi; + LeaveCriticalSection(&phantom_symlinks_cs); + break; + } + case PHANTOM_SYMLINK_DIRECTORY: + /* if we created a dir symlink, process other phantom symlinks */ + process_phantom_symlinks(); + break; + default: + break; + } return 0; } @@ -3424,6 +3587,7 @@ int wmain(int argc, const wchar_t **wargv) /* initialize critical section for waitpid pinfo_t list */ InitializeCriticalSection(&pinfo_cs); + InitializeCriticalSection(&phantom_symlinks_cs); /* set up default file mode and file modes for stdin/out/err */ _fmode = _O_BINARY; From d3b89c29b01e5ed19b40f45714b3a1db800afa5d Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Tue, 30 May 2017 21:50:57 +0200 Subject: [PATCH 31/33] mingw: try to create symlinks without elevated permissions As of Windows 10 Build 14972 in Developer Mode, a new flag is supported by `CreateSymbolicLink()` to create symbolic links even when running outside of an elevated session (which was previously required). This new flag is called `SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE` and has the numeric value 0x02. Previous Windows 10 versions will not understand that flag and return an `ERROR_INVALID_PARAMETER`, therefore we have to be careful to try passing that flag only when the build number indicates that it is supported. For more information about the new flag, see this blog post: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/ Signed-off-by: Johannes Schindelin --- compat/mingw.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index 59a32e454ed7d1..3e2110a87ae35d 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -331,6 +331,8 @@ static const wchar_t *make_relative_to(const wchar_t *path, return out; } +static DWORD symlink_file_flags = 0, symlink_directory_flags = 1; + enum phantom_symlink_result { PHANTOM_SYMLINK_RETRY, PHANTOM_SYMLINK_DONE, @@ -381,7 +383,8 @@ process_phantom_symlink(const wchar_t *wtarget, const wchar_t *wlink) return PHANTOM_SYMLINK_DONE; /* otherwise recreate the symlink with directory flag */ - if (DeleteFileW(wlink) && CreateSymbolicLinkW(wlink, wtarget, 1)) + if (DeleteFileW(wlink) && + CreateSymbolicLinkW(wlink, wtarget, symlink_directory_flags)) return PHANTOM_SYMLINK_DIRECTORY; errno = err_win_to_posix(GetLastError()); @@ -2846,7 +2849,7 @@ int symlink(const char *target, const char *link) wtarget[len] = '\\'; /* create file symlink */ - if (!CreateSymbolicLinkW(wlink, wtarget, 0)) { + if (!CreateSymbolicLinkW(wlink, wtarget, symlink_file_flags)) { errno = err_win_to_posix(GetLastError()); return -1; } @@ -3523,6 +3526,24 @@ static void maybe_redirect_std_handles(void) GENERIC_WRITE, FILE_FLAG_NO_BUFFERING); } +static void adjust_symlink_flags(void) +{ + /* + * Starting with Windows 10 Build 14972, symbolic links can be created + * using CreateSymbolicLink() without elevation by passing the flag + * SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE (0x02) as last + * parameter, provided the Developer Mode has been enabled. Some + * earlier Windows versions complain about this flag with an + * ERROR_INVALID_PARAMETER, hence we have to test the build number + * specifically. + */ + if (GetVersion() >= 14972 << 16) { + symlink_file_flags |= 2; + symlink_directory_flags |= 2; + } + +} + #ifdef _MSC_VER #ifdef _DEBUG #include @@ -3558,6 +3579,7 @@ int wmain(int argc, const wchar_t **wargv) #endif maybe_redirect_std_handles(); + adjust_symlink_flags(); /* determine size of argv and environ conversion buffer */ maxlen = wcslen(wargv[0]); From 2e73ab42217496739adc5881d900552bce006c05 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Mon, 2 Mar 2020 21:54:29 +0100 Subject: [PATCH 32/33] mingw: emulate `stat()` a little more faithfully When creating directories via `safe_create_leading_directories()`, we might encounter an already-existing directory which is not readable by the current user. To handle that situation, Git's code calls `stat()` to determine whether we're looking at a directory. In such a case, `CreateFile()` will fail, though, no matter what, and consequently `mingw_stat()` will fail, too. But POSIX semantics seem to still allow `stat()` to go forward. So let's call `mingw_lstat()` to the rescue if we fail to get a file handle due to denied permission in `mingw_stat()`, and fill the stat info that way. We need to be careful to not allow this to go forward in case that we're looking at a symbolic link: to resolve the link, we would still have to create a file handle, and we just found out that we cannot. Therefore, `stat()` still needs to fail with `EACCES` in that case. This fixes https://github.com/git-for-windows/git/issues/2531. Signed-off-by: Johannes Schindelin --- compat/mingw.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/compat/mingw.c b/compat/mingw.c index 3e2110a87ae35d..628a3941d24647 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -1273,7 +1273,19 @@ int mingw_stat(const char *file_name, struct stat *buf) FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); if (hnd == INVALID_HANDLE_VALUE) { - errno = err_win_to_posix(GetLastError()); + DWORD err = GetLastError(); + + if (err == ERROR_ACCESS_DENIED && + !mingw_lstat(file_name, buf) && + !S_ISLNK(buf->st_mode)) + /* + * POSIX semantics state to still try to fill + * information, even if permission is denied to create + * a file handle. + */ + return 0; + + errno = err_win_to_posix(err); return -1; } result = get_file_info_by_handle(hnd, buf); From 817f488523099b784432742328ea69646c6abafc Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Thu, 4 Jun 2020 23:16:07 +0200 Subject: [PATCH 33/33] mingw: special-case index entries for symlinks with buggy size In https://github.com/git-for-windows/git/pull/2637, we fixed a bug where symbolic links' target path sizes were recorded incorrectly in the index. The downside of this fix was that every user with tracked symbolic links in their checkouts would see them as modified in `git status`, but not in `git diff`, and only a `git add ` (or `git add -u`) would "fix" this. Let's do better than that: we can detect that situation and simply pretend that a symbolic link with a known bad size (or a size that just happens to be that bad size, a _very_ unlikely scenario because it would overflow our buffers due to the trailing NUL byte) means that it needs to be re-checked as if we had just checked it out. Signed-off-by: Johannes Schindelin --- read-cache.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/read-cache.c b/read-cache.c index 990d4ead0d8ae4..260f4b3b2fd55c 100644 --- a/read-cache.c +++ b/read-cache.c @@ -470,6 +470,17 @@ int ie_modified(struct index_state *istate, * then we know it is. */ if ((changed & DATA_CHANGED) && +#ifdef GIT_WINDOWS_NATIVE + /* + * Work around Git for Windows v2.27.0 fixing a bug where symlinks' + * target path lengths were not read at all, and instead recorded + * as 4096: now, all symlinks would appear as modified. + * + * So let's just special-case symlinks with a target path length + * (i.e. `sd_size`) of 4096 and force them to be re-checked. + */ + (!S_ISLNK(st->st_mode) || ce->ce_stat_data.sd_size != MAX_PATH) && +#endif (S_ISGITLINK(ce->ce_mode) || ce->ce_stat_data.sd_size != 0)) return changed;