Skip to content

Conversation

@SteveL-MSFT
Copy link
Member

@SteveL-MSFT SteveL-MSFT commented May 1, 2019

PR Summary

The current logic of Get-ChildItem -Recurse for the FileSystemProvider is that if it's a symlink, we don't recurse into it unless -FollowSymLink is specified. However, OneDrive reparse points aren't symlinks (in that they don't link to the local filesystem). There is another flag (on Windows) to indicate if the reparse point is a named surrogate which means it IS a symlink. So the fix is to check for this flag and if it's not a named surrogate (aka symlink), it is ok to recurse into it.

Manually verified with OneDrive.

PR Context

Fix #9461

PR Checklist

@iSazonov iSazonov added the CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log label May 2, 2019
@iSazonov iSazonov added this to the 7.0.0-preview.1 milestone May 2, 2019
@sba923
Copy link
Contributor

sba923 commented May 3, 2019

Impressive fix... relying on very undocumented stuff if you ask me... unless "named surrogates" and "data.dwReserved0 & 0x20000000" is documented somewhere. @SteveL-MSFT?

  • my understanding is that there won't be a 6.3 before 7.0, correct?

@SteveL-MSFT
Copy link
Member Author

@sba923 correct, next version is 7.0-preview.1. As noted in the comments in the change, the use of dwReserved0 & 0x20000000 is documented via the macro published in the Windows SDK header. I agree that it would have been better as an API (or even better just another FileAttribute), but that's what we have to work with right now.

@sba923
Copy link
Contributor

sba923 commented May 3, 2019

@SteveL-MSFT Thanks for the confirmation about 7.0. I had missed the comment with the reference to the IsReparseTagNameSurrogate macro, sorry for that. Reparse point tags are documented here, this documents the N (Name surrogate) bit with value 0x20000000, and links to another documentation for the macro.

The documentation for WIN32_FIND_DATAA documents that symlinks use a reparse point tag of IO_REPARSE_TAG_SYMLINK (0xA000000C) that has the N bit set. To satisfy my curiosity, I will check what reparse point tag (with the N bit not set) is used by OneDrive (and request that it be added to the documentation).

{
#if !UNIX
var data = new WIN32_FIND_DATA();
using (SafeFileHandle handle = FindFirstFileEx(filePath, FINDEX_INFO_LEVELS.FindExInfoBasic, ref data, FINDEX_SEARCH_OPS.FindExSearchNameMatch, IntPtr.Zero, 0))
Copy link
Collaborator

@iSazonov iSazonov May 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could use method with fewer parameters
https://github.com/dotnet/corefx/blob/87089ef4559605f8a34ca8959a0ef9c71aa1e02e/src/System.IO.FileSystem/src/System/IO/FileSystem.Windows.cs#L241

I do very wonder that CoreFX uses IsNameSurrogateReparsePoint() method only for recurse directory remove but not for recurse directory read.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CoreFx uses FindFirstFileEx() as well, they just name their wrapper FindFirstFile()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the second point, I agree it might be an oversight that they don't have symmetric behavior.

@adityapatwardhan adityapatwardhan merged commit 1d28144 into PowerShell:master May 7, 2019
@sba923
Copy link
Contributor

sba923 commented May 12, 2019

I've collected my findings about the reparse tags used by OneDrive in this spreadsheet.

https://docs.microsoft.com/en-us/windows/desktop/w8cookbook/placeholder-files is definitely outdated, as Windows 10 OneDrive doesn't use IO_REPARSE_TAG_FILE_PLACEHOLDER anymore. Instead, it uses a number of tags from IO_REPARSE_TAG_CLOUD_1 to IO_REPARSE_TAG_CLOUD_F (on my system I found only 3 values being used), with a IO_REPARSE_TAG_CLOUD_MASK that apparently can be used to compute (tag & IO_REPARSE_TAG_CLOUD_MASK) >> 12 and get some value between 1 and 15 that the OneDrive sync client probably uses as information about the item...

I'm not sure where to post the request to document all this. @SteveL-MSFT: can you help?

@sba923
Copy link
Contributor

sba923 commented May 12, 2019

For now I've opened MicrosoftDocs/feedback#1484 and MicrosoftDocs/feedback#1483

@SteveL-MSFT SteveL-MSFT deleted the ondrive-recurse branch May 12, 2019 20:04
IntPtr hTemplateFile);

[DllImport(PinvokeDllNames.FindFirstFileDllName, EntryPoint = "FindFirstFileExW", SetLastError = true, CharSet = CharSet.Unicode)]
private static extern SafeFileHandle FindFirstFileEx(string lpFileName, FINDEX_INFO_LEVELS fInfoLevelId, ref WIN32_FIND_DATA lpFindFileData, FINDEX_SEARCH_OPS fSearchOp, IntPtr lpSearchFilter, int dwAdditionalFlags);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This returns a SafeFileHandle, which eventually calls CloseHandle.
According to the FindFirstFileExW documentation, the handle should be closed with FindClose, not CloseHandle.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! Will submit a new PR to address.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Get-ChildItem -Recurse doesn't work on the OneDrive sync folder

5 participants