-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
onedrive: Error reading destination directory with >100000 files #2707
Comments
@Cnly have you got time to look through the forum thread and give us your opinion? It looks like to me that this directory with a large number of files takes more than 20s for the initial listing so times out. I wonder if we can set the timeout any higher at onedrive? |
Note that there is a thread here #2723 (comment) which summarised says 100,000 files is too many, but 50,000 files is OK. I'll update the docs with that in a moment |
Thanks for sorting that out... I think I should also open an issue in the OneDrive API repo to get some clarification. In that case, it'd be the best if we have a Request ID. @gualdo0 I saw your earlier forum post but the Request ID part is removed. Do you mind providing some log with that information so we can open an issue here (https://github.com/OneDrive/onedrive-api-docs/issues)? |
...can we move discussion of the listing number problem back into #2707 where it belongs? |
@Cnly sorry what is the Request ID? If you mean the post in the rclone forum, the link is |
Oh yes, of course, please @ncw provide that log file if it is useful to better describe the issue.
Regards.
…________________________________
De: Alex Chen <[email protected]>
Enviado: miércoles, 28 de noviembre de 2018 11:00
Para: ncw/rclone
Cc: González Alfonso Eduardo; Mention
Asunto: Re: [ncw/rclone] onedrive: Error reading destination directory with >100000 files (#2707)
Sorry for the delay! I looked at the forum post again and it seems @gualdo0<https://github.com/gualdo0> you had sent your log privately to @ncw<https://github.com/ncw>. If @gualdo0<https://github.com/gualdo0> you agree, I'd like to ask @ncw<https://github.com/ncw> to help provide a snippet of your original log containing the Request ID.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#2707 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AqdiHzTblkpWueI1NwI0aF_Zeq4UfWxwks5uzl6ogaJpZM4X9ZNi>.
|
I've forwarded the log files to @Cnly via email :-) |
Thanks! I've opened an upstream issue. |
@gualdo0 There's an update from the upstream. I see in your previous log that the error happens multiple times. But does it still happen now? If it does, can you reproduce that and share your logs again? Some more recent logs may help solve this better. If you don't want to post the logs publicly, you can also send them to me at [email protected]. Thanks! |
@gualdo0 Does the problem reproduce with rclone 1.54? |
No answer since 2018. Closing. |
Output of
rclone version
rclone v1.44-022-ge606d456-fix-onedrive-list-beta
Describe the issue
Hello.
The issue I have with rclone, when trying to sync a local folder with many files (>100000) and a remote folder on OneDrive, is described here:
https://forum.rclone.org/t/error-reading-destination-directory/7316
As you will see, there is apparently a timeout, so the operation at microsoft took longer than some timeout - which looks like 20s (thanks to Nick for looking at this).
Any help would be greatly appreciated.
The text was updated successfully, but these errors were encountered: