Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: ArchiveBox doesn't work on NFS/SMB/FUSE drives that disallow root ownership or have root_squash set #1304

Closed
gnattu opened this issue Dec 30, 2023 · 6 comments
Closed

Bug: ArchiveBox doesn't work on NFS/SMB/FUSE drives that disallow root ownership or have root_squash set #1304

gnattu opened this issue Dec 30, 2023 · 6 comments

Comments

@gnattu
Copy link
Contributor

gnattu commented Dec 30, 2023

Describe the bug

If the docker-entry point failed to chown the archive folder, it quits immediately, so that the container does not run at all, make it impossible to run archivebox with an nfs-mounted volume as the archive folder.

I'm currently using this with a modified entrypoint which removed the line to chmod everything under DATA dir.

Steps to reproduce

  1. Create an NFS volume with docker: docker volume create --driver local --opt type=nfs --opt o=addr=[ip-address],rw --opt device=:[path-to-directory] [volume-name]

  2. Use this as the archivefolder in docker-compose.yml: - volume-name:/data/archive

  3. Start container. The conatiner quit right after it failed to chown the nfs mounted folder

Screenshots or log output

[+] Creating 1/0
 ✔ Container archivebox-sonic-1  Running                                                                     0.0s
chown: changing ownership of '/data/archive': Operation not permitted

ArchiveBox version

[+] Creating 1/0
 ✔ Container archivebox-sonic-1  Running                                                                     0.0s
0.7.1
ArchiveBox v0.7.1+editable BUILD_TIME=2023-12-18 06:57:08 1702882628
IN_DOCKER=True IN_QEMU=False ARCH=aarch64 OS=Linux PLATFORM=Linux-6.5.13-orbstack-00121-ge428743e4e98-aarch64-with-glibc2.36 PYTHON=Cpython
FS_ATOMIC=True FS_REMOTE=True FS_USER=1000:1000 FS_PERMS=644
DEBUG=False IS_TTY=True TZ=UTC SEARCH_BACKEND=sonic LDAP=False

[i] Dependency versions:
 √  PYTHON_BINARY         v3.11.7         valid     /usr/local/bin/python3.11                                     
 √  SQLITE_BINARY         v2.6.0          valid     /usr/local/lib/python3.11/sqlite3/dbapi2.py                   
 √  DJANGO_BINARY         v3.1.14         valid     /usr/local/lib/python3.11/site-packages/django/__init__.py    
 √  ARCHIVEBOX_BINARY     v0.7.1          valid     /usr/local/bin/archivebox                                     

 √  CURL_BINARY           v8.4.0          valid     /usr/bin/curl                                                 
 √  WGET_BINARY           v1.21.3         valid     /usr/bin/wget                                                 
 √  NODE_BINARY           v21.4.0         valid     /usr/bin/node                                                 
 √  SINGLEFILE_BINARY     v1.1.18         valid     /app/node_modules/single-file-cli/single-file                 
 √  READABILITY_BINARY    v0.0.9          valid     /app/node_modules/readability-extractor/readability-extractor 
 √  MERCURY_BINARY        v1.0.0          valid     /app/node_modules/@postlight/parser/cli.js                    
 √  GIT_BINARY            v2.39.2         valid     /usr/bin/git                                                  
 √  YOUTUBEDL_BINARY      v2023.11.16     valid     /usr/local/bin/yt-dlp                                         
 √  CHROME_BINARY         v120.0.6099.28  valid     /browsers/chromium-1091/chrome-linux/chrome                   
 √  RIPGREP_BINARY        v13.0.0         valid     /usr/bin/rg                                                   

[i] Source-code locations:
 √  PACKAGE_DIR           24 files        valid     /app/archivebox                                               
 √  TEMPLATES_DIR         4 files         valid     /app/archivebox/templates                                     
 -  CUSTOM_TEMPLATES_DIR  -               disabled  None                                                          

[i] Secrets locations:
 -  CHROME_USER_DATA_DIR  -               disabled  None                                                          
 -  COOKIES_FILE          -               disabled  None                                                          

[i] Data locations:
 √  OUTPUT_DIR            9 files @       valid     /data                                                         
 √  SOURCES_DIR           77 files        valid     ./sources                                                     
 √  LOGS_DIR              1 files         valid     ./logs                                                        
 √  ARCHIVE_DIR           361 files @     valid     ./archive                                                     
 √  CONFIG_FILE           162.0 Bytes     valid     ./ArchiveBox.conf                                             
 √  SQL_INDEX             4.7 MB          valid     ./index.sqlite3       ```
<!-- Tickets without full version info will closed until it is provided,
we need the full output here to help you solve your issue -->
@pirate
Copy link
Member

pirate commented Jan 1, 2024

This only happens if your NFS volume disables or remaps permissions. The solution is not to remove the chmod but rather to set PUID & PGID environment variables to the same values that your NFS server enforces.

@gnattu
Copy link
Contributor Author

gnattu commented Jan 1, 2024

set PUID & PGID environment variables to the same values that your NFS server enforces.

This is not always do-able because my server enforces the owner group id to 0 for all files and in this case, you cannot just set PGID=0 in env to fix this issue because you will have to allow root access for this NFS client as well. The PUID is already set to the correct value and that PUID does have read/write permission in the mounted dir. (And actually, the owner of the folder)

@pirate
Copy link
Member

pirate commented Jan 3, 2024 •
edited
Loading

hmm that's somewhat usual but I guess I can allow PGID=0.

I just pushed a commit to :dev to remove the PGID!=0 check, can you try it out?

You can get it with docker pull archivebox/archivebox:dev or follow these instructions: https://github.com/ArchiveBox/ArchiveBox#install-and-run-a-specific-github-branch

@pirate pirate changed the title Bug: docker-entrypoint would not excecute if it cannot chown the archive folder Bug: PGID=0 is not allowed, which breaks when FUSE/NFS drives try to force-set group ownership to 0 Jan 3, 2024
@gnattu
Copy link
Contributor Author

gnattu commented Jan 3, 2024

The strange thing is, even after set PGID=0 using archivebox/archivebox:dev, the container still failed with chown: changing ownership of '/data/archive': Operation not permitted.

@pirate
Copy link
Member

pirate commented Jan 3, 2024 •
edited
Loading

Ah it looks like you're using the root_squash option on the server. This prevents all chown calls from working, and is unfortunately not easily supported by ArchiveBox at the moment.

AB in Docker starts running as root to create the data dir and set it up correctly, so when it drops down to a sub-user with fewer permissions it wont be able to modify the root_squashed files it just created as they'll now be owned by the NFS anonymous/nobody user. If you're able to disable that option on the server or set no_root_squash, it should work.

The security benefit that root_squash provides is limited anyway, as root on NFS is effectively able to impersonate any UID. root_squashing only prevents a specific SUID escalation attack but doesn't stop clients from impersonating other users.

https://superuser.com/questions/1737302/root-squashing-for-nfs-and-smb-clarification
https://serverfault.com/questions/88114/cant-modify-or-chown-files-on-readynas-nfs-share

@pirate pirate changed the title Bug: PGID=0 is not allowed, which breaks when FUSE/NFS drives try to force-set group ownership to 0 Bug: ArchiveBox doesn't work on NFS/SMB/FUSE drives that disallow root ownership or have root_squash set Jan 4, 2024
@jamesob
Copy link
Contributor

jamesob commented Jan 4, 2024

In my case, the fix here was to disable root_squash on my NFS-via-ZFS share (on the NAS server), like so:

zfs set sharenfs="no_root_squash,[email protected]/24,[email protected]/24" tank/nas

and then verify that the NFS server exports reflects the new option with # exportfs -v. I then recreated the archivebox docker container and stopped getting this permission error.

@pirate pirate closed this as completed Jan 5, 2024
@gnattu gnattu mentioned this issue Jan 5, 2024
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jamesob @pirate @gnattu