Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Add WGET_EXTRA_ARGS, CURL_EXTRA_ARGS, SINGLEFILE_EXTRA_ARGS to extend default args without overriding defaults #1025

Open
notevenaperson opened this issue Sep 12, 2022 · 1 comment
Labels
expected: maybe someday good first ticket help wanted size: easy status: backlog Work is planned someday but is not the highest priority at the moment touches: configuration why: functionality Intended to improve ArchiveBox functionality or features

Comments

@notevenaperson
Copy link
Contributor

notevenaperson commented Sep 12, 2022

These WGET_ARGS, CURL_ARGS, etc. options let the user shoot themselves in the foot, silently. I think the documentation or the variables themselves should be changed to be more ergonomic.

Did you need to add a header to wget?

WGET_ARGS=['--header=Accept-Language: en-US,en']

Look at this configuration, it looks inoffensive. Checking the documention, nothing ticks you off you're using it wrong. But no.
After this, your 🆆 button won't take you to the main html you archived, and wget archives things slighlty differently, silently.

Why? By setting WGET_ARGS you overwrote vital settings, as it turns out they're also stored in WGET_ARGS. The documentation doesn't tell you about this. This happened to me, I just happily overwrote the variable. When I should've written something like this:

WGET_ARGS=['--header=Accept-Language: en-US,en;q=0.5', '--no-verbose', '--adjust-extension', '--convert-links', '--force-directories', '--backup-converted', '--span-hosts', '--no-parent', '-e', 'robots=off']

Proposal:

  1. The documentation for *_ARGS should have a good warning or display the default value, so that users can suspect they're overwriting something.
  2. Or, create EXTRA_WGET_ARGS, EXTRA_CURL_ARGS, and so on. These won't overwrite the now-considered low-level *_ARGS. EXTRA_*_ARGS shall be the more user-facing option and more promoted in the documentation.
@notevenaperson notevenaperson changed the title Footguns: WGET_ARGS, CURL_ARGS, SINGLEFILE_ARGS Footguns: WGET_ARGS, CURL_ARGS, SINGLEFILE_ARGS Sep 12, 2022
@pirate
Copy link
Member

pirate commented Sep 28, 2022

I'm ok with the EXTRA_ options as I want users to be able to override the defaults.

@pirate pirate changed the title Footguns: WGET_ARGS, CURL_ARGS, SINGLEFILE_ARGS Enhancement: Add WGET_EXTRA_ARGS, CURL_EXTRA_ARGS, SINGLEFILE_EXTRA_ARGS to extend default args without overriding defaults Jun 13, 2023
@pirate pirate added touches: configuration why: functionality Intended to improve ArchiveBox functionality or features size: easy status: backlog Work is planned someday but is not the highest priority at the moment expected: maybe someday good first ticket help wanted labels Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
expected: maybe someday good first ticket help wanted size: easy status: backlog Work is planned someday but is not the highest priority at the moment touches: configuration why: functionality Intended to improve ArchiveBox functionality or features
Projects
None yet
Development

No branches or pull requests

2 participants