Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature Request: New extractor to download javascript/ts sourcemap files for any compiled/minified .js assets used in archived pages #1291

Open
jensolsson opened this issue Dec 17, 2023 · 3 comments
Open

Feature Request: New extractor to download javascript/ts sourcemap files for any compiled/minified .js assets used in archived pages #1291

jensolsson opened this issue Dec 17, 2023 · 3 comments
Labels
expected: unlikely unless contributed This change is unlikely to be made unless someone contributes a PR for review. help wanted size: medium status: idea-phase Work is tentatively approved and is being planned / laid out, but is not ready to be implemented yet touches: configuration touches: data/schema/architecture touches: dependencies/packaging Issues or changes that add/remove/affect dependencies touches: docs why: functionality Intended to improve ArchiveBox functionality or features

Comments

@jensolsson
Copy link

jensolsson commented Dec 17, 2023 •
edited
Loading

I love how ArchiveBox is downloading javascript files from a specific site, however to make minified javascript readable it would be great to also include the .map file (called javascript source maps. If I understand correctly they are the same name but ends in .js.map instead of .js). Can this be added easily ?

If someone would give me some pointers on where to start I could probably do the work
@jensolsson jensolsson changed the title Feature Request: ... Feature Request: Try to download javascript.map also if javascript.js is downloaded Dec 17, 2023
@pirate
Copy link
Member

pirate commented Dec 17, 2023 •
edited
Loading

This would likely require a new extractor, since none of our existing extractors try to download or are even aware of .map files.

You can check out our docs on that process here: https://github.com/ArchiveBox/ArchiveBox#contributing-a-new-extractor

Though to be honest this one would be fairly low on my priority list as it's just for developer experience and doesn't visually impact replay fidelity.

If you're interested in contributing an extractor, we'd love to have help adding one of these higher-impact ones, and I'm offering $250~$1000+ bounties for contributions at the moment:

  • an extractor to save discussion threads from forums, comment sections, reddit, twitter, etc as markdown/json
  • an extractor to fetch galleries of images e.g. gallery-dl
  • an extractor to fetch linked 3D assets like meshes, shaders, STL files, and other CAD models
  • an extractor to fetch research papers referenced by DOI numbers from scihub/jstor/etc

@pirate pirate changed the title Feature Request: Try to download javascript.map also if javascript.js is downloaded Feature Request: New extractor to download javascript/ts sourcemap files for any compiled/minified .js assets used in archived pages Dec 17, 2023
@pirate pirate added status: idea-phase Work is tentatively approved and is being planned / laid out, but is not ready to be implemented yet size: medium touches: configuration why: functionality Intended to improve ArchiveBox functionality or features touches: data/schema/architecture help wanted touches: dependencies/packaging Issues or changes that add/remove/affect dependencies touches: docs type: enhancement expected: unlikely unless contributed This change is unlikely to be made unless someone contributes a PR for review. labels Dec 18, 2023
@Myestery
Copy link

Hello @pirate, if this bounty is still up I'll like to work on it

@pirate
Copy link
Member

pirate commented Oct 24, 2024

The bounty is unfortunatley no longer available, but once the plugin ecosystem is released in v0.9.0 https://docs.sweeting.me/s/archivebox-plugin-ecosystem-announcement it should be much easier to contribute new extractors, please check back in a bit!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
expected: unlikely unless contributed This change is unlikely to be made unless someone contributes a PR for review. help wanted size: medium status: idea-phase Work is tentatively approved and is being planned / laid out, but is not ready to be implemented yet touches: configuration touches: data/schema/architecture touches: dependencies/packaging Issues or changes that add/remove/affect dependencies touches: docs why: functionality Intended to improve ArchiveBox functionality or features
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pirate @jensolsson @Myestery