Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Documentation: document how to pull bookmarks from Firefox Sync #648

Closed
khimaros opened this issue Feb 6, 2021 · 3 comments
Closed

Documentation: document how to pull bookmarks from Firefox Sync #648

khimaros opened this issue Feb 6, 2021 · 3 comments
Labels
good first ticket help wanted size: easy touches: docs

Comments

@khimaros
Copy link

khimaros commented Feb 6, 2021

Wiki Page URL

https://github.com/ArchiveBox/ArchiveBox/wiki/Quickstart#2-get-your-list-of-urls-to-archive

Suggested Edit

document how to pull bookmarks and/or history from the Firefox Sync server.

this enables the user to simply bookmark pages they want to archive on their desktop and mobile browser and they will automatically be archived by archivebox.

here's what worked for me.

determine your Firefox Sync client id:

$ python3 -m pip install PyFxA

$ fxa-client --bearer -u <YOUR EMAIL> \
    --account-server https://api.accounts.firefox.com/v1 \
    --oauth-server https://oauth.accounts.firefox.com/v1
# ---- BEARER TOKEN INFO ----
# User: <YOUR EMAIL>
# Scopes: profile
# Account: https://api.accounts.firefox.com/v1
# Oauth: https://oauth.accounts.firefox.com/v1
# Client ID: <YOUR CLIENT ID>
# ---------------------------

import all of your bookmarks into archivebox:

$ git clone https://github.com/eNote-GmbH/syncclient

$ cd syncclient

$ python3 -m pip install -e .

$ python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> \
    get_records bookmarks \
        | jq '.[].payload | fromjson | select(.type == "bookmark") | .bmkUri)' \
        | archivebox add

this also works with a self hosted Firefox Sync server by setting the env var TOKENSERVER_URL="https://<YOUR SYNC SERVER>/token". you can configure this to run on a regular schedule eg. via cron.

with a bit of work, you could make this only archive bookmarks with a specific tag or folder.
@kominoshja
Copy link

Note: This requires disabling 2FA while setting it up

@kominoshja
Copy link

kominoshja commented Nov 30, 2021 •
edited
Loading

When you issue python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> get_records bookmarks, you get something like

"6J***********3Vk"
"3_***********Iw7"
"U1***********ah"

therefore, jq fails. You'd need to run python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> get_record bookmarks --decrypt 6J***********3Vk | jq '.payload | fromjson | select(.type == "bookmark") | .bmkUri'

I created the following script to iterate over the response and add bookmarks to archivebox (Keep in mind you need to change BATCH_SIZE to archive more than 4 at a time): https://gist.github.com/kominoshja/484dd83f978079daf6c3bacebb21fe37

@pirate
Copy link
Member

pirate commented Jan 19, 2024

Going to close this for now, I'll add a link to this issue from the README in case it helps ppl in the future trying to set this up.

@pirate pirate closed this as completed Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first ticket help wanted size: easy touches: docs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@khimaros @pirate @kominoshja