Adding ol-web3 to web head cluster#12433
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds ol-web3 as an additional production web head and updates deployment/provisioning tooling to support bringing up a fresh Debian-based node and optionally deploying a specific git ref.
Changes:
- Extend host lists (deploy/restart/sync/test scripts) and docker compose profiles to include
ol-web3. - Add a new Debian-oriented provisioning script under
scripts/deployment/and remove the olderscripts/setup_olserver.sh. - Update
deploy.shto tolerate missing/opt/*repos during provisioning and addGIT_BRANCHsupport + move deploy tagging into the deploy wizard.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/solr_builder/utils.sh | Include ol-web3 in search/solr-switch helper loops. |
| scripts/setup_olserver.sh | Remove legacy Ubuntu-oriented provisioning script. |
| scripts/deployment/restart_servers.sh | Add ol-web3 to default restart set. |
| scripts/deployment/provision_olserver.sh | New Debian provisioning checklist/script for new nodes. |
| scripts/deployment/pre_deploy.sh | Add ol-web3 to pre-deploy status checks. |
| scripts/deployment/patchdeploy.sh | Update usage/defaults to include ol-web3. |
| scripts/deployment/patch_test_new_memcache_server.sh | Add warning that the script is outdated. |
| scripts/deployment/deploy.sh | Add ol-web3, handle missing repo dirs, add GIT_BRANCH, adjust tagging behavior. |
| scripts/deployment/are_servers_in_sync.sh | Add ol-web3 to sync checks. |
| compose.production.yaml | Enable web, fast_web, and monitoring profiles on ol-web3. |
Comment on lines
+670
to
+672
| echo "Creating git tag for deploy... " | ||
| tag_deploy | ||
|
|
There was a problem hiding this comment.
deploy_wizard calls tag_deploy unconditionally, but tag_deploy assumes $DEPLOY_DIR/openlibrary exists. If the user answers "n" to running deploy_openlibrary, the tag step will fail (and with set -e will abort the wizard). Gate tagging on whether the openlibrary deploy actually ran / the repo directory exists, or move tagging into the branch that runs deploy_openlibrary.
Comment on lines
+378
to
+383
| echo -ne "Cloning openlibrary repo (branch: $git_branch) ... " | ||
| git clone --depth=1 "https://github.com/internetarchive/openlibrary.git" openlibrary 2> /dev/null | ||
| if [ -n "$git_branch" ]; then | ||
| git -C openlibrary fetch --depth=1 origin $git_branch | ||
| git -C openlibrary checkout FETCH_HEAD | ||
| fi |
There was a problem hiding this comment.
The new GIT_BRANCH support claims to allow deploying a SHA, but git fetch origin $git_branch treats the value as a remote ref. For a raw commit SHA this will typically fail with "couldn't find remote ref". Consider handling SHAs explicitly (e.g., full clone or a fallback fetch strategy) and validating the ref before checkout so GIT_BRANCH reliably supports tag/branch/SHA as intended.
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
IvanPisquiy06
pushed a commit
to IvanPisquiy06/openlibrary
that referenced
this pull request
Apr 27, 2026
* Adding ol-web3 to web head cluster * Rename server setup script + move to deployment directory * Update provision script + tweak some deploy.sh configs
IvanPisquiy06
pushed a commit
to IvanPisquiy06/openlibrary
that referenced
this pull request
Apr 27, 2026
* Adding ol-web3 to web head cluster * Rename server setup script + move to deployment directory * Update provision script + tweak some deploy.sh configs
4 tasks
Sadashii
pushed a commit
to Sadashii/openlibrary
that referenced
this pull request
May 11, 2026
* Adding ol-web3 to web head cluster * Rename server setup script + move to deployment directory * Update provision script + tweak some deploy.sh configs
pull Bot
pushed a commit
to ppker/openlibrary
that referenced
this pull request
May 12, 2026
- tag_deploy/tag_release: fall back to the script's own repo when $DEPLOY_DIR/openlibrary doesn't exist, so './deploy.sh tag' works standalone after cleanup and the wizard no longer aborts if the openlibrary deploy step is skipped (fixes regression from internetarchive#12433) - deploy_openlibrary: move tag_deploy call inside the function (after the transfer block) so it runs for standalone deploys, not just the wizard - Add SKIP_TRANSFER env var: set SKIP_TRANSFER=1 to skip file copy, git-init-on-servers, prune, and image pulls -- useful when rerunning after a partial failure where servers already have current files - Fix wizard regex typo ^[Yy$] -> ^[Yy]$ (would have matched literal $) - Remove dead commented-out tag_deploy in wizard and redundant git describe call (tag_release already sets $LATEST_TAG_NAME)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding a new web head to deal with increased traffic and near-constant webpy worker saturation.
Technical
Testing
Screenshot
Stakeholders