Skip to content

Comments

Auto-discover news HTML files in Vite config to fix missing S3 uploads#427

Merged
pethers merged 5 commits intomainfrom
copilot/investigate-s3-upload-issue
Feb 22, 2026
Merged

Auto-discover news HTML files in Vite config to fix missing S3 uploads#427
pethers merged 5 commits intomainfrom
copilot/investigate-s3-upload-issue

Conversation

Copy link
Contributor

Copilot AI commented Feb 22, 2026

  • vite.config.js: Replace all hardcoded HTML entries with discoverHtmlEntries() — recursively scans the entire repo (root, dashboard/, news/) for *.html files, ignoring source-only dirs. New pages added anywhere are automatically included.
  • deploy-s3.yml: Add "Copy static assets to dist" step — copies non-HTML deployable files (CNAME, robots.txt, sitemap.xml, manifest.json, cia-data/, js/, data/, news/metadata/) into dist/ so S3 sync picks them up
  • deploy-s3.yml: Add "Verify deployment completeness" step — FAILS the build if ANY repo HTML file or static asset is missing from dist/. Never silently skip files again.
  • deploy-s3.yml: Simplify S3 sync — remove fragile exclude patterns from catch-all, add CSV cache headers, add final sync for everything remaining
  • s3-troubleshoot.yml: Add static asset copy step for consistent diagnostics
  • Verified: dist/ now contains all 530 HTML + 1256 total files (including cia-data CSVs, js/, robots.txt, CNAME, etc.)
  • All 1216 tests pass
  • Code review and security checks

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Copilot AI and others added 2 commits February 22, 2026 03:58
…rticles in S3 deployment

Root cause: vite.config.js used hardcoded rollupOptions.input entries for news articles.
336 news HTML files (including 2026-02-22-justice-reform-*) existed in news/ but were
not listed, so Vite excluded them from dist/ and S3 deploy never uploaded them.

Fix: replaced hardcoded entries with dynamic fs.readdirSync() discovery of news/*.html.
Also added s3-troubleshoot.yml workflow for manual AWS CLI diagnostics.

Co-authored-by: pethers <[email protected]>
Copilot AI changed the title [WIP] Investigate missing S3 uploads for justice reform files Auto-discover news HTML files in Vite config to fix missing S3 uploads Feb 22, 2026
Copilot AI requested a review from pethers February 22, 2026 04:03
@pethers pethers requested a review from Copilot February 22, 2026 04:05
@github-actions github-actions bot added javascript JavaScript code changes workflow GitHub Actions workflows ci-cd CI/CD pipeline changes deployment Deployment configuration performance Performance optimization size-xl Extra large change (> 1000 lines) labels Feb 22, 2026
@github-actions
Copy link
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes missing S3 uploads for newly added news article HTML pages by ensuring Vite includes all news/*.html files in the build output (so they land in dist/ and are eligible for S3 sync), and adds a manual S3 diagnostic workflow to help detect mismatches between dist/ and the bucket.

Changes:

  • Replace hardcoded rollupOptions.input news article entries with filesystem auto-discovery in vite.config.js.
  • Add .github/workflows/s3-troubleshoot.yml (workflow_dispatch) for listing missing/orphaned files and basic bucket stats.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 5 comments.

File Description
vite.config.js Auto-discovers news/*.html article pages and spreads them into Rollup inputs, avoiding omissions from the Vite build.
.github/workflows/s3-troubleshoot.yml Adds a manual workflow to compare dist/ vs S3 and check for missing news files.

Comment on lines +25 to +30
for (const file of fs.readdirSync(newsDir)) {
if (file.endsWith('.html') && !file.startsWith('index')) {
const name = file.replace('.html', '');
entries[`news/${name}`] = `./news/${file}`;
}
}
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fs.readdirSync(newsDir) does not guarantee a consistent order across filesystems/OSes. Sorting the filenames before populating entries will keep rollupOptions.input stable and help avoid hard-to-diff build output changes.

Copilot uses AI. Check for mistakes.
- name: Harden Runner
uses: step-security/harden-runner@5ef0c079ce82195b2a36a210272d6b661572d83e # v2.14.2
with:
egress-policy: audit
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow assumes an AWS role, but harden-runner is set to egress-policy: audit, which doesn't prevent outbound exfiltration if a step is compromised. Consider switching to egress-policy: block with an allowed-endpoints list similar to .github/workflows/deploy-s3.yml, since the network destinations here are well-known (npm registry, GitHub, and AWS APIs).

Suggested change
egress-policy: audit
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
uploads.github.com:443
objects.githubusercontent.com:443
registry.npmjs.org:443
nodejs.org:443
sts.amazonaws.com:443
s3.amazonaws.com:443
s3.us-east-1.amazonaws.com:443

Copilot uses AI. Check for mistakes.
Comment on lines +100 to +111
FILE="${{ inputs.file_pattern }}"
if [ -z "$FILE" ]; then
echo "❌ No file pattern specified"
exit 1
fi

echo "🔍 Checking file: $FILE"
echo ""

echo "=== In local dist/ ==="
if [ -f "dist/$FILE" ]; then
echo "✅ EXISTS in dist/$FILE"
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inputs.file_pattern is interpolated into file paths (dist/$FILE, $FILE) without validation. To avoid accidental path traversal (e.g., ../../...) and to keep the workflow focused on news diagnostics, validate that the input matches an expected pattern like ^news/.*\.html$ before using it.

Copilot uses AI. Check for mistakes.
ls -la "dist/$FILE"
else
echo "❌ MISSING from dist/$FILE"
echo " Check if the file is listed in vite.config.js rollupOptions.input"
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log hint here still refers to checking whether the file is listed in rollupOptions.input, but this PR switches news articles to auto-discovery. Update the message to point to the discovery rules (e.g., file must be a top-level news/*.html that does not start with index).

Suggested change
echo " Check if the file is listed in vite.config.js rollupOptions.input"
echo " Check that the file matches the news auto-discovery rules: top-level news/*.html and filename does not start with index"

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <[email protected]>
Signed-off-by: James Pether Sörling <[email protected]>
import { defineConfig } from 'vite';
import sri from 'vite-plugin-sri-gen';
import fs from 'node:fs';
import path from 'node:path';
@pethers pethers marked this pull request as ready for review February 22, 2026 04:18
@pethers pethers merged commit d049748 into main Feb 22, 2026
11 checks passed
@pethers pethers deleted the copilot/investigate-s3-upload-issue branch February 22, 2026 04:19
Copilot stopped work on behalf of pethers due to an error February 22, 2026 04:19
@github-actions
Copy link
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd CI/CD pipeline changes deployment Deployment configuration javascript JavaScript code changes performance Performance optimization size-xl Extra large change (> 1000 lines) workflow GitHub Actions workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants