chore: Add ruff autofix step to CI workflows#1640
Merged
Merged
Conversation
Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThis PR expands the autofix CI workflow to apply both ruff safe lint fixes and formatting on changed Python files. The main workflow ( ChangesCI Workflow Enhancement
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
ci
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/autofix.ci.yml:
- Line 80: The workflow pins the GitHub Action using a mutable tag ("uses:
autofix-ci/[email protected]"); replace that with the action's full commit SHA to
ensure immutability. Find the workflow step containing "uses:
autofix-ci/[email protected]", fetch the corresponding full commit SHA from the
autofix-ci/action repository (e.g., via the repo's commits page or `git
ls-remote`), and update the uses line to "uses:
autofix-ci/action@<FULL_COMMIT_SHA>" so the action is referenced by an exact
immutable commit.
- Around line 66-72: The workflow currently expands `${{
steps.changed.outputs.files }}` directly into the `run:` shell commands for the
"Ruff lint fix (in-place)" and "Ruff format (in-place)" steps, which breaks on
filenames with whitespace and is unsafe; instead change the producer step to
emit a multiline/heredoc output and in the consumer steps read that output using
`mapfile` (or `readarray`) into a bash array, then iterate the array or pass the
array safely to `uv run ruff ...` so each filename is preserved and not subject
to word-splitting or shell injection—update references to
`steps.changed.outputs.files` to the new multiline output and use `mapfile -t
files <<< "${{ steps.changed.outputs.files }}"` (or equivalent) before invoking
`uv run ruff check --fix "${files[@]}"` and `uv run ruff format "${files[@]}"`
in the respective steps.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 268334a2-344b-4b7e-ab92-259b6a42879b
📒 Files selected for processing (2)
.github/workflows/autofix.ci.yml.github/workflows/lint-backend.yml
Comment on lines
+66
to
72
| - name: Ruff lint fix (in-place) | ||
| if: steps.changed.outputs.files != '' | ||
| run: uv run ruff check --fix ${{ steps.changed.outputs.files }} | ||
|
|
||
| - name: Ruff format (in-place) | ||
| if: steps.changed.outputs.files != '' | ||
| run: uv run ruff format ${{ steps.changed.outputs.files }} |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Verify direct template expansion and fragile space-joined output are present.
rg -n '\${{\s*steps\.changed\.outputs\.files\s*}}' .github/workflows/autofix.ci.yml -C2
rg -n 'echo "files=\$\{files\[\*\]\}"' .github/workflows/autofix.ci.yml -C2Repository: langflow-ai/openrag
Length of output: 191
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the workflow file to see current state
echo "=== Content of autofix.ci.yml ==="
cat .github/workflows/autofix.ci.yml
echo ""
echo "=== Searching for steps.changed.outputs.files pattern ==="
# Use a simpler grep pattern
grep -n 'steps.changed.outputs.files' .github/workflows/autofix.ci.yml || echo "Pattern not found"
echo ""
echo "=== Searching for echo 'files=' pattern ==="
grep -n 'echo.*files=' .github/workflows/autofix.ci.yml || echo "Pattern not found"Repository: langflow-ai/openrag
Length of output: 3319
Use environment variables with proper array handling to safely pass filenames to shell commands.
Lines 68 and 72 expand ${{ steps.changed.outputs.files }} directly into the run: command without quoting, which breaks on filenames containing whitespace and creates a shell injection vulnerability. Additionally, line 64 stores files using space-joined format (${files[*]}), which is fragile for filenames with special characters.
Use a heredoc-based multiline output format and mapfile to safely parse files:
🔧 Proposed hardening patch
@@
- echo "files=${files[*]}" >> "$GITHUB_OUTPUT"
+ {
+ echo "files<<EOF"
+ printf '%s\n' "${files[@]}"
+ echo "EOF"
+ } >> "$GITHUB_OUTPUT"
@@
- name: Ruff lint fix (in-place)
if: steps.changed.outputs.files != ''
- run: uv run ruff check --fix ${{ steps.changed.outputs.files }}
+ env:
+ CHANGED_FILES: ${{ steps.changed.outputs.files }}
+ run: |
+ mapfile -t files <<< "$CHANGED_FILES"
+ uv run ruff check --fix "${files[@]}"
@@
- name: Ruff format (in-place)
if: steps.changed.outputs.files != ''
- run: uv run ruff format ${{ steps.changed.outputs.files }}
+ env:
+ CHANGED_FILES: ${{ steps.changed.outputs.files }}
+ run: |
+ mapfile -t files <<< "$CHANGED_FILES"
+ uv run ruff format "${files[@]}"🧰 Tools
🪛 zizmor (1.25.2)
[info] 68-68: code injection via template expansion (template-injection): may expand into attacker-controllable code
(template-injection)
[info] 72-72: code injection via template expansion (template-injection): may expand into attacker-controllable code
(template-injection)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/autofix.ci.yml around lines 66 - 72, The workflow
currently expands `${{ steps.changed.outputs.files }}` directly into the `run:`
shell commands for the "Ruff lint fix (in-place)" and "Ruff format (in-place)"
steps, which breaks on filenames with whitespace and is unsafe; instead change
the producer step to emit a multiline/heredoc output and in the consumer steps
read that output using `mapfile` (or `readarray`) into a bash array, then
iterate the array or pass the array safely to `uv run ruff ...` so each filename
is preserved and not subject to word-splitting or shell injection—update
references to `steps.changed.outputs.files` to the new multiline output and use
`mapfile -t files <<< "${{ steps.changed.outputs.files }}"` (or equivalent)
before invoking `uv run ruff check --fix "${files[@]}"` and `uv run ruff format
"${files[@]}"` in the respective steps.
| - name: Apply ruff format autofix via autofix.ci | ||
| - name: Apply ruff autofix via autofix.ci | ||
| if: steps.changed.outputs.files != '' | ||
| uses: autofix-ci/[email protected] |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Resolve the commit behind the tag before pinning.
git ls-remote https://github.com/autofix-ci/action 'refs/tags/v1.3.1' 'refs/tags/v1.3.1^{}'Repository: langflow-ai/openrag
Length of output: 121
Pin autofix-ci/action to a full commit SHA.
Using @v1.3.1 is mutable and weakens supply-chain guarantees.
🔒 Suggested change
- - name: Apply ruff autofix via autofix.ci
- uses: autofix-ci/[email protected]
+ - name: Apply ruff autofix via autofix.ci
+ uses: autofix-ci/action@2891949f3779a1cafafae1523058501de3d4e944🧰 Tools
🪛 zizmor (1.25.2)
[error] 80-80: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/autofix.ci.yml at line 80, The workflow pins the GitHub
Action using a mutable tag ("uses: autofix-ci/[email protected]"); replace that with
the action's full commit SHA to ensure immutability. Find the workflow step
containing "uses: autofix-ci/[email protected]", fetch the corresponding full commit
SHA from the autofix-ci/action repository (e.g., via the repo's commits page or
`git ls-remote`), and update the uses line to "uses:
autofix-ci/action@<FULL_COMMIT_SHA>" so the action is referenced by an exact
immutable commit.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the CI workflows to let autofix.ci apply Ruff’s in-place auto-fixes in addition to formatting, while keeping the main lint workflow strict (--no-fix) and running mypy.
Changes:
- Add a
uv run ruff check --fixstep to.github/workflows/autofix.ci.ymlfor changed Python files. - Rename the autofix job / step labels and adjust the autofix commit message to reflect “ruff autofix”.
- Update workflow comments in both workflows to clarify responsibilities between autofix vs strict lint/mypy.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| .github/workflows/lint-backend.yml | Updates comments to indicate autofix.ci auto-applies safe fixes/formatting while lint-backend stays strict. |
| .github/workflows/autofix.ci.yml | Adds Ruff lint auto-fix step and renames autofix labels/commit message accordingly. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+66
to
+67
| # Safe lint fixes and formatting are auto-applied by | ||
| # `.github/workflows/autofix.ci.yml`. |
|
|
||
| - name: Ruff lint fix (in-place) | ||
| if: steps.changed.outputs.files != '' | ||
| run: uv run ruff check --fix ${{ steps.changed.outputs.files }} |
Collaborator
There was a problem hiding this comment.
(a) [Major] ruff check --fix auto-applies F401 removals, contradicting the documented decision in lint-backend.yml
- 👍 Concur with the Copilot feedback
Problem
- The
lint-backend.ymlworkflow contains an explicit inline rationale for whyruff check --fixis intentionally not enabled (lines 61–65) - No restriction on which rules are eligible for auto-fix
- The project's ruff config (
pyproject.tomlline 97) selects theFrule-set, which includesF401(unused imports); ruff classifies the F401 fix as "safe", meaning it will be applied by--fix src/main.pydemonstrates that the project has side-effect imports that rely on# noqa: F401to survive linting (lines 21, 29):
- The project's ruff config (
Potential Solution
- Restrict the auto-fix step to rules that are safe unconditionally for this codebase by using
--selectto an explicit safe subset, or by ignoring F401 specifically:- name: Ruff lint fix (in-place) if: steps.changed.outputs.files != '' run: uv run ruff check --fix --extend-ignore F401 ${{ steps.changed.outputs.files }}
- Also update the comment in
lint-backend.ymlto reflect thatautofix.ciapplies F401-excluded fixes, so the two comments are no longer contradictory
mpawlow
requested changes
May 20, 2026
|
|
||
| - name: Ruff lint fix (in-place) | ||
| if: steps.changed.outputs.files != '' | ||
| run: uv run ruff check --fix ${{ steps.changed.outputs.files }} |
Collaborator
There was a problem hiding this comment.
(a) [Major] ruff check --fix auto-applies F401 removals, contradicting the documented decision in lint-backend.yml
- 👍 Concur with the Copilot feedback
Problem
- The
lint-backend.ymlworkflow contains an explicit inline rationale for whyruff check --fixis intentionally not enabled (lines 61–65) - No restriction on which rules are eligible for auto-fix
- The project's ruff config (
pyproject.tomlline 97) selects theFrule-set, which includesF401(unused imports); ruff classifies the F401 fix as "safe", meaning it will be applied by--fix src/main.pydemonstrates that the project has side-effect imports that rely on# noqa: F401to survive linting (lines 21, 29):
- The project's ruff config (
Potential Solution
- Restrict the auto-fix step to rules that are safe unconditionally for this codebase by using
--selectto an explicit safe subset, or by ignoring F401 specifically:- name: Ruff lint fix (in-place) if: steps.changed.outputs.files != '' run: uv run ruff check --fix --extend-ignore F401 ${{ steps.changed.outputs.files }}
- Also update the comment in
lint-backend.ymlto reflect thatautofix.ciapplies F401-excluded fixes, so the two comments are no longer contradictory
|
|
||
| - name: Ruff lint fix (in-place) | ||
| if: steps.changed.outputs.files != '' | ||
| run: uv run ruff check --fix ${{ steps.changed.outputs.files }} |
Collaborator
There was a problem hiding this comment.
(b) [Normal] Unfixable violations cause ruff check --fix to exit non-zero, skipping format and autofix-commit steps
Problem
ruff check --fixexits with code1if any violations remain after applying safe fixes (i.e. violations that have no auto-fix, or that are markedunfixable)- When this step exits non-zero, GitHub Actions skips all subsequent steps that do not have
if: always() - Result: the safe fixes that ruff did apply (in-place on the runner filesystem) are never committed back to the PR branch, and the
autofix-ciApp produces no commit; the developer must fix all violations — including the automatically-fixable ones — by hand
Potential Solution
- Add
continue-on-error: trueto the lint-fix step so that the format and commit-back steps always run after it, regardless of remaining violations:- name: Ruff lint fix (in-place) if: steps.changed.outputs.files != '' continue-on-error: true run: uv run ruff check --fix ${{ steps.changed.outputs.files }}
- This preserves the correct division of responsibility: autofix.ci commits what it can;
lint-backend.ymlis the authoritative gate that fails the PR if any violations remain
mpawlow
approved these changes
May 20, 2026
ricofurtado
pushed a commit
that referenced
this pull request
May 22, 2026
* Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml
ricofurtado
added a commit
that referenced
this pull request
May 22, 2026
…on (#1625) * feat: Implement in-process JWT claims caching with TTL and LRU eviction * style: ruff format (auto) * refactor: Update cache initialization to use settings for max size and TTL * feat: code cleanup * fix: Improve public key file reading and enhance token scheme handling * style: ruff autofix (auto) * fix: Handle user insert races and add test timeout (#1618) * Handle user insert races and add test timeout Increase TypeScript integration test timeout to 120s to reduce flakiness during slow CI runs. Enhance user_service.ensure_user_row IntegrityError handling to explicitly handle concurrent-insert races: detect a (oauth_provider, oauth_subject) race and return the existing row, detect an email_lookup_hash race by looking up the email and returning the concurrent identity when it matches, and handle PK collisions by retrying the insert with a new UUID. Add explanatory comments about which collisions are recoverable and when errors should propagate to the caller. * style: ruff format (auto) * Use PEP 604 union for agent config return type Replace typing.Optional[dict] with PEP 604 union syntax (dict | None) for get_effective_agent_config and remove the now-unused Optional import. This is a pure type-annotation cleanup (no runtime behavior changes); note it requires Python 3.10+ for the `|` union syntax. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: display file names and fix ingestion for onedrive (#1609) * fixed onedrive not working * style: ruff format (auto) * removed sites.read.all from onedrive * style: ruff format (auto) * fixed allowed users and groups * fix lint error * fixed lint * fixed mypy lint * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: Skip deleted connector files; return orphan IDs (#1592) * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Add *.db to .gitignore Add a '*.db' pattern to .gitignore to prevent local database files (e.g., SQLite) from being committed to the repository. * Add orphan reconcile and bulk-delete helper Introduce a reconcile pass and supporting bulk-delete utility to remove OpenSearch chunks for files that no longer exist remotely. - Add reconcile_orphans_for_connector_type (src/api/connectors.py): lists all active connections for a connector type, strictly gates on unauthenticated connections or listing errors (abort with 0 deletes), aggregates paginated remote file IDs, computes orphans (indexed IDs not present remotely) and invokes delete_chunks_by_document_ids. Integrated into connector_sync and sync_all_connectors (skips reconcile when sync is capped). - Add delete_chunks_by_document_ids (src/api/documents.py): issues a single delete_by_query with terms(document_id, ...) and conflicts="proceed", returns deleted count, and short-circuits on empty input. - Update connector metadata painless params (src/connectors/service.py): include filename param and assign ctx._source.filename = params.filename in the update script so renamed files update indexed chunks. - Add unit tests: cover delete_chunks_by_document_ids, reconcile_orphans_for_connector_type (gating, pagination, multi-connection union, error handling), and connector metadata filename behavior. Behavior notes: reconcile is conservative to avoid false-positive deletions; bulk delete is defensive and returns 0 on unexpected responses or failures. * fix lint * style: ruff format (auto) * fix lint * Use max_files param and add typing Add an explicit type annotation for files_to_process and update the connector.list_files call to use the max_files parameter instead of limit to match the connector API. Also remove redundant `# type: ignore` comments on connector.cfg assignments as they are no longer necessary. Minor cleanup to improve type clarity and API consistency. * Add connector sync preview UI and API Introduce sync-preview functionality for connectors and sync-all flows. Backend: add ID→filename aggregation, compute_orphans (non-destructive orphan detection), orphan deletion helper, and preview endpoints (connector_sync_preview, connectors_sync_all_preview) plus wiring in internal routes. Frontend: add useSyncConnectorPreview/useSyncAllConnectorsPreview mutations, wire preview flows into Knowledge pages/components, and add SyncConfirmDialog component to show orphan lists, per-connector synced counts, availability flags, and confirm deletion+sync interactions. Also update UI buttons to open the preview dialog and handle loading/syncing states. * style: ruff format (auto) * Hoist and consolidate imports in processors Move many inline imports to module scope and consolidate repeated imports across TaskProcessor and its subclasses. Adds top-level imports (asyncio, datetime, mimetypes, os, time), config/settings and utility functions (clients, get_embedding_model, get_index_name, get_openrag_config, extract_relevant, process_text_file, resplit_chunks_character_windows, ensure_embedding_field_exists, auto_cleanup_tempfile, hash_id, build_filename_delete_body, build_filename_search_body) and imports TaskStatus. Removes redundant per-function imports to improve readability and reduce repeated import overhead; no functional changes intended. * Type UseQueryOptions with SearchResult Specify UseQueryOptions<SearchResult> for the useGetSearchQuery hook's options parameter to improve TypeScript type inference and ensure the query options expect SearchResult data. This change tightens typings without altering runtime behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: add reset db on command, expose error on e2e tests (#1627) * fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * refactor: handler for file upload for context (#1624) * file upload for context * coderabbit suggestions * chore: add operator make commands (#1628) * Add kind build/load targets and kind sample Add Makefile targets to build and load app images into a kind cluster (KIND_CLUSTER_NAME default: openrag): kind-load-app-images and kind-build-load-apps, and expose them in the help output. Update kubernetes/operator/README.md with instructions to run the operator locally, apply a kind-local sample, build/load images into kind, and restart deployments after rebuilding. Add a new config sample openrag_v1alpha1_openrag-kind-local.yaml tuned for 2-CPU kind/Colima clusters (lower resource requests, imagePullPolicy: Never). Also annotate the default sample to point users to the kind-local file for local clusters. * Add Makefile help for operator & kind Add a new help_operator target and OPERATOR_DIR variable to the Makefile, and include it in the .PHONY list. The new help output documents Kubernetes operator and kind-related commands (build/load app images into kind, operator deps/install/run/build/test/lint/manifests/generate, deploy/undeploy, docker-build, helm install) and provides a typical kind + local images workflow and sample CR usage. This makes it easier for developers to discover and run local operator/cluster tasks from the repo root. * redirect on logout (#1440) (#1636) * fix: Copy flows directory into Docker image (#1632) Add a COPY instruction to include the local flows/ directory at /app/flows in the image so bundled flows are available at runtime. Placed before the entrypoint/ownership handling to ensure the files are copied with the intended UID/permissions. * feat: settings saas tabs menu (#1637) * remove logic for previous tab selected, need to be discussed with Ana * reduce repeated code and use darkmodelight class * reduce repeated code and use darkmodelight class * remove unecessary !important * remove unecessary !important * remove duplicate css blocks * remove !important and merge active and hover state * put back TabsContent anf fix hover issue --------- Co-authored-by: Olfa Maslah <[email protected]> * chore: Add ruff autofix step to CI workflows (#1640) * Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml * fix: Enum IDs and delete OpenSearch docs by _id (#1638) * Enum IDs and delete OpenSearch docs by _id Add DLS-safe OpenSearch delete helpers and use them to avoid silent no-ops from delete_by_query. Introduces utils/opensearch_delete.py with collect_visible_document_ids and delete_document_ids (enumerate visible _ids via search/scroll, then delete by primary _id). Update delete_chunks_by_document_ids to enumerate chunk IDs and delete each by _id. Ensure langflow_connector_service and TaskProcessor clear stale chunks (by document_id) before re-ingest/re-index to prevent duplicate or trailing chunks after renames. Improve connector listing by scoping cfg.file_ids/folder_ids when available to avoid false orphan detection. Add and update unit tests to assert the new enumeration-and-delete behavior and ordering. * style: ruff format (auto) * ruff fix * ruff fix --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * make probes more robust (#1642) * feat: Show google_drive connector for cloud brand (#1650) Remove google_drive from the cloud-brand exclusion so Google Drive connectors are visible when isCloudBrand is true. Applied the change in connector-cards.tsx and knowledge-dropdown.tsx (OneDrive remains excluded). * operator ubi base build (#1652) * remove flow download initContainer when flowRef is removed (#1653) * fix: cr deletion (#1656) * fix cr deletion * refactoring * fix unit test * fix lint * fix cross namespace CR deletion * Potential fix for pull request finding 'CodeQL / Information exposure through an exception' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Edwin Jose <[email protected]> Co-authored-by: Lucas Oliveira <[email protected]> Co-authored-by: Mike Fortman <[email protected]> Co-authored-by: Wallgau <[email protected]> Co-authored-by: Olfa Maslah <[email protected]> Co-authored-by: ming <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
ricofurtado
pushed a commit
that referenced
this pull request
May 23, 2026
* Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml
ricofurtado
added a commit
that referenced
this pull request
May 23, 2026
…on (#1625) * feat: Implement in-process JWT claims caching with TTL and LRU eviction * style: ruff format (auto) * refactor: Update cache initialization to use settings for max size and TTL * feat: code cleanup * fix: Improve public key file reading and enhance token scheme handling * style: ruff autofix (auto) * fix: Handle user insert races and add test timeout (#1618) * Handle user insert races and add test timeout Increase TypeScript integration test timeout to 120s to reduce flakiness during slow CI runs. Enhance user_service.ensure_user_row IntegrityError handling to explicitly handle concurrent-insert races: detect a (oauth_provider, oauth_subject) race and return the existing row, detect an email_lookup_hash race by looking up the email and returning the concurrent identity when it matches, and handle PK collisions by retrying the insert with a new UUID. Add explanatory comments about which collisions are recoverable and when errors should propagate to the caller. * style: ruff format (auto) * Use PEP 604 union for agent config return type Replace typing.Optional[dict] with PEP 604 union syntax (dict | None) for get_effective_agent_config and remove the now-unused Optional import. This is a pure type-annotation cleanup (no runtime behavior changes); note it requires Python 3.10+ for the `|` union syntax. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: display file names and fix ingestion for onedrive (#1609) * fixed onedrive not working * style: ruff format (auto) * removed sites.read.all from onedrive * style: ruff format (auto) * fixed allowed users and groups * fix lint error * fixed lint * fixed mypy lint * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: Skip deleted connector files; return orphan IDs (#1592) * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Add *.db to .gitignore Add a '*.db' pattern to .gitignore to prevent local database files (e.g., SQLite) from being committed to the repository. * Add orphan reconcile and bulk-delete helper Introduce a reconcile pass and supporting bulk-delete utility to remove OpenSearch chunks for files that no longer exist remotely. - Add reconcile_orphans_for_connector_type (src/api/connectors.py): lists all active connections for a connector type, strictly gates on unauthenticated connections or listing errors (abort with 0 deletes), aggregates paginated remote file IDs, computes orphans (indexed IDs not present remotely) and invokes delete_chunks_by_document_ids. Integrated into connector_sync and sync_all_connectors (skips reconcile when sync is capped). - Add delete_chunks_by_document_ids (src/api/documents.py): issues a single delete_by_query with terms(document_id, ...) and conflicts="proceed", returns deleted count, and short-circuits on empty input. - Update connector metadata painless params (src/connectors/service.py): include filename param and assign ctx._source.filename = params.filename in the update script so renamed files update indexed chunks. - Add unit tests: cover delete_chunks_by_document_ids, reconcile_orphans_for_connector_type (gating, pagination, multi-connection union, error handling), and connector metadata filename behavior. Behavior notes: reconcile is conservative to avoid false-positive deletions; bulk delete is defensive and returns 0 on unexpected responses or failures. * fix lint * style: ruff format (auto) * fix lint * Use max_files param and add typing Add an explicit type annotation for files_to_process and update the connector.list_files call to use the max_files parameter instead of limit to match the connector API. Also remove redundant `# type: ignore` comments on connector.cfg assignments as they are no longer necessary. Minor cleanup to improve type clarity and API consistency. * Add connector sync preview UI and API Introduce sync-preview functionality for connectors and sync-all flows. Backend: add ID→filename aggregation, compute_orphans (non-destructive orphan detection), orphan deletion helper, and preview endpoints (connector_sync_preview, connectors_sync_all_preview) plus wiring in internal routes. Frontend: add useSyncConnectorPreview/useSyncAllConnectorsPreview mutations, wire preview flows into Knowledge pages/components, and add SyncConfirmDialog component to show orphan lists, per-connector synced counts, availability flags, and confirm deletion+sync interactions. Also update UI buttons to open the preview dialog and handle loading/syncing states. * style: ruff format (auto) * Hoist and consolidate imports in processors Move many inline imports to module scope and consolidate repeated imports across TaskProcessor and its subclasses. Adds top-level imports (asyncio, datetime, mimetypes, os, time), config/settings and utility functions (clients, get_embedding_model, get_index_name, get_openrag_config, extract_relevant, process_text_file, resplit_chunks_character_windows, ensure_embedding_field_exists, auto_cleanup_tempfile, hash_id, build_filename_delete_body, build_filename_search_body) and imports TaskStatus. Removes redundant per-function imports to improve readability and reduce repeated import overhead; no functional changes intended. * Type UseQueryOptions with SearchResult Specify UseQueryOptions<SearchResult> for the useGetSearchQuery hook's options parameter to improve TypeScript type inference and ensure the query options expect SearchResult data. This change tightens typings without altering runtime behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: add reset db on command, expose error on e2e tests (#1627) * fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * refactor: handler for file upload for context (#1624) * file upload for context * coderabbit suggestions * chore: add operator make commands (#1628) * Add kind build/load targets and kind sample Add Makefile targets to build and load app images into a kind cluster (KIND_CLUSTER_NAME default: openrag): kind-load-app-images and kind-build-load-apps, and expose them in the help output. Update kubernetes/operator/README.md with instructions to run the operator locally, apply a kind-local sample, build/load images into kind, and restart deployments after rebuilding. Add a new config sample openrag_v1alpha1_openrag-kind-local.yaml tuned for 2-CPU kind/Colima clusters (lower resource requests, imagePullPolicy: Never). Also annotate the default sample to point users to the kind-local file for local clusters. * Add Makefile help for operator & kind Add a new help_operator target and OPERATOR_DIR variable to the Makefile, and include it in the .PHONY list. The new help output documents Kubernetes operator and kind-related commands (build/load app images into kind, operator deps/install/run/build/test/lint/manifests/generate, deploy/undeploy, docker-build, helm install) and provides a typical kind + local images workflow and sample CR usage. This makes it easier for developers to discover and run local operator/cluster tasks from the repo root. * redirect on logout (#1440) (#1636) * fix: Copy flows directory into Docker image (#1632) Add a COPY instruction to include the local flows/ directory at /app/flows in the image so bundled flows are available at runtime. Placed before the entrypoint/ownership handling to ensure the files are copied with the intended UID/permissions. * feat: settings saas tabs menu (#1637) * remove logic for previous tab selected, need to be discussed with Ana * reduce repeated code and use darkmodelight class * reduce repeated code and use darkmodelight class * remove unecessary !important * remove unecessary !important * remove duplicate css blocks * remove !important and merge active and hover state * put back TabsContent anf fix hover issue --------- Co-authored-by: Olfa Maslah <[email protected]> * chore: Add ruff autofix step to CI workflows (#1640) * Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml * fix: Enum IDs and delete OpenSearch docs by _id (#1638) * Enum IDs and delete OpenSearch docs by _id Add DLS-safe OpenSearch delete helpers and use them to avoid silent no-ops from delete_by_query. Introduces utils/opensearch_delete.py with collect_visible_document_ids and delete_document_ids (enumerate visible _ids via search/scroll, then delete by primary _id). Update delete_chunks_by_document_ids to enumerate chunk IDs and delete each by _id. Ensure langflow_connector_service and TaskProcessor clear stale chunks (by document_id) before re-ingest/re-index to prevent duplicate or trailing chunks after renames. Improve connector listing by scoping cfg.file_ids/folder_ids when available to avoid false orphan detection. Add and update unit tests to assert the new enumeration-and-delete behavior and ordering. * style: ruff format (auto) * ruff fix * ruff fix --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * make probes more robust (#1642) * feat: Show google_drive connector for cloud brand (#1650) Remove google_drive from the cloud-brand exclusion so Google Drive connectors are visible when isCloudBrand is true. Applied the change in connector-cards.tsx and knowledge-dropdown.tsx (OneDrive remains excluded). * operator ubi base build (#1652) * remove flow download initContainer when flowRef is removed (#1653) * fix: cr deletion (#1656) * fix cr deletion * refactoring * fix unit test * fix lint * fix cross namespace CR deletion * Potential fix for pull request finding 'CodeQL / Information exposure through an exception' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Edwin Jose <[email protected]> Co-authored-by: Lucas Oliveira <[email protected]> Co-authored-by: Mike Fortman <[email protected]> Co-authored-by: Wallgau <[email protected]> Co-authored-by: Olfa Maslah <[email protected]> Co-authored-by: ming <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow.
Summary by CodeRabbit