fix: add reset db on command, expose error on e2e tests#1627
Merged
Conversation
github-actions
Bot
added
frontend
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughMakefile factory-reset now removes local ChangesFactory Reset Data Clearing
Onboarding Error Detection and Upload Handling
Monitor Screen Code Formatting
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
bug
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/tui/screens/monitor.py (1)
3-25:⚠️ Potential issue | 🟠 Major | ⚡ Quick winFix import ordering to unblock Ruff I001.
CI is failing on this import block (
I001). Reorder/sort imports (including names insidefrom ... import (...)) to match isort/ruff expectations.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/tui/screens/monitor.py` around lines 3 - 25, Reorder and sort the import block to satisfy Ruff/I001: group standard library imports first (asyncio, re, shutil, Path, typing names) sorted alphabetically, then third-party/Textual/Rich imports (textual.app, textual.containers, textual.screen, textual.widgets, textual.timer, rich.text, rich.table) sorted, and finally local package imports (from ..managers.container_manager import ...) with the names inside the parentheses alphabetically sorted (ContainerManager, ServiceInfo, ServiceStatus, format_port_conflict_message). Also move the ButtonVariant Literal definition after the typing imports if needed so all typing/standard-library imports remain grouped.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Makefile`:
- Around line 568-571: The Makefile removal step trusts OPENRAG_DATA_PATH and
may delete dangerous locations; add a hard safety guard before the rm -rf call:
verify OPENRAG_DATA_PATH is non-empty, is an absolute path, is not "/" or "." or
the project root or current working directory, and (preferably) resides under an
expected base directory (e.g., a dedicated data dir) before running rm -rf; if
any check fails, print an error and skip deletion. Reference the
OPENRAG_DATA_PATH variable and the rm -rf "$$OPENRAG_DATA_PATH" invocation to
locate where to add these pre-checks.
In `@src/tui/screens/monitor.py`:
- Around line 503-511: The reset flow currently calls
expand_path(env_manager.config.openrag_data_path) and then
clear_directory_with_container / shutil.rmtree without validating the resolved
path; add the same resolved-path safety guard used for langflow-data to ensure
data_path is inside the expected app data directory and not a root/important
filesystem path before attempting deletion. Specifically, resolve and normalize
data_path (via expand_path/resolved), check it is within the trusted base
directory (compare parents or use pathlib.Path.commonpath with the configured
APP_DATA_DIR/trusted base), and only call
container_manager.clear_directory_with_container(data_path) or
shutil.rmtree(data_path) if the guard passes; otherwise abort the reset and log
an error. Reference symbols: openrag_data_path, expand_path, data_path,
clear_directory_with_container, shutil.rmtree, env_manager.config.
---
Outside diff comments:
In `@src/tui/screens/monitor.py`:
- Around line 3-25: Reorder and sort the import block to satisfy Ruff/I001:
group standard library imports first (asyncio, re, shutil, Path, typing names)
sorted alphabetically, then third-party/Textual/Rich imports (textual.app,
textual.containers, textual.screen, textual.widgets, textual.timer, rich.text,
rich.table) sorted, and finally local package imports (from
..managers.container_manager import ...) with the names inside the parentheses
alphabetically sorted (ContainerManager, ServiceInfo, ServiceStatus,
format_port_conflict_message). Also move the ButtonVariant Literal definition
after the typing imports if needed so all typing/standard-library imports remain
grouped.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: fb0ba436-5e30-4c3d-9a6b-8167a7d9f3bc
📒 Files selected for processing (4)
Makefilefrontend/app/onboarding/_components/onboarding-card.tsxfrontend/tests/utils/onboarding.tssrc/tui/screens/monitor.py
Comment on lines
+568
to
+571
| if [ -n "$$OPENRAG_DATA_PATH" ] && [ -d "$$OPENRAG_DATA_PATH" ]; then \ | ||
| echo "Removing $$OPENRAG_DATA_PATH..."; \ | ||
| rm -rf "$$OPENRAG_DATA_PATH"; \ | ||
| echo "$(PURPLE)$$OPENRAG_DATA_PATH removed$(NC)"; \ |
Contributor
There was a problem hiding this comment.
Add a hard safety guard before deleting OPENRAG_DATA_PATH.
rm -rf "$$OPENRAG_DATA_PATH" currently trusts a raw env path. A mis-set value (e.g. /, ., or project root) can wipe unrelated data.
🛡️ Suggested guard
- if [ -n "$$OPENRAG_DATA_PATH" ] && [ -d "$$OPENRAG_DATA_PATH" ]; then \
- echo "Removing $$OPENRAG_DATA_PATH..."; \
- rm -rf "$$OPENRAG_DATA_PATH"; \
- echo "$(PURPLE)$$OPENRAG_DATA_PATH removed$(NC)"; \
- fi; \
+ if [ -n "$$OPENRAG_DATA_PATH" ] && [ -d "$$OPENRAG_DATA_PATH" ]; then \
+ RESOLVED_PATH=$$(cd "$$OPENRAG_DATA_PATH" 2>/dev/null && pwd -P); \
+ if [ -z "$$RESOLVED_PATH" ] || [ "$$RESOLVED_PATH" = "/" ] || [ "$$RESOLVED_PATH" = "$$(pwd -P)" ] || [ "$$RESOLVED_PATH" = "$$HOME" ]; then \
+ echo "$(RED)Refusing to delete unsafe OPENRAG_DATA_PATH: $$OPENRAG_DATA_PATH$(NC)"; \
+ exit 1; \
+ fi; \
+ echo "Removing $$RESOLVED_PATH..."; \
+ rm -rf "$$RESOLVED_PATH"; \
+ echo "$(PURPLE)$$RESOLVED_PATH removed$(NC)"; \
+ fi; \📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if [ -n "$$OPENRAG_DATA_PATH" ] && [ -d "$$OPENRAG_DATA_PATH" ]; then \ | |
| echo "Removing $$OPENRAG_DATA_PATH..."; \ | |
| rm -rf "$$OPENRAG_DATA_PATH"; \ | |
| echo "$(PURPLE)$$OPENRAG_DATA_PATH removed$(NC)"; \ | |
| if [ -n "$$OPENRAG_DATA_PATH" ] && [ -d "$$OPENRAG_DATA_PATH" ]; then \ | |
| RESOLVED_PATH=$$(cd "$$OPENRAG_DATA_PATH" 2>/dev/null && pwd -P); \ | |
| if [ -z "$$RESOLVED_PATH" ] || [ "$$RESOLVED_PATH" = "/" ] || [ "$$RESOLVED_PATH" = "$$(pwd -P)" ] || [ "$$RESOLVED_PATH" = "$$HOME" ]; then \ | |
| echo "$(RED)Refusing to delete unsafe OPENRAG_DATA_PATH: $$OPENRAG_DATA_PATH$(NC)"; \ | |
| exit 1; \ | |
| fi; \ | |
| echo "Removing $$RESOLVED_PATH..."; \ | |
| rm -rf "$$RESOLVED_PATH"; \ | |
| echo "$(PURPLE)$$RESOLVED_PATH removed$(NC)"; \ | |
| fi; \ |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Makefile` around lines 568 - 571, The Makefile removal step trusts
OPENRAG_DATA_PATH and may delete dangerous locations; add a hard safety guard
before the rm -rf call: verify OPENRAG_DATA_PATH is non-empty, is an absolute
path, is not "/" or "." or the project root or current working directory, and
(preferably) resides under an expected base directory (e.g., a dedicated data
dir) before running rm -rf; if any check fails, print an error and skip
deletion. Reference the OPENRAG_DATA_PATH variable and the rm -rf
"$$OPENRAG_DATA_PATH" invocation to locate where to add these pre-checks.
Comment on lines
+503
to
+511
| # Clear backend data directory (database, session ownership, conversations, oauth tokens) | ||
| data_path = expand_path(env_manager.config.openrag_data_path) | ||
| if data_path.exists(): | ||
| success, msg = await self.container_manager.clear_directory_with_container( | ||
| data_path | ||
| ) | ||
| if not success: | ||
| shutil.rmtree(data_path) | ||
| data_path.mkdir(parents=True, exist_ok=True) |
Contributor
There was a problem hiding this comment.
Validate openrag_data_path before recursive delete.
This newly added factory-reset step can recursively delete any configured path without a safety boundary. Add the same kind of resolved-path guard used for langflow-data deletion before clear_directory_with_container/shutil.rmtree.
🛡️ Suggested guard pattern
# Clear backend data directory (database, session ownership, conversations, oauth tokens)
data_path = expand_path(env_manager.config.openrag_data_path)
if data_path.exists():
+ resolved = data_path.resolve()
+ home = Path.home().resolve()
+ if resolved == Path("/") or not str(resolved).startswith(str(home) + "/"):
+ self.notify(
+ f"Refusing to delete unsafe data path: {resolved}",
+ severity="error",
+ )
+ return
success, msg = await self.container_manager.clear_directory_with_container(
data_path
)
if not success:
shutil.rmtree(data_path)
data_path.mkdir(parents=True, exist_ok=True)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/tui/screens/monitor.py` around lines 503 - 511, The reset flow currently
calls expand_path(env_manager.config.openrag_data_path) and then
clear_directory_with_container / shutil.rmtree without validating the resolved
path; add the same resolved-path safety guard used for langflow-data to ensure
data_path is inside the expected app data directory and not a root/important
filesystem path before attempting deletion. Specifically, resolve and normalize
data_path (via expand_path/resolved), check it is within the trusted base
directory (compare parents or use pathlib.Path.commonpath with the configured
APP_DATA_DIR/trusted base), and only call
container_manager.clear_directory_with_container(data_path) or
shutil.rmtree(data_path) if the guard passes; otherwise abort the reset and log
an error. Reference symbols: openrag_data_path, expand_path, data_path,
clear_directory_with_container, shutil.rmtree, env_manager.config.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontend/app/onboarding/_components/onboarding-upload.tsx`:
- Around line 96-99: On upload failure (the block that calls setError,
setCurrentStep(null), setUploadedTaskId(null)), also clear the filter-intent
state by resetting shouldCreateFilter and uploadedFilename to safe empty values
(e.g., call setShouldCreateFilter(false) and setUploadedFilename(null or empty
string)); apply the same change in the other failure block around the setError
at the 251-253 area so retries can't reuse stale
shouldCreateFilter/uploadedFilename state.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 8538c873-207b-4b51-a7fb-cce396f34992
📒 Files selected for processing (2)
frontend/app/onboarding/_components/onboarding-upload.tsxfrontend/tests/utils/onboarding.ts
Comment on lines
+96
to
+99
| setError(errorMessage); | ||
| setCurrentStep(null); | ||
| setUploadedTaskId(null); | ||
| return; |
Contributor
There was a problem hiding this comment.
Clear filter-intent state on failure to avoid retry cross-contamination.
After a failed ingestion/upload, shouldCreateFilter and uploadedFilename remain set. A later retry can accidentally create a filter for stale data.
Suggested fix
if (failedTask || hasFailedFile) {
let errorMessage = "Document ingestion failed. Please try again.";
@@
setError(errorMessage);
setCurrentStep(null);
setUploadedTaskId(null);
+ setShouldCreateFilter(false);
+ setUploadedFilename(null);
return;
}
@@
} catch (error) {
@@
// Reset on error
setCurrentStep(null);
setUploadedTaskId(null);
+ setShouldCreateFilter(false);
+ setUploadedFilename(null);
} finally {Also applies to: 251-253
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/app/onboarding/_components/onboarding-upload.tsx` around lines 96 -
99, On upload failure (the block that calls setError, setCurrentStep(null),
setUploadedTaskId(null)), also clear the filter-intent state by resetting
shouldCreateFilter and uploadedFilename to safe empty values (e.g., call
setShouldCreateFilter(false) and setUploadedFilename(null or empty string));
apply the same change in the other failure block around the setError at the
251-253 area so retries can't reuse stale shouldCreateFilter/uploadedFilename
state.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontend/tests/utils/onboarding.ts`:
- Around line 150-154: The global page.getByText("Done") can match a previous
step and cause false-positive waits; update the waits that create doneLocator
and its uses (the doneLocator variable and the expect call that uses
doneLocator.or(errorLocator)) to scope the "Done" locator to the current step
container (e.g., use the specific step locator or a parent element's locator and
call .getByText("Done") on that container) so the check only resolves for the
active step; apply the same change for the other occurrence around lines 199-203
where page.getByText("Done") is used.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: f5b1e63e-be01-4754-83d6-abbd59f74189
📒 Files selected for processing (2)
frontend/tests/utils/onboarding.tssrc/tui/screens/monitor.py
🚧 Files skipped from review as they are similar to previous changes (1)
- src/tui/screens/monitor.py
Comment on lines
+150
to
154
| const doneLocator = page.getByText("Done"); | ||
| const errorLocator = page.getByTestId("onboarding-error"); | ||
|
|
||
| await expect(doneLocator.or(errorLocator)).toBeVisible({ | ||
| timeout: isEmbedding ? 120000 : 60000, |
Contributor
There was a problem hiding this comment.
Scope the "Done" locator to avoid false-positive waits.
Using page.getByText("Done") globally can resolve against a previously completed step, so these waits may pass before the current async action actually finishes.
Suggested fix
- const doneLocator = page.getByText("Done");
+ const doneLocator = page.getByText(/^Done$/).last();
const errorLocator = page.getByTestId("onboarding-error");
@@
- const uploadDoneLocator = page.getByText("Done");
+ const uploadDoneLocator = page.getByText(/^Done$/).last();
const uploadErrorLocator = page.getByTestId("onboarding-upload-error");Also applies to: 199-203
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/tests/utils/onboarding.ts` around lines 150 - 154, The global
page.getByText("Done") can match a previous step and cause false-positive waits;
update the waits that create doneLocator and its uses (the doneLocator variable
and the expect call that uses doneLocator.or(errorLocator)) to scope the "Done"
locator to the current step container (e.g., use the specific step locator or a
parent element's locator and call .getByText("Done") on that container) so the
check only resolves for the active step; apply the same change for the other
occurrence around lines 199-203 where page.getByText("Done") is used.
mfortman11
approved these changes
May 19, 2026
ricofurtado
pushed a commit
that referenced
this pull request
May 20, 2026
* fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
ricofurtado
pushed a commit
that referenced
this pull request
May 22, 2026
* fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
ricofurtado
added a commit
that referenced
this pull request
May 22, 2026
…on (#1625) * feat: Implement in-process JWT claims caching with TTL and LRU eviction * style: ruff format (auto) * refactor: Update cache initialization to use settings for max size and TTL * feat: code cleanup * fix: Improve public key file reading and enhance token scheme handling * style: ruff autofix (auto) * fix: Handle user insert races and add test timeout (#1618) * Handle user insert races and add test timeout Increase TypeScript integration test timeout to 120s to reduce flakiness during slow CI runs. Enhance user_service.ensure_user_row IntegrityError handling to explicitly handle concurrent-insert races: detect a (oauth_provider, oauth_subject) race and return the existing row, detect an email_lookup_hash race by looking up the email and returning the concurrent identity when it matches, and handle PK collisions by retrying the insert with a new UUID. Add explanatory comments about which collisions are recoverable and when errors should propagate to the caller. * style: ruff format (auto) * Use PEP 604 union for agent config return type Replace typing.Optional[dict] with PEP 604 union syntax (dict | None) for get_effective_agent_config and remove the now-unused Optional import. This is a pure type-annotation cleanup (no runtime behavior changes); note it requires Python 3.10+ for the `|` union syntax. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: display file names and fix ingestion for onedrive (#1609) * fixed onedrive not working * style: ruff format (auto) * removed sites.read.all from onedrive * style: ruff format (auto) * fixed allowed users and groups * fix lint error * fixed lint * fixed mypy lint * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: Skip deleted connector files; return orphan IDs (#1592) * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Add *.db to .gitignore Add a '*.db' pattern to .gitignore to prevent local database files (e.g., SQLite) from being committed to the repository. * Add orphan reconcile and bulk-delete helper Introduce a reconcile pass and supporting bulk-delete utility to remove OpenSearch chunks for files that no longer exist remotely. - Add reconcile_orphans_for_connector_type (src/api/connectors.py): lists all active connections for a connector type, strictly gates on unauthenticated connections or listing errors (abort with 0 deletes), aggregates paginated remote file IDs, computes orphans (indexed IDs not present remotely) and invokes delete_chunks_by_document_ids. Integrated into connector_sync and sync_all_connectors (skips reconcile when sync is capped). - Add delete_chunks_by_document_ids (src/api/documents.py): issues a single delete_by_query with terms(document_id, ...) and conflicts="proceed", returns deleted count, and short-circuits on empty input. - Update connector metadata painless params (src/connectors/service.py): include filename param and assign ctx._source.filename = params.filename in the update script so renamed files update indexed chunks. - Add unit tests: cover delete_chunks_by_document_ids, reconcile_orphans_for_connector_type (gating, pagination, multi-connection union, error handling), and connector metadata filename behavior. Behavior notes: reconcile is conservative to avoid false-positive deletions; bulk delete is defensive and returns 0 on unexpected responses or failures. * fix lint * style: ruff format (auto) * fix lint * Use max_files param and add typing Add an explicit type annotation for files_to_process and update the connector.list_files call to use the max_files parameter instead of limit to match the connector API. Also remove redundant `# type: ignore` comments on connector.cfg assignments as they are no longer necessary. Minor cleanup to improve type clarity and API consistency. * Add connector sync preview UI and API Introduce sync-preview functionality for connectors and sync-all flows. Backend: add ID→filename aggregation, compute_orphans (non-destructive orphan detection), orphan deletion helper, and preview endpoints (connector_sync_preview, connectors_sync_all_preview) plus wiring in internal routes. Frontend: add useSyncConnectorPreview/useSyncAllConnectorsPreview mutations, wire preview flows into Knowledge pages/components, and add SyncConfirmDialog component to show orphan lists, per-connector synced counts, availability flags, and confirm deletion+sync interactions. Also update UI buttons to open the preview dialog and handle loading/syncing states. * style: ruff format (auto) * Hoist and consolidate imports in processors Move many inline imports to module scope and consolidate repeated imports across TaskProcessor and its subclasses. Adds top-level imports (asyncio, datetime, mimetypes, os, time), config/settings and utility functions (clients, get_embedding_model, get_index_name, get_openrag_config, extract_relevant, process_text_file, resplit_chunks_character_windows, ensure_embedding_field_exists, auto_cleanup_tempfile, hash_id, build_filename_delete_body, build_filename_search_body) and imports TaskStatus. Removes redundant per-function imports to improve readability and reduce repeated import overhead; no functional changes intended. * Type UseQueryOptions with SearchResult Specify UseQueryOptions<SearchResult> for the useGetSearchQuery hook's options parameter to improve TypeScript type inference and ensure the query options expect SearchResult data. This change tightens typings without altering runtime behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: add reset db on command, expose error on e2e tests (#1627) * fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * refactor: handler for file upload for context (#1624) * file upload for context * coderabbit suggestions * chore: add operator make commands (#1628) * Add kind build/load targets and kind sample Add Makefile targets to build and load app images into a kind cluster (KIND_CLUSTER_NAME default: openrag): kind-load-app-images and kind-build-load-apps, and expose them in the help output. Update kubernetes/operator/README.md with instructions to run the operator locally, apply a kind-local sample, build/load images into kind, and restart deployments after rebuilding. Add a new config sample openrag_v1alpha1_openrag-kind-local.yaml tuned for 2-CPU kind/Colima clusters (lower resource requests, imagePullPolicy: Never). Also annotate the default sample to point users to the kind-local file for local clusters. * Add Makefile help for operator & kind Add a new help_operator target and OPERATOR_DIR variable to the Makefile, and include it in the .PHONY list. The new help output documents Kubernetes operator and kind-related commands (build/load app images into kind, operator deps/install/run/build/test/lint/manifests/generate, deploy/undeploy, docker-build, helm install) and provides a typical kind + local images workflow and sample CR usage. This makes it easier for developers to discover and run local operator/cluster tasks from the repo root. * redirect on logout (#1440) (#1636) * fix: Copy flows directory into Docker image (#1632) Add a COPY instruction to include the local flows/ directory at /app/flows in the image so bundled flows are available at runtime. Placed before the entrypoint/ownership handling to ensure the files are copied with the intended UID/permissions. * feat: settings saas tabs menu (#1637) * remove logic for previous tab selected, need to be discussed with Ana * reduce repeated code and use darkmodelight class * reduce repeated code and use darkmodelight class * remove unecessary !important * remove unecessary !important * remove duplicate css blocks * remove !important and merge active and hover state * put back TabsContent anf fix hover issue --------- Co-authored-by: Olfa Maslah <[email protected]> * chore: Add ruff autofix step to CI workflows (#1640) * Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml * fix: Enum IDs and delete OpenSearch docs by _id (#1638) * Enum IDs and delete OpenSearch docs by _id Add DLS-safe OpenSearch delete helpers and use them to avoid silent no-ops from delete_by_query. Introduces utils/opensearch_delete.py with collect_visible_document_ids and delete_document_ids (enumerate visible _ids via search/scroll, then delete by primary _id). Update delete_chunks_by_document_ids to enumerate chunk IDs and delete each by _id. Ensure langflow_connector_service and TaskProcessor clear stale chunks (by document_id) before re-ingest/re-index to prevent duplicate or trailing chunks after renames. Improve connector listing by scoping cfg.file_ids/folder_ids when available to avoid false orphan detection. Add and update unit tests to assert the new enumeration-and-delete behavior and ordering. * style: ruff format (auto) * ruff fix * ruff fix --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * make probes more robust (#1642) * feat: Show google_drive connector for cloud brand (#1650) Remove google_drive from the cloud-brand exclusion so Google Drive connectors are visible when isCloudBrand is true. Applied the change in connector-cards.tsx and knowledge-dropdown.tsx (OneDrive remains excluded). * operator ubi base build (#1652) * remove flow download initContainer when flowRef is removed (#1653) * fix: cr deletion (#1656) * fix cr deletion * refactoring * fix unit test * fix lint * fix cross namespace CR deletion * Potential fix for pull request finding 'CodeQL / Information exposure through an exception' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Edwin Jose <[email protected]> Co-authored-by: Lucas Oliveira <[email protected]> Co-authored-by: Mike Fortman <[email protected]> Co-authored-by: Wallgau <[email protected]> Co-authored-by: Olfa Maslah <[email protected]> Co-authored-by: ming <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
ricofurtado
pushed a commit
that referenced
this pull request
May 23, 2026
* fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
ricofurtado
added a commit
that referenced
this pull request
May 23, 2026
…on (#1625) * feat: Implement in-process JWT claims caching with TTL and LRU eviction * style: ruff format (auto) * refactor: Update cache initialization to use settings for max size and TTL * feat: code cleanup * fix: Improve public key file reading and enhance token scheme handling * style: ruff autofix (auto) * fix: Handle user insert races and add test timeout (#1618) * Handle user insert races and add test timeout Increase TypeScript integration test timeout to 120s to reduce flakiness during slow CI runs. Enhance user_service.ensure_user_row IntegrityError handling to explicitly handle concurrent-insert races: detect a (oauth_provider, oauth_subject) race and return the existing row, detect an email_lookup_hash race by looking up the email and returning the concurrent identity when it matches, and handle PK collisions by retrying the insert with a new UUID. Add explanatory comments about which collisions are recoverable and when errors should propagate to the caller. * style: ruff format (auto) * Use PEP 604 union for agent config return type Replace typing.Optional[dict] with PEP 604 union syntax (dict | None) for get_effective_agent_config and remove the now-unused Optional import. This is a pure type-annotation cleanup (no runtime behavior changes); note it requires Python 3.10+ for the `|` union syntax. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: display file names and fix ingestion for onedrive (#1609) * fixed onedrive not working * style: ruff format (auto) * removed sites.read.all from onedrive * style: ruff format (auto) * fixed allowed users and groups * fix lint error * fixed lint * fixed mypy lint * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: Skip deleted connector files; return orphan IDs (#1592) * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Add *.db to .gitignore Add a '*.db' pattern to .gitignore to prevent local database files (e.g., SQLite) from being committed to the repository. * Add orphan reconcile and bulk-delete helper Introduce a reconcile pass and supporting bulk-delete utility to remove OpenSearch chunks for files that no longer exist remotely. - Add reconcile_orphans_for_connector_type (src/api/connectors.py): lists all active connections for a connector type, strictly gates on unauthenticated connections or listing errors (abort with 0 deletes), aggregates paginated remote file IDs, computes orphans (indexed IDs not present remotely) and invokes delete_chunks_by_document_ids. Integrated into connector_sync and sync_all_connectors (skips reconcile when sync is capped). - Add delete_chunks_by_document_ids (src/api/documents.py): issues a single delete_by_query with terms(document_id, ...) and conflicts="proceed", returns deleted count, and short-circuits on empty input. - Update connector metadata painless params (src/connectors/service.py): include filename param and assign ctx._source.filename = params.filename in the update script so renamed files update indexed chunks. - Add unit tests: cover delete_chunks_by_document_ids, reconcile_orphans_for_connector_type (gating, pagination, multi-connection union, error handling), and connector metadata filename behavior. Behavior notes: reconcile is conservative to avoid false-positive deletions; bulk delete is defensive and returns 0 on unexpected responses or failures. * fix lint * style: ruff format (auto) * fix lint * Use max_files param and add typing Add an explicit type annotation for files_to_process and update the connector.list_files call to use the max_files parameter instead of limit to match the connector API. Also remove redundant `# type: ignore` comments on connector.cfg assignments as they are no longer necessary. Minor cleanup to improve type clarity and API consistency. * Add connector sync preview UI and API Introduce sync-preview functionality for connectors and sync-all flows. Backend: add ID→filename aggregation, compute_orphans (non-destructive orphan detection), orphan deletion helper, and preview endpoints (connector_sync_preview, connectors_sync_all_preview) plus wiring in internal routes. Frontend: add useSyncConnectorPreview/useSyncAllConnectorsPreview mutations, wire preview flows into Knowledge pages/components, and add SyncConfirmDialog component to show orphan lists, per-connector synced counts, availability flags, and confirm deletion+sync interactions. Also update UI buttons to open the preview dialog and handle loading/syncing states. * style: ruff format (auto) * Hoist and consolidate imports in processors Move many inline imports to module scope and consolidate repeated imports across TaskProcessor and its subclasses. Adds top-level imports (asyncio, datetime, mimetypes, os, time), config/settings and utility functions (clients, get_embedding_model, get_index_name, get_openrag_config, extract_relevant, process_text_file, resplit_chunks_character_windows, ensure_embedding_field_exists, auto_cleanup_tempfile, hash_id, build_filename_delete_body, build_filename_search_body) and imports TaskStatus. Removes redundant per-function imports to improve readability and reduce repeated import overhead; no functional changes intended. * Type UseQueryOptions with SearchResult Specify UseQueryOptions<SearchResult> for the useGetSearchQuery hook's options parameter to improve TypeScript type inference and ensure the query options expect SearchResult data. This change tightens typings without altering runtime behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: add reset db on command, expose error on e2e tests (#1627) * fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * refactor: handler for file upload for context (#1624) * file upload for context * coderabbit suggestions * chore: add operator make commands (#1628) * Add kind build/load targets and kind sample Add Makefile targets to build and load app images into a kind cluster (KIND_CLUSTER_NAME default: openrag): kind-load-app-images and kind-build-load-apps, and expose them in the help output. Update kubernetes/operator/README.md with instructions to run the operator locally, apply a kind-local sample, build/load images into kind, and restart deployments after rebuilding. Add a new config sample openrag_v1alpha1_openrag-kind-local.yaml tuned for 2-CPU kind/Colima clusters (lower resource requests, imagePullPolicy: Never). Also annotate the default sample to point users to the kind-local file for local clusters. * Add Makefile help for operator & kind Add a new help_operator target and OPERATOR_DIR variable to the Makefile, and include it in the .PHONY list. The new help output documents Kubernetes operator and kind-related commands (build/load app images into kind, operator deps/install/run/build/test/lint/manifests/generate, deploy/undeploy, docker-build, helm install) and provides a typical kind + local images workflow and sample CR usage. This makes it easier for developers to discover and run local operator/cluster tasks from the repo root. * redirect on logout (#1440) (#1636) * fix: Copy flows directory into Docker image (#1632) Add a COPY instruction to include the local flows/ directory at /app/flows in the image so bundled flows are available at runtime. Placed before the entrypoint/ownership handling to ensure the files are copied with the intended UID/permissions. * feat: settings saas tabs menu (#1637) * remove logic for previous tab selected, need to be discussed with Ana * reduce repeated code and use darkmodelight class * reduce repeated code and use darkmodelight class * remove unecessary !important * remove unecessary !important * remove duplicate css blocks * remove !important and merge active and hover state * put back TabsContent anf fix hover issue --------- Co-authored-by: Olfa Maslah <[email protected]> * chore: Add ruff autofix step to CI workflows (#1640) * Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml * fix: Enum IDs and delete OpenSearch docs by _id (#1638) * Enum IDs and delete OpenSearch docs by _id Add DLS-safe OpenSearch delete helpers and use them to avoid silent no-ops from delete_by_query. Introduces utils/opensearch_delete.py with collect_visible_document_ids and delete_document_ids (enumerate visible _ids via search/scroll, then delete by primary _id). Update delete_chunks_by_document_ids to enumerate chunk IDs and delete each by _id. Ensure langflow_connector_service and TaskProcessor clear stale chunks (by document_id) before re-ingest/re-index to prevent duplicate or trailing chunks after renames. Improve connector listing by scoping cfg.file_ids/folder_ids when available to avoid false orphan detection. Add and update unit tests to assert the new enumeration-and-delete behavior and ordering. * style: ruff format (auto) * ruff fix * ruff fix --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * make probes more robust (#1642) * feat: Show google_drive connector for cloud brand (#1650) Remove google_drive from the cloud-brand exclusion so Google Drive connectors are visible when isCloudBrand is true. Applied the change in connector-cards.tsx and knowledge-dropdown.tsx (OneDrive remains excluded). * operator ubi base build (#1652) * remove flow download initContainer when flowRef is removed (#1653) * fix: cr deletion (#1656) * fix cr deletion * refactoring * fix unit test * fix lint * fix cross namespace CR deletion * Potential fix for pull request finding 'CodeQL / Information exposure through an exception' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Edwin Jose <[email protected]> Co-authored-by: Lucas Oliveira <[email protected]> Co-authored-by: Mike Fortman <[email protected]> Co-authored-by: Wallgau <[email protected]> Co-authored-by: Olfa Maslah <[email protected]> Co-authored-by: ming <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
lucaseduoli
added a commit
that referenced
this pull request
May 25, 2026
* fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces improvements to the reset and onboarding flows, focusing on more thorough data cleanup and enhanced error detection during onboarding. The most important changes are as follows:
Factory Reset Improvements:
factory-resettarget in theMakefilenow deletes thedatadirectory (which contains the database and session configs) and also removes any directory specified by theOPENRAG_DATA_PATHenvironment variable. This ensures a more complete cleanup of user and session data. [1] [2]Onboarding Error Handling Enhancements:
data-testid="onboarding-error"attribute on error messages, making them easier to target in automated tests.Summary by CodeRabbit
Improvements
New Features
Tests
Chores