feat: Helm chart changes to import Flow and document samples files from git#986
Conversation
… for parameterized ingestion flow in backend and langflow deployments, update langflow and backend initContainers to pull large flow and sample documents directly from git repo (remove mounting of files through configmaps), updated ingestion flow
…ions of the Langflow flow files and OpenRAG sample documents (in addition to specifying a branch)
|
@rodageve can you check the conflicts also? |
…tible with langflow - 1.7.0.dev21
There was a problem hiding this comment.
Pull request overview
This PR migrates from ConfigMap-based file distribution to a git-based approach using initContainers that download Flow and document files directly from GitHub. This eliminates the need for separate ConfigMap deployment steps and allows handling larger files that exceed ConfigMap size limits.
Changes:
- Replaced ConfigMap-based flow and document distribution with initContainer downloads from GitHub
- Added git configuration options in values.yaml for specifying repository, branch/commit, and file lists
- Changed backend documents volume from PersistentVolumeClaim to emptyDir for ephemeral storage
- Updated backend readiness probe to check external service dependencies (OpenSearch, Langflow, Docling)
- Added new Kubernetes resource templates (PVCs, Secrets, Services, Ingress) for a complete Helm chart structure
Reviewed changes
Copilot reviewed 9 out of 31 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| kubernetes/helm/openrag/values.yaml | Added git configuration for flows and documents, updated OpenSearch/Docling defaults, added git commit hash pinning |
| kubernetes/helm/openrag/templates/langflow/deployment.yaml | Replaced ConfigMap volume mounts with initContainer that downloads flow files from GitHub |
| kubernetes/helm/openrag/templates/backend/deployment.yaml | Added initContainer for document downloads, changed readiness probe to check dependencies, changed documents volume to emptyDir |
| kubernetes/helm/openrag/templates/ingress/ingress.yaml | Added complete Ingress resource with TLS and cert-manager support |
| kubernetes/helm/openrag/templates/storage/*.yaml | Added PVC templates for langflow, keys, and config volumes |
| kubernetes/helm/openrag/templates/secrets/*.yaml | Added Secret templates for opensearch, oauth, llm-providers, langflow, and aws credentials |
| kubernetes/helm/openrag/templates/frontend/*.yaml | Added frontend service, HPA, and deployment templates |
| kubernetes/helm/openrag/templates/dashboards/*.yaml | Added dashboards service and deployment templates |
| kubernetes/helm/openrag/templates/configmaps/*.yaml | Added flow-ids and app-config ConfigMaps |
| kubernetes/helm/openrag/templates/_helpers.tpl | Added docling URL helper template |
| helm/openrag/templates/storage/documents-pvc.yaml | Deleted - replaced with emptyDir approach |
| helm/openrag/templates/configmaps/flows/*.yaml | Deleted - replaced with git download approach |
| flows/ingestion_flow.json | Removed unused form fields and edges, updated component versions and configurations |
Comments suppressed due to low confidence (9)
kubernetes/helm/openrag/templates/backend/deployment.yaml:338
- The backend readiness probe now checks external service health (OpenSearch, Langflow, Docling) instead of checking the backend service itself at /health. This means the backend pod won't be marked ready until all dependencies are healthy, which creates a circular dependency with Langflow. The backend waits for Langflow to be ready (line 322), but Langflow may depend on the backend being ready as well. This can cause both pods to be stuck in a not-ready state during startup.
Consider either:
- Checking only the backend's own /health endpoint in the readiness probe and let Kubernetes service meshes handle dependency health
- Making the dependency checks optional/non-blocking, or
- Using a startup probe for dependency checks instead of a readiness probe
kubernetes/helm/openrag/templates/backend/deployment.yaml:343
- The documents volume has been changed from a PersistentVolumeClaim to an emptyDir, which means document data will be lost when the pod restarts. This is inconsistent with the PR description which states documents should be loaded from git on first deployment. Since documents are downloaded to this volume by the initContainer and the volume is now ephemeral, all downloaded documents will be lost on pod restart, requiring re-download every time.
The initContainer code checks if the directory is empty (line 61), so on every pod restart, it will download files again since emptyDir starts empty. This may be intentional for stateless operation, but conflicts with the persistence configuration in values.yaml which still defines document persistence settings.
kubernetes/helm/openrag/values.yaml:23
- The default OpenSearch host is set to "opensearch-coordinating.opensearch.svc.cluster.local" which assumes a specific cluster-internal OpenSearch deployment. However, the PR description and comments indicate this chart is designed to work with "External OpenSearch SaaS connection" (line 21 comment). This default value conflicts with the external SaaS use case and should either be empty (requiring explicit configuration) or be a placeholder value that clearly indicates it needs to be replaced.
kubernetes/helm/openrag/values.yaml:87 - The git hash is hardcoded to a specific commit "f5635c9906df581384c62b570afb55ace65ad1eb". This means the chart will always download files from this specific commit instead of using the latest from the main branch. If this is intentional for version pinning, it should be documented why this specific commit is used. If not, the hash field should be empty by default to use the branch head.
kubernetes/helm/openrag/values.yaml:180 - The same git hash "f5635c9906df581384c62b570afb55ace65ad1eb" is hardcoded for document downloads. This means the chart will always download documents from this specific commit. If this is intentional for version pinning, it should be documented. If not, the hash field should be empty by default to use the branch head.
kubernetes/helm/openrag/templates/langflow/deployment.yaml:39 - The initContainer uses a hardcoded curl image version "curlimages/curl:8.5.0". This version may become outdated or have security vulnerabilities over time. Consider either using a more recent version, using a parameterized value from values.yaml, or using a version tag that gets automatic security updates (though this reduces reproducibility).
kubernetes/helm/openrag/templates/backend/deployment.yaml:38 - The initContainer uses the same hardcoded curl image version "curlimages/curl:8.5.0". This version may become outdated or have security vulnerabilities over time. Consider either using a more recent version, using a parameterized value from values.yaml, or using a version tag that gets automatic security updates.
kubernetes/helm/openrag/templates/backend/deployment.yaml:103 - The initContainer downloads files from GitHub over HTTP without any integrity verification (no checksums, signatures, or hash validation). This means if GitHub is compromised or a man-in-the-middle attack occurs, malicious files could be downloaded and used. While the specific commit hash helps with reproducibility, there's no cryptographic verification that the downloaded content matches what's expected.
Consider adding SHA256 checksum verification for downloaded files, or use a more secure distribution method for production deployments.
kubernetes/helm/openrag/templates/langflow/deployment.yaml:107
- The initContainer downloads files from GitHub over HTTP without any integrity verification (no checksums, signatures, or hash validation). This means if GitHub is compromised or a man-in-the-middle attack occurs, malicious flow files could be downloaded and executed. While the specific commit hash helps with reproducibility, there's no cryptographic verification that the downloaded content matches what's expected.
Consider adding SHA256 checksum verification for downloaded files, or use a more secure distribution method for production deployments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…rgeve_fix_helm_chart_feb6
…working OpenRag flows
Description:
Alternatives for Cloud deployment: