Skip to content

feat: Helm chart changes to import Flow and document samples files from git#986

Merged
rodageve merged 10 commits into
mainfrom
rgeve_fix_helm_chart_feb6
Feb 23, 2026
Merged

feat: Helm chart changes to import Flow and document samples files from git#986
rodageve merged 10 commits into
mainfrom
rgeve_fix_helm_chart_feb6

Conversation

@rodageve

Copy link
Copy Markdown
Collaborator

Description:

  • Import large Flow and Sample document files into the langflow and backend pods respectively by downloading the files from git in an initContainer
  • Remove importing these files as a separate step using ConfigMaps
  • Updates to the Ingestion Flow from cloud testing

Alternatives for Cloud deployment:

  1. Consider importing the files from object storage such as S3
  2. An operator can deploy the respective files from S3 as required into the backend and langflow pdos

… for parameterized ingestion flow in backend and langflow deployments, update langflow and backend initContainers to pull large flow and sample documents directly from git repo (remove mounting of files through configmaps), updated ingestion flow
rodageve added 3 commits February 18, 2026 17:34
Comment thread flows/ingestion_flow.json
@edwinjosechittilappilly

Copy link
Copy Markdown
Collaborator

@rodageve can you check the conflicts also?

@edwinjosechittilappilly edwinjosechittilappilly changed the title Helm chart changes to import Flow and document samples files from git feat: Helm chart changes to import Flow and document samples files from git Feb 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates from ConfigMap-based file distribution to a git-based approach using initContainers that download Flow and document files directly from GitHub. This eliminates the need for separate ConfigMap deployment steps and allows handling larger files that exceed ConfigMap size limits.

Changes:

  • Replaced ConfigMap-based flow and document distribution with initContainer downloads from GitHub
  • Added git configuration options in values.yaml for specifying repository, branch/commit, and file lists
  • Changed backend documents volume from PersistentVolumeClaim to emptyDir for ephemeral storage
  • Updated backend readiness probe to check external service dependencies (OpenSearch, Langflow, Docling)
  • Added new Kubernetes resource templates (PVCs, Secrets, Services, Ingress) for a complete Helm chart structure

Reviewed changes

Copilot reviewed 9 out of 31 changed files in this pull request and generated no comments.

Show a summary per file
File Description
kubernetes/helm/openrag/values.yaml Added git configuration for flows and documents, updated OpenSearch/Docling defaults, added git commit hash pinning
kubernetes/helm/openrag/templates/langflow/deployment.yaml Replaced ConfigMap volume mounts with initContainer that downloads flow files from GitHub
kubernetes/helm/openrag/templates/backend/deployment.yaml Added initContainer for document downloads, changed readiness probe to check dependencies, changed documents volume to emptyDir
kubernetes/helm/openrag/templates/ingress/ingress.yaml Added complete Ingress resource with TLS and cert-manager support
kubernetes/helm/openrag/templates/storage/*.yaml Added PVC templates for langflow, keys, and config volumes
kubernetes/helm/openrag/templates/secrets/*.yaml Added Secret templates for opensearch, oauth, llm-providers, langflow, and aws credentials
kubernetes/helm/openrag/templates/frontend/*.yaml Added frontend service, HPA, and deployment templates
kubernetes/helm/openrag/templates/dashboards/*.yaml Added dashboards service and deployment templates
kubernetes/helm/openrag/templates/configmaps/*.yaml Added flow-ids and app-config ConfigMaps
kubernetes/helm/openrag/templates/_helpers.tpl Added docling URL helper template
helm/openrag/templates/storage/documents-pvc.yaml Deleted - replaced with emptyDir approach
helm/openrag/templates/configmaps/flows/*.yaml Deleted - replaced with git download approach
flows/ingestion_flow.json Removed unused form fields and edges, updated component versions and configurations
Comments suppressed due to low confidence (9)

kubernetes/helm/openrag/templates/backend/deployment.yaml:338

  • The backend readiness probe now checks external service health (OpenSearch, Langflow, Docling) instead of checking the backend service itself at /health. This means the backend pod won't be marked ready until all dependencies are healthy, which creates a circular dependency with Langflow. The backend waits for Langflow to be ready (line 322), but Langflow may depend on the backend being ready as well. This can cause both pods to be stuck in a not-ready state during startup.

Consider either:

  1. Checking only the backend's own /health endpoint in the readiness probe and let Kubernetes service meshes handle dependency health
  2. Making the dependency checks optional/non-blocking, or
  3. Using a startup probe for dependency checks instead of a readiness probe
    kubernetes/helm/openrag/templates/backend/deployment.yaml:343
  • The documents volume has been changed from a PersistentVolumeClaim to an emptyDir, which means document data will be lost when the pod restarts. This is inconsistent with the PR description which states documents should be loaded from git on first deployment. Since documents are downloaded to this volume by the initContainer and the volume is now ephemeral, all downloaded documents will be lost on pod restart, requiring re-download every time.

The initContainer code checks if the directory is empty (line 61), so on every pod restart, it will download files again since emptyDir starts empty. This may be intentional for stateless operation, but conflicts with the persistence configuration in values.yaml which still defines document persistence settings.
kubernetes/helm/openrag/values.yaml:23

  • The default OpenSearch host is set to "opensearch-coordinating.opensearch.svc.cluster.local" which assumes a specific cluster-internal OpenSearch deployment. However, the PR description and comments indicate this chart is designed to work with "External OpenSearch SaaS connection" (line 21 comment). This default value conflicts with the external SaaS use case and should either be empty (requiring explicit configuration) or be a placeholder value that clearly indicates it needs to be replaced.
    kubernetes/helm/openrag/values.yaml:87
  • The git hash is hardcoded to a specific commit "f5635c9906df581384c62b570afb55ace65ad1eb". This means the chart will always download files from this specific commit instead of using the latest from the main branch. If this is intentional for version pinning, it should be documented why this specific commit is used. If not, the hash field should be empty by default to use the branch head.
    kubernetes/helm/openrag/values.yaml:180
  • The same git hash "f5635c9906df581384c62b570afb55ace65ad1eb" is hardcoded for document downloads. This means the chart will always download documents from this specific commit. If this is intentional for version pinning, it should be documented. If not, the hash field should be empty by default to use the branch head.
    kubernetes/helm/openrag/templates/langflow/deployment.yaml:39
  • The initContainer uses a hardcoded curl image version "curlimages/curl:8.5.0". This version may become outdated or have security vulnerabilities over time. Consider either using a more recent version, using a parameterized value from values.yaml, or using a version tag that gets automatic security updates (though this reduces reproducibility).
    kubernetes/helm/openrag/templates/backend/deployment.yaml:38
  • The initContainer uses the same hardcoded curl image version "curlimages/curl:8.5.0". This version may become outdated or have security vulnerabilities over time. Consider either using a more recent version, using a parameterized value from values.yaml, or using a version tag that gets automatic security updates.
    kubernetes/helm/openrag/templates/backend/deployment.yaml:103
  • The initContainer downloads files from GitHub over HTTP without any integrity verification (no checksums, signatures, or hash validation). This means if GitHub is compromised or a man-in-the-middle attack occurs, malicious files could be downloaded and used. While the specific commit hash helps with reproducibility, there's no cryptographic verification that the downloaded content matches what's expected.

Consider adding SHA256 checksum verification for downloaded files, or use a more secure distribution method for production deployments.
kubernetes/helm/openrag/templates/langflow/deployment.yaml:107

  • The initContainer downloads files from GitHub over HTTP without any integrity verification (no checksums, signatures, or hash validation). This means if GitHub is compromised or a man-in-the-middle attack occurs, malicious flow files could be downloaded and executed. While the specific commit hash helps with reproducibility, there's no cryptographic verification that the downloaded content matches what's expected.

Consider adding SHA256 checksum verification for downloaded files, or use a more secure distribution method for production deployments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@rodageve rodageve merged commit 32f6e91 into main Feb 23, 2026
3 checks passed
@mpawlow mpawlow removed their request for review February 23, 2026 20:41
@edwinjosechittilappilly edwinjosechittilappilly added the enhancement 🔵 New feature or request label Feb 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement 🔵 New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants