Component
API Server / GraphQL
Infrahub version
1.9.3 (present in earlier versions too — defect was introduced when the Prefect Redis result-storage block was added).
Current Behavior
When Infrahub is configured to use an external TLS-only Redis (INFRAHUB_CACHE_TLS_ENABLED=true, plus optionally INFRAHUB_CACHE_TLS_INSECURE and INFRAHUB_CACHE_TLS_CA_FILE), background flows that persist results through Prefect — for example git_repositories_sync, clean-up-deadlocks, and schema_validate_migrations — start, run their internal task runs, and then hang in RUNNING indefinitely. The flow never reaches a terminal state.
Startup itself looks healthy, because RedisStorageContainer.asave(...) only writes the block document to Prefect's metadata DB; it does not connect to Redis. The failure only surfaces later, when a flow first tries to persist a result through the block.
Cache (InfrahubCache) and distributed-lock (InfrahubLock) traffic continue to work, because those code paths construct redis.Redis(..., ssl=..., ssl_cert_reqs=..., ssl_check_hostname=..., ssl_ca_certs=...) directly and honor INFRAHUB_CACHE_TLS_*. The Prefect result-storage block is the only Redis client in the codebase that does not.
Expected Behavior
INFRAHUB_CACHE_TLS_ENABLED, INFRAHUB_CACHE_TLS_INSECURE, and INFRAHUB_CACHE_TLS_CA_FILE should apply uniformly to every Redis client Infrahub creates — including the Prefect result-storage block — so that flows complete successfully against a TLS-only Redis.
Steps to Reproduce
- Provision a TLS-only Redis reachable to Infrahub (a managed TLS-only Redis, or a local
stunnel/testcontainer wrapper in front of a plaintext Redis).
- Set on the Infrahub backend processes:
INFRAHUB_CACHE_ADDRESS=<tls-redis-host>
INFRAHUB_CACHE_TLS_ENABLED=true
PREFECT_RESULTS_PERSIST_BY_DEFAULT=true
- Start Infrahub and trigger any flow that persists results (e.g. wait for
clean-up-deadlocks on its schedule, or run git_repositories_sync).
- Observe in Prefect: the flow's task runs complete, but the parent flow run stays in
RUNNING indefinitely.
Workaround that confirms the diagnosis: patch the saved block document in Prefect's DB to use connection_string: rediss://... instead of host/port/db/username/password. All affected flows complete immediately afterwards.
Additional Information
Root cause. backend/infrahub/workflows/initialization.py::setup_blocks() builds the block via RedisStorageContainer.from_host(...). Prefect's from_host has no TLS parameter, and the internal client path used in that branch (_client()) calls aioredis.Redis(host=..., port=..., username=..., password=..., db=...) with no ssl= argument. The only TLS-capable path through RedisStorageContainer is connection_string, which is then resolved via redis.Redis.from_url(...).
Sites that already honor the setting (for parity reference):
backend/infrahub/lock.py — redis.Redis(..., ssl=config.SETTINGS.cache.tls_enabled, ssl_cert_reqs=..., ssl_check_hostname=..., ssl_ca_certs=...)
backend/infrahub/services/adapters/cache/redis.py — same pattern
Proposed fix. Always construct a connection_string URL in setup_blocks():
- Scheme switches between
redis:// and rediss:// based on tls_enabled.
tls_insecure=true encodes as ?ssl_cert_reqs=none&ssl_check_hostname=False query params.
tls_ca_file encodes as ?ssl_ca_certs=<path> query param.
redis.Redis.from_url parses these query params and applies them to the underlying SSLConnection, so all three TLS knobs become deterministic across every Redis client in the codebase — no upstream Prefect change needed and no separate "this flag doesn't apply here" warning.
PR to follow.
Component
API Server / GraphQL
Infrahub version
1.9.3 (present in earlier versions too — defect was introduced when the Prefect Redis result-storage block was added).
Current Behavior
When Infrahub is configured to use an external TLS-only Redis (
INFRAHUB_CACHE_TLS_ENABLED=true, plus optionallyINFRAHUB_CACHE_TLS_INSECUREandINFRAHUB_CACHE_TLS_CA_FILE), background flows that persist results through Prefect — for examplegit_repositories_sync,clean-up-deadlocks, andschema_validate_migrations— start, run their internal task runs, and then hang inRUNNINGindefinitely. The flow never reaches a terminal state.Startup itself looks healthy, because
RedisStorageContainer.asave(...)only writes the block document to Prefect's metadata DB; it does not connect to Redis. The failure only surfaces later, when a flow first tries to persist a result through the block.Cache (
InfrahubCache) and distributed-lock (InfrahubLock) traffic continue to work, because those code paths constructredis.Redis(..., ssl=..., ssl_cert_reqs=..., ssl_check_hostname=..., ssl_ca_certs=...)directly and honorINFRAHUB_CACHE_TLS_*. The Prefect result-storage block is the only Redis client in the codebase that does not.Expected Behavior
INFRAHUB_CACHE_TLS_ENABLED,INFRAHUB_CACHE_TLS_INSECURE, andINFRAHUB_CACHE_TLS_CA_FILEshould apply uniformly to every Redis client Infrahub creates — including the Prefect result-storage block — so that flows complete successfully against a TLS-only Redis.Steps to Reproduce
stunnel/testcontainer wrapper in front of a plaintext Redis).INFRAHUB_CACHE_ADDRESS=<tls-redis-host>INFRAHUB_CACHE_TLS_ENABLED=truePREFECT_RESULTS_PERSIST_BY_DEFAULT=trueclean-up-deadlockson its schedule, or rungit_repositories_sync).RUNNINGindefinitely.Workaround that confirms the diagnosis: patch the saved block document in Prefect's DB to use
connection_string: rediss://...instead of host/port/db/username/password. All affected flows complete immediately afterwards.Additional Information
Root cause.
backend/infrahub/workflows/initialization.py::setup_blocks()builds the block viaRedisStorageContainer.from_host(...). Prefect'sfrom_hosthas no TLS parameter, and the internal client path used in that branch (_client()) callsaioredis.Redis(host=..., port=..., username=..., password=..., db=...)with nossl=argument. The only TLS-capable path throughRedisStorageContainerisconnection_string, which is then resolved viaredis.Redis.from_url(...).Sites that already honor the setting (for parity reference):
backend/infrahub/lock.py—redis.Redis(..., ssl=config.SETTINGS.cache.tls_enabled, ssl_cert_reqs=..., ssl_check_hostname=..., ssl_ca_certs=...)backend/infrahub/services/adapters/cache/redis.py— same patternProposed fix. Always construct a
connection_stringURL insetup_blocks():redis://andrediss://based ontls_enabled.tls_insecure=trueencodes as?ssl_cert_reqs=none&ssl_check_hostname=Falsequery params.tls_ca_fileencodes as?ssl_ca_certs=<path>query param.redis.Redis.from_urlparses these query params and applies them to the underlyingSSLConnection, so all three TLS knobs become deterministic across every Redis client in the codebase — no upstream Prefect change needed and no separate "this flag doesn't apply here" warning.PR to follow.