Skip to content

Fix malformed MySQL FORCE INDEX hint with schema-qualified table prefix#3086

Merged
lahma merged 1 commit into
3.xfrom
fix/3084-mysql-schema-prefix-3x
May 14, 2026
Merged

Fix malformed MySQL FORCE INDEX hint with schema-qualified table prefix#3086
lahma merged 1 commit into
3.xfrom
fix/3084-mysql-schema-prefix-3x

Conversation

@lahma

@lahma lahma commented May 14, 2026

Copy link
Copy Markdown
Member

Summary

Fix the FORCE INDEX hint added in #2894 so it produces valid SQL when the configured table prefix includes a schema (e.g. common.QRTZ_).

Details

#2894 added FORCE INDEX hints to four MySQLDelegate methods using the SQL template placeholder {0} for both the table reference and the index name:

.Replace("{0}TRIGGERS t", "{0}TRIGGERS t FORCE INDEX (IDX_{0}T_NFT_ST)")

When the configured tablePrefix contains a schema (e.g. common.QRTZ_), string.Format substitutes {0} everywhere and the index identifier inherits the schema dot:

... FROM common.QRTZ_TRIGGERS t FORCE INDEX (IDX_common.QRTZ_T_NFT_ST) ...

MySQL/MariaDB index names cannot contain . — schema qualification belongs only on the table reference. This is the cause of the JobPersistenceException: An error occurred while scanning for the next trigger to fire reported in #3084 (regression from 3.15.1).

Approach

Introduce a second string.Format placeholder {1} in AdoJobStoreUtil.ReplaceTablePrefix that resolves to the unqualified portion of the prefix (everything after the last ., or the whole prefix when there is no schema). All four MySQLDelegate FORCE INDEX templates now use {1} for the index name. Templates that only use {0} are unaffected; the change is non-breaking.

Why this design

  • Centralizes schema-stripping in one place instead of every delegate re-implementing it.
  • No new protected surface on StdAdoDelegate.
  • Future-proof: any other delegate (Oracle, PostgreSQL, …) that later needs a schema-stripped identifier can use the same {1}.

Linked issue

Fixes #3084

Test plan

  • Added or updated unit tests
  • Ran dotnet test src/Quartz.Tests.Unit locally — 1378/1378 passed (9 MySQLDelegateTest including 4 new WithSchemaQualifiedPrefix_ProducesValidIndexHint cases)
  • Ran integration tests where relevant (build.cmd Compile UnitTest IntegrationTest, Docker required)
  • Manually verified against a MariaDB instance with tablePrefix = common.QRTZ_

Breaking change?

No. Existing SQL templates only use {0}; widening string.Format to two args is invisible to them. Grep across src/Quartz/Impl/AdoJobStore confirms no template currently contains a literal {1}.

🤖 Generated with Claude Code

When the configured tablePrefix contains a schema (e.g. "common.QRTZ_"),
the FORCE INDEX hints added in #2894 produced invalid SQL because the
index identifier inherited the schema dot:

  FORCE INDEX (IDX_common.QRTZ_T_NFT_ST)  -- invalid

MySQL/MariaDB index names cannot contain a '.' - schema qualification
belongs only on the table reference, not on the index name.

Add a second string.Format placeholder {1} to AdoJobStoreUtil that
resolves to the unqualified portion of the table prefix (everything
after the last '.', or the whole prefix when there is no schema), and
switch all four MySQLDelegate FORCE INDEX hints to use it. Existing
templates using only {0} are unaffected.

Fixes #3084

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@sonarqubecloud

Copy link
Copy Markdown

@lahma
lahma merged commit e0d56da into 3.x May 14, 2026
16 checks passed
@lahma
lahma deleted the fix/3084-mysql-schema-prefix-3x branch May 14, 2026 08:33
This was referenced Jun 27, 2026
lahma added a commit that referenced this pull request Jul 14, 2026
Implements control over which cluster node runs a trigger, for use cases
like in-memory caching between job runs.

Core feature:
- TriggerBuilder.WithPreferredNode() API for pinning triggers to nodes
- Manual pin (specific instance id) and auto-pin ("*" sentinel)
- SQL-level acquisition filter with checkin-time-aware liveness check
- INextVersion* internal interface pattern — no breaking changes on 3.x
- Optional PREFERRED_NODE column on QRTZ_TRIGGERS, probed at startup

Auto-pin prefix ("auto:"):
- Auto-pinned triggers stored as "auto:nodeA" to distinguish from
  explicit pins ("nodeA") in the PREFERRED_NODE column
- Steal-on-failover: a node firing a trigger whose raw pin is "*" or
  "auto:<other-node>" claims it as "auto:<self>" — acquisition only
  releases a foreign auto-pin via the liveness fallback, so the claim
  converges to a live, eligible node. Also fixes a write-back race
  where a fire acquired before ClusterRecover's reset would persist
  the stale "auto:<deadNode>" back after the state row was deleted,
  permanently pinning the trigger to a nonexistent node
- Dirty-flag persistence: UpdateTrigger only writes PREFERRED_NODE
  when the value was changed on that instance (setter, builder,
  deserialization, auto-pin claim) — never when merely loaded at
  acquire time. Prevents routine fires from reverting concurrent
  UpdateTriggerDetails re-pins or ClusterRecover resets
- ClusterRecover resets auto-pins to "*" sentinel so any eligible node
  can claim — correctly handles execution group limits
- Explicit pins preserved through failover; original node reclaims
  when it returns
- GetTriggerBuilder() strips prefix, produces explicit pin for
  clone/reschedule flows
- Validation rejects "auto:" anywhere in user-supplied values
- Clustered schedulers throw at startup if InstanceId contains "auto:"
  (gated on HasPreferredNodeColumn)
- SetPreferredNodeRaw() internal bypass for Quartz's own writes

Additional:
- PreferredNode/ExecutionGroup added to JSON scheduling configuration
- Cached INextVersionDelegate field for hot-path performance
- Missing-column log level changed from DEBUG to WARN
- WARN when UpdateTriggerDetails cannot persist a preferred-node change
  because the column is missing
- MySQL acquisition SQL uses the IDX_{1} unqualified-prefix FORCE INDEX
  pattern from #3086 (schema-qualified table prefixes)

Tests:
- Unit tests incl. RAMJobStore metadata-only behavior
- SQLite: old-schema graceful degradation, ghost-node pin still fires,
  stale-auto-pin steal regression, misfire preserves pin,
  acquire-without-fire does not claim
- PostgreSQL cluster (2-3 real nodes): exclusion, failover, write-back
  race regression, execution-group-saturated survivor, runtime re-pin
  redirect; base class unbinds nodes from SchedulerRepository (name-only
  non-proxy lookup previously collapsed concurrent nodes to one),
  cleans DB state per test, and uses a short idleWaitTime so nodes
  notice remote changes quickly

Co-Authored-By: Claude Fable 5 <[email protected]>
lahma added a commit that referenced this pull request Jul 14, 2026
Implements control over which cluster node runs a trigger, for use cases
like in-memory caching between job runs.

Core feature:
- TriggerBuilder.WithPreferredNode() API for pinning triggers to nodes
- Manual pin (specific instance id) and auto-pin ("*" sentinel)
- SQL-level acquisition filter with checkin-time-aware liveness check
- INextVersion* internal interface pattern — no breaking changes on 3.x
- Optional PREFERRED_NODE column on QRTZ_TRIGGERS, probed at startup

Auto-pin prefix ("auto:"):
- Auto-pinned triggers stored as "auto:nodeA" to distinguish from
  explicit pins ("nodeA") in the PREFERRED_NODE column
- Steal-on-failover with compare-and-swap: a node firing a trigger whose
  raw pin is "*" or "auto:<other-node>" claims it as "auto:<self>" via a
  conditional UPDATE against the acquire-time value, so concurrent
  re-pins/clears (UpdateTriggerDetails, ClusterRecover reset) win over
  the claim instead of being clobbered
- Dirty-flag persistence: UpdateTrigger writes PREFERRED_NODE only when
  the value was changed on that instance (setter, builder,
  deserialization) — never when merely loaded at acquire time; loading
  clears the flag so blob triggers do not stay permanently dirty. The
  flag is serialized ([OptionalField]) so explicit changes survive
  net472 remoting by-value marshaling
- Builder-built triggers fully define the pin: a definition without
  WithPreferredNode clears a stored pin on replace (matches
  EXECUTION_GROUP semantics)
- ClusterRecover resets auto-pins to "*" sentinel so any eligible node
  can claim — correctly handles execution group limits
- Explicit pins preserved through failover; original node reclaims
  when it returns
- GetTriggerBuilder() strips prefix, produces explicit pin for
  clone/reschedule flows
- Validation rejects "auto:" anywhere in user-supplied values
- Clustered schedulers throw at startup if InstanceId contains "auto:"
  (gated on HasPreferredNodeColumn)
- UpdateTriggerDetails throws when a pin change cannot be persisted
  because the column is missing (instead of reporting success)

Robustness (deep-review round):
- Optional-column probing tolerates connectivity failures: Initialize
  no longer requires a live database and no longer latches transient
  errors as "column missing" — the probe retries on the first database
  operation and at Start()
- Blob-stored ExecutionGroup/PreferredNode values are preserved (not
  scrubbed) when the optional column is absent, preventing permanent
  erasure on blob re-serialization
- Positional-provider safety: named parameters appear exactly once per
  statement (liveness subquery correlates on t.SCHED_NAME) and new
  statements add parameters in SQL token order
- preferredNode is [OptionalField] so pre-upgrade binary blobs still
  deserialize
- One-time WARN when a custom delegate's SelectTriggerToAcquire
  override is bypassed by the preferred-node-aware overload

Additional:
- PreferredNode/ExecutionGroup added to JSON scheduling configuration
- Cached INextVersionDelegate field for hot-path performance
- Missing-column log level changed from DEBUG to WARN
- MySQL acquisition SQL uses the IDX_{1} unqualified-prefix FORCE INDEX
  pattern from #3086 (schema-qualified table prefixes)

Tests:
- Unit tests incl. RAMJobStore metadata-only behavior
- SQLite: old-schema graceful degradation, ghost-node pin still fires,
  stale-auto-pin steal regression, misfire preserves pin,
  acquire-without-fire does not claim
- PostgreSQL cluster (2-3 real nodes): exclusion, failover, write-back
  race regression, execution-group-saturated survivor, runtime re-pin
  redirect; base class unbinds nodes from SchedulerRepository (name-only
  non-proxy lookup previously collapsed concurrent nodes to one),
  cleans DB state per test, and uses a short idleWaitTime so nodes
  notice remote changes quickly

Co-Authored-By: Claude Fable 5 <[email protected]>
lahma added a commit that referenced this pull request Jul 15, 2026
Implements control over which cluster node runs a trigger, for use cases
like in-memory caching between job runs.

Core feature:
- TriggerBuilder.WithPreferredNode() API for pinning triggers to nodes
- Manual pin (specific instance id) and auto-pin ("*" sentinel)
- SQL-level acquisition filter with checkin-time-aware liveness check
- INextVersion* internal interface pattern — no breaking changes on 3.x
- Optional PREFERRED_NODE column on QRTZ_TRIGGERS, probed at startup

Auto-pin prefix ("auto:"):
- Auto-pinned triggers stored as "auto:nodeA" to distinguish from
  explicit pins ("nodeA") in the PREFERRED_NODE column
- Steal-on-failover with compare-and-swap: a node firing a trigger whose
  raw pin is "*" or "auto:<other-node>" claims it as "auto:<self>" via a
  conditional UPDATE against the acquire-time value, so concurrent
  re-pins/clears (UpdateTriggerDetails, ClusterRecover reset) win over
  the claim instead of being clobbered
- Dirty-flag persistence: UpdateTrigger writes PREFERRED_NODE only when
  the value was changed on that instance (setter, builder,
  deserialization) — never when merely loaded at acquire time; loading
  clears the flag so blob triggers do not stay permanently dirty. The
  flag is serialized ([OptionalField]) so explicit changes survive
  net472 remoting by-value marshaling
- ClusterRecover resets auto-pins to "*" sentinel so any eligible node
  can claim — correctly handles execution group limits
- Explicit pins preserved through failover; original node reclaims
  when it returns
- GetTriggerBuilder() strips prefix, produces explicit pin for
  clone/reschedule flows
- Validation of the reserved "auto:" token shared across TriggerBuilder,
  AbstractTrigger, and TriggerDetailsUpdate (single helper)
- Clustered schedulers throw at startup if InstanceId contains "auto:"
  (gated on HasPreferredNodeColumn)

Robustness (deep-review rounds):
- Optional-column probing tolerates connectivity failures: a canary
  query against an always-present column distinguishes "table
  unreachable" (transient — retried, not latched) from "column missing"
  (latched). Initialize no longer requires a live database; the probe
  retries on the first database operation and at Start(), serialized by
  a semaphore so concurrent early operations don't double-probe
- Custom delegates that override the legacy acquisition SQL but not the
  preferred-node overload keep acquisition on the legacy path (their
  override still runs; preferred-node filtering not applied, warned)
  instead of being silently bypassed
- UpdateTriggerDetails degrades gracefully (warns, applies other
  changes) when the column is missing, instead of throwing and losing
  bundled description/priority updates
- Blob-stored ExecutionGroup/PreferredNode values are preserved (not
  scrubbed) when the optional column is absent
- Positional-provider safety: named parameters appear exactly once per
  statement (liveness subquery correlates on t.SCHED_NAME) and new
  statements add parameters in SQL token order
- preferredNode is [OptionalField] so pre-upgrade binary blobs still
  deserialize
- Acquisition SQL no longer projects/materializes the unread
  PREFERRED_NODE column (filtered in WHERE only)

Additional:
- PreferredNode/ExecutionGroup added to JSON scheduling configuration
- Cached INextVersionDelegate field for hot-path performance
- Missing-column log level changed from DEBUG to WARN
- MySQL acquisition SQL uses the IDX_{1} unqualified-prefix FORCE INDEX
  pattern from #3086 (schema-qualified table prefixes)

Tests:
- Unit tests incl. RAMJobStore metadata-only behavior
- SQLite: old-schema graceful degradation, ghost-node pin still fires,
  stale-auto-pin steal regression, misfire preserves pin,
  acquire-without-fire does not claim; CreateScheduler unbinds from
  SchedulerRepository so a failing test cannot leak a scheduler
- PostgreSQL cluster (2-3 real nodes): exclusion, failover, write-back
  race regression, execution-group-saturated survivor, runtime re-pin
  redirect; base class unbinds nodes from SchedulerRepository (name-only
  non-proxy lookup previously collapsed concurrent nodes to one),
  cleans DB state per test, uses a short idleWaitTime so nodes notice
  remote changes quickly

Co-Authored-By: Claude Fable 5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant