Repository navigation
fix: share MySQL/OceanBase initialization and reconcile fresh seeds - #7529
Conversation
wy471x
left a comment
There was a problem hiding this comment.
Reviewed locally at 05ae928, checked against the merge base with master (76122a4). I verified the seed reconciliation statically rather than by re-running the container matrix:
- CI scripts:
python3 .github/scripts/test-native-sql-matrix.py(23 tests OK) and.github/scripts/test-agent-gateway-migrations.py(10 tests OK) both pass on this head. - Permission cleanup: all 50 removed
permissionrows indb/init/mysql/schema.sqlreference aresource_idthat does not exist in the same file'sresourcetable (the base file had exactly 50 such orphans, the head has 0). Comparing the deleteddb/init/ob/schema.sqlagainst the reconciled MySQL seed: every OceanBase-only grant either duplicates a(role, resource)pair that already exists under a different permission id, or points at a resource that is absent from both files. So pointing OceanBase at the shared seed does not drop an effective grant. - The two dropped
plugin_handlerows (plugin 50, fieldprompt) are not consumed anywhere:AiProxyHandlehas nopromptfield, and MySQL/openGauss never shipped those rows.
Two things I would ask for before merge:
- Please rebase.
git merge-tree --write-tree gh/master <head>reports a modify/delete conflict ondb/init/ob/schema.sql: master has since removed 68 orphan permission rows from that same file (the same class of cleanup). The conflict is mechanical, but the rebase needs to keep the check scripts' expectations consistent with master's ob/og seeds. - State the scope. This PR edits the canonical
db/init/mysql/schema.sql, i.e. it changes what a fresh MySQL install gets (50 permission rows removed, plus the loggingKafka plugin row and anamespace_plugin_relrow). The title/description frame the change as an OceanBase fix, and that MySQL-visible part is easy to miss in a 2.5k-line deletion diff. Please call it out explicitly, or split it.
Also verified while diffing the seeds — shenyu_dict: the OceanBase-only DEFAULT_KEY_RESOLVER row (id 1679002911061737490, dict_name default) is not carried forward, and MySQL keeps CONTEXT_PATH_KEY_RESOLVER under that same id. Since AiTokenLimiterEnum has no default value and getByName falls back to CONTEXT_PATH, behaviour is unchanged and the guide's "unsupported default dictionary entry" note is accurate. But the H2 schema still ships DEFAULT_KEY_RESOLVER and no CONTEXT_PATH_KEY_RESOLVER, so all three schema families now disagree; a follow-up issue (or one line in the initialization guide) would keep that from being rediscovered later.
The added native-matrix checks (column/nullability, PK/index/uniqueness, orphan relations, duplicate natural keys, invalid JSON, grants and namespace defaults) are a real improvement — that is the right place to catch this class of seed regression.
05ae928 to
5e876de
Compare
Aias00
left a comment
There was a problem hiding this comment.
Review: share MySQL/OceanBase initialization and reconcile fresh seeds
This is a well-scoped, thoroughly validated consolidation. The design is sound and the CI evidence is comprehensive.
What is correct
- Shared fresh-install source, immutable upgrade baseline. Fresh OceanBase now uses
db/init/mysql/schema.sql(native-sql-matrix.py:43), while theupgradeflow still reads the historicaldb/init/ob/schema.sqlviagit show {BASELINE}:...(schema_source()atnative-sql-matrix.py:51-56). Deleting the duplicate full OB schema from the tree is therefore safe — the released baseline is pinned by commit and only consumed for upgrade compatibility. This is a clean way to avoid drift between the two copies. - Fresh-seed contract is now machine-checked.
check_fresh_mysql_schema()(native-sql-matrix.py:77-140) validates tables, secondary indexes, unique constraints, referential integrity (orphanpermission/resource/user_role/namespace_*_rel), duplicate natural keys, admin resource permission, default namespace plugin, and theloggingKafkabootstrapServervalue.test-native-sql-matrix.pyadds positive/negative coverage for both the shared-source routing and the validator. Good defense-in-depth. mysql/schema.sqledits match the stated reconciliation. 50permissionrows referencing absentresource_ids are removed (verified as orphans by the new validator'spermission.resource_id → resourcecheck);loggingKafka(pluginid33) is corrected from the non-existentnamesrvAddrtobootstrapServer, and itsnamespace_plugin_rel(id1801816010882822171) now carries the full config aligned with the plugin row — both asserted bycheck_fresh_mysql_schema. All 447 effective grants are retained.- CI repair is targeted and correct.
integrated-test.ymldrops themvndpath (which silently skipped the entire subreactor and reported BUILD SUCCESS in 0.036s) in favor of the Maven wrapper with-Dmaven.test.skip=truefor image construction, adocker image inspectgate before compose, and-DfailIfNoTests=truefor the actual run.native-sql-matrix.py:66-69sets--ulimit nofile=65536:65536only for the disposable OB container, without touching host/daemon limits. - Test-only fixture repairs are clean. The four
mockStatic(LoggerFactory.class, CALLS_REAL_METHODS)changes (ExceptionHandlers/WebsocketCollector/MetaDataService/UpstreamCheckService tests), thediscoveryDO.setNamespaceIdaddition, and theApplicationEventPublishermock inProxySelectorServiceImplTestalign the inherited master fixtures without altering production behavior or existing assertions.
Evidence
pr_build,native-sql-matrix(mysql/ob/pg/og/oracle, fresh+upgrade),agent_gateway_migrations,rat_check,check-license-header, and all 21shenyu-integrated-test-*jobs are green at head342955c0.- The shared-source routing and the validator are covered by new Python unit tests.
Non-blocking suggestions
- The H2 dictionary
DEFAULT_KEY_RESOLVERvsCONTEXT_PATH_KEY_RESOLVERdivergence is correctly documented as a separate follow-up; please keep that tracked so the H2 seed does not silently diverge further. - Consider adding a short release note that fresh MySQL installs also change (orphan permission removal +
loggingKafkafield rename), since the impact is not OB-only — existing deployments must use the version-specificdb/upgradescripts, which is already called out ininit-guide.md.
Approving. Nice work on the reconciliation rigor.
Fixes #7521
OceanBase fresh installations in MySQL mode now use the canonical
db/init/mysql/schema.sql. Remove the duplicate full schema and update the native runner, Agent Gateway migration checker and packaged initialization guide.This also changes fresh MySQL installations: remove 50 permission seeds referencing absent resources, fix loggingKafka's
bootstrapServerfield, and align its default-namespace configuration with the plugin configuration. All 447 effective MySQL role/resource grants are retained, and every effective grant in the former OceanBase seed is present in the shared schema. Existing databases continue to use the separate version-specific upgrade scripts.Scope and reconciliation
DEFAULT_KEY_RESOLVER(default), andAiTokenLimiterEnum.getByNamefalls back toCONTEXT_PATH. H2 alignment remains a separate follow-up.ad38688, preserving upstream SOFA/TARS seed cleanup. Keep the duplicate OceanBase initialization schema deleted.Test fixture repairs after rebase
A separate test-only commit repairs Admin fixtures inherited from master: supply the event publisher mock, set the discovery namespace in the listener-update test, and let LoggerFactory static mocks call real methods for unstubbed loggers. This prevents JsonUtils from permanently initializing a null logger. Production Java behavior and existing assertions are unchanged; no test ordering workaround is needed.
CI diagnosis and workflow repair
The failed SDK HTTP job at
e742dc5reported every one of the 21 integrated-test reactor modules asSKIPPEDunder mvnd, then returnedBUILD SUCCESSin 0.036 seconds. The gateway image was therefore absent locally. Compose tried to pullapache/shenyu-integrated-test-sdk-http:latestand failed withpull access denied. Health checks and tests were skipped; successful preceding build steps are not evidence that this test passed.The workflow, integrated-test POMs and SDK HTTP Compose file at the failed head match master
ad386882. An earlier master SDK HTTP job built its modules and ran four tests successfully. The daemon's internal reason for the all-skipped reactor has not been isolated; the failed CI log does not show a Maven Central or Docker Hub timeout.The separate CI repair at
342955c0uses the Maven wrapper for the selected integrated-test module and its reactor dependencies, skips test execution during image construction, verifies the local gateway image before Compose, and uses the wrapper withfailIfNoTests=truefor the actual test step. No SQL changes are made by this repair.YAML/shell syntax checks, the CI routing regression suite in Linux, root Apache RAT and
git diff --checkpassed. The local wrapper run compiled and packaged SDK HTTP and invoked Docker image construction, then stopped on an explicit local Docker base-image pull timeout foreclipse-temurin:17-centos7. Local Compose and the full SDK HTTP suite were not completed. The updated Linux SDK HTTP job passed at this head: integrated-image build, image inspection, Compose startup, health checks, test execution and result checking all completed successfully.Validation of SQL and fixture changes at e742dc5
pr_build / Build with Maven, the aggregate build check, Agent Gateway migration consistency, license headers and Apache RAT../mvnw -B -ntp -pl shenyu-admin -am test -Prelease -DskipRemoteResources=true -Djacoco.skip=truepassed all 34 modules: 2,586 tests, zero failures/errors and two existing skips. Admin accounts for 1,775 tests and one skip. Checkstyle passed.git diff --checkpassed.The original pre-rebase full 213-module Maven reactor validation (5,122 tests, zero failures/errors, eight existing skips) remains historical evidence; current-head checks are listed separately above.
Contribution checklist