Skip to content

Commit 38cfb0e

Browse files
suryaiyer95claude
andcommitted
fix: address PR #705 bot review findings (coderabbitai + cubic + copilot)
Addresses the remaining issues raised by coderabbitai, cubic-dev-ai, and the Copilot PR reviewer on top of the multi-model consensus fix. ### CRITICAL - **`@azure/identity` peer dep removed** (`drivers/package.json`) `mssql@12` → `tedious@19` bundles `@azure/identity ^4.2.1` as a regular dependency. Declaring it here as an optional peer was redundant and caused transitive-version-drift concerns. Users get the correct version automatically through the tedious chain; our lazy import handles the browser-bundle edge case itself. ### MAJOR - **Cross-dialect date partition literal normalization** (`data-diff.ts`) `buildPartitionDiscoverySQL` on MSSQL returns a JS `Date` object, stringified upstream as `"Mon Jan 01 2024 …"`. `CONVERT(DATE, …, 23)` rejects that format. Normalize `partitionValue` to ISO `yyyy-mm-dd` before dialect casting so the T-SQL/Fabric path works end-to-end on dates discovered from MSSQL sources. - **`crossWarehouse` uses resolved warehouse identity** (`data-diff.ts`) Previous commit gated on dialect compare, which treated two independent MSSQL instances as "same warehouse" and would have let `joindiff` route a JOIN through a warehouse that can't resolve the other side's base tables. Now resolves both sides' warehouse name (falling back to the default warehouse when a side is omitted) and compares identities — identity-based gating handles both the "undefined vs default" case (cubic) and the "same-dialect, different instance" case (Copilot). - **Drop `mssql.connect()` fallback** (`sqlserver.ts`) `mssql@^12` guarantees `ConnectionPool` as a named export. The fallback silently re-introduced the global-shared-pool bug this branch was added to fix. Now throws a descriptive error if `ConnectionPool` is missing — cross-database pool interference cannot regress. - **Non-string `config.authentication` guarded** (`sqlserver.ts`) Caller passing a pre-built `{ type, options }` block (or `null`) previously crashed with `TypeError: rawAuth.toLowerCase is not a function`. Now only applies the shorthand lookup when `rawAuth` is a string; other values pass through so tedious can handle them or reject them with its own error. - **Unknown `azure-active-directory-*` subtype fails fast** (`sqlserver.ts`) Typos or future tedious subtypes previously dropped through all `else if` branches, producing a config with `encrypt: true` but no `authentication` block. tedious then surfaced an opaque error far from the root cause. Now throws with the offending subtype and the supported list. - **`execSync` replaced with async `exec`** (`sqlserver.ts`) The `az account get-access-token` CLI fallback previously blocked the event loop for up to 15s. Switched to `util.promisify(exec)` so the connection path stays non-blocking. - **Mixed named + unnamed column derivation preserves headers** (`sqlserver.ts`) Previously `SELECT name, COUNT(*), SUM(x)` produced either `["name", ""]` (blank header) or `["col_0", "col_1", "col_2"]` (lost `name`). Rewrote column/row derivation to iterate in one pass, preserving known named columns and synthesizing `col_N` only for expanded `""`-key positions. ### MINOR - **`(no values)` fallback for empty `diff_row.values` array** (`tools/data-diff.ts`) `[].join(" | ") ?? "(no values)"` never fires because `""` is falsy-but-not- nullish. Gate on `d.values?.length` instead. ### Test / docs - `sqlserver-unit.test.ts`: token-cache client-id test now counts actual `getToken` invocations (previous version only verified both got the same mocked token, which proved nothing about keying). - `sqlserver-unit.test.ts`: "empty result" test now mirrors the real mssql shape (`recordset.columns` is a property *on* the recordset array, not a sibling key). - `sqlserver-unit.test.ts`: added mixed-column regression tests — "name + COUNT + SUM" and "single unnamed column" — to lock in the derivation fix. - `sqlserver-unit.test.ts`: stubbed async `exec` via `util.promisify.custom` so tests drive both the `execSync` legacy path and the new async path. - `SKILL.md`: Fabric config fenced block now declares `yaml` (markdownlint MD040). All tests: 43/43 sqlserver driver + 238/238 opencode test suite. Attribution: findings identified by coderabbitai, cubic-dev-ai, and the Copilot PR reviewer. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
1 parent 872e082 commit 38cfb0e

6 files changed

Lines changed: 213 additions & 61 deletions

File tree

‎.opencode/skills/data-parity/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -450,7 +450,7 @@ Even when tables match perfectly, state what was checked:
450450

451451
Fabric uses the same TDS protocol as SQL Server — no separate driver needed. Configuration:
452452

453-
```
453+
```yaml
454454
type: "fabric"
455455
host: "<workspace-id>-<item-id>.datawarehouse.fabric.microsoft.com"
456456
database: "<warehouse-name>"

‎packages/drivers/package.json‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,5 @@
2222
"duckdb": "^1.0.0",
2323
"mongodb": "^6.0.0",
2424
"@clickhouse/client": "^1.0.0"
25-
},
26-
"peerDependencies": {
27-
"@azure/identity": ">=4.0.0"
28-
},
29-
"peerDependenciesMeta": {
30-
"@azure/identity": {
31-
"optional": true
32-
}
3325
}
3426
}

‎packages/drivers/src/sqlserver.ts‎

Lines changed: 76 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -109,8 +109,15 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
109109
"managed-identity": "azure-active-directory-msi-vm",
110110
msi: "azure-active-directory-msi-vm",
111111
}
112-
const rawAuth = config.authentication as string | undefined
113-
const authType = rawAuth ? (AUTH_SHORTHANDS[rawAuth.toLowerCase()] ?? rawAuth) : undefined
112+
// `config.authentication` is typed as unknown upstream — accept only
113+
// strings here. A caller passing a non-string (object, null, pre-built
114+
// auth block) shouldn't crash with "toLowerCase is not a function";
115+
// treat as "no shorthand requested" and leave authType undefined.
116+
const rawAuth = config.authentication
117+
const authType =
118+
typeof rawAuth === "string"
119+
? (AUTH_SHORTHANDS[rawAuth.toLowerCase()] ?? rawAuth)
120+
: undefined
114121

115122
if (authType?.startsWith("azure-active-directory")) {
116123
;(mssqlConfig.options as any).encrypt = true
@@ -169,11 +176,18 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
169176

170177
if (!token) {
171178
try {
172-
const { execSync } = await import("node:child_process")
173-
const out = execSync(
179+
// Use async `exec` (not `execSync`) so the connection path stays
180+
// non-blocking — `az account get-access-token` can take several
181+
// seconds to round-trip and execSync would block the entire
182+
// event loop for that duration.
183+
const childProcess = await import("node:child_process")
184+
const { promisify } = await import("node:util")
185+
const execAsync = promisify(childProcess.exec)
186+
const { stdout } = await execAsync(
174187
`az account get-access-token --resource ${resourceUrl} --query accessToken -o tsv`,
175-
{ encoding: "utf-8", timeout: 15000, stdio: ["pipe", "pipe", "pipe"] },
176-
).trim()
188+
{ encoding: "utf-8", timeout: 15000 },
189+
)
190+
const out = stdout.trim()
177191
if (out) {
178192
token = out
179193
expiresAt = parseTokenExpiry(out)
@@ -245,6 +259,18 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
245259
tenantId: config.azure_tenant_id,
246260
},
247261
}
262+
} else {
263+
// Any other `azure-active-directory-*` subtype (typo or future
264+
// tedious addition). Fail fast — otherwise we'd silently connect
265+
// with no `authentication` block and tedious would surface an
266+
// opaque error far from the root cause.
267+
throw new Error(
268+
`Unsupported Azure AD authentication subtype: "${authType}". ` +
269+
"Supported subtypes: azure-active-directory-default, " +
270+
"azure-active-directory-password, azure-active-directory-access-token, " +
271+
"azure-active-directory-msi-vm, azure-active-directory-msi-app-service, " +
272+
"azure-active-directory-service-principal-secret.",
273+
)
248274
}
249275
} else {
250276
// Standard SQL Server user/password
@@ -254,12 +280,17 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
254280

255281
// Use an explicit ConnectionPool (not the global mssql.connect()) so
256282
// multiple simultaneous connections to different servers are isolated.
257-
if (MssqlConnectionPool) {
258-
pool = new MssqlConnectionPool(mssqlConfig)
259-
await pool.connect()
260-
} else {
261-
pool = await mssql.connect(mssqlConfig)
283+
// `mssql@^12` guarantees ConnectionPool as a named export — if it's
284+
// missing, the installed driver version is too old. Fail fast rather
285+
// than silently use the global shared pool (which reintroduces the
286+
// cross-database interference bug this branch was added to fix).
287+
if (!MssqlConnectionPool) {
288+
throw new Error(
289+
"mssql.ConnectionPool is not available — the installed `mssql` package is too old. Upgrade to mssql@^12.",
290+
)
262291
}
292+
pool = new MssqlConnectionPool(mssqlConfig)
293+
await pool.connect()
263294
},
264295

265296
async execute(sql: string, limit?: number, _binds?: any[], options?: ExecuteOptions): Promise<ConnectorResult> {
@@ -288,28 +319,45 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
288319

289320
// mssql merges unnamed columns (e.g. SELECT COUNT(*), SUM(...)) into a
290321
// single array under the empty-string key: row[""] = [val1, val2, ...].
291-
// Flatten only the empty-string key to restore positional column values;
292-
// legitimate array values from other keys are preserved as-is.
293-
const flattenRow = (row: any): any[] => {
322+
// When a query mixes named and unnamed columns (e.g.
323+
// SELECT name, COUNT(*), SUM(x) → { name: "alice", "": [42, 100] }),
324+
// we must preserve the known header for `name` and synthesize col_N only
325+
// for the unnamed positions. Build columns and rows in a single pass so
326+
// they stay aligned regardless of how many unnamed values the row
327+
// contains.
328+
let columns: string[] = []
329+
let columnsBuilt = false
330+
const flatten = (row: any): any[] => {
294331
const vals: any[] = []
295-
for (const [k, v] of Object.entries(row)) {
296-
if (k === "" && Array.isArray(v)) vals.push(...v)
297-
else vals.push(v)
332+
let unnamedCounter = 0
333+
const entries = Object.entries(row)
334+
for (const [k, v] of entries) {
335+
if (k === "" && Array.isArray(v)) {
336+
for (const inner of v) {
337+
if (!columnsBuilt) columns.push(`col_${unnamedCounter}`)
338+
unnamedCounter++
339+
vals.push(inner)
340+
}
341+
} else if (k === "") {
342+
// Empty-string key with non-array value — rare edge case, give it
343+
// a synthetic name rather than producing a column named "".
344+
if (!columnsBuilt) columns.push(`col_${unnamedCounter}`)
345+
unnamedCounter++
346+
vals.push(v)
347+
} else {
348+
if (!columnsBuilt) columns.push(k)
349+
vals.push(v)
350+
}
298351
}
352+
columnsBuilt = true
299353
return vals
300354
}
301355

302-
const rows = limitedRecordset.map(flattenRow)
303-
const sampleFlat = rows.length > 0 ? rows[0] : []
304-
const namedKeys = recordset.length > 0 ? Object.keys(recordset[0]) : []
305-
const columns =
306-
namedKeys.length === sampleFlat.length
307-
? namedKeys
308-
: sampleFlat.length > 0
309-
? sampleFlat.map((_: any, i: number) => `col_${i}`)
310-
: (result.recordset?.columns
311-
? Object.keys(result.recordset.columns)
312-
: [])
356+
const rows = limitedRecordset.map(flatten)
357+
if (!columnsBuilt) {
358+
// No rows — fall back to driver-reported column metadata.
359+
columns = result.recordset?.columns ? Object.keys(result.recordset.columns) : []
360+
}
313361

314362
return {
315363
columns,

‎packages/drivers/test/sqlserver-unit.test.ts‎

Lines changed: 97 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,31 @@ const cliState = {
9191
throwError: null as null | { stderr?: string; message?: string },
9292
}
9393
const realChildProcess = await import("node:child_process")
94+
const realUtil = await import("node:util")
95+
// Stub `exec` with a custom `util.promisify.custom` so `promisify(exec)`
96+
// yields { stdout, stderr } exactly as the real implementation does. Also
97+
// keep the legacy callback form of `execSync` for tests that still use it.
98+
const execStub: any = (cmd: string, optsOrCb: any, maybeCb?: any) => {
99+
cliState.lastCmd = cmd
100+
const cb = typeof optsOrCb === "function" ? optsOrCb : maybeCb
101+
if (cliState.throwError) {
102+
const e: any = new Error(cliState.throwError.message ?? "az failed")
103+
e.stderr = cliState.throwError.stderr
104+
if (cb) cb(e, "", cliState.throwError.stderr ?? "")
105+
return { on() {}, stdout: null, stderr: null }
106+
}
107+
if (cb) cb(null, cliState.output, "")
108+
return { on() {}, stdout: null, stderr: null }
109+
}
110+
execStub[realUtil.promisify.custom] = (cmd: string, _opts?: any) => {
111+
cliState.lastCmd = cmd
112+
if (cliState.throwError) {
113+
const e: any = new Error(cliState.throwError.message ?? "az failed")
114+
e.stderr = cliState.throwError.stderr
115+
return Promise.reject(e)
116+
}
117+
return Promise.resolve({ stdout: cliState.output, stderr: "" })
118+
}
94119
mock.module("node:child_process", () => ({
95120
...realChildProcess,
96121
execSync: (cmd: string) => {
@@ -102,6 +127,7 @@ mock.module("node:child_process", () => ({
102127
}
103128
return cliState.output
104129
},
130+
exec: execStub,
105131
}))
106132

107133
// Import after mocking
@@ -174,7 +200,11 @@ describe("SQL Server driver unit tests", () => {
174200
})
175201

176202
test("empty result returns correctly", async () => {
177-
mockQueryResult = { recordset: [], recordset_columns: {} }
203+
// mssql exposes column metadata as `recordset.columns` (a property ON
204+
// the recordset array), not as a sibling key — mirror the real shape.
205+
const recordset: any[] = []
206+
;(recordset as any).columns = {}
207+
mockQueryResult = { recordset }
178208
const result = await connector.execute("SELECT * FROM t")
179209
expect(result.rows).toEqual([])
180210
expect(result.truncated).toBe(false)
@@ -507,8 +537,33 @@ describe("SQL Server driver unit tests", () => {
507537
recordset: [{ name: "alice", "": [42] }],
508538
}
509539
const result = await connector.execute("SELECT * FROM t")
540+
// Named header preserved; single unnamed aggregate synthesized.
541+
expect(result.columns).toEqual(["name", "col_0"])
510542
expect(result.rows).toEqual([["alice", 42]])
511543
})
544+
545+
test("mixed named + MULTIPLE unnamed aggregates keep named header", async () => {
546+
// SELECT name, COUNT(*), SUM(x) FROM t → { name: "alice", "": [42, 100] }.
547+
// Regression: previous implementation fell back to col_0..col_N for all
548+
// columns, erasing the known `name` header.
549+
mockQueryResult = {
550+
recordset: [{ name: "alice", "": [42, 100] }],
551+
}
552+
const result = await connector.execute("SELECT name, COUNT(*), SUM(x) FROM t")
553+
expect(result.columns).toEqual(["name", "col_0", "col_1"])
554+
expect(result.rows).toEqual([["alice", 42, 100]])
555+
})
556+
557+
test("single unnamed column gets synthetic name (no blank header)", async () => {
558+
// SELECT COUNT(*) FROM t → { "": [5] }
559+
mockQueryResult = {
560+
recordset: [{ "": [5] }],
561+
}
562+
const result = await connector.execute("SELECT COUNT(*) FROM t")
563+
expect(result.columns).toEqual(["col_0"])
564+
expect(result.columns).not.toContain("")
565+
expect(result.rows).toEqual([[5]])
566+
})
512567
})
513568

514569
// --- Azure token caching (Fix #2) ---
@@ -574,25 +629,48 @@ describe("SQL Server driver unit tests", () => {
574629
})
575630

576631
test("different clientIds cache separately", async () => {
632+
// Prove cache keying by counting distinct getToken invocations: with
633+
// separate clientIds we expect 2 calls (one per key); with a shared
634+
// clientId we expect 1 on the second connect.
635+
let getTokenCalls = 0
577636
azureIdentityState.tokenOverride = { token: "shared-token", expiresOnTimestamp: Date.now() + 3600_000 }
578-
resetMocks()
579-
const a = await connect({
580-
host: "h.database.windows.net", database: "d",
581-
authentication: "azure-active-directory-default",
582-
azure_client_id: "client-1",
583-
})
584-
await a.connect()
585-
const b = await connect({
586-
host: "h.database.windows.net", database: "d",
587-
authentication: "azure-active-directory-default",
588-
azure_client_id: "client-2",
589-
})
590-
await b.connect()
591-
// Both embed the mock token but the cache is keyed separately; both calls
592-
// hit getToken (not 1) — easiest way to assert is that both configs got
593-
// a token with no thrown "expired" behavior.
594-
expect(mockConnectCalls[0].authentication.options.token).toBe("shared-token")
595-
expect(mockConnectCalls[1].authentication.options.token).toBe("shared-token")
637+
const origCredential = (await import("@azure/identity")).DefaultAzureCredential
638+
const origGetToken = origCredential.prototype.getToken
639+
origCredential.prototype.getToken = async function (scope: string) {
640+
getTokenCalls++
641+
return origGetToken.call(this, scope)
642+
}
643+
try {
644+
resetMocks()
645+
const a = await connect({
646+
host: "h.database.windows.net", database: "d",
647+
authentication: "azure-active-directory-default",
648+
azure_client_id: "client-1",
649+
})
650+
await a.connect()
651+
const b = await connect({
652+
host: "h.database.windows.net", database: "d",
653+
authentication: "azure-active-directory-default",
654+
azure_client_id: "client-2",
655+
})
656+
await b.connect()
657+
// Two distinct client IDs → two distinct cache entries → two getToken
658+
// calls. If the cache were keyed only on resource URL this would be 1.
659+
expect(getTokenCalls).toBe(2)
660+
expect(mockConnectCalls[0].authentication.options.token).toBe("shared-token")
661+
expect(mockConnectCalls[1].authentication.options.token).toBe("shared-token")
662+
663+
// Reconnect with client-1 again — should hit the cache, no new getToken
664+
const c = await connect({
665+
host: "h.database.windows.net", database: "d",
666+
authentication: "azure-active-directory-default",
667+
azure_client_id: "client-1",
668+
})
669+
await c.connect()
670+
expect(getTokenCalls).toBe(2)
671+
} finally {
672+
origCredential.prototype.getToken = origGetToken
673+
}
596674
})
597675
})
598676

‎packages/opencode/src/altimate/native/connections/data-diff.ts‎

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -591,7 +591,21 @@ function buildPartitionWhereClause(
591591

592592
// date mode
593593
const expr = dateTruncExpr(granularity!, quotedCol, dialect)
594-
const escaped = partitionValue.replace(/'/g, "''")
594+
// Normalize the partition value to ISO yyyy-mm-dd. The mssql driver returns
595+
// date columns as JS Date objects which get `String()`-coerced upstream,
596+
// producing output like "Mon Jan 01 2024 00:00:00 GMT+0000 (UTC)" —
597+
// T-SQL `CONVERT(DATE, …, 23)` and Postgres date literals both reject that
598+
// format. Parsing once here keeps the downstream SQL dialect-safe.
599+
const isoDate = (() => {
600+
const trimmed = partitionValue.trim()
601+
// Already looks like yyyy-mm-dd — preserve as-is so pre-formatted values
602+
// (e.g. from Postgres, BigQuery, DATE_FORMAT MySQL output) flow through
603+
// without surprising timezone shifts.
604+
if (/^\d{4}-\d{2}-\d{2}(\s|T|$)/.test(trimmed)) return trimmed.slice(0, 10)
605+
const d = new Date(trimmed)
606+
return Number.isNaN(d.getTime()) ? trimmed : d.toISOString().slice(0, 10)
607+
})()
608+
const escaped = isoDate.replace(/'/g, "''")
595609

596610
// Cast the literal appropriately per dialect
597611
switch (dialect) {
@@ -849,6 +863,16 @@ export async function runDataDiff(params: DataDiffParams): Promise<DataDiffResul
849863
}
850864
}
851865

866+
// Resolve warehouse identity (fall back to the default warehouse when the
867+
// caller omits a side). Returns the canonical warehouse name so we can detect
868+
// cross-warehouse mode even when both sides share a dialect (e.g. two
869+
// independent MSSQL instances).
870+
const resolveWarehouseName = (warehouse: string | undefined): string | undefined => {
871+
if (warehouse) return warehouse
872+
const warehouses = Registry.list().warehouses
873+
return warehouses[0]?.name
874+
}
875+
852876
// Resolve dialect from warehouse config
853877
const resolveDialect = (warehouse: string | undefined): string => {
854878
if (warehouse) {
@@ -859,15 +883,21 @@ export async function runDataDiff(params: DataDiffParams): Promise<DataDiffResul
859883
return warehouseTypeToDialect(warehouses[0]?.type ?? "generic")
860884
}
861885

886+
const resolvedSource = resolveWarehouseName(params.source_warehouse)
887+
const resolvedTarget = resolveWarehouseName(params.target_warehouse ?? params.source_warehouse)
888+
862889
const dialect1 = resolveDialect(params.source_warehouse)
863890
const dialect2 = resolveDialect(params.target_warehouse ?? params.source_warehouse)
864891

865892
// Cross-warehouse mode requires side-specific CTE injection: T-SQL / Fabric
866893
// parse-bind every CTE body even when unreferenced, so sending the combined
867894
// prefix to a warehouse that lacks the other side's base table fails at parse.
868-
// Detect by dialect compare — more robust than warehouse-name compare, and
869-
// matches the underlying invariant that we care about (SQL-text divergence).
870-
const crossWarehouse = dialect1 !== dialect2
895+
// Gate on resolved warehouse identity, not dialect — two independent
896+
// same-dialect warehouses (e.g. two MSSQL instances) still can't resolve each
897+
// other's base tables. Identity comparison after resolving the default
898+
// warehouse avoids misclassifying `source_warehouse=undefined` vs the
899+
// explicit default-warehouse name as different.
900+
const crossWarehouse = resolvedSource !== resolvedTarget
871901

872902
// Explicit JoinDiff cannot work across warehouses: it emits one FULL OUTER
873903
// JOIN task referencing both CTE aliases, but side-aware injection only

0 commit comments

Comments
 (0)