feat(core): Brand token extraction — Tailwind config + CSS vars + W3C DTCG#30
feat(core): Brand token extraction — Tailwind config + CSS vars + W3C DTCG#30
Conversation
There was a problem hiding this comment.
Findings
-
[Major] Tailwind v4 parser drops tokens after the first
@themeblock — this causes silent token loss when files contain multiple theme blocks, evidencepackages/core/src/brand/tailwindExtractor.ts:158
Suggested fix:function extractFromV4Theme(source: string): DesignToken[] { const results: DesignToken[] = []; const themeBlockRe = /@theme\s*\{/g; for (const m of source.matchAll(themeBlockRe)) { const blockStart = (m.index ?? 0) + m[0].length - 1; const body = extractBodyAt(source, blockStart); if (!body) continue; const declRe = /--([\w-]+)\s*:\s*([^;]+);/g; for (const dm of body.matchAll(declRe)) { const prop = dm[1]; const rawValue = dm[2]?.trim(); if (!prop || !rawValue) continue; const tokenType = inferTypeFromCssProp(prop, rawValue); if (!tokenType) continue; results.push({ schemaVersion: 1, type: tokenType, name: prop, value: rawValue, origin: 'tailwind-config', group: prop.split('-').slice(0, 2).join('.') }); } } return results; }
-
[Major] Brace matching is not string/comment-aware, so parser can terminate early on
}inside quoted values — this can truncate section parsing and produce incorrect/missing tokens, evidencepackages/core/src/brand/tailwindExtractor.ts:61
Suggested fix:function extractBodyAt(text: string, bracePos: number): string | null { if (text[bracePos] !== '{') return null; let depth = 0; let quote: '"' | "'" | null = null; let escape = false; for (let i = bracePos; i < text.length; i++) { const ch = text[i]!; if (quote) { if (escape) { escape = false; continue; } if (ch === '\\') { escape = true; continue; } if (ch === quote) quote = null; continue; } if (ch === '"' || ch === "'") { quote = ch; continue; } if (ch === '{') depth++; else if (ch === '}') { depth--; if (depth === 0) return text.slice(bracePos + 1, i); } } return null; }
-
[Minor] Tailwind extractor tests do not cover the new v4
@themepath or multi-block behavior, so regressions in claimed v4 support are currently undetected, evidencepackages/core/src/brand/tailwindExtractor.test.ts:7
Suggested fix:it('extracts from multiple @theme blocks', async () => { const tokens = await extractFromTailwindConfig(V4_MULTI_THEME_FIXTURE); expect(tokens.map((t) => t.name)).toContain('color-brand'); expect(tokens.map((t) => t.name)).toContain('font-size-base'); });
Summary
Review mode: initial. 3 issues found in the latest diff (2 Major parser correctness/data-loss risks, 1 Minor test coverage gap). docs/VISION.md and docs/PRINCIPLES.md were not available in this checkout (Not found in repo/docs), so review used repo-visible constraints and changed lines only.
Testing
Not run (automation)
open-codesign Bot
| function extractFromV4Theme(source: string): DesignToken[] { | ||
| const results: DesignToken[] = []; | ||
|
|
||
| const themeBlockRe = /@theme\s*\{/g; |
There was a problem hiding this comment.
[Major] extractFromV4Theme only processes the first @theme block (exec once + immediate return path). In files with multiple @theme blocks, later tokens are silently dropped. Please iterate all matches and merge extracted declarations.
| if (text[bracePos] !== '{') return null; | ||
| let depth = 0; | ||
| let i = bracePos; | ||
| while (i < text.length) { |
There was a problem hiding this comment.
[Major] extractBodyAt counts {/} without ignoring quoted strings/comments. A } inside a string can close the block early and corrupt extraction. Please make brace scanning quote/comment-aware.
| import { extractFromTailwindConfig } from './tailwindExtractor.js'; | ||
|
|
||
| const __dir = dirname(fileURLToPath(import.meta.url)); | ||
| const FIXTURE = resolve(__dir, '__fixtures__/tailwind.config.js'); |
There was a problem hiding this comment.
[Minor] Current tests only use the v3 fixture and do not exercise the new v4 @theme path (including multi-block cases). Please add at least one v4 fixture test to lock behavior.
There was a problem hiding this comment.
Findings
-
[Major] Brace parser is not quote/comment aware, so
}inside string/comment can terminate blocks early and silently drop tokens; this can corrupt extraction output from valid Tailwind files, evidencepackages/core/src/brand/tailwindExtractor.ts:57
Suggested fix:function extractBodyAt(text: string, bracePos: number): string | null { if (text[bracePos] !== '{') return null; let depth = 0; let quote: '"' | "'" | null = null; let escape = false; let inLineComment = false; let inBlockComment = false; for (let i = bracePos; i < text.length; i++) { const ch = text[i]!; const next = text[i + 1]; if (inLineComment) { if (ch === '\n') inLineComment = false; continue; } if (inBlockComment) { if (ch === '*' && next === '/') { inBlockComment = false; i++; } continue; } if (!quote && ch === '/' && next === '/') { inLineComment = true; i++; continue; } if (!quote && ch === '/' && next === '*') { inBlockComment = true; i++; continue; } if (quote) { if (escape) { escape = false; continue; } if (ch === '\\') { escape = true; continue; } if (ch === quote) quote = null; continue; } if (ch === '"' || ch === "'") { quote = ch; continue; } if (ch === '{') depth++; else if (ch === '}') { depth--; if (depth === 0) return text.slice(bracePos + 1, i); } } return null; }
-
[Major] Tailwind v4 detection only matches bare
@theme {and misses valid variants like@theme inline {/@theme static {, causing wrong parser path and token loss, evidencepackages/core/src/brand/tailwindExtractor.ts:139
Suggested fix:function hasTailwindV4Theme(source: string): boolean { return /@theme\b[^{}]*\{/.test(source); }
Summary
Review mode: follow-up after new commits. 2 Major issues remain in the latest PR diff (Tailwind parsing correctness/data-loss risks). docs/VISION.md and docs/PRINCIPLES.md are Not found in repo/docs in this checkout.
Testing
Not run (automation)
open-codesign Bot
| } | ||
|
|
||
| // Extract the body of a `{` brace at position `bracePos` in `text`. | ||
| function extractBodyAt(text: string, bracePos: number): string | null { |
There was a problem hiding this comment.
[Major] extractBodyAt counts braces without ignoring strings/comments. A } in quoted values or comments can close blocks early and silently drop tokens. Please make the scanner quote/comment-aware.
| } | ||
|
|
||
| function hasTailwindV4Theme(source: string): boolean { | ||
| return /@theme\s*\{/.test(source); |
There was a problem hiding this comment.
[Major] hasTailwindV4Theme only matches bare @theme { and misses @theme inline { / @theme static {, so v4 files can be routed to v3 parsing and lose tokens. Please align this regex with extractFromV4Theme.
fb2c1b6 to
65eef4f
Compare
There was a problem hiding this comment.
Findings
-
[Major] DTCG
dimensionis forced tospacing, which misclassifies valid non-spacing dimension tokens (for exampletypography.fontSize.*, radius, line-height) and can corrupt downstream token usage, evidencepackages/core/src/brand/dtcgImporter.ts:14
Suggested fix:const DTCG_TYPE_MAP: Partial<Record<string, TokenType>> = { color: 'color', fontFamily: 'fontFamily', fontSize: 'fontSize', shadow: 'shadow', }; function resolveType(dtcgType: string | undefined, path: string): TokenType | null { if (dtcgType === 'dimension' || dtcgType === 'number') { return resolveTypeFromPath(path); } if (dtcgType) { const mapped = DTCG_TYPE_MAP[dtcgType]; if (mapped) return mapped; } return resolveTypeFromPath(path); }
-
[Major] CSS var block extraction stops at the first
}because[^}]is not quote/comment aware, so valid declarations after}inside strings/comments are silently dropped, evidencepackages/core/src/brand/cssVarExtractor.ts:39
Suggested fix:function extractDeclarations(source: string): Array<{ prop: string; value: string }> { const results: Array<{ prop: string; value: string }> = []; const blockRe = /(?::root|\[data-theme[^\]]*\])\s*\{/g; for (const m of source.matchAll(blockRe)) { const start = (m.index ?? 0) + m[0].length - 1; const body = extractBodyAt(source, start); // reuse brace scanner from tailwind extractor if (!body) continue; const declRe = /--([\w-]+)\s*:\s*([^;]+);/g; for (const dm of body.matchAll(declRe)) { const prop = dm[1]?.trim(); const value = dm[2]?.trim(); if (prop && value) results.push({ prop, value }); } } return results; }
Summary
Review mode: follow-up after new commits. 2 Major issues remain in the latest head, both in newly added extractor logic that can silently produce incorrect token outputs. docs/VISION.md and docs/PRINCIPLES.md are Not found in repo/docs in this checkout.
Testing
Not run (automation). Suggested additions: one Vitest for DTCG dimension under typography.fontSize.* and one for CSS declarations containing } in string/comment.
open-codesign Bot
| color: 'color', | ||
| fontFamily: 'fontFamily', | ||
| fontSize: 'fontSize', | ||
| dimension: 'spacing', |
There was a problem hiding this comment.
[Major] Mapping DTCG dimension directly to spacing misclassifies many valid dimension tokens (e.g. typography.fontSize.*). Prefer resolving dimension/number from token path instead of hard-mapping to spacing.
| function extractDeclarations(source: string): Array<{ prop: string; value: string }> { | ||
| const results: Array<{ prop: string; value: string }> = []; | ||
|
|
||
| const blockRe = /(?::root|\[data-theme[^\]]*\])\s*\{([^}]*)\}/g; |
There was a problem hiding this comment.
[Major] [^}] truncates blocks at the first } even when it appears inside string/comment content, which drops later declarations. Parse from the opening { with a quote/comment-aware brace scanner.
65eef4f to
1689aec
Compare
There was a problem hiding this comment.
Findings
-
[Major] Tailwind v4
--text-*size tokens are misclassified ascolorbecausetextis matched before font-size checks, which corrupts downstream token typing, evidencepackages/core/src/brand/tailwindExtractor.ts:185
Suggested fix:function inferTypeFromCssProp(prop: string, value: string): TokenType | null { if (/font-size|text-size|^text-(xs|sm|base|lg|xl|2xl|3xl|4xl)$/.test(prop)) return 'fontSize'; if (/font-family/.test(prop)) return 'fontFamily'; if (/line-height|leading/.test(prop)) return 'lineHeight'; if (/radius|rounded/.test(prop)) return 'radius'; if (/shadow/.test(prop)) return 'shadow'; if (/spacing|gap|padding|margin|space/.test(prop)) return 'spacing'; if (/color|bg|text-color|border-color|fill|stroke|ring/.test(prop) || looksLikeColor(value)) return 'color'; return null; }
-
[Major] CSS var inference classifies any
--border-*token ascolorbefore radius/spacing checks, causing non-color tokens (for example--border-radius-md) to be typed incorrectly, evidencepackages/core/src/brand/cssVarExtractor.ts:9
Suggested fix:function inferType(prop: string, value: string): TokenType | null { const p = prop.toLowerCase(); if (/radius|rounded|border-radius/.test(p)) return 'radius'; if (/font-size|text-size|text-sm|text-base|text-lg|text-xl/.test(p)) return 'fontSize'; if (/font-family|font-sans|font-mono|font-serif|typeface/.test(p)) return 'fontFamily'; if (/spacing|space|gap|padding|margin|indent|offset/.test(p)) return 'spacing'; if (/shadow|elevation/.test(p)) return 'shadow'; if (/line-height|leading/.test(p)) return 'lineHeight'; if (/color|palette|brand|accent|fg|bg|foreground|background|fill|stroke|ring|border-color/.test(p)) return 'color'; if (looksLikeColor(value)) return 'color'; return null; }
Summary
- Review mode: follow-up after new commits. 2 Major issues remain in the latest head, both introduced in new token-extractor logic and both can silently produce wrong token types.
docs/VISION.mdanddocs/PRINCIPLES.md: Not found in repo/docs.
Testing
- Not run (automation):
pnpmis unavailable in this runner.
open-codesign Bot
| } | ||
|
|
||
| function inferTypeFromCssProp(prop: string, value: string): TokenType | null { | ||
| if (/color|bg|text|border|fill|stroke|ring/.test(prop)) return 'color'; |
There was a problem hiding this comment.
[Major] inferTypeFromCssProp() matches text as color before size checks, so v4 tokens like --text-sm are typed as color instead of fontSize. Please prioritize font-size/text-size patterns before generic color matching (or narrow color regex to text-color).
| function inferType(prop: string, value: string): TokenType | null { | ||
| const p = prop.toLowerCase(); | ||
|
|
||
| if (/color|palette|brand|accent|fg|bg|foreground|background|fill|stroke|ring|border/.test(p)) |
There was a problem hiding this comment.
[Major] inferType() currently treats any border token as color, which misclassifies non-color vars such as --border-radius-* or border widths. Please check radius/spacing first and narrow the color matcher to border-color.
1689aec to
c3938a9
Compare
There was a problem hiding this comment.
Findings
- [Major] Tailwind v4 mode detection misses valid
@themevariants (for example@theme inline { ... }/@theme static { ... }), so those files fall back to v3 parsing and can return incomplete or empty token results. Evidencepackages/core/src/brand/tailwindExtractor.ts:185,packages/core/src/brand/tailwindExtractor.ts:293.
Suggested fix:function hasTailwindV4Theme(source: string): boolean { // Match plain `@theme {}` and keyword forms like `@theme inline {}`. return /@theme\b[^\{]*\{/.test(source); }
Summary
- Review mode: follow-up after new commits.
- 1 Major issue found in the latest head.
docs/VISION.mdanddocs/PRINCIPLES.md: Not found in repo/docs.
Testing
- Not run (automation):
pnpmis unavailable in this runner.
open-codesign Bot
| } | ||
|
|
||
| function hasTailwindV4Theme(source: string): boolean { | ||
| return /@theme\s*\{/.test(source); |
There was a problem hiding this comment.
[Major] hasTailwindV4Theme() only matches @theme { and misses valid v4 forms like @theme inline { / @theme static {. This can route v4 files into extractFromV3Config() and return incomplete/empty tokens. Please widen detection to match any @theme prelude before {, e.g. /@theme\b[^\{]*\{/.
c3938a9 to
ddbe0d8
Compare
There was a problem hiding this comment.
Findings
-
[Major] Silent token drop on unresolved DTCG type inference can hide import failures and lose user token data, conflicting with the project rule to surface errors with context; evidence
packages/core/src/brand/dtcgImporter.ts:63.
Suggested fix:const tokenType = resolveType($type, name); if (!tokenType) { throw new Error(`Unsupported DTCG token type/path: type=${$type ?? 'unknown'} path=${name}`); }
-
[Major]
$valuecan beundefinedbut still gets serialized and emitted, producing an invalid runtime token payload (valuemay becomeundefined) instead of a validated string; evidencepackages/core/src/brand/dtcgImporter.ts:57,packages/core/src/brand/dtcgImporter.ts:60.
Suggested fix:const rawValue = record['$value']; if (rawValue === undefined) { throw new Error(`DTCG token missing $value at path: ${pathSegments.join('.')}`); } const value = serializeValue(rawValue);
Summary
- Review mode: follow-up after new commits
- 2 issues found in current head diff, both in DTCG import error handling/validation.
docs/VISION.mdanddocs/PRINCIPLES.md: Not found in repo/docs.
Testing
- Not run (automation):
pnpmis unavailable in this runner environment.
| const value = serializeValue(rawValue); | ||
| const name = pathSegments.join('.'); | ||
| const tokenType = resolveType($type, name); | ||
| if (!tokenType) return; |
There was a problem hiding this comment.
[Major] This silently drops tokens when type inference fails (if (!tokenType) return;). That can hide import errors and lose data without user feedback. Suggest throwing with path/type context instead of silently returning.
| const rawValue = record['$value']; | ||
| const $type = typeof record['$type'] === 'string' ? record['$type'] : inherited$type; | ||
|
|
||
| const value = serializeValue(rawValue); |
There was a problem hiding this comment.
[Major] rawValue can be undefined ($value key present but undefined), and serializeValue then propagates an invalid runtime value. Add an explicit rawValue === undefined guard and throw with path context before serializing.
… DTCG (PR-A)
Implements the core data model and three text-only extractors for Brand
Token Inheritance (research 13). No UI, no SQLite, no Figma — pure
backend extraction layer.
- packages/shared: DesignTokenV1 + DesignTokenSet zod schemas (W3C DTCG
compatible, schemaVersion: 1), re-exported from shared index
- packages/core/src/brand/tailwindExtractor.ts: parses tailwind.config.{js,ts}
as plain text (never require/eval); handles both v3 JS object literal and
v4 CSS @theme block; extracts colors, fontSize, fontFamily, spacing,
borderRadius, boxShadow, lineHeight
- packages/core/src/brand/cssVarExtractor.ts: extracts :root and
[data-theme] CSS custom properties; heuristic type inference from prop
name and value
- packages/core/src/brand/dtcgImporter.ts: recursive W3C DTCG 2025.10
JSON walker; supports $type inheritance from group nodes; maps dimension
→ spacing, shadow, color, fontFamily, fontSize
- packages/core/src/brand/index.ts: public API barrel
Security: Tailwind config is read as text only — no require(), no eval(),
no new Function(). Prevents arbitrary code execution from user-supplied
config files.
Tests: 28 new tests (8 tailwindExtractor + 8 cssVarExtractor + 12
dtcgImporter). All 53 core tests pass; full workspace green.
New deps: none (zero new dependencies added).
Signed-off-by: hqhq1025 <[email protected]>
Signed-off-by: hqhq1025 <[email protected]>
Signed-off-by: hqhq1025 <[email protected]>
The previous extractBodyAt walked braces blindly, so a `}` inside a string literal or CSS comment could close a block early and silently drop tokens from valid Tailwind v3 / v4 sources. Track string state (`'`, `"`, backtick) with escape handling and CSS/JS comment state (`//` line and `/* */` block) so only structural braces affect depth. Signed-off-by: hqhq1025 <[email protected]>
…orcing spacing DTCG `dimension` (and `number`) can mean spacing, font-size, line-height, radius, border-width, etc. The previous map blindly bucketed every dimension token under `spacing`, misclassifying typography.fontSize.*, radius.*, and similar tokens. Now ambiguous DTCG types defer to the path-based resolver, which already understands fontSize/lineHeight/radius/spacing hints. Tokens without a usable hint are dropped rather than silently mislabeled. Signed-off-by: hqhq1025 <[email protected]>
The Tailwind v4 `--text-*` tokens were misclassified as `color` because the broad `text` keyword matched before the `font-size` and value-shape checks. Same shape bug for `--border-radius-*` which the broad `border` keyword swallowed as `color`. Reorder both inferType passes so specific shape patterns (font-size, font-family, line-height, radius, shadow, spacing) run first, then disambiguate the ambiguous `--text-*` namespace by value: color values become `color`, length values become `fontSize`, and the conventional size names (xs/sm/base/lg/xl/2xl/...) fall back to `fontSize`. Adds vitest coverage for both extractors: - `--text-lg: 1.125rem` -> fontSize - `--text-primary: oklch(...)` -> color - `--border-radius-md: 8px` -> radius - `--border-color-muted: #ddd` -> color Signed-off-by: hqhq1025 <[email protected]>
hasTailwindV4Theme only matched bare `@theme {`, so files using
`@theme inline`, `@theme static`, or `@theme reference` fell back
to the v3 parser and produced empty/incomplete tokens. Match
`@theme\b[^{]*\{` to align with the block scanner already in
extractFromV4Theme.
Signed-off-by: hqhq1025 <[email protected]>
…own` Previously, leaf tokens whose `$type` and path both failed type inference were silently dropped from the imported set. This could hide user-authored brand tokens with no warning, conflicting with the project rule to surface errors with context. Now we keep them with `type: 'unknown'` and emit a single `console.warn` listing every unresolved token name, so the caller can either ignore them or correct the input. Adds `'unknown'` to the DesignToken `type` enum so the schema can carry the bucket without lying about what we know. Addresses Codex Major on PR #30. Signed-off-by: hqhq1025 <[email protected]>
ddbe0d8 to
74fd100
Compare
There was a problem hiding this comment.
Findings
- No high-confidence issues found in added/modified lines of this update.
Summary
- Review mode: follow-up after new commits
- No new issues identified in the latest head commit range.
- Constraint checks on this diff: direct provider SDK imports not found, new dependencies/license changes not found, and no UI-token hardcoding violations found in app source touched by this update.
docs/VISION.mdanddocs/PRINCIPLES.md: Not found in repo/docs.
Testing
- Not run (automation):
pnpmis unavailable in this runner, so tests could not be executed here.
open-codesign Bot
Summary
First PR of 3 implementing Brand Token Inheritance (research 13). Backend extractors only — no UI, no SQLite, no Figma.
DesignTokenschema (packages/shared/src/design-token.ts) — zod v1,schemaVersionfield, 7 token typestailwindExtractor.ts) — handles both v3 JS object literal and v4 CSS@themeblock; neverrequire()/eval()— prevents arbitrary code execution from user-supplied filescssVarExtractor.ts) — parses:rootand[data-theme]blocks; heuristic type inference from prop name + valuedtcgImporter.ts) — recursive walker with$typeinheritance from group nodes; mapsdimension→spacing, etc.packages/core/src/brand/index.ts)Follow-up:
Security
Tailwind config is parsed as text only (no
require(), noeval()). Prevents arbitrary code execution from user-supplied config files.New dependencies
None. Zero new runtime or dev dependencies added.
License impact
None (no new dependencies).
Test plan
tailwindExtractor: 8 tests — fixturetailwind.config.js→ colors, fontSize, fontFamily, borderRadius extracted; v3 theme.colors + theme.extend.colors both collected; no duplicates; correct origin/schemaVersioncssVarExtractor: 8 tests —--color-*→ color,--font-family-*→ fontFamily,--font-size-*→ fontSize,--space-*→ spacing,--radius-*→ radius,--shadow-*→ shadow;[data-theme]overrides collecteddtcgImporter: 12 tests — nested$value/$typetree → flat tokens with correct group;$typeinherited from group nodes; shadow/fontFamily/spacing/color all classified; graceful handling of non-object inputAll 53 core tests pass. Full workspace green (
pnpm -r test,pnpm -r typecheck).