fix(knowledge): keep long non-ASCII folder names valid UTF-8 - #3056

Description

NormalizeKnowledgeFolderPath caps a directory name with segment[:MaxKnowledgeFolderSegmentLength] at internal/types/knowledge_folder.go:51. That index counts bytes, so a name longer than 128 bytes is cut wherever byte 128 happens to land. In a Chinese folder name that is inside the 43rd character, and the leading bytes of the half-cut character stay in the string, so the folder_path written to knowledges is not valid UTF-8.

The damage shows up downstream. json.Marshal replaces the broken bytes with U+FFFD, so the folder path the client reads back is not the one stored in the row. Send that value to a rename and RenameKnowledgeFolder (internal/application/service/knowledge.go:611) normalizes it into a source that no longer equals the stored folder_path, so RenameKnowledgeFolderPath matches nothing. The sidebar label for the folder also ends in a replacement character.

The cut now backs off to the nearest rune start. The cap stays a byte budget, so the invariant the constant exists for (one segment cannot eat the whole 1024-byte path budget) is unchanged, and an ASCII name truncates at exactly the same place it did before. A 60-character Chinese name keeps its first 42 characters, 126 bytes, instead of 128 bytes ending in half a character.

Type of Change

• 🐛 Bug fix
• ✨ New feature
• 💥 Breaking change
• 📚 Documentation update
• 🎨 Refactor
• ⚡ Performance improvement
• 🧪 Test
• 🔧 Configuration / Build / CI

Related Issue

No issue filed. Found while reading the folder path normalizer.

Testing

New test TestNormalizeKnowledgeFolderPathKeepsSegmentsValidUTF8 in internal/types/knowledge_folder_test.go covers a 60-character Chinese segment and an ASCII name with a two-byte character straddling the cap. Scoped to the changed package, as the README's Validation section suggests for a focused PR. Go 1.26.4 on macOS arm64.

$ gofmt -l internal/types/knowledge_folder.go internal/types/knowledge_folder_test.go
(no output)

$ go vet ./internal/types/
(no output)

$ go test ./internal/types/ -count=1
ok  	github.com/Tencent/WeKnora/internal/types	0.78s

$ go build ./cmd/server
(no output)

$ git diff --check origin/main...HEAD
(no output)

Restoring only knowledge_folder.go from main and keeping the new test gives the failure this fixes:

--- FAIL: TestNormalizeKnowledgeFolderPathKeepsSegmentsValidUTF8 (0.00s)
    knowledge_folder_test.go:56: Should be true
        Messages: segment truncated mid-rune: e4 b8 ad ... e4 b8 ad e4 b8
    knowledge_folder_test.go:68: Not equal:
        expected: "aaa...aaa"
        actual  : "aaa...aaa\xc3"

Checklist

• git diff --check origin/main...HEAD passes
• Changed source files are formatted
• Targeted tests for the changed packages/components pass
• Diff-scoped lint passes where applicable (for Go: golangci-lint run --new-from-rev=origin/main ./...)
• Full-repository checks were run, or any unrelated/environment-dependent failures are documented above
• Self-reviewed the code
• Added/updated tests covering the change
• Updated related documentation (README, docs/, Swagger annotations, etc.)
• Breaking changes are clearly called out in the description above

Screenshots / Recordings

None. No UI change.