fix: Qwen model JSON preamble, force-recompile overview pages, health false positives#529
Merged
Merged
Conversation
added 3 commits
May 27, 2026 15:40
…artup scripts - Fix UnicodeEncodeError 'gbk' on Windows by setting encoding='utf-8' on stdout/stderr reconfigure in run_server.py - Guard Deep Research send to prevent 'settings are incomplete' crash - Add lazy import for notebook_manager in services/__init__.py - Add start_backend.bat with PYTHONIOENCODING=utf-8 and PYTHONUTF8=1 - Add start_frontend.bat convenience script - Document all patches in PATCHES.md for re-apply after upstream sync
…h false positives - _llm_writer.py: strip thinking preamble from Qwen output, remove unsupported response_format, increase max_tokens to 2600, fix reasoning_effort 'minimal' -> 'low' - engine.py: allow force-recompile on overview pages (previously a no-op), add _maybe_finalize_book to regenerate_block for book status propagation - kb_health.py: filter repeated_failures to only count error/fail log entries - PageReader.tsx: loading spinner + disabled state on regenerate buttons - page.tsx: error notifications via notify() for compile/regenerate failures - layout.tsx: suppressHydrationWarning for browser extension attribute injection
alvinets
marked this pull request as ready for review
May 27, 2026 18:29
Collaborator
|
Thanks for your contribution! |
pancacake
added a commit
that referenced
this pull request
Jun 11, 2026
#493 (sqlite): route _initialize() through the _connect() contextmanager (the one remaining raw sqlite3.connect FD leak); document the auto-commit contract so the redundant explicit commits are understood. #529 (book): - _strip_thinking_preamble: guard `{ not in text` before rindex() so plain prose (refusals/errors) no longer raises ValueError -> ERROR block; hoist json import to module top. - llm_json: honor caller max_tokens as max(max_tokens, 2600) instead of discarding it; bump default to 2600; fix stale 'minimal' docstring -> 'low'. - engine.compile_page: rebuild OVERVIEW pages deterministically via _materialize_overview_page on force, instead of running the generic LLM compiler over hand-built intro/concept-graph/index blocks. - kb_health.scan_log_health: key repeated-failure detection on the op, not the free-text message, removing false positives/negatives + dead block_error. - services/__init__: drop orphaned notebook_manager lazy-import + __all__ entry. - PageReader.tsx: restore t() i18n on regenerate buttons; add 'Regenerating…'. - remove PATCHES.md (developer-local scratch notes). #484 (zulip): also match the disambiguated @**Name|user_id** mention form; make the docstring truthful; add coverage. #500: remove stray PR_DESCRIPTION.md committed to repo root. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
vaskoyudha
added a commit
to vaskoyudha/deeptutor-for-programmer-fork
that referenced
this pull request
Jul 25, 2026
…rview pages, health false positives Book Engine fixes for local/Qwen models: robust JSON parsing, force-recompile now resets overview pages, regenerate_block finalizes the book, and scan_log_health no longer flags repeated successes as failures. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
vaskoyudha
added a commit
to vaskoyudha/deeptutor-for-programmer-fork
that referenced
this pull request
Jul 25, 2026
…#529/HKUDS#484 HKUDS#493 (sqlite): route _initialize() through the _connect() contextmanager (the one remaining raw sqlite3.connect FD leak); document the auto-commit contract so the redundant explicit commits are understood. HKUDS#529 (book): - _strip_thinking_preamble: guard `{ not in text` before rindex() so plain prose (refusals/errors) no longer raises ValueError -> ERROR block; hoist json import to module top. - llm_json: honor caller max_tokens as max(max_tokens, 2600) instead of discarding it; bump default to 2600; fix stale 'minimal' docstring -> 'low'. - engine.compile_page: rebuild OVERVIEW pages deterministically via _materialize_overview_page on force, instead of running the generic LLM compiler over hand-built intro/concept-graph/index blocks. - kb_health.scan_log_health: key repeated-failure detection on the op, not the free-text message, removing false positives/negatives + dead block_error. - services/__init__: drop orphaned notebook_manager lazy-import + __all__ entry. - PageReader.tsx: restore t() i18n on regenerate buttons; add 'Regenerating…'. - remove PATCHES.md (developer-local scratch notes). HKUDS#484 (zulip): also match the disambiguated @**Name|user_id** mention form; make the docstring truthful; add coverage. HKUDS#500: remove stray PR_DESCRIPTION.md committed to repo root. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes three categories of issues in the Book Engine encountered when running with local/Qwen models (Qwen3.6-35B-A3B via vLLM):
1.
_llm_writer.py— Local model JSON output handlingresponse_format=json_object. The existingparse_json_responsecould not recover JSON reliably.reasoning_effort: "minimal"— only"low","medium","high"are accepted.max_tokenswas too low (900) for local models that consume tokens on thinking text.2.
engine.py— Force-recompile & book finalization gapscompile_page(force=True)excluded overview pages from block reset, making "Force regenerate" a no-op for the book intro page.regenerate_block()did not trigger_maybe_finalize_book(), so fixing the last failing block left the book incompiling/partialstate.3.
kb_health.py— False-positive "Recurring issue" warningsscan_log_health()flagged every repeated log entry as a "recurring failure", including repeated successfulcompile_page → readyentries, making the health banner useless.4. Frontend UX — Missing feedback on long operations
compilePageandregenerateBlock— failures silently swallowed.Related Issues
Module(s) Affected
book(Book Engine —_llm_writer.py,engine.py,kb_health.py)web(Frontend —PageReader.tsx,page.tsx,layout.tsx)Checklist
pre-commit run --all-filesand fixed any issues.Detailed Changes
deeptutor/book/blocks/_llm_writer.py_strip_thinking_preamble()— finds the last{in the response (actual JSON after thinking preamble). Validates withjson.loads(), falls back to first{with heuristic prefix check.max_tokensfrom 900 → 2600 to accommodate thinking preamble + JSON output.response_format={"type": "json_object"}— local models ignore this parameter.reasoning_effortretry from"minimal"→"low"(vLLM only acceptslow/medium/high)._strip_thinking_preamblefirst, then fallback toparse_json_responseon raw text.deeptutor/book/engine.pypage.content_type != ContentType.OVERVIEWguard fromcompile_page(force=True)— overview pages now reset blocks on force-recompile.await self._maybe_finalize_book(book_id)toregenerate_block()for proper book status propagation.deeptutor/book/kb_health.py("error" in k.lower() or "fail" in k.lower())filter torepeated_failures— only actual errors/failures are reported, not successful compilations.web/app/(workspace)/book/components/PageReader.tsxdisabled+disabled:cursor-not-allowed disabled:opacity-50to regenerate buttons during loading.RefreshCcwwith<Loader2 className="animate-spin" />spinner.web/app/(workspace)/book/page.tsxcatchclauses tocompilePageandhandleRegenerateBlockthat callnotify()with error message.finallyfor partial progress reflection.web/app/layout.tsxsuppressHydrationWarningto<body>for browser extension attribute injection.Testing
All changes verified end-to-end with Qwen3.6-35B-A3B (vLLM) on Windows:
How to read this bookforce-recompiles (3/3 blocks in ~32s).regenerate_block()on last failing block correctly transitions book toready.scan_log_healthreturnsrepeated_failures: []for book with 36 successfulcompile_pageentries.notify(); loading spinner shown during compile.Files Changed