Repository navigation
Commit fbbaa75
authored
fix: preserve paragraph breaks as sentence boundaries in normalizer (#525)
The normalizer replaced all newlines with spaces unconditionally,
merging paragraph-separated sections (headings, chapter titles, body
text) into one continuous phrase with no prosodic pause.
Convert double-newline paragraph breaks to a period before the
newline-to-space pass: if the preceding non-whitespace character is
already sentence-ending punctuation the period is omitted to avoid
duplication. Single newlines continue to become spaces.
Add tests covering the exact failure case from the issue report, no-
double-punctuation, single newline, and multiple consecutive blank
lines.
Fixes #519
Signed-off-by: Christian-Sidak <[email protected]>1 parent 34d694c commit fbbaa75
2 files changed
Lines changed: 63 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
485 | 485 | | |
486 | 486 | | |
487 | 487 | | |
488 | | - | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
489 | 495 | | |
490 | 496 | | |
491 | 497 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
394 | 394 | | |
395 | 395 | | |
396 | 396 | | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
0 commit comments