You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 794a27b
Browse filesBrowse the repository at this point in the historyBrowse files
feat(normalizer): caps_normalization option, spoken phone numbers, hertz plurals, drop the 'S pass, emoji tag flags and joiners, web toggles (remsky#353)
Copy file name to clipboardExpand all lines: CHANGELOG.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,7 @@ Per-PR attribution and contributor credits are published automatically on the co
7
7
## [Unreleased]
8
8
### Added
9
9
-`normalization_options.remove_emoji` drops emoji before synthesis instead of reading them by name, any language (#353). Off by default.
10
+
-`normalization_options.caps_normalization` reads all-caps headers and names (`ARNE SAKNUSSEMM`) as words instead of letter by letter. On by default. Short acronyms (`FBI`, `US GDP`) are still spelled.
10
11
11
12
### Changed
12
13
- Text normalization refactored towards multi-language support: `Normalizer` base class w/ neutral passes, `EnglishNormalizer` implements the rest, registry keyed by lang code. Adding a language is a subclass + table test, see CONTRIBUTING.md.
@@ -20,6 +21,9 @@ Per-PR attribution and contributor credits are published automatically on the co
20
21
- Digits glued to letters (`MP3`, `B2B`, `v1.0`, `1.5x`) pass through as written. `COVID-19` is no longer COVID minus nineteen.
21
22
-`.5` reads as zero point five, `1980S` as nineteen eighty S.
22
23
- Long digit runs no longer stall or 500 the request.
24
+
- Phone numbers read as spoken digits, `555-123-4567` as five five five, one two three, four five six seven.
25
+
-`3.5 GHz` reads gigahertz and `1 min` reads one minute with `unit_normalization` on.
26
+
- Plural and possessive acronyms (`DVDs`, `DVD's`) are no longer rewritten to `DVD'S`, which was spelled out as dee-vee-dee-ess.
23
27
- Times with seconds keep their am/pm (`12:30:15 pm`).
24
28
- Blank input, or emoji-only input with `remove_emoji`, is a 400 on the streaming path too, not an empty 200.
25
29
-`--` and `---` read as a dash. Previously the words on either side fused and word timestamps stopped for the rest of the chunk (#249).
0 commit comments