Log inSign up
Generative History
352 posts
user avatar
Generative History
@HistoryGPT
History Professor (Mark Humphries) exploring how historians can engage with Generative AI
Waterloo
generativehistory.substack.com
Joined March 2023
104
Following
1,686
Followers
  • Pinned
    user avatar
    Generative History
    @HistoryGPT
    May 27
    Using a new method of automated/human verification which uses multiple models to identify and highlight potential errors, we’ve reduced meaningful error rates on handwriting recognition to 0.33% WER and 0.23%. 18 months ago the gold standard was around 50x worse.
    904
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Google is A/B testing a new model (Gemini 3?) in AI Studio. I tried my hardest 18th century handwritten document. Terrible writing and full of spelling and grammatical errors that predictive LLMs want to correct. The new model was very nearly perfect. No other model is close.
    634K
  • user avatar
    Generative History
    @HistoryGPT
    May 14, 2024
    GPT-4o is truly remarkable on 18th handwriting. I gave it the following letter and asked it for a transcription. A couple of very minor errors…amazing!
    1.4M
  • user avatar
    Generative History
    @HistoryGPT
    Oct 17, 2025
    Replying to @HistoryGPT
    Read my full analysis here: generativehistory.substack.com/p/has-google-q…
    330K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Replying to @HistoryGPT
    Some additional context: the spelling errors and names are important t for two reasons. First, obviously, accuracy, More important (from a technical point of view): LLMs are predictive and misspelled words (and names) are out of distribution results.
    25K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 17, 2025
    Has Google quietly solved two of AI’s oldest problems? Their mysterious new model seems to have achieved expert level handwriting recognition. But it also shows evidence of spontaneous, abstract symbolic reasoning. No other model does those things.
    16K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Replying to @HistoryGPT
    To me, this result is significant because the model has to repeatedly choose a low probability output that is actually more correct for the task at hand. Very hard to do for LLMs (up until now).
    15K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Replying to @HistoryGPT
    To this point, models have had great difficulty correctly transcribing handwrittten text where the capitalization, punctuation, spelling, and grammar are incorrect. Getting the models to ~95% accuracy was a vision problem. iMO, above that is a reasoning problem.
    18K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 17, 2025
    Replying to @HistoryGPT
    Yes the prompt is very simple and is the same one we’ve used on all models for consistency:
    6.9K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Replying to @HistoryGPT
    I have no idea what model this actually is, but whatever it is seems to have overcome this major issue.
    6.8K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Replying to @pizzacritic999
    My prompt asked it to accurately transcribe the document retaining the original spelling, punctuation, capitalization, and grammar. It’s what we naturally default to as historians and I forget that might not be obvious (or even desirable) outside my narrow bubble!
    663
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Replying to @Fixlation7
    That looks like a pretty typical result for GPT5 or Gemini 2.5: it does not get the names right and corrects spelling.
    3.5K
  • user avatar
    Generative History
    @HistoryGPT
    Oct 14, 2025
    Replying to @Fixlation7
    So GPT-5 and Gemini 2.5 both achieve something like 95% accuracy. But the 5% is really important for historians. If you are making a searchable document and the names are wrong, it is a useless result as we routinely search for names.
    938
  • user avatar
    Generative History
    @HistoryGPT
    Nov 11, 2025
    Good question and of, course, I am not sure but this is my intuition. The reason is that no other item on the page is quantified in lbs and ozs nor are any other items quantified after the description. They are all written as X gallons of Y etc. This one is also the only one with
    2.2K

New to X?

Sign up now to get your own personalized timeline!

Create account

By signing up, you agree to the Terms of Service and Privacy Policy, including Cookie Use.

Terms of Service|Privacy Policy|Cookie Policy|Accessibility|Ads info|© 2026 X Corp.
Don't miss what's happening
People on X are the first to know.
Log inSign up