Skip to content

bufio.Scanner: token too long #3409

New issue
New issue
Closed
Closed
bufio.Scanner: token too long#3409
Assignees
rekram1-node
Labels
bugSomething isn't working

Description

@anthonymq
anthonymq
opened on Oct 24, 2025

Description

Hi,
When working on a small repo, opencode is stuck after i get a notification "bufio.Scanner: token too long"
After that it's stuck on "Generating..."

in the logs i see:

ERROR 2025-10-24T12:46:16 +40ms service=tui error=bufio.Scanner: token too long Error streaming events
INFO  2025-10-24T12:46:16 +43ms service=snapshot hash=1a759a0149a0c3129b2cc0c5c0bd1da90352e796
 cwd=/Users/antmarqu3/workspace-perso/qa-gitlab git=/Users/antmarqu3/.local/share/opencode/snapshot/d07b7ad4b93274723e8059e42f36a2fca2e70f17 tracking
INFO  2025-10-24T12:46:16 +1ms service=bus type=message.part.updated publishing
INFO  2025-10-24T12:46:16 +0ms service=session.prompt session=ses_5e9bf06d4ffeCJMWSIdbfCDK7v type=reasoning-start part
INFO  2025-10-24T12:46:16 +5ms service=session.prompt session=ses_5e9bf06d4ffeCJMWSIdbfCDK7v type=reasoning-delta part
INFO  2025-10-24T12:46:16 +2ms service=bus type=message.part.updated publishing
INFO  2025-10-24T12:46:16 +0ms service=session.prompt session=ses_5e9bf06d4ffeCJMWSIdbfCDK7v type=reasoning-delta part
INFO  2025-10-24T12:46:16 +0ms service=bus type=message.part.updated publishing
INFO  2025-10-24T12:46:16 +0ms service=session.prompt session=ses_5e9bf06d4ffeCJMWSIdbfCDK7v type=reasoning-delta part
INFO  2025-10-24T12:46:16 +1ms service=bus type=message.part.updated publishing
INFO  2025-10-24T12:46:16 +0ms service=session.prompt session=ses_5e9bf06d4ffeCJMWSIdbfCDK7v type=reasoning-delta part
INFO  2025-10-24T12:46:16 +0ms service=bus type=message.part.updated publishing
INFO  2025-10-24T12:46:16 +0ms service=session.prompt session=ses_5e9bf06d4ffeCJMWSIdbfCDK7v type=reasoning-delta part
INFO  2025-10-24T12:46:16 +0ms service=bus type=message.part.updated publishing
INFO  2025-10-24T12:46:16 +0ms service=session.prompt session=ses_5e9bf06d4ffeCJMWSIdbfCDK7v type=reasoning-delta part

OpenCode version

No response

Steps to reproduce

No response

Screenshot and/or share link

No response

Operating System

No response

Terminal

No response

Activity

  1. anthonymq
    added
    bugSomething isn't working
    on Oct 24, 2025

  2. github-actions commented on Oct 24, 2025

    @github-actions
    github-actionsBot
    on Oct 24, 2025 – with GitHub Actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    Feel free to ignore if none of these address your specific case.

  3. rekram1-node commented on Oct 24, 2025

    @rekram1-node
    rekram1-node
    on Oct 24, 2025
    Collaborator

    Hm that's interesting. This will be fixed very soon because all the go code is about to die

  4. rekram1-node
    self-assigned this
    on Oct 24, 2025

  5. melMass commented on Oct 26, 2025

    @melMass
    melMass
    on Oct 26, 2025

    @rekram1-node is there anything to do? This is among a lot of things that currently stale whole sessions (exiting/resuming it doesn't work)

  6. melMass commented on Oct 26, 2025

    @melMass
    melMass
    on Oct 26, 2025

    It's actually sneaky.. It does run the request/edit file if in edit mode. The UI is just stuck, queued message are also ran, but the UI is "stuck" (only seing my messages stacking up), exiting opencode and going back to the same session I can see the answers/tool calls

    Panic Stack on exit:

    here 
    panic: reflect: NumField of non-struct type tea.Model

goroutine 257 [running]:
reflect.(*rtype).NumField(0x0?)
        /opt/hostedtoolcache/go/1.24.7/x64/src/reflect/type.go:782 +0x59
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newStructTypeEncoder.func1({0x1152890, 0xcbeea0}, {0xc015b89d10, 0x1, 0x1})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:221 +0xc2
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newStructTypeEncoder.func1({0x1152890, 0xd719e0}, {0x174f000, 0x0, 0x0})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:230 +0x236
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newStructTypeEncoder(0xc002802048, {0x1152890, 0xd719e0})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:264 +0x149
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newTypeEncoder(0xc002802048, {0x1152890, 0xd719e0})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:130 +0x33f
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).typeEncoder(0xc002802048, {0x1152890, 0xd719e0})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:93 +0x1d2
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newTypeEncoder.encoder.newInterfaceEncoder.func4({0xc8d100?, 0xc004745ac0?, 0xc8d100?})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:333 +0x68
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).encodeMapEntries(0xc00000e108, {0xc015b89cf0, 0x2, 0x2}, {0xcc0b20?, 0xc005f1c330?, 0x10?})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:372 +0x53b
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newTypeEncoder.(*encoder).newMapEncoder.func3({0xcc0b20?, 0xc005f1c330?, 0xc002a09298?})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:392 +0x65
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newFieldTypeEncoder.func1({0xd3bc80?, 0xc005f1c320?, 0xc0004340e0?})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:316 +0x1d6
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).newStructTypeEncoder.func3({0xd39380?, 0xc005f1c240?, 0xd39380?})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:276 +0x145
github.com/sst/opencode-sdk-go/internal/apijson.(*encoder).marshal(0xd39380?, {0xd39380?, 0xc005f1c240?})
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:61 +0xd5
github.com/sst/opencode-sdk-go/internal/apijson.MarshalRoot(...)
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/apijson/encoder.go:32
github.com/sst/opencode-sdk-go.AppLogParams.MarshalJSON({{{0x0, 0x0}, {0xd8b325, 0x4}, 0x0, 0x1, {0x0, 0x0}}, {{0x0, 0x0}, ...}, ...})
        /home/runner/work/opencode/opencode/packages/sdk/go/app.go:229 +0x57
github.com/sst/opencode-sdk-go/internal/requestconfig.NewRequestConfig({0x1146b78, 0x174f000}, {0xd8b2f9, 0x4}, {0xd8a69e, 0x3}, {0xd39380, 0xc005f1c120}, {0xc56f80, 0xc016538248}, ...)
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/requestconfig/requestconfig.go:98 +0xa5
github.com/sst/opencode-sdk-go/internal/requestconfig.ExecuteNewRequest({0x1146b78?, 0x174f000?}, {0xd8b2f9?, 0x0?}, {0xd8a69e?, 0xcaef60?}, {0xd39380?, 0xc005f1c120?}, {0xc56f80, 0xc016538248}, ...)
        /home/runner/work/opencode/opencode/packages/sdk/go/internal/requestconfig/requestconfig.go:563 +0x85
github.com/sst/opencode-sdk-go.(*AppService).Log(_, {_, _}, {{{0x0, 0x0}, {0xd8b325, 0x4}, 0x0, 0x1, {0x0, ...}}, ...}, ...)
        /home/runner/work/opencode/opencode/packages/sdk/go/app.go:41 +0x2be
github.com/sst/opencode/internal/util.NewAPILogHandler.func1()
        /home/runner/work/opencode/opencode/packages/tui/internal/util/apilogger.go:55 +0x136
created by github.com/sst/opencode/internal/util.NewAPILogHandler in goroutine 1
        /home/runner/work/opencode/opencode/packages/tui/internal/util/apilogger.go:49 +0x145

  7. rekram1-node commented on Oct 26, 2025

    @rekram1-node
    rekram1-node
    on Oct 26, 2025
    Collaborator

    what version are you on?

  8. melMass commented on Oct 26, 2025

    @melMass
    melMass
    on Oct 26, 2025

    latest: 0.15.18 on linux and macos

  9. rekram1-node commented on Oct 26, 2025

    @rekram1-node
    rekram1-node
    on Oct 26, 2025
    Collaborator

    hm okay Ill try to find cause so we can fix it

  10. rekram1-node
    marked bufio.Scanner: token too long #3502 as a duplicate of this issue on Oct 28, 2025

  11. yudgnahk commented on Oct 28, 2025

    @yudgnahk
    yudgnahk
    on Oct 28, 2025
    Contributor

    hm okay Ill try to find cause so we can fix it

    Hi @rekram1-node,
    I already tried and made a PR to fix this bug.
    I hope you can review my PR. Thanks!

  12. rekram1-node commented on Oct 28, 2025

    @rekram1-node
    rekram1-node
    on Oct 28, 2025
    Collaborator

    I checked it out, unfortunately editing the sdk directly wont' work since it is auto generated

  13. ShadowArcanist commented on Oct 29, 2025

    @ShadowArcanist
    ShadowArcanist
    on Oct 29, 2025

    I am also experiencing the same issue on opencode v0.15.23 (macOS 13.7.8 (22H730))

  14. rekram1-node commented on Oct 30, 2025

    @rekram1-node
    rekram1-node
    on Oct 30, 2025
    Collaborator

    should be fixed by: #3531

    Will be in next release

  15. benceferdinandy-signifyd commented on Nov 27, 2025

    @benceferdinandy-signifyd
    benceferdinandy-signifyd
    on Nov 27, 2025

    I had a similar error "prompt is too long: 218083 tokens > 200000 maximum" but I think this is from Anthropic? Yet opencode stops completely instead of trying to resolve

  16. benceferdinandy-signifyd commented on Nov 27, 2025

    @benceferdinandy-signifyd
    benceferdinandy-signifyd
    on Nov 27, 2025
    Image

    it's not possible to recover from here it seems

  17. rekram1-node commented on Dec 1, 2025

    @rekram1-node
    rekram1-node
    on Dec 1, 2025
    Collaborator

    You can "recover" using /undo one or more times but we will fix this in a better way

  18. rekram1-node
    changed the title [-]Opencode stuck after token too long[/-] [+]bufio.Scanner: token too long[/+] on Dec 1, 2025

  19. rekram1-node commented on Dec 1, 2025

    @rekram1-node
    rekram1-node
    on Dec 1, 2025
    Collaborator

    Also @benceferdinandy-signifyd this is about an error w/ bufio, not prompt lengths this was a scanner thingy.

    Closing as done since we don't have Go as a dep anymore

  20. rekram1-node
    closed this as completedon Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions