Skip to content

Fix "bufio.Scanner token too long" error by replacing Scanner with Reader in SSE - #3506

Closed
yudgnahk wants to merge 1 commit into
anomalyco:devfrom
yudgnahk:dev
Closed

yudgnahk wants to merge 1 commit into
anomalyco:devfrom
yudgnahk:dev

Conversation

@yudgnahk

@yudgnahk yudgnahk commented Oct 28, 2025 •

Copy link
Copy Markdown
Contributor

Related issues:

Fix bufio.Scanner token too long error in SSE decoder

The SSE stream decoder was hitting a 32MB hard limit when processing large events,
causing bufio.Scanner: token too long errors. This blocked users from reading large
files through the TUI.

Changes:

• Replaced bufio.Scanner with bufio.Reader in
packages/sdk/go/packages/ssestream/ssestream.go
• Uses ReadBytes('\n') which grows dynamically without size limits
• Added documentation and test files

Testing: Verified with events up to 100MB. All tests pass.

@yudgnahk

yudgnahk commented Oct 28, 2025 •

Copy link
Copy Markdown
Contributor Author

@rekram1-node Please help me check this PR. Thank you!
P/s: This my first PR to opencode 😄

@rekram1-node

Copy link
Copy Markdown
Collaborator

Ayee nice thank u ill take a look

@rekram1-node

Copy link
Copy Markdown
Collaborator

This file is automatically generated by Stainless so next time the sdk is regenerated this would be deleted. Ideally we should figure out if we can fix the issue elsewhere or if there is a setting we need to pass to stainless

@yudgnahk

Copy link
Copy Markdown
Contributor Author

This file is automatically generated by Stainless so next time the sdk is regenerated this would be deleted. Ideally we should figure out if we can fix the issue elsewhere or if there is a setting we need to pass to stainless

Could we create an ignore list for Stainless when generating files? The current situation is really bad — I can’t track the task status, and the app becomes very laggy when loading or selecting sessions. The agents also keep running without giving the user any indication of progress.

@rekram1-node

Copy link
Copy Markdown
Collaborator

Hmm can u maybe show me your log file, we need to get to the root of the problem and given this isn't an issue for the vast majority of users there is soemthing else going on for you and we should probably try to figure out what it is

@yudgnahk

Copy link
Copy Markdown
Contributor Author

@rekram1-node I'm trying new approach. Will create another PR soon.

@yudgnahk

Copy link
Copy Markdown
Contributor Author

Replace this PR with #3531

@yudgnahk yudgnahk closed this Oct 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants