Skip to content

Support read image in read file tool #5169

@samhvw8

Description

@samhvw8

What specific problem does this solve?

When user need to send image to Roo but currently we only support via paste to chat area

It will be hard for user to send multiple image with description (eg: 1st image is screen shoot for screen1 view, 2nd image is screen shot for screen2 view, … )

Current behavior is user copy image it to clipboard, and add description to it. Thus with this approach, it will be hard to indicate which image corresponding with this description.

Additional context (optional)

No response

Request checklist

  • I've searched existing Issues and Discussions for duplicates
  • This describes a specific problem with clear impact and context

Interested in implementing this?

  • Yes, I'd like to help implement this feature

Implementation requirements

  • I understand this needs approval before implementation begins

How should this be solved? (REQUIRED if contributing, optional otherwise)

What will change:
Add handle read image when file is binary in read file tool

How user interact with it
User can request roo to read image file

How will we know it works? (Acceptance Criteria - REQUIRED if contributing, optional otherwise)

Given i have 10 image in a folder and an provider that support image
When i request Roo read all of them in the folder and run OCR for each of them
Then Roo should return text that run ocr from them

Technical considerations (REQUIRED if contributing, optional otherwise)

In readFileTool in section if file is binary then we check if file is supported image or not

If it is supported image then we will read that file in based64 we will add to file result array ( we need create specific field in FileResult type to store based64 data )

Then we will push this to pushToResult function

Trade-offs and risks (REQUIRED if contributing, optional otherwise)

The implementation should handle when image file size is big if not extension host my crash or out of provider context

Metadata

Metadata

Assignees

Labels

EnhancementNew feature or requestFeature RequestFeature request, not a bugIssue - In ProgressSomeone is actively working on this. Should link to a PR soon.proposal

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions