Many use cases often involve integrating files as part of a the task

• write an analysis on the insights in this image

How

• modify UI to support file upload. The idea is to yield a sequence of string and files
  e.g., if a command and an image are uploaded -> ["what is in this image", image, ""]
• Update task endpoint to process files and add them to the task

We need to convert the the sequence from frontend a sequence from

# Note: task is of type `task: str | ChatMessage | Sequence[ChatMessage] | None = None,`

ChatMessage = Annotated[
    TextMessage | MultiModalMessage | StopMessage | ToolCallSummaryMessage | HandoffMessage, Field(discriminator="type")
]

• Support a specific set of filetypes and determine how they are processed before passed to model

  • images, convert them with pil and add directly
  • txt/pdf/ etc... save to a location , convert to markdown and add as string sequence