News, tutorials, and updates from the Potato team.
When who annotates matters, a validated questionnaire beats a question you invented. A tour of Potato's 55 built-in survey instruments and when each one earns a place in your study.
How to find why a multi-agent LLM system failed using Potato: the interaction graph, failure attribution, handoff review, per-agent scorecards, tool-contention timeline, and emergent-behavior tagging.
Potato 2.6 is out: QDA Mode for qualitative coding, an LLM-as-judge calibration and alignment workflow, trajectory editing that produces SFT and DPO training data, a 3x faster boot, and a relicense to GPL-3.0-or-later.
Get notified about new tutorials, releases, and community highlights.
Annotator identity shapes labels, so demographics are worth collecting, but only with consent, care, and a reason. A guide to what to ask, what to leave alone, and how to do it in Potato.
Most agent evaluation stops at a score. Potato 2.6's trajectory_edit schema lets annotators rewrite a wrong step instead of rating it, and exports each correction as supervised fine-tuning targets and DPO preference pairs.
An honest look at how to pick an open-source data annotation tool, what questions actually narrow the choice, and where Potato fits among Label Studio, Prodigy, Doccano, brat, and Argilla.
How to write an annotation codebook an LLM can actually follow, validate its labels against human coders, and keep a person in the loop, with a worked Potato config.
Step-by-step guide to collecting per-step reward signals for PRM training using Potato. Covers first-error mode, per-step annotation, and export to training pipelines.
Tutorial for setting up live coding agent observation with Ollama, Anthropic API, or Claude Agent SDK. Includes pause, rollback, branching, and trajectory export.
Set up multi-criteria rubric evaluation with custom criteria, configurable rating scales, and dimension weights for systematic AI agent evaluation using Potato's rubric_eval.
Potato 2.4.0 ships web agent trace review, real-time live agent evaluation, an LLM chat sidebar, HuggingFace Hub export, webhooks, SSO/OAuth, and five active learning strategies.
Potato 2.2.0 adds 9 new annotation schemas, a pluggable export system, MACE competence estimation, 55 validated survey instruments, and remote data sources.
Potato 2.1.0 brings the instance display system, visual AI support for image and video annotation, span linking, multi-field spans, and layout customization.
Best practices for annotating medical images in Potato, DICOM display, radiology report labeling, adverse event extraction, and IRB-compliant self-hosted deployment.
Configure polygon drawing tools in Potato for image segmentation tasks, with tips for complex shapes, overlapping regions, multi-class polygons, and export to COCO format.
Potato 2.0 ships AI-powered pre-annotation with OpenAI and Claude, multimedia support for audio and video, active learning, bounding box annotation, and a redesigned UI.
Configure AI-powered keyword highlighting in Potato to draw annotator attention to important terms. Covers OpenAI, Claude, and custom keyword list configuration.
Build a pronunciation quality annotation task in Potato with audio playback, waveform visualization, Likert rating scales, and per-recording free-text feedback fields.
Build a speaker identification task in Potato with interactive audio waveforms, timestamp markers, speaker label assignment, and inter-annotator agreement measurement.
Best practices for ensuring annotation quality in annotation projects, including practical strategies you can implement with and beyond Potato.
Set up image classification in Potato with thumbnail previews, zoom controls, multi-label checkbox schemas, and keyboard shortcuts for high-throughput visual annotation.
Create an audio emotion classification task in Potato with interactive waveform display, playback speed controls, Likert scales, and configurable emotion label sets.
Step-by-step guide to building a named entity recognition task in Potato, span annotation config, label colors, overlapping spans, keyboard shortcuts, and CoNLL export.