How to Build Vision Data Agents with Tools, Skills, and MCP
Workshop · Editorial classification · Sep 22, 2026
Machine learning teams spend most of their time on repetitive dataset work. Finding duplicates, running inference, evaluating models, and curating the next round of training data all require custom scripts, constant context switching, and hours of manual effort that slow down every iteration. In this session, you will learn how to build vision data agents with tools, skills, and the Model Context Protocol (MCP), starting with the FiftyOne Agent, an AI assistant built directly into FiftyOne that turns plain-language requests into real dataset operations. Ask it to find and remove duplicate images, run object detection and surface low-confidence predictions, or evaluate a model and summarize where it fails, and the agent handles execution end to end. We will also walk through the newest capabilities shipping with this release, including code generation and plugin generation. You will see how to go from a conversational prompt to a custom dashboard, visualization, or full FiftyOne application, and how to package multi-step workflows as reusable skills the agent can call on demand. Because the agent runs inside FiftyOne's secure guardrails, teams keep full control. Connect your preferred models from over 100 LLM providers, route requests through your own enterprise gateway, and maintain audit logging and user attribution for every action the agent takes. Whether you manage large-scale visual AI datasets, build computer vision applications, or are exploring how agents fit into your machine learning stack, this session will show you how to put an agent to work on your data from day one. What you’ll learn: How the FiftyOne Agent works under the hood: from natural language request to executed dataset operationAutomating everyday dataset work: import data, find duplicates, run inference, evaluate predictions, and export datasets through conversationGenerating code and plugins with the new release: build custom dashboards, visualizations, and full FiftyOne applications from a single promptExtending the agent with custom skills: package your organization’s multi-step workflows into reusable capabilitiesConnecting your stack with MCP: use the FiftyOne Model Context Protocol server with coding assistants like Claude Code, Cursor, and CopilotDeploying agents responsibly in the enterprise: model configuration, gateway integration, audit logging, and guardrails
Official agenda