Automating MCP Server Testing: Engineering Reliability for Agentic Systems

AI Builders Global Conference 2026

Date to be announced · Time to be announced

AI agents don’t fail like traditional software. They don’t just throw exceptions, they drift, misinterpret tools, invoke the wrong functions or behave differently across environments. When deploying Arm’s Open-Source custom MCP server to power AI assistants for architecture development, migration, & optimization, we faced a critical question: how do we test a system built for nondeterministic interaction? In this talk, I’ll share how we moved from manual validation to a repeatable, CI-enforced testing strategy using Pytest & Testcontainers. We spin up real MCP server in Docker during tests, validating tool discovery, invocation, & protocol compliance end-to-end. This isn’t about mocking LLM output. It’s about testing the contract between agents & tools. The key insight: treat your MCP server like production infrastructure, not experimental glue code. Because “it worked on my machine” is not a deployment strategy. This session contributes practical, open, & reproducible testing approach that any project can adopt to improve reliability & trust in Open-Source MCP implementations. Thus, making agent infrastructure production-ready.

Session status: scheduled

Track: AI Engineering, Architecture & Infrastructure

Level: All Levels

Official session / agenda

Saving a session is not registration. Check the official agenda for availability, eligibility and any additional fees.

Speakers and their event affiliations

Parent event

Virtual · Online event

This describes the parent event. Confirm the session’s attendance requirements with the organizer.

Companies participating in the parent event

These are event-level relationships, not confirmed sponsorship of this session.

Showing 24 of 209 company relationships. View all on the event page.