Microsoft Foundry's continuous-improvement approach gives teams a way to define what an AI agent should accomplish, evaluate its results, and optimize the system around measurable business goals. It is aimed at developers and technology teams building agents for business processes, rather than people looking only to select a model. Microsoft Events demonstrates the approach with a travel-agent scenario and a rubric-based evaluation workflow.
The central idea is to treat an AI application as a system that needs ongoing evaluation and improvement, not as a finished product once it can answer a prompt. An agent may respond correctly yet fail to resolve a user's problem or complete the intended business action. The session's example is a travel portal where employees need trips submitted and approved, compliance managers need policy adherence, and the CFO needs spending kept within budget.
The proposed process has three stages: make it work, make it better, and make it scale. Start by defining the business outcome and what counts as success; then build a working prototype. Next, improve quality, cost, and performance, with people and agents working together. Finally, monitor models in production and use built-in controls to continue improving and governing them.
This puts the business specification ahead of model selection. In the travel example, the stated goal is to make the right policy decision for every request, quickly and at the lowest cost to serve. Those criteria can become an evaluation rubric, so a team can compare agent behavior against its requirements rather than relying on an impression that one model or prompt is better.
The approach is for developers and technology builders who are implementing AI systems around business outcomes. The demonstration uses Microsoft Foundry and a travel-agent prototype, but the presenters describe the scenario as applicable to other business processes as well. Its practical starting point is a written specification: the outcome the agent should produce and the criteria that define a good result.
The example uses GitHub Copilot connected to the Foundry MCP server to build from the agent specification. The evaluator is then created from the agent's instructions, with proposed dimensions such as task completion, policy compliance, and grounding. Teams should treat those dimensions as a starting point to align with their own process and success criteria.
The demonstration turns the agent's requirements into a repeatable evaluation rather than testing a few answers manually. A rubric evaluator scores responses across defined dimensions; grounding, in this context, means whether the answer is supported by the relevant information. A combined score can help teams track whether a change improves the result against the rubric.
Model choice is one part of that work, not the whole architecture. The session describes Foundry as bringing together models, agent capabilities, knowledge, tools, and safety systems. It also presents the model catalog as a way to evaluate options against consistent quality, safety, and business criteria without rebuilding the AI stack every time a model choice changes.
A rubric is only useful when its criteria reflect the outcome the organization actually needs. A high score against a poorly chosen rubric could still represent a poor business result, so the specification should be grounded in the intended task and policy. The session's advice is to define the criteria before selecting a model, rather than retrofitting success measures after development.
Improvement also involves trade-offs: the stated goals include quality, cost, and performance, and the session calls out latency as an optimization target. A change that helps one measure may not be the right choice for the overall business outcome. Use the rubric and success criteria to evaluate changes instead of assuming that a newer or larger model is automatically better.
The workflow described is an engineering process for defining, evaluating, and optimizing an agent. It does not mean an agent independently changes its own production behavior without oversight. The session frames scaling as monitoring models and using controls to keep improving and governing them. Teams should therefore make the evaluation criteria and intended business result explicit before they use scores to guide changes.
Define success criteria in the agent specification, create a rubric evaluator, then evaluate simulated conversations against dimensions such as task completion, policy compliance, and grounding.
Define the business outcome and plain-English grading criteria first. These criteria become the rubric for evaluating candidate models and later changes.
The session lists prompt optimization, content grounding, task decomposition, model routing and post-training, caching, and system architecture.