Agent Operative: Quick User Feedback | Mission 11

Key insights

• Mission 11 in Agent Academy: Operative (inside Copilot Studio) teaches how to add user feedback to agents so they improve over time.
  It focuses on practical steps to capture ratings, comments, and telemetry after agent interactions.


• Use the thumbs up/thumbs down built-in reactions to gather quick user signals.
  Review the built-in analytics to spot trends and measure agent response quality.


• Build a custom feedback flow using an adaptive card to collect structured ratings and text comments from users.
  Custom forms let you capture richer feedback than simple reactions.


• Log feedback and events to Azure Application Insights to centralize telemetry and run queries for analysis.
  Telemetry helps you find failure points and prioritize fixes or prompt updates.


• Design agents with clear parent-child roles and active feedback loops so updates propagate across tasks.
  Enable moderation and safety settings to limit risky content and keep agents compliant.


• Collecting feedback yields improved performance, faster iteration, and stronger governance for production agents.
  Teams can use these signals to tune prompts, retrain models, and monitor agent behavior at scale.

Overview of the video

The YouTube video, published by Microsoft, shows Mission 11 of the Agent Academy: Operative series and focuses on collecting user feedback for agents built in Copilot Studio. The session walks through both built-in reactions such as thumbs up and thumbs down and custom feedback collection with adaptive cards. Furthermore, the presenter demonstrates how to view thumbs analytics and how to log custom feedback to Azure Application Insights for deeper telemetry. As a result, viewers get a practical playbook for closing the loop between user interactions and agent improvements.

What the demo shows

First, the tutorial explains feedback basics and why structured responses matter, beginning with simple reactions and then moving to richer inputs. It then walks viewers through creating an adaptive card to collect ratings and comments, showing how that card appears in conversation flows. Next, the video reveals how the built-in reactions generate analytics you can review directly in the studio, which helps teams spot patterns quickly. Finally, it covers the steps to send custom entries into Azure Application Insights for querying and long-term analysis.

The presenter uses clear timestamps to mark each segment, which makes it easy for viewers to jump to specific techniques. For example, the video separates the explanation of built-in interactions, analytics review, adaptive card construction, and telemetry logging into distinct chapters. Consequently, developers can follow along at their own pace and apply the specific part they need. The structure also reflects common development workflows from quick validation to production observability.

How to implement feedback in Copilot Studio

The video outlines practical steps: enable built-in reactions on the agent, design an adaptive card for richer responses, and wire events to telemetry sinks. Additionally, it shows mapping user context—like email or chat metadata—so feedback ties back to the correct session and user scenario. Then, it demonstrates basic code snippets or configuration options to push data into Azure Application Insights, where teams can run queries and build dashboards. By following these steps, teams can instrument agents without heavy custom backends, which reduces initial engineering work.

However, the tutorial also indicates that some setup work is necessary for production: you need proper configuration of instrumentation keys, retention policies, and role-based access. In practice, this means operations and security teams must coordinate to ensure telemetry follows organizational policies. Moreover, developers should validate that adaptive cards render correctly across client platforms to avoid collection gaps. Thus, implementation mixes quick wins with important operational tasks.

Benefits and tradeoffs

Collecting feedback delivers clear advantages: it helps improve response quality, surfaces safety concerns, and provides data for prioritizing model or prompt updates. In addition, telemetry allows teams to measure trends, detect regressions, and correlate user sentiment with agent changes. Yet, tradeoffs exist because richer feedback can increase friction for users; too many prompts may reduce engagement and bias results toward vocal users. Therefore, teams must balance depth of feedback with user experience to avoid skewed or sparse data.

Another tradeoff involves cost and complexity. Sending high-volume telemetry to Azure Application Insights improves analysis but raises ingestion and retention expenses. Furthermore, storing detailed user comments requires thought around privacy, access controls, and retention policies. Consequently, organizations must weigh the value of deeper observability against budget and compliance constraints, and adopt sampling or aggregation strategies when needed.

Challenges and governance

The video highlights several challenges that teams commonly face when building feedback loops. For example, feedback quality varies and may include noise, sarcasm, or ambiguous comments that are hard to interpret automatically. Thus, automated pipelines require careful filtering and may need human-in-the-loop review to label important cases accurately. Meanwhile, scaling feedback across multiple agents and workflows compounds these issues, as different agents may need different evaluation criteria and dashboards.

Governance poses another challenge because feedback data often contains sensitive information or personally identifiable details captured from conversations. Accordingly, the presenter stresses configuring moderation settings and disabling broad general-knowledge access when appropriate. Additionally, setting up access controls in tools like Agent 365 or within organizational telemetry systems helps ensure only authorized teams can query and export raw feedback. In short, good governance reduces legal and ethical risk while enabling useful insights.

Takeaways for builders

Overall, the video from Microsoft provides a practical, step-by-step guide for adding feedback loops to agents built in Copilot Studio, and it balances implementation tips with operational concerns. Moreover, it makes clear that while feedback systems are powerful, they require deliberate design to manage user friction, cost, and privacy. Therefore, teams should prototype with built-in reactions first, then expand to custom adaptive cards and telemetry as needs and capacity grow.

Finally, the tutorial encourages cross-team collaboration between developers, security, and product owners to turn feedback into meaningful improvements. By doing so, organizations can iterate faster while controlling risks and costs, enabling agents to become more reliable and aligned with user needs over time. As a result, Mission 11 serves as a useful reference for any team aiming to operationalize feedback in a production agent environment.

Keywords

user feedback collection, collecting user feedback, in-app feedback tools, user feedback survey, customer feedback methods, UX feedback best practices, feedback analysis tools, NPS measurement