Azure AD Conditional Access: How to Test

Key insights

• Microsoft Entra What If tool is built into tenants and free to use.
  In the video the presenter runs it across real sign-ins to show how quickly admins can validate Conditional Access outcomes.
• Common misconfigurations appear often: a legacy authentication block may exist but be turned off, guest MFA policies can sit in report-only and do nothing, and layered admin protection is frequently passive or misconfigured.
  These failures leave gaps until someone tests them.
• The demo covers three concrete scenarios: legacy auth sign-ins, guest user access, and Global Admin sign-in behavior.
  Each scenario shows how missing or disabled controls let risky sign-ins bypass intended protections.
• Simple, practical steps: run the What If checks for representative users and apps, move validated policies from report-only to enforce, block legacy clients, and require MFA for high-risk roles.
  Test again after changes to confirm the policy behaves as expected.
• Important 2026 change: All resources policies will evaluate low-privilege OIDC scopes that previously bypassed enforcement, closing a loophole.
  Rollout starts March 27–June 2026, so review policies now to avoid surprises.
• Why this matters: Conditional Access gives granular security, supports zero-trust controls, and integrates with device management like Intune for stronger enforcement.
  Regular testing prevents outages and protects admins, guests, and user devices from unintended access gaps.

In a recent YouTube video by Jonathan Edwards, the reliability of enterprise Conditional Access setups is put under scrutiny, and the piece serves as a practical wake-up call for administrators.

Video and Context

The video opens by noting that many Conditional Access policies are never thoroughly tested, and indeed admins commonly enable or leave policies in unintended states. Edwards examines a tenant with dozens of policies and explains why surface-level checks can miss critical gaps. Importantly, he frames the issue not as blame but as a systems problem: policy complexity and default behaviors create blind spots.

Moreover, the timing matters because changes in policy evaluation, rolling out in 2026, reduce prior exceptions and expand enforcement across more sign-in flows. Thus, organizations that assume earlier behavior will persist may find enforcement changing under them, and so routine validation becomes essential. As a result, testing tools become central to operational security rather than optional extras.

How the Microsoft Entra "What If" Tool Works

Edwards demonstrates the What If tool by simulating sign-ins and evaluating policies against three targeted scenarios: legacy authentication, guest user access, and Global Admin sign-ins. In practice, the tool lets admins choose a user, application, and conditions to see which Conditional Access policies would apply, helping validate expected outcomes before real users are impacted. Consequently, the tool reduces the need to rely on post-incident analysis and encourages proactive verification.

Additionally, the tool is free and already available in tenants, so the biggest barrier is often awareness and habit rather than cost or licensing. Edwards stresses that many administrators have never used it, and so simply running a few tests can reveal surprising misconfigurations. Therefore, integrating this testing into regular operational checklists provides immediate value with minimal effort.

Key Findings from Jonathan Edwards' Test

In the tenant Edwards analyzes, three recurring problems surface: a legacy auth block policy that exists but is switched off, guest MFA policies left in report-only mode and therefore not enforcing anything, and multiple admin protections that are passive or misconfigured. These issues illustrate how policies can give a false sense of security when they are present but not active in the intended way. Consequently, organizations may be relying on policies that do not actually prevent risky sign-ins.

Furthermore, the video highlights how layered protections can fail if each layer is only partially configured, because attackers often exploit the weakest path. For example, legacy authentication clients that cannot enforce MFA can bypass protections unless policies explicitly target those flows. Thus, consistent application and testing across different client types and identity scenarios are crucial.

Tradeoffs and Operational Challenges

Balancing security and usability is an ongoing tradeoff that Edwards addresses indirectly: stricter policies reduce risk but can also disrupt users and business processes if applied without testing. For instance, blocking legacy authentication helps security but may break legacy applications and services that have not migrated, forcing teams to plan staged rollouts. Therefore, testing with the What If tool lets teams anticipate breakage and coordinate remediation.

Another challenge is complexity: Conditional Access rules can interact in unexpected ways, and exclusions or layered policies create evaluation nuances that are easy to overlook. Consequently, administrators must reconcile operational demands like guest collaboration with the need for strong controls for privileged accounts. In practice, this means adopting an iterative approach: test, adjust, monitor, and repeat.

Practical Recommendations and Next Steps

First, Edwards recommends that admins regularly run the What If tool across representative scenarios, including legacy clients, guest users, and high-privilege sign-ins, to confirm intended outcomes. Next, move verification into routine processes: schedule periodic reviews, document policy intent versus behavior, and treat report-only findings as actionable items rather than records to ignore. Doing so helps teams close the gap between policy design and real-world enforcement.

Finally, as Microsoft tightens enforcement in 2026, organizations should reassess exclusions and legacy flows that previously bypassed policies, because enforcement will be broader and more consistent. Consequently, teams should combine technical checks with staged communication and remediation plans to avoid operational surprises. In short, active testing and incremental change management provide the most practical path to stronger, dependable Conditional Access security.

Keywords

Conditional Access testing, Azure AD Conditional Access, test Conditional Access policies, validate Conditional Access rules, Conditional Access policy simulator, Conditional Access troubleshooting, how to check Conditional Access, Conditional Access monitoring