Conditional Access Policies are one of the most valuable tools in the arsenal of Entra ID/Microsoft 365 administrators. That is not to say that the feature doesn’t come with its own set of problems, and Microsoft has introduced a set of related reports, audit log enhancements and standalone tools to help customers make the most of CA. The What If tool is one such example, and is now getting an overhaul with a new, Graph API based experience. And we’re also getting access to the underlying API endpoints ourselves, so let’s take a look.
The new What If UI experience
Let’s start with the UI experience first. To access it, navigate to the usual home of the What If tool in the portal, then click the Preview features button on top to toggle the Enhanced ‘What if’ evaluation experience. The screenshot below gives you a side by side comparison between the “legacy” (left) and “enhanced” (right) UI, which at first glance differ mostly in the arrangement of the various controls.
The differences however go beyond that, as the new experience is based on the Graph API methods of the What If tool, which we will discuss in more detail in the next section. For the time being, the important thing to note is that the new experience expects you to provide more details before hitting the What If button to get the evaluation results. Whereas previously you could only point to a user object (or a workload identity) and request the evaluation, under the new experience there are few additional required bits – the Target resource, the Device platform and the Client app. As you can see on the screenshot above, all those are marked with an asterisk, and failing to provide values will prevent you from running the evaluation.
As we will see in the next section, this does not necessarily apply to the API itself, so it’s more of an UI limitation. And yes, all of these are required even for evaluations against Workload identities, even though the latter two categories of conditions don’t make sense in such scenarios. The remaining conditions are largely the same between the two experiences, apart from the now deprecated Device state condition. In addition, the new experience allows you to also specify the Insider risk and/or the Authentication flow conditions, both of which are unavailable in the legacy experience.
In terms of the evaluation results, the UI bits remain largely unchanged between the “legacy” and the “new” experiences, so I will not bother detailing them here. Refer to the official documentation if you need more details on that.
What If methods in the Graph API
The more interesting part of the preview is the introduction of a new Graph API resource and method for the What If tool. In contrast with the previously utilized main.iam.ad.ext.azure.com endpoint, which was never officially supported and thus was largely undocumented, we now have a properly documented, public and officially supported API. And as it is all part of the Graph, you get to reap the “standard” benefits, such as being able to authenticate with a single token.
The documentation articles linked above cover only some of the needed details, so we will go over more details below. In a nutshell, to run a What If evaluation, we issue a POST request against the /identity/conditionalAccess/evaluate endpoint, with the request body containing a set of conditions, packaged in a JSON payload. As with all things Graph, you will need to handle the authentication and permissions first. The evaluate method requires the Policy.Read.ConditionalAccess permission at minimum, and supports for both application and delegate permissions.
As discussed in the previous section, the Graph based implementation comes with changed requirements for the set of sign-in details and conditions you need to provide for a successful evaluation. In other words, the request’s payload will need to contain at least the following:
- signInIdentity – the GUID (and type) of the identity for which we are doing the evaluation. Mandatory. You cannot use UPN or any other identifier, apart from GUID. Note that only a single value can be provided!
The documentation currently lacks detail on how to cover the Guest or external user scenario, so if you are interested in those, your best option is to capture the request made via the UI.
Here’s an example on how the signInIdentity resource looks like:- For user object
"@odata.type": "#microsoft.graph.userSignIn", "userId": "15dc174b-f34c-4588-ac45-61d6e05dce93"
- For user object
-
- For service principal object
"@odata.type": "#microsoft.graph.servicePrincipalSignIn", "servicePrincipalId": "c65b94a5-0049-439a-a6fd-bce307077730"
- For service principal object
-
- For Guest or external user
"@odata.type": "#microsoft.graph.userSignIn" "externalTenantId": "0be8a6c3-4078-43e5-a912-2ddb8b253b38" "externalUserType": "internalGuest"
- For Guest or external user
- signInContext – defines the application, user action or authentication context for the sign-in request. Mandatory.
Here are some examples on how to provide the relevant payload:- For applications
"@odata.type": "#microsoft.graph.applicationContext", "includeApplications": [ "00000003-0000-0ff1-ce00-000000000000" ] - For user actions
"@odata.type": "#microsoft.graph.userActionContext", "userAction": "registerSecurityInformation"
- For authentication context
"@odata.type": "#microsoft.graph.authContext", "authenticationContextValue": "c37"
- For applications
- signInConditions – the set of conditions for the sign-in request. At least one condition must be provided, thus this is a required part of the payload.
Unlike the UI-based evaluations that require both the devicePlatform and the clientAppType conditions to be used, for API-based evaluations the aforementioned are not mandatory. Neither is any of the remaining conditions, as long as one is provided. The list of supported conditions can be found in the documentation on the signInConditions resource.
Here are some examples:- Using the UI-required conditions
"signInConditions": { "devicePlatform": "windows", "clientAppType": "browser" } - With added IP address condition
"signInConditions": { "devicePlatform": "windows", "clientAppType": "browser", "ipAddress": "1.2.3.4" } - Just the Country condition
"signInConditions": { "country": "BG" }
- Using the UI-required conditions
In addition, there is one optional parameter you can specify:
- appliedPoliciesOnly – control whether the evaluation result will only contain Conditional access policies that apply to the given request, or all CA policies. Optional.
Let’s see some examples in action, first using the (now crippled) Graph explorer tool:
POST https://graph.microsoft.com/beta/identity/conditionalAccess/evaluate
{
"signInIdentity": {
"@odata.type": "#microsoft.graph.userSignIn",
"userId": "15dc174b-f34c-1234-ac45-61d6e05dce93"
},
"signInContext": {
"@odata.type": "#microsoft.graph.applicationContext",
"includeApplications": [
"00000003-0000-0ff1-ce00-000000000000"
]
},
"signInConditions": {
"devicePlatform": "windows",
"clientAppType": "browser",
"country": "BG"
},
"appliedPoliciesOnly": false
}
Here’s also a request done via application permissions and PowerShell:
#prepare the payload
$body = @{
"signInIdentity" = @{
"@odata.type"= "#microsoft.graph.userSignIn"
"userId"= "15dc174b-f34c-1234-ac45-61d6e05dce93"
}
"signInContext" = @{
"@odata.type" = "#microsoft.graph.applicationContext"
"includeApplications" = @(
"00000003-0000-0ff1-ce00-000000000000"
)
}
"signInConditions" = @{
"country" = "BG"
}
"appliedPoliciesOnly" = $false
} | ConvertTo-Json
#Make the request
$uri = 'https://graph.microsoft.com/beta/identity/conditionalAccess/evaluate'
$res = Invoke-WebRequest -Method Post -Uri $uri -Headers $authHeader1 -Body $body -ContentType "application/json"
#Prepare the output
$res = ($res.Content | ConvertFrom-Json).value
$res | ? {$_.policyApplies -eq $false} | ft id,displayName,state,policyApplies,analysisReasons
id displayName state policyApplies analysisReasons
-- ----------- ----- ------------- ---------------
e616b71c-b952-4901-899a-095e52fc6f2f Block Flow disabled False policyNotEnabled
ba9f6a95-1743-42ae-bf79-d6a05d2ab713 Block Gosho externally enabledForReportingButNotEnforced False application
f02df45d-66cd-4e92-9e9c-bb2f32c67d8a Service Principal Policy enabled False users
e731463f-ed5c-476a-9832-2a3cfd895a43 Session control enabled False users
ea9b1e35-d307-42c3-bcde-fcb954c31aa6 Force MFA for unknown locations enabled False notEnoughInformation
817a28eb-a7eb-4ea0-9c26-9098db8e71ba Use application enforced restrictions for O365 apps disabled False policyNotEnabled
2f4f1395-5245-4ef3-988d-c76f25ea0afe Prevent Hard Delete enabled False application
The output of our What If evaluation request was then filtered to display all the Conditional access policies that didn’t trigger for one reason or another. The analysisReasons field will give you the reason why a given policy wasn’t applied. The values include, but are not limited to: policyNotEnabled (the CA policy is in disabled state), application (the cloud app value didn’t match the request), users (the user’s GUID didn’t match), notEnoughInformation (we didn’t provide sufficient input). For a full list of possible reasons, refer to the official documentation.
If you are looking to use the Graph SDK for PowerShell instead of “raw” HTTPS requests, the relevant cmdlet will be Test-MgBetaIdentityConditionalAccess.
Before I forget, a note. In my tests, the Policy.Read.ConditionalAccess application permission did NOT work, despite the documentation listing it as the minimum supported one. Instead, I had to grant the Policy.Read.All one.
Additional notes and summary
In this article we did a quick evaluation of the Graph API based implementation of the Conditional Access What If tool. Both the UI and API experiences were examined, with the focus on the latter.
As this is a preview release (so only /beta endpoints on the Graph), there are some rough edges, both in the implementation and the documentation. Most importantly, there are some differences with the “legacy” experience that you need to be aware of. Those mostly revolve around the need to provide additional input, or risk getting an error, or notEnoughInformation evaluation result. As Microsoft points out in the documentation:
The What-if API expects all sign-in parameters to be defined for the evaluation to provide the most accurate results. If your tenant has policies with specific conditions and the sign-in details for those conditions aren’t provided, the What If API can’t evaluate those conditions.
Few other things to note. The tool does not evaluate classic Conditional access policies, though by now you should have left those behind. If also does not allow you to use the “bundle” client app values, such as Office 365 or Microsoft Admin Portals.