Tuning SIEM Correlation Rules to Cut False Positives on Azure AD Sign‑Ins

Azure AD sign‑in logs are a goldmine for detecting compromise, but they also flood SIEMs with benign events that look suspicious at first glance. A well‑tuned correlation rule set can reduce noise while preserving detection quality. Below is a step‑by‑step guide that balances precision, maintainability, and operational overhead.

1. Start with a Baseline: Understand the Noise Profile

Before tweaking any rule, run a 30‑day baseline to capture normal activity. Export the raw sign‑in logs to a CSV or a SIEM table and run a quick aggregation:

Get-AzureADAuditSignInLogs -Top 1000 | 
  Export-Csv -Path .\signins_sample.csv -NoTypeInformation

In the exported data, look for:

Geographic clusters: Are most logins from a single country or IP range?
Device types: How many are from corporate devices vs. personal devices?
Application mix: Which SaaS apps are accessed most often?
Time‑of‑day patterns: Do logins spike during business hours only?

Document these patterns. They will inform the thresholds you set later.

2. Leverage Built‑in Azure AD Conditional Access Insights

Azure AD’s Conditional Access (CA) policies already enforce many controls. Correlation rules should treat CA‑denied events as low‑risk unless they’re part of a broader attack pattern. For example, a CA policy that blocks sign‑ins from unfamiliar locations will generate many “blocked” events. Instead of treating every blocked event as an alert, create a rule that only fires when a blocked event is followed by a successful sign‑in from the same IP within 5 minutes.

SigninLogs
| where ResultType == 0
| where ConditionalAccessStatus == "blocked"
| join kind=inner (
    SigninLogs
    | where ResultType == 0
    | where ConditionalAccessStatus == "success"
) on IPAddress
| where Timestamp - prev(Timestamp) < 5m

This rule surfaces a potential “brute‑force” bypass without flagging every CA block.

3. Apply Contextual Filters

3.1. Trusted IP Ranges

Maintain a list of corporate VPN and data center IP ranges. Exclude sign‑ins from these ranges unless the user is accessing a high‑risk application.

let trusted = dynamic(["10.0.0.0/8","172.16.0.0/12"]);
SigninLogs
| where IPAddress !in (trusted)

3.2. Known Good User Agents

Many legitimate sign‑ins come from browsers or Microsoft Authenticator. Flag only uncommon user agents unless paired with other risk signals.

let goodAgents = dynamic(["Mozilla/5.0 (Windows NT 10.0; Win64; x64)", "Authenticator"]);
SigninLogs
| where UserAgent !in (goodAgents)

4. Use Rate‑Based Thresholds

Instead of a binary “alert if X” rule, use sliding windows to detect anomalies. For example, more than 10 sign‑ins from a single IP within 10 minutes is suspicious, but 3 sign‑ins from a corporate VPN are normal.

SigninLogs
| summarize Count = count() by IPAddress, bin(Timestamp, 10m)
| where Count > 10

Adjust the Count threshold per environment. A small SaaS company may set it to 5; a large enterprise may need 20.

5. Correlate with Threat Intelligence

Integrate external threat feeds (e.g., MITRE ATT&CK Tactics or known malicious IPs from CISA). If a sign‑in originates from an IP in the malicious list, raise a high‑severity alert regardless of other factors.

let maliciousIPs = externaldata(IP:string)
    [@"https://raw.githubusercontent.com/cisagov/malicious-ip-list/master/ips.txt"]
    with (format="txt");
SigninLogs
| where IPAddress in (maliciousIPs)

The cost of a false positive here is low compared to the benefit of catching a compromised credential.

6. Implement User‑Level Baselines

Create a per‑user baseline of normal sign‑in behavior. If a user normally logs in from a single location but suddenly signs in from a distant country, flag it.

SigninLogs
| summarize AvgLocations = dcount(IPAddress) by UserPrincipalName, bin(Timestamp, 30d)
| where AvgLocations > 1

Store these baselines in a lookup table and reference them in your rules. This approach reduces noise from users who legitimately travel.

7. Test Rules in a Sandbox

Before deploying, run each rule against a historical dataset. Measure:

True positives: Known compromise events that trigger the rule.
False positives: Benign events that trigger the rule.

Adjust thresholds until the false‑positive rate is acceptable (e.g., <5% of total alerts). Document the tuning process; future analysts will benefit from the rationale.

8. Automate Rule Maintenance

Use IaC (Infrastructure as Code) to version‑control your SIEM rules. For example, store Kusto queries in a GitHub repo and deploy them via Azure Automation or Terraform. This practice ensures that rule changes are auditable and reversible.

resource "azurerm_log_analytics_workspace" "example" {
  name                = "example-ws"
  location            = "East US"
  resource_group_name = azurerm_resource_group.example.name
  sku                 = "PerGB2018"
}

9. Monitor Rule Effectiveness

Set up a dashboard that shows alert volume per rule over time. If a rule’s alert count spikes, investigate whether a new benign pattern has emerged (e.g., a new application rollout) and adjust the rule accordingly.

10. Trade‑Offs to Consider

Trade‑off	Impact	Mitigation
Lower thresholds	Higher detection rate, more noise	Use multi‑factor correlation (e.g., combine IP, user agent, and time)
Higher thresholds	Fewer alerts, risk of missed attacks	Periodically review missed incidents and adjust
Complex rules	Better precision, harder to maintain	Document logic, use modular queries, version‑control
External feeds	Adds context, may lag	Keep feeds up‑to‑date, use multiple sources

11. Real‑World Example

A mid‑size company noticed a surge in sign‑in alerts after a phishing campaign. Their initial rule was:

SigninLogs
| where ResultType == 0 and ConditionalAccessStatus == "blocked"

This produced 1,200 alerts per day, most of which were legitimate CA blocks. After tuning:

Excluded corporate VPN IPs.
Added a rate‑based threshold of 5 blocked events per IP per hour.
Correlated with a CISA malicious IP list.

Result: Alerts dropped to 120 per day, with 95% being true positives.

12. Quick Checklist Before Deployment

Baseline noise profile documented.
Trusted IP ranges and user agents defined.
Rate‑based thresholds set per environment.
Threat intelligence feeds integrated.
User baselines created.
Rules tested against historical data.
IaC deployment pipeline established.
Monitoring dashboard in place.

By following this structured approach, you can keep your Azure AD sign‑in alerts actionable, reduce analyst fatigue, and maintain a robust security posture.