Microsoft Azure Virtual Network Flow Logs
Ingests Azure Virtual Network Flow Logs from Azure Blob Storage to analyze network traffic patterns and security events.
Sync Type: Incremental
Requirements
Before setting up the Azure Virtual Network Flow Logs input, you need to:
- Have a Microsoft Account with an active Azure Subscription
- Create an Azure Entra ID application with appropriate permissions
- Configure Azure Virtual Network Flow Logs to be stored in an Azure Storage Account
- Grant the application access to the Azure Storage Account containing flow logs
The input reads only VNet flow logs that use Azure's default flow log name, created in the NetworkWatcherRG resource group and targeting a whole virtual network.
Flow logs with a custom name, in another resource group, or targeting a subnet or network interface are not found.
Network access (restricted storage accounts)
If your Azure Storage account restricts public network access with a firewall (Networking → Firewalls and virtual networks), Monad's cluster must be explicitly allowed to reach it.
Why IP allowlisting fails for same-region clusters
If your Monad cluster runs in the same Azure region as your Azure Storage account, adding Monad's public IP to the allowlist will not work. Azure routes same-region requests over its internal network, so the storage account sees a private source IP that a public-IP allowlist rule can never match. This is documented Azure platform behavior:
"You can't use IP network rules to restrict access to clients in the same Azure region as the storage account. IP network rules have no effect on requests that originate from the same Azure region. Use Virtual network rules to allow same-region requests."
Resolution: allow Monad's subnet with a virtual-network rule
Because same-region traffic arrives with a private IP, you must allow Monad by subnet identity rather than by IP, using a VNet Service Endpoint:
- Monad provisions a VNet Service Endpoint for
Microsoft.Storageon the subnet of the Monad cluster running your connector. Contact Monad Customer Support to have this enabled and to obtain the subnet resource ID (the subnet lives in a Monad-managed VNet). - Add that subnet to your Storage Account's allowed virtual networks: Storage Account → Networking → Firewalls and virtual networks → Add existing virtual network. With the Service Endpoint in place, requests from Monad are recognized as coming from an allowed virtual network instead of an unmatchable private IP.
- Save. Access from Monad should succeed once the rule propagates.
If the portal won't let you add the subnet
The Azure portal only lets you add a virtual network/subnet you have read access to. Because the Monad cluster's VNet is Monad-managed, you likely won't see it in the portal picker. Add the rule with the Azure CLI instead:
Code
Monad Customer Support will provide the <monad-cluster-subnet-resource-id>.
This affects any Monad deployment running on Azure (AKS) — shared-tenancy SaaS as well as dedicated clusters — whenever your Azure Storage account is in the same Azure region as the AKS cluster running the connector. Deployments where Monad and the storage account are in different regions are unaffected (public-IP allowlisting works normally there).
Details
The Azure Virtual Network Flow Logs input allows you to collect and ingest network flow logs generated by Azure Virtual Networks. These logs provide detailed information about network traffic flowing through your Azure virtual networks, including source and destination IP addresses, ports, protocols, and traffic decisions (allow/deny).
Flow logs are stored in Azure Blob Storage in a structured format and are automatically partitioned by date and time. The input processes these logs incrementally, ensuring that only new data is ingested since the last successful run.
Note: The first sync starts at 00:00 UTC on the current day; there is no backfill setting.
How It Works
When the input is run for the first time, it will process flow logs from the current day. After every successful sync, a timestamp of the last successful fetch is saved. This timestamp is used to incrementally pull only new data, ensuring efficient processing and avoiding duplicates.
The input:
- Connects to the specified Azure Storage Account
- Reads flow log files from the
insights-logs-flowlogfloweventcontainer - Processes JSON-formatted flow log records
- Maintains state to track progress and enable incremental syncing
- Includes a 75-minute data availability delay to account for Azure's log processing time
Setting up Azure Virtual Network Flow Logs
Step 1: Enable Flow Logs on Your Virtual Network
- Navigate to the Azure portal
- Go to Network Watcher service
- In the left sidebar, select Flow logs under Logs
- Click + Add to create a new flow log
- Configure the flow log:
- Target resource: Select your whole Virtual Network (not a subnet or network interface)
- Flow log name: Keep the default name Azure suggests
- Storage account: Choose or create a storage account for storing logs
- Retention (days): Set retention period as needed
- Enable Traffic Analytics: Optional, for additional insights
- Click Create to enable flow logging
Step 2: Create Azure Entra ID Application
- Navigate to App Registrations in the Azure portal
- Click New Registration
- Provide a name for the application (e.g., "Monad Flow Logs Reader")
- Select appropriate account types (typically "Accounts in this organizational directory only")
- Click Register
- Save the Application (client) ID and Directory (tenant) ID from the overview page
- Go to Certificates & secrets in the left sidebar
- Under Client secrets, click New client secret
- Add a description and set expiration period
- Click Add and save the Value (this is your client secret)
Step 3: Grant Storage Account Access
The application needs permission to read from the storage account containing flow logs:
- Navigate to your Storage Account in the Azure portal
- Click Access Control (IAM) in the left sidebar
- Click + Add > Add role assignment
- Select the Storage Blob Data Reader role
- Click Next
- Under Assign access to, select User, group, or service principal
- Click + Select members
- Search for your application name and select it
- Click Select, then Review + assign
- Click Review + assign again to confirm
Step 4: Verify Flow Log Storage Structure
Flow logs are automatically stored in your storage account with the following structure:
Code
Configuration
Settings
| Setting | Type | Required | Default | Description |
|---|---|---|---|---|
tenant_id | string | Yes | - | The tenant ID of the Entra ID directory associated with your Azure subscription. |
subscription_id | string | Yes | - | The unique identifier of your Azure subscription, used to manage resources and billing in Azure Cloud. |
storage_account_url | string | Yes | - | The Azure storage account URL where flow logs are stored (format: https://accountname.blob.core.windows.net) |
region | string | Yes | - | The Azure region where the virtual network is located Allowed values: eastus — (US) East US, southcentralus — (US) South Central US, westus2 — (US) West US 2, westus3 — (US) West US 3, australiaeast — (Asia Pacific) Australia East, southeastasia — (Asia Pacific) Southeast Asia, northeurope — (Europe) North Europe, swedencentral — (Europe) Sweden Central, uksouth — (Europe) UK South, westeurope — (Europe) West Europe, centralus — (US) Central US, southafricanorth — (Africa) South Africa North, centralindia — (Asia Pacific) Central India, eastasia — (Asia Pacific) East Asia, indonesiacentral — (Asia Pacific) Indonesia Central, japaneast — (Asia Pacific) Japan East, japanwest — (Asia Pacific) Japan West, koreacentral — (Asia Pacific) Korea Central, malaysiawest — (Asia Pacific) Malaysia West, newzealandnorth — (Asia Pacific) New Zealand North, canadacentral — (Canada) Canada Central, francecentral — (Europe) France Central, germanywestcentral — (Europe) Germany West Central, italynorth — (Europe) Italy North, norwayeast — (Europe) Norway East, polandcentral — (Europe) Poland Central, spaincentral — (Europe) Spain Central, switzerlandnorth — (Europe) Switzerland North, mexicocentral — (Mexico) Mexico Central, uaenorth — (Middle East) UAE North, brazilsouth — (South America) Brazil South, chilecentral — (South America) Chile Central, eastus2euap — (US) East US 2 EUAP, israelcentral — (Middle East) Israel Central, qatarcentral — (Middle East) Qatar Central, centralusstage — (US) Central US (Stage), eastusstage — (US) East US (Stage), eastus2stage — (US) East US 2 (Stage), northcentralusstage — (US) North Central US (Stage), southcentralusstage — (US) South Central US (Stage), westusstage — (US) West US (Stage), westus2stage — (US) West US 2 (Stage), asia, asiapacific (Asia Pacific), australia, brazil, canada, europe, france, germany, global, india, indonesia, israel, italy, japan, korea, malaysia, mexico, newzealand (New Zealand), norway, poland, qatar, singapore, southafrica (South Africa), spain, sweden, switzerland, taiwan, uae (United Arab Emirates), uk (United Kingdom), unitedstates (United States), unitedstateseuap (United States EUAP), eastasiastage — (Asia Pacific) East Asia (Stage), southeastasiastage — (Asia Pacific) Southeast Asia (Stage), brazilus — (South America) Brazil US, eastus2 — (US) East US 2, eastusstg — (US) East US STG, northcentralus — (US) North Central US, westus — (US) West US, jioindiawest — (Asia Pacific) Jio India West, centraluseuap — (US) Central US EUAP, southcentralusstg — (US) South Central US STG, westcentralus — (US) West Central US, southafricawest — (Africa) South Africa West, australiacentral — (Asia Pacific) Australia Central, australiacentral2 — (Asia Pacific) Australia Central 2, australiasoutheast — (Asia Pacific) Australia Southeast, jioindiacentral — (Asia Pacific) Jio India Central, koreasouth — (Asia Pacific) Korea South, southindia — (Asia Pacific) South India, westindia — (Asia Pacific) West India, canadaeast — (Canada) Canada East, francesouth — (Europe) France South, germanynorth — (Europe) Germany North, norwaywest — (Europe) Norway West, switzerlandwest — (Europe) Switzerland West, ukwest — (Europe) UK West, uaecentral — (Middle East) UAE Central, brazilsoutheast — (South America) Brazil Southeast. |
virtual_network_name | string | Yes | - | The name of the virtual network for which flow logs are being collected |
resource_group_name | string | Yes | - | The name of the resource group containing the virtual network |
use_synthetic_data | bool | No | false | Generate synthetic data for testing, instead of connecting to a real data source. Defaults to an hourly cron schedule for cron-based inputs. |
Secrets
| Secret | Type | Required | Description |
|---|---|---|---|
client_id | secret | Yes | The client ID of the Entra ID application |
client_secret | secret | Yes | The client secret of the Entra ID application |
Troubleshooting
Common Issues
- Connection Failed: Verify that the storage account URL is correct and the application has proper permissions
- No Data Found: Ensure that flow logs are enabled on your virtual network and data is being written to the storage account
- Authentication Errors: Check that the Tenant ID, Client ID, and Client Secret are correct and the application exists in the specified tenant
- Permission Denied: Verify that the application has "Storage Blob Data Reader" role on the storage account
Data Availability
- Input Connector Delay: This input connector includes a 75-minute delay before processing flow logs. While Azure flow logs are typically available in the storage account within a few minutes, this connector waits 75 minutes to ensure all data for a given time period has been fully written and is available for processing.
- The delay helps ensure data completeness and consistency during ingestion
Related Articles
- Azure Network Watcher Documentation
- Azure Storage Account Management
- Azure Entra ID App Registration
Sample Record
Code
Sync frequency
By default this input polls approximately every 10 seconds, with each sync beginning after the previous one completes. A cron schedule configured on the pipeline overrides this cadence. See Input Sync Frequency for details.