Cloudflare HTTP Requests
Beta Feature
This input is currently in beta. If you run into any issues, please reach out to product@monad.com.
Ingests Cloudflare HTTP request analytics using the GraphQL Analytics API. This input collects detailed information about HTTP requests processed by Cloudflare, including client information, response metrics, caching details, and security data from the httpRequestsAdaptive dataset.
Sync Type: Incremental
Requirements
- A Cloudflare account with access to the Analytics API
- An API Token with the
Zone>Analytics>Readpermission for the zone (this dataset is zone-scoped, soAccount Analytics:Readis not enough) - Your Cloudflare Zone ID
Data Retention by Plan
Cloudflare retains HTTP request analytics based on your plan:
| Plan | Data Retention |
|---|---|
| Free | 24 hours |
| Pro | 3 days |
| Business | 3-30 days |
| Enterprise | 30-90 days |
The input automatically detects your plan's retention limits and adjusts queries accordingly.
Creating an API Token
- Log in to your Cloudflare dashboard
- Navigate to My Profile > API Tokens
- Click Create Token
- Use the Create Custom Token option
- Configure the token:
- Token name: Give it a descriptive name (e.g., "Monad HTTP Requests Analytics")
- Permissions: Add
Zone>Analytics>Read - Zone Resources: Select the specific zone(s) or "All zones"
- Click Continue to summary and then Create Token
- Copy the token value (you won't be able to see it again)
Finding Your Zone ID
- Log in to your Cloudflare dashboard
- Select the domain/zone you want to monitor
- The Zone ID is displayed in the right sidebar on the overview page
- It's a 32-character alphanumeric string
Details
- State Management: Uses timestamp-based incremental sync. Monad stores the last processed event timestamp to fetch only new events on subsequent runs.
- API Endpoint: GraphQL Analytics API (
/graphql) - Dataset:
httpRequestsAdaptive - Pagination: Time-window based pagination. The API defines maximum query duration (
maxDuration) to prevent timeout. The input automatically chunks requests into time windows to handle large time ranges. - Rate Limits: 300 requests per 5 minutes (1 request per second). The input automatically enforces this limit.
- Field Selection: The dataset has over 120 available fields.
The input always adds
datetimeandrayName, and keeps up to 48 user-selected fields alongside them (50 fields total). Selected fields beyond that are dropped. At run time, fields that are not available to your account are dropped without an error. - First sync: If Lookback Duration is not set, the first sync starts 5 minutes before the input first runs.
Set a lookback such as
24hor168hto collect older requests, up to your plan's retention. Day units such as7dare not accepted. - Late-arriving events: Each sync reads up to the current time, so an event that Cloudflare makes available later than newer events already collected can be missed.
- Adaptive Sampling: During high-traffic periods, Cloudflare may apply sampling to this dataset. Not all events may be returned during very high volume periods.
Configuration
Settings
| Setting | Type | Required | Default | Description |
|---|---|---|---|---|
zone_id | string | Yes | - | Cloudflare Zone ID |
fields | array of string | No | - | Fields to include in the query. Leave empty to use the default curated list. Only fields available to your account are included. Maximum 48 fields (the input always adds datetime and rayName). Allowed values: clientIP, clientAsn, clientCountryName, clientDeviceType, clientIPClass, clientMTLSAuthCertFingerprint, clientMTLSAuthStatus, clientRefererHost, clientRefererPath, clientRefererQuery, clientRefererScheme, clientRequestBytes, clientRequestHTTPHost, clientRequestHTTPMethodName, clientRequestHTTPProtocol, clientRequestPath, clientRequestQuery, clientRequestReferer, clientRequestScheme, clientRequestSource, clientRequestURI, clientSSLCipher, clientSSLProtocol, clientSrcPort, clientTCPRTTMs, clientXRequestedWith, coloCode, edgeColoCode, edgeColoID, edgeColoName, edgeEndTimestamp, edgeRateLimitAction, edgeRateLimitID, edgeRequestHost, edgeResponseBodyBytes, edgeResponseBytes, edgeResponseCompressionRatio, edgeResponseContentType, edgeResponseStatus, edgeServerIP, edgeStartTimestamp, edgeTimeToFirstByteMs, cacheStatus, cacheCacheStatus, cacheTieredFill, cacheReserveUsed, originASN, originIP, originResponseBytes, originResponseDurationMs, originResponseHTTPExpires, originResponseHTTPLastModified, originResponseStatus, originResponseTime, originSSLProtocol, botScore, botScoreSrcName, botManagementDecision, ja3Hash, ja4, securityAction, securityActions, securityRuleDescription, securityRuleID, securityRuleIDs, securitySources, wafAttackScore, wafFlags, wafMatchedVar, wafProfile, wafRCEAttackScore, wafSQLiAttackScore, wafXSSAttackScore, requestHeaders, responseHeaders, upperTierColoName, userAgent, workerCPUTime, workerStatus, workerSubrequest, workerSubrequestCount, workerWallTimeUs, xRequestedWith, zoneID, zoneName, tlsCipher, tlsClientHelloLength, tlsClientRandom, tlsVersion, firewallMatchesActions, firewallMatchesRuleIDs, firewallMatchesSources, contentScanHasMaliciousPayload, contentScanNumMaliciousObj, contentScanNumObj, contentScanObjResults, contentScanObjTypes, leakyBucketActions, leakyBucketConfigurationIDs, leakyBucketMitigations, smartColoConnectTimeMs, smartColoID. |
lookback_duration | string | No | - | Initial lookback duration for first sync (e.g., "24h", "168h"). Respects API retention limits. |
use_synthetic_data | bool | No | false | Generate synthetic data for testing, instead of connecting to a real data source. Defaults to an hourly cron schedule for cron-based inputs. |
rate_limit | object | No | - | Maximum outbound source-API request rate. A blank or zero value falls back to the connector's default rate-limiting behavior. Maximum 1 requests per second. See API Rate Limiting for the field format. |
Secrets
| Secret | Type | Required | Description |
|---|---|---|---|
api_token | secret | Yes | Cloudflare API Token with Analytics Read permissions |
Rate Limits
| Limit | Value | Notes |
|---|---|---|
| Requests per 5 minutes | 300 | GraphQL Analytics API limit |
| Requests per second | 1 | Enforced by rate limiter |
| Max records per query | Plan-dependent | Discovered automatically via Settings API |
Source: Cloudflare GraphQL Analytics API Limits
Troubleshooting
Common Issues
Issue: GraphQL error with authentication message
Cause: The API token is invalid, expired, or lacks the required permissions.
Solution: Verify your API token is correct and has the Zone > Analytics > Read permission for the zone.
Create a new token if needed.
Issue: Zone ID is required validation error
Cause: The Zone ID field was left empty.
Solution: Enter your Cloudflare Zone ID in the settings. You can find this in your Cloudflare dashboard under the zone's overview page.
Issue: invalid settings response error
Cause: The Zone ID is incorrect or the API token doesn't have access to that zone.
Solution: Verify the Zone ID is correct and that your API token has permissions for that specific zone.
Issue: No data returned despite having HTTP traffic Cause: The lookback duration may exceed your plan's data retention, or there may be no requests in the time range. Solution: Check your plan's data retention limits. Free plans only retain 24 hours of data.
Issue: unknown field validation error
Cause: A requested field name is not one Monad recognises, usually because it is misspelled.
Solution: Check that all field names are spelled correctly and appear in the allowed values for Fields.
Issue: A selected field is missing from records Cause: The field is not available to your account, so the input drops it at run time without an error. Some fields require specific plan levels or add-ons. Solution: Check which fields your Cloudflare plan and add-ons make available for this dataset.
Issue: maximum 49 fields allowed error
Cause: More than 49 fields were selected.
Solution: Reduce the number of selected fields.
Because the input always adds datetime and rayName, only the first 48 selected fields are kept, so select 48 or fewer.
Issue: Requests appear to be missing during high traffic Cause: Cloudflare applies adaptive sampling to this dataset during high-volume periods. Solution: This is expected behavior. The dataset is designed for monitoring and analysis, not exact request counting.
Issue: invalid lookback_duration format error
Cause: The lookback duration is not in a valid Go duration format.
Solution: Use valid duration formats like 24h, 168h, or 1h30m.
Day units such as 7d are not accepted.
Related Articles
- Cloudflare GraphQL Analytics API
- Querying HTTP Request Analytics with GraphQL
- Cloudflare API Tokens
- GraphQL Analytics API Limits
Sample Record
Code
Sync frequency
By default this input polls approximately every 10 seconds, with each sync beginning after the previous one completes. A cron schedule configured on the pipeline overrides this cadence. See Input Sync Frequency for details.
Community transforms
The Monad Transforms Repository has ready-made transforms for this input. Import one into your organization, then add it to a pipeline after this input.
| Format | Transform |
|---|---|
| OCSF v1.9.0 | Cloudflare HTTP Requests to OCSF v1.9.0 |
| ECS v9.5.0 | Cloudflare HTTP Requests to ECS v9.5.0 |
| CEF | Cloudflare HTTP Requests to CEF |
CEF transforms emit an intermediate record; add the CEF Serializer transform after one to produce the CEF:0 string.