Cloudflare URL Scanner
Beta Feature
This input is currently in beta. If you run into any issues, please reach out to product@monad.com.
Ingests Cloudflare URL Scanner scan results for security monitoring. This input collects security scan data from URLs submitted to Cloudflare's URL Scanner, providing insights into page safety, SSL status, and security threats detected during scanning.
Sync Type: Incremental
Requirements
- A Cloudflare account with access to the URL Scanner API
- An API Token with the
Account>URL Scanner>Readpermission - Your Cloudflare Account ID
Creating an API Token
- Log in to your Cloudflare dashboard
- Navigate to My Profile > API Tokens
- Click Create Token
- Use the Create Custom Token option
- Configure the token:
- Token name: Give it a descriptive name (e.g., "Monad URL Scanner")
- Permissions: Add
Account>URL Scanner>Read - Account Resources: Select the specific account(s) or "All accounts"
- Click Continue to summary and then Create Token
- Copy the token value (you won't be able to see it again)
Finding Your Account ID
- Log in to your Cloudflare dashboard
- Select the account you want to monitor
- The Account ID is displayed in the right sidebar on the overview page
- Alternatively, find it in the URL:
https://dash.cloudflare.com/{account_id}/...
Details
- State Management: Uses ascending cursor-based pagination with timestamp tracking. Monad stores the last processed timestamp to fetch only new scans on subsequent runs.
- API Endpoint:
GET /accounts/{account_id}/urlscanner/v2/search - Query Syntax: Uses Elasticsearch query syntax for filtering scan results. Supports filters like
date:[start TO end]andapikey:me. - Daily queries: The input queries one UTC day at a time, from the last collected day up to today.
- 100 scans per day: Cloudflare's scan search returns at most 100 results per query. As a result, the input collects at most the 100 most recent scans for each UTC day, and additional scans that day are not collected.
- Rate Limits: 4 requests per second (1,200 requests per 5 minutes). The input automatically enforces this limit.
Configuration
Settings
| Setting | Type | Required | Default | Description |
|---|---|---|---|---|
account_id | string | Yes | - | Cloudflare Account ID |
filter_my_scans | bool | No | - | Filter to only show scans created by the current API token |
use_synthetic_data | bool | No | false | Generate synthetic data for testing, instead of connecting to a real data source. Defaults to an hourly cron schedule for cron-based inputs. |
backfill_start_time | date | No | - | Date to start fetching data from (RFC3339 format). Historical data availability depends on your Cloudflare plan: Free/Radar (last 50 scans), Self Serve (30 days), Enterprise (12 months), Cloudforce One (unlimited) |
rate_limit | object | No | - | Maximum outbound source-API request rate. A blank or zero value falls back to the connector's default rate-limiting behavior. Maximum 4 requests per second. See API Rate Limiting for the field format. |
Secrets
| Secret | Type | Required | Description |
|---|---|---|---|
api_token | secret | Yes | API Token with URL Scanner Edit permission |
Rate Limits
| Limit | Value | Notes |
|---|---|---|
| Requests per second | 4 | 1,200 requests per 5 minutes |
| Records per page | 100 | Maximum records returned per API request |
Headers: Authorization: Bearer {api_token}, Content-Type: application/json
Source: Cloudflare URL Scanner API Documentation
Troubleshooting
Common Issues
Issue: API Token is required validation error
Cause: The API token field was left empty.
Solution: Enter your Cloudflare API token in the secrets configuration.
The token must have the URL Scanner Read permission.
Issue: Account ID is required validation error
Cause: The Account ID field was left empty.
Solution: Enter your Cloudflare Account ID in the settings. You can find this in your Cloudflare dashboard.
Issue: API error with authentication failure
Cause: The API token is invalid, expired, or lacks the required URL Scanner Read permission.
Solution: Verify your API token is correct and has the necessary permissions. Create a new token if needed.
Issue: No scan results returned Cause: The backfill start time may be set to a future date, or no scans exist in the specified time range. If the backfill start time is left empty, the input starts collecting from the day it first runs and does not fetch older scans. Solution: Verify the backfill start time is set correctly. To collect older scans, set an earlier backfill start time.
Issue: Rate limit errors Cause: Multiple integrations or manual API calls are consuming the rate limit quota. Solution: The input automatically handles rate limiting at 4 requests per second. If you see persistent errors, reduce the frequency of other API calls to your Cloudflare account.
Related Articles
- Cloudflare URL Scanner API Documentation
- Cloudflare URL Scanner Limits
- Cloudflare API Tokens
- Cloudflare API Rate Limits
Sample Record
Code
Sync frequency
By default this input polls approximately every 10 seconds, with each sync beginning after the previous one completes. A cron schedule configured on the pipeline overrides this cadence. See Input Sync Frequency for details.
Community transforms
The Monad Transforms Repository has ready-made transforms for this input. Import one into your organization, then add it to a pipeline after this input.
| Format | Transform |
|---|---|
| OCSF v1.9.0 | Cloudflare URL Scanner to OCSF v1.9.0 |
| ECS v9.5.0 | Cloudflare URL Scanner to ECS v9.5.0 |
| CEF | Cloudflare URL Scanner to CEF |
CEF transforms emit an intermediate record; add the CEF Serializer transform after one to produce the CEF:0 string.