# Data Routing

Data routing allows you to conditionally send data records to different destinations based on the content of each record. In the pipeline UI, routing is configured through **edges** - the connections between your inputs and outputs that determine how data flows through your system.

## Overview

Each edge in your pipeline represents a routing rule. When you connect an input to an output, you create an edge that can include conditions to control which records pass through. Data flows from inputs through edges to outputs based on the conditions you define.

Edges use **logical conditions** to evaluate records. For complete documentation on conditionals, operators, and condition types, see [Conditionals](../conditionals/).

## Understanding Edges

In the pipeline interface:
- **Edges** are the arrow connections between nodes
- Each edge has a name label (e.g., "always", "high-priority")
- Clicking on an edge opens its configuration panel
- You can have multiple edges from a single input to different outputs

### Pipeline Structure
Your pipelines follow these important structural rules:
- **Single Input**: Each component accepts exactly one incoming edge
- **Multiple Outputs**: Components can have multiple outgoing edges to different destinations
- **Tree Structure**: Edges cannot create loops - data flows forward only, preventing circular routing

This design ensures predictable data flow and makes it easy to trace how records move through your pipeline.

## Configuring an Edge

### Accessing Edge Configuration

1. Click on any edge in your pipeline to select it
2. The configuration panel appears showing:
    - **Name**: The edge identifier
    - **Description**: Optional description of the edge's purpose
    - **Conditions**: Current operator and rules summary
3. Click "Edit edge" to modify the configuration

### Edit Modes

The edge editor provides two ways to configure routing:

#### Form View
The visual interface for configuring edges:
- **Operator dropdown**: Select how conditions are combined
- **Rules section**: Add and configure individual conditions
- Each rule can be edited or removed independently

#### Code View
Direct JSON editing for advanced users:
```json
{
  "operator": "and",
  "conditions": [
    {
      "type_id": "equals_any",
      "config": {
        "key": "status",
        "values": ["active"]
      }
    }
  ]
}
```

### Edge Components

Each edge configuration is a [logical condition](../conditionals/#structure) consisting of:
1. **Operator** - How multiple conditions are combined (`always`, `and`, `or`, `nor`, `xor`)
2. **Conditions** - The leaf conditions or nested logical conditions that records must meet

## Operators

Select an operator from the dropdown to determine how rules are evaluated. See [Logical Operators](../conditionals/#logical-operators) for complete details.

| Operator | Description | Conditions Required |
|----------|-------------|---------------------|
| `always` | Routes all records without any conditions | 0 |
| `never`  | Blocks all records | 0 |
| `and`    | All conditions must be true | At least 1 |
| `or`     | At least one condition must be true | At least 2 |
| `nor`    | None of the conditions should be true | At least 1 |
| `xor`    | Exactly one condition must be true | At least 2 |

## Available Conditions

The following condition types can be used in edge routing. See [Leaf Condition Types](../conditionals/#leaf-condition-types) for complete configuration details.

| Condition | Description |
|-----------|-------------|
| [`key_exists`](../conditionals/key-exists) | Check if a field exists in the record |
| [`equals`](../conditionals/#equals) | Check if a field equals a specific value |
| [`equals_any`](../conditionals/equals-any) | Check if a field matches any value in a list |
| [`contains`](../conditionals/#contains) | Check if a string contains a substring |
| [`starts_with`](../conditionals/#starts_with) | Check if a string starts with a prefix |
| [`ends_with`](../conditionals/#ends_with) | Check if a string ends with a suffix |
| [`matches_regex`](../conditionals/#matches_regex) | Check if a string matches a regex pattern |
| [`greater_than`](../conditionals/#greater_than) | Check if a number is greater than a threshold |
| [`less_than`](../conditionals/#less_than) | Check if a number is less than a threshold |
| [`sample`](../conditionals/sample) | Sample a percentage of records randomly or by key |

## Creating and Managing Edges

### Adding a New Edge
1. In the pipeline view, drag from an input's output port to an output's input port
2. A new edge is created with default settings
3. Click the edge to configure its conditions

### Configuring Conditions
1. Click "Edit edge" to open the editor
2. Select your preferred view (Form or Code)
3. In Form view:
    - Choose an operator from the dropdown
    - Click "Add new rule" to add conditions
    - Configure each rule's type and parameters
4. In Code view:
    - Edit the JSON directly
    - Ensure valid JSON syntax
5. Click "Update" to save changes

### Multiple Edges from One Input
You can create multiple edges from a single input to different outputs:
- Each edge evaluates independently
- Records can match multiple edges and be sent to multiple destinations
- All edges are evaluated for every record - order doesn't matter

## Common Patterns

### Simple Pass-Through
```json
{
  "operator": "always",
  "conditions": []
}
```
All data flows through without filtering.

### Multi-Criteria Filtering
```json
{
  "operator": "and",
  "conditions": [
    {
      "type_id": "key_exists",
      "config": {"key": "customer_id"}
    },
    {
      "type_id": "equals_any",
      "config": {
        "key": "status",
        "values": ["active", "pending"]
      }
    }
  ]
}
```
Records must have a customer_id AND status of active or pending.

### Priority Routing
Create multiple edges with different conditions (all evaluate independently):
- Edge 1: "critical" - Routes severity:critical to alerts.urgent
- Edge 2: "standard" - Routes severity:warning to alerts.normal
- Edge 3: "archive" - Routes all records (using "always") to storage.archive

Note: A single record can match multiple edges and be routed to multiple destinations.

## Best Practices

1. **Name edges descriptively**: Use names that describe the routing purpose (e.g., "high-value-customers" not "edge1")

2. **Use the Code view for complex logic**: While the Form view is user-friendly, Code view is faster for complex configurations and is a great way to copy and paste your edge configurations

3. **Test edge conditions**: Use test data to verify your routing logic before connecting to production outputs

4. **Document with descriptions**: Add descriptions to edges explaining their purpose for team members

5. **Consider edge interactions**: Since all edges are evaluated, ensure your conditions work well together when records may match multiple edges

## Visual Pipeline Tips

- **Edge colors**: Selected edges are highlighted for clarity
- **Edge labels**: Always visible to understand data flow at a glance
- **Hover information**: Hover over edges to see quick configuration details
- **Remove edges**: Use the "Remove" button in the configuration panel to delete unwanted edges

## Pipeline Constraints

Understanding these constraints helps you design effective pipelines:

### Single Input Rule
Each component can receive data from only one source through a single incoming edge, but can distribute that data to multiple destinations through multiple outgoing edges. This creates a fan-out pattern where data flows from one source to potentially many destinations.

### No Circular Routing
Components can have multiple outgoing edges but cannot receive edges that create loops back to themselves. This constraint ensures all pipelines maintain a tree-like structure without cycles, preventing infinite loops and ensuring predictable data flow.

### Practical Implications
- **Data flow**: Always flows forward through the tree structure, never backwards
- **Debugging**: The tree structure makes it easier to trace data lineage and troubleshoot issues
- **Performance**: Prevents infinite loops and ensures efficient processing

## Validation

The UI validates your configuration:
- JSON syntax in Code view must be valid
- Required fields must be populated
- Operators have minimum condition requirements (see [Operators](#operators) table)

## Troubleshooting

**Edge not passing data:**
- Check if conditions are too restrictive
- Verify field names match your data exactly (see [Key Path Syntax](../conditionals/#key-path-syntax))
- Test with a simple "always" operator first

**Cannot save edge configuration:**
- Ensure JSON is valid in Code view
- Check all required fields are filled
- Verify condition count meets operator requirements

**Unexpected routing behavior:**
- Remember all edges evaluate independently
- A record matching multiple edges will be sent to all matching destinations
- Use mutually exclusive conditions if you need records to go to only one destination