Pipeline Configuration

This document provides a comprehensive reference for all AutoTrigger pipeline components. These concepts apply to all job types (PDF reports, API calls, data processing, etc.).

Schema Structure

The AutoTrigger schema defines the structure for configuring automated jobs. Below is a breakdown of each field and its purpose.

---

Data Selection

Data selection defines where the job gets its data from. Currently, only Elasticsearch is supported.

Use Elasticsearch for querying time-series data, logs, or structured data stored in ES.

Basic ES Query

{
  "dataSource": "es",
  "variableName": "myData",
  "parameters": [
    { "index": "calls", "ignore_unavailable": true },
    {
      "size": 0,
      "query": { ... },
      "aggs": { ... }
    }
  ]
}

Required Parameters

Parameter	Required	Description
index	No	Elasticsearch index pattern (e.g., calls, logs-*)
ignore_unavailable	No	Ignore missing indices
query	Yes	Elasticsearch query DSL
aggs	No	Aggregations for data analysis

Example: Query with Sort

Query with Sort

{
  "dataSource": "es",
  "variableName": "RecentCalls",
  "parameters": [
    { "index": "calls" },
    {
      "size": 100,
      "sort": [{ "@timestamp": { "order": "desc" } }],
      "query": {
        "bool": {
          "must": [
            { "query_string": { "query": "attrs.type:call-end" } },
            { "range": { "@timestamp": { "gte": "now-24h", "lte": "now" } } }
          ]
        }
      }
    }
  ]
}

---

Transformations

Transformations process and manipulate data before output. Only JavaScript transformations are supported. Python support is planned for a future release.

Transformation Overview

Execute custom JavaScript code to transform data.

Inline Code

"transform": {
  "transformationType": "js",
  "code": "return data.map(item => { item.processed = true; return item; });"
}

Inline Code

• Use the code field for custom JavaScript
• Data is passed as the data variable
• Return the transformed data

Named Function Reference

Use predefined transformation functions.

Named Function Reference

"transform": {
  "transformationType": "js",
  "codeFunctionReference": "anonymization"
}

Available Function References:

• anonymization - Apply data anonymization rules

Transformation Examples

Filter Data

"transform": {
  "transformationType": "js",
  "code": "return data.filter(item => item.duration > 60);"
}

Add Calculated Field

"transform": {
  "transformationType": "js",
  "code": "return data.map(item => ({ ...item, durationMinutes: item.duration / 60 }));"
}

Group and Aggregate

"transform": {
  "transformationType": "js",
  "code": `
    const grouped = {};
    data.forEach(item => {
      if (!grouped[item.agent]) grouped[item.agent] = { count: 0, total: 0 };
      grouped[item.agent].count++;
      grouped[item.agent].total += item.duration;
    });
    return Object.entries(grouped).map(([agent, stats]) => ({ agent, ...stats }));
  `
}

---

Output Formats

Output formats define how the data is formatted and presented. Four output types are supported: JSON, CSV, PDF, and HTML.

Output Types

JSON

Raw JSON output for programmatic use.

JSON Output

{
  "transformationType": "toJson",
  "parameters": {}
}

CSV

Comma-separated values for spreadsheet import.

CSV Output

{
  "transformationType": "toCsv",
  "parameters": {
    "delimiter": ",",
    "includeHeaders": true
  }
}

PDF

PDF report with visualizations.

PDF Output

{
  "transformationType": "toPdf",
  "parameters": {
    "pageSize": "A4",
    "orientation": "landscape",
    "charts": [
      {
        "variableName": "myData",
        "chartTitle": "My Chart",
        "chartType": "line",
        "maxLength": 30
      }
    ]
  }
}

See PDF Report Generation for detailed chart options.

HTML

HTML/MHTML report with visualizations (same options as PDF).

HTML Output

{
  "transformationType": "toHtml",
  "parameters": {
    "pageSize": "A4",
    "orientation": "landscape",
    "charts": [
      {
        "variableName": "myData",
        "chartTitle": "My Chart"
      }
    ]
  }
}

Output Format Parameters

Format	Parameters	Description
toJson	Any	Raw JSON output
toCsv	delimiter, includeHeaders	CSV formatting options
toPdf	pageSize, orientation, charts	PDF with optional charts
toHtml	pageSize, orientation, charts	HTML with optional charts

For chart configuration, page settings, and chart type inference, see PDF Report Generation.

---

Delivery Channels

Delivery channels define where the generated output is sent. Configure multiple channels for flexible distribution.

Supported Channel Types

Channel	Description
Email	SMTP email delivery for sending reports
Matrix	Matrix chat integration for notifications
Browser	Browser-based notification delivery
SNMP	SNMP trap notification for monitoring
API	HTTP POST to external endpoint for integrations

For configuration details, see the Channels documentation.

Channel Configuration

Channel Configuration

"delivery": {
  "channels": ["emailChannelId1", "apiChannelId2"]
}

Channel IDs are defined in the Channels configuration section of the application. Each channel must be pre-configured with connection details.

Example: Email and API

Multiple Delivery Channels

{
  "name": "Daily Report",
  "pipeline": {
    "output": {
      "formats": [
        {
          "transformationType": "toPdf",
          "parameters": { "pageSize": "A4", "orientation": "landscape", "charts": [...] }
        }
      ]
    },
    "delivery": {
      "channels": ["email-daily-team", "api-monitoring"]
    }
  }
}

---

Actions

Actions are executed after job processing. Currently, one action type is supported: saveToEs.

Save to Elasticsearch

Save processed data to an Elasticsearch stream for further analysis or storage.

Save to Elasticsearch Action

"action": [
  {
    "type": "saveToEs",
    "parameters": {
      "stream": "my-report-data",
      "timestampField": "@timestamp"
    }
  }
]

Parameters

Field	Required	Description
stream	Yes	Elasticsearch stream name (lowercase, no spaces)
timestampField	No	Field to use as document timestamp (default: @timestamp)

warning

Reserved Stream Prefixes: The following stream names are reserved and cannot be used:

• calls
• collectd
• exceeded
• rtp
• alarms
• status
• syslog

Example: Save Call Records

Save Call Records to ES

"action": [
  {
    "type": "saveToEs",
    "parameters": {
      "stream": "call-aggregates",
      "timestampField": "periodEnd"
    }
  }
]

---

See Also

• PDF Report Generation - Specific guide for PDF reports
• Create New Report - Getting started guide