> For the complete documentation index, see [llms.txt](https://docs.rierino.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rierino.com/data-science/complex-event-processing.md).

# Complex Event Processing

Data flows provide a standardized form of defining data integration flows, which can easily be migrated between systems.

![Data Flow UI](https://1659095931-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FcnDk3J1AzTgg2NFrGPlh%2Fuploads%2FEwZsawIKy4WCnECl2tig%2FUI_Data_Flow.png?alt=media\&token=14a2b3fd-e878-4750-ab52-64213105bfa6)

Data flows are typically used by data flow handlers such as [Siddhi](https://docs.rierino.com/data-science/complex-event-processing/siddhi-data-flows.md).

Data flows share the following attributes:

* **Name:** Descriptive name of the data flow
* **Description:** Detailed description of the flow
* **Domain:** Business domain / group for the data flow
* **Platform:** Target execution platform for the data flow
* **Command:** Complete platform specific command for the data flow definition
* **Parameters:** Additional platform specific parameters

Data flows consist of mappings between data sources and tables through mutations.

## Data Sources

Data sources define origins (such as streams) from which data is fed into data flows. These sources share the following attributes:

* **Name:** Name of the source (such as Kafka topic)
* **Description:** Detailed description of the source
* **For Each:** Expression for repeating / denormalizing data from the source during feed
* **Fields:** Mapping of Json paths of data in source to target fields
* **Target:** Name of the target table for feeding from the source
* **Parameters:** Additional platform specific parameters

## Tables

Tables define data stores, which are used as destinations from data sources and mutations, as well as the origins for queries. These tables share the following attributes:

* **Name:** Name of the table
* **Description:** Detailed description of the table
* **Type:** Platform specific type of the table
* **Fields:** List of table fields and their types
* **DDL:** Platform specific statement for creating the table
* **Parameters:** Additional platform specific parameters

## Mutations

Mutations define transformations of data between and on different tables (e.g. delete, insert operations). These mutations share the following attributes:

* **Name:** Name of the mutation
* **Description:** Detailed description of the mutation
* **Type:** Type of the mutation (DELETE, INSERT, UPDATE, UPSERT, COMMAND)
* **Target:** Name of the target table for the mutation
* **Query:** [Query](https://docs.rierino.com/configuration/queries.md) to apply on the target table
* **Command:** Platform specific command for the mutation (if type is COMMAND)
* **Priority:** Execution priority for the mutation
* **Set Fields:** Field mappings from query results to target, if different than query field names
* **Parameters:** Additional platform specific parameters


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.rierino.com/data-science/complex-event-processing.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
