Run Python Procedure

This handler (com.rierino.handler.py4j.Py4JEventHandler) provides ability to handle events using Python libraries.

Handler Parameters

This handler has no special parameters.

This handler requires the following dependency added to deployment contents:

implementation (group:'com.rierino.custom', name: 'py4j', version:"${rierinoVersion}")

Actions

Process / ProcessPython

Passes requested event to Python interface for processing. Event metadata fields applicable for this action are as follows:

Table

Field	Definition	Example	Default
Input Element	Json path of payload input to pass	parameters	-
Output Element	Json path for output to add on payload	$	-

JSON Schema

With event metadata parameters as:

Table

Parameter	Definition	Example	Default
Input Pattern	JMESPath pattern to apply on input element	{data:contents}	-
Python Package	Name of the Python package to use	rierino_runner	-
Python Module	Name of the Python module to use	CustomHandler	-
Python Action	Name of the Python function to call	Calculate	-

JSON Schema

Python function receives java handler, process input as dict, and event metadata parameters as input.

Using Py4JEventHandler requires deployment of an extra container along with the runner, such as "ghcr.io/rierino-open/py-runner" image, configured from useExtraContainer and extraImage parameters of Deployment UI.

Python Modules

The following python packages and modules are currently available out of box:

NoopEventHandler

Used for Py4J gateway testing purposes only.

• pyPackage: rierino_runner

• pyModule: NoopEventHandler

• pyAction: hello

ProcessEventHandler

Generic handler for triggering any Python Process class (such as rierino_runner.IterateProcess, rierino_media.MediaEventProcess, rierino_tensor.TFModelProcess).

• pyPackage: rierino_runner

• pyModule: ProcessEventHandler

• pyAction: execute

Table

Parameter	Definition	Example	Default
sync	Whether handler should wait for task completion to return a response or not	true	false
systems	Comma separated list of system names for which parameters shall be received from Java runner	fs_media,s3_model	-
connections	Json array containing details of connections to pass to handler (system configurations on Java runner is usually preferred)	[{name: "fs_test", type: "fs", dir: "/tmp"}]	-
silent	Whether handler should log process updates or not	true	false
module	Process module to execute (view list)	rierino_runner.IterateProcess	-
[module_params]	Process module specific parameters	{"iterator":{ "module": "rierino_runner.iterator.DataIterator", "element": "list" }}	-

JSON Schema

DQEventHandler

Performs data quality assessment of given inputs against a configured list of checks.

• pyPackage: rierino_dq

• pyModule: DQEventHandler

• pyAction: eval

Has the following set of additional event metadata parameters:

Table

Parameter	Definition	Example	Default
configPath	Json path in input payload for DQ check configuration (if not using a config state)	parameters.dq_config	-
configState	Name of the runner state manager storing DQ check configs	dq_main	dq
configId	Id of the config stored in state manager	product_dq	default
reload	Whether config should be reloaded from state	true	false
summarize	Whether results should be summarized	true	false

JSON Schema

Example

Input

{
    "parameters": {
        "id": "product-1",
        "data": {
             "name": "Test Product",
             "description": "Too Short"
        }
    }
}

Event Metadata

Additional python classes can be implemented and deploy using py-runner library for use with Py4JEventHandler, simply having the following two functions:

• def init(self, javaHandler)

• def [FUNCTION NAME](self, javaHandler, processDict, parameters)