Score ML Models

This set of handlers provide ability to run real-time inference using various machine learning libraries, which share the same set of parameters and actions.

Handler Parameters

This handler is cacheable.

Parameter
Definition
Example
Default

model.state

Name of the state manager with model configurations

model

-

model.id

Id of the model to run on this instance

recommendation

-

model.store.system

Name of the system to store model assets

hdfs_default

-

model.local.dir

Local directory to store model assets for execution

/tmp

java.io.tmpdir

Example Tensorflow Model Handler Definition (Can be Imported on Element Screen)

Actions

Score / ScoreML

Scores input data using pretrained model managed by this handler. Event metadata fields applicable for this action are as follows:

Field
Definition
Example
Default

inputElement

Json path for the input in request event payload

parameters

-

outputElement

Json path for the output in response event payload

$.score

-

With event metadata parameters as:

Parameter
Definition
Example
Default

inputPattern

JMESPath pattern to apply on data input

[ [text] ]

-

outputPattern

JMESPath pattern to apply on data output, before returning response

{totalPrice:basket.totalPrice}

-

Versions

TensorScoreEventHandler

This handler (com.rierino.handler.tensor.TensorScoreEventHandler) uses Tensorflow library for inference using saved models.

This handler requires the following dependency added to deployment contents:

implementation (group:'com.rierino.custom', name: 'tensor', version:"${rierinoVersion}")

OnnxScoreEventHandler

This handler (com.rierino.handler.onnx.OnnxScoreEventHandler) uses ONNX Runtime library for inference using models saved using this common format.

This handler requires the following dependency added to deployment contents:

implementation (group:'com.rierino.custom', name: 'onnx', version:"${rierinoVersion}")

PMMLScoreEventHandler

This handler (com.rierino.handler.pmml.PMMLScoreEventHandler) uses PMML formatted models for inference.

This handler requires the following dependency added to deployment contents:

implementation (group:'com.rierino.custom', name: 'pmml', version:"${rierinoVersion}")

OpenNLPScoreEventHandler

This handler (com.rierino.handler.opennlp.OpenNLPScoreEventHandler) uses OpenNLP library for inference using common NLP tools, saved dictionaries and models.

This handler requires the following dependency added to deployment contents:

implementation (group:'com.rierino.custom', name: 'opennlp', version:"${rierinoVersion}")

Last updated