GenAI Models
In addition to general ML models, Rierino provides facility for governing LLM models for GenAI use cases
GenAI model definitions allow configuration of AI agents and general purpose LangChain models using different providers such as Amazon Bedrock, Anthropic, Azure, Google Gemini, Mistral, OpenAI, as well as local LLM models.
To enable use of AI agents on a runner, you can simply add GenAI base runner to its configuration, set your API Keys for your preferred LLM provider in secrets and start allowing specific GenAI models to be executed on that runner.
Using AI agents on a runner increases its memory requirements, with minimum requirement becoming 1024mb.
To keep out of box runners lightweight, GenAI base runner is not added to them by default (even for core and training runners), which can be added after initial installation as preferred.
It is recommended to keep AI agents running on dedicated runners, if possible, as they can increase build time and consume more resources than the rest of event handlers on the runner.
All agents generated using this approach can be governed by RBAC and corporate governance rules, based on access rights of the users interacting with the agents.
Initial definition of a GenAI model includes 4 main tabs:
Definition
Name: A descriptive name
Version: Current version of the model
Domain: Business domain for grouping of the model
Allowed For: List of runners which can provide the AI agent
Tags: Descriptive tags for the model
Status: Whether this model should be deployed or not
Instructions: Instructions to send to agent for its initialization (such as main purpose and persona)
Model
Provider: LLM provider that will be used by the agent
Memory Name: State manager which will be used as the chat memory by the agent (distributed states for distributed agents)
Memory Size: Maximum memory size to be kept for chat history
Parameters: Model level parameters specific to LLM provider, including sagas that are allowed access for the AI agents as tools
Toolkit
Tool Sagas: List of sagas that can be executed by the agent for custom business flows & logic, which allows agents to have access to all capabilities available within Rierino
Tool States: List of state managers that the agent is allowed to use for read and/or write operations (individual actions such as Delete can be also disabled)
Tool Systems: External systems that the agent is allowed to interact with, automatically discovering endpoints from Open API specification of the target system and using already configured authentication logic
Tool Things: Web of things that the agent is allowed to control, automatically discovering properties and actions from WOT discovery endpoint, using already configured 'thing' state or given inline 'thing' definition
Tool Scripts: List of programming languages the agent is allowed to use for creating custom scripts and executing them on the fly (such as for data transformations or calculations)
Prompts
ID: Unique identifier of the prompt used when calling AI agent
Name: Descriptive name of the prompt
Description: Detailed description of the prompt and its use cases
Template: Handlebars template for producing prompt with given structured inputs
Comments
List of comments for reference to model capabilities or changes over time.
Last updated