SDKActive

ONNX Runtime GenAI Java API

Java API for ONNX Runtime GenAI, providing model loading, tokenization, and generation.

Open source page ↗

Field note

What it does

The Java API for ONNX Runtime GenAI enables AI model inference with features like model management, tokenization, and text generation. It is in preview and subject to change.

Capabilities

Available capabilities

No use cases filed yet.

Tags

Tags

JavaGenerative AIONNX Runtime

Ways to use it

Ways to use it

sdk

Java / ai.onnxruntime.genai

Product features

Product features

Configuration

Manage providers and options via the Config class.

Model Loading

Load a model from a path using the Model class.

Streaming

Stream tokenization using TokenizerStream.

Text Generation

Generate output sequences using the Generator class.

Tokenization

Encode and decode text using the Tokenizer class.