SDKActive

ONNX Runtime GenAI Python API

Python API for ONNX Runtime GenAI, enabling generative AI model inference with classes for models, tokenizers, generators, and more.

Open source page ↗

Field note

What it does

Python API for ONNX Runtime GenAI, delivered via the onnxruntime-genai package. Provides Model, Config, GeneratorParams, Generator, Tokenizer, TokenizerStream, NamedTensors, Tensor, Adapters, and MultiModalProcessor classes for loading and running generative AI models.

Capabilities

Available capabilities

No use cases filed yet.

Adapters

Load and unload adapters for model fine-tuning.

Generation

Generate tokens with configurable search options and guidance.

Model Loading

Load a model from a config path or Config object.

Multimodal Processing

Create a MultiModalProcessor for handling images and audio.

Tokenization

Encode and decode text using Tokenizer and TokenizerStream.