React Native LiteRT-LM

Requirements

Installation

npm install react-native-litert-lm react-native-nitro-modules

Example App

The example/ directory contains a fully functional test app with a dark-themed diagnostic UI that demonstrates:

• Model downloading with progress tracking
• Text inference (blocking and streaming)
• Multi-turn conversation with context retention
• Performance benchmarking (tokens/sec, latency)
• Real-time memory tracking
• Speculative decoding & tool calling settings toggles
• Zero-copy multimodal inference loading images/audio directly into ArrayBuffers

Running the Example

1. Build the library (compiles TypeScript to lib/):

   npm run build
   

1. Install example dependencies:

   cd example
   npm install
   

1. Create a development build and run:

   npx expo prebuild --clean
   npx expo run:android  # Android
   npx expo run:ios      # iOS (pre-linked with CLiteRTLM.xcframework)
   

Note: If you change native code (Swift/Kotlin), you must run npx expo prebuild --clean again before rebuilding.

Usage

Manual Usage

import { createLLM } from "react-native-litert-lm";

const llm = createLLM();

// Load a model from URL (auto-downloads) or local path
await llm.loadModel("https://example.com/model.litertlm", {
  backend: "gpu",
  systemPrompt: "You are a helpful assistant.",
});

// Generate a response
const response = await llm.sendMessage("What is the capital of France?");
console.log(response);

// Clean up
llm.close();

`getMemoryUsage(): MemoryUsage`

Returns real OS-level memory usage.

Demo

Gemma 4 E2B running on-device on a Samsung Galaxy S22 (Snapdragon 8 Gen 1, 4 GB RAM) — CPU backend, streaming inference.

<video src="https://github.com/user-attachments/assets/1da527ce-0432-4f8b-8899-474f81b2feea" width="300" controls></video>

`createLLM(options?): LiteRTLM`

Creates a new LLM inference engine instance.

• options.enableMemoryTracking — enable automatic memory snapshot recording
• options.maxMemorySnapshots — max number of snapshots to retain (default: 256)

`loadModel(path, config?): Promise<void>`

Loads a model from a local path or HTTPS URL.

Backend Options

iOS: Both 'cpu' and 'gpu' (Metal) are supported. The engine automatically tries fallback backend combinations if the primary one fails. Android GPU: The GPU backend requires OpenCL, which is not available on most Samsung and Qualcomm devices. Use checkBackendSupport('gpu') to check before loading. The engine will throw a clear error if GPU is unsupported.

API Reference

Supported Models

All exported model URLs are public — no authentication required. Pass them directly to useModel() or loadModel() for automatic downloading with progress tracking and local caching.

Recommended: Use GEMMA_4_E2B_IT for most use cases — multimodal (text + vision + audio) and the best quality-to-size ratio. iOS Note: Models larger than ~2 GB require the com.apple.developer.kernel.extended-virtual-addressing entitlement. See iOS Entitlements below. Gemma 3n E2B (~1.3 GB) works without it.

Other compatible models (download .litertlm files manually from HuggingFace):

iOS FFI Architecture & Integration

The library uses a highly optimized Swift Direct-FFI bridge that links directly with the pre-compiled C library CLiteRTLM.xcframework.