GPU Llama3

Prerequisites

• Java 21 — required for the Vector API & TornadoVM (Java 25 supported via the -jdk25 artifact / llamaTornado script).
• TornadoVM with an OpenCL, PTX, CUDA, or Metal backend. llama-tornado/llamaTornado auto-detect whichever backend your installed SDK was built with.
• GCC/G++ 13+ — to build TornadoVM's native components.

`llamaTornado` — zero-dependency Java 25 script

Same backend auto-detection as llama-tornado. A single-file Java 25 launcher that replaces the Python script (needs java 25+ on your PATH):

./llamaTornado --gpu --verbose-init --metal \
  --model Mistral-7B-Instruct-v0.3.Q8_0.gguf --prompt "what is java"

1. Install a TornadoVM SDK (bundles the GPU runtime)

curl -s "https://get.sdkman.io" | bash && source "$HOME/.sdkman/bin/sdkman-init.sh" sdk install tornadovm tornado --devices # confirm your GPU is listed

2. Run a model on the GPU — no build required, via JBang

jbang gpullama3@beehive-lab -m beehive-llama-3.2-1b-instruct-fp16.gguf -p "Explain GPU acceleration in one sentence." ```

Grab a ready-to-run model from the Hugging Face collections below.

-----------

🛠️ Install & build

🚀 JBang — run without building

Script-like startup à la Jlama, powered by JBang:

```bash curl -Ls https://sh.jbang.dev | bash -s - app setup

🐳 Docker

Fully containerized GPU inference via pre-built images (docker-gpullama3.java):

docker run --rm -it --gpus all -v "$PWD":/data \
  beehivelab/gpullama3.java-nvidia-openjdk-opencl \
  /gpullama3/GPULlama3.java/llama-tornado \
  --gpu --verbose-init \
  --model /data/Llama-3.2-1B-Instruct.FP16.gguf --prompt "Tell me a joke"

-----------

⏱️ Quickstart (60 seconds)

```bash

Get TornadoVM (SDKMAN!, recommended)

TornadoVM is distributed via the official website and SDKMAN!. Pick a package matching your OS, architecture, and backend (opencl, ptx).

sdk install tornadovm
tornado --devices        # verify

Force a specific backend (only needed for multi-backend SDKs)

./llama-tornado --gpu --cuda \ --model beehive-llama-3.2-1b-instruct-fp16.gguf \ --prompt "Explain the benefits of GPU acceleration." ```

Swap in any tested model — e.g. beehive-llama-3.2-3b-instruct-fp16.gguf or ...-8b-....