AIMA AMD395 Qwen3.6 35B Linux Engine — Native Inference Engine

Why build this engine

The AMD Ryzen AI Max+ 395 packs a Radeon 8060S GPU and 128 GB of unified memory into a desktop-class device. It can run 35B-class BF16 models locally, but the Python environments, PyTorch versions, ROCm versions, and dynamic-library dependencies that come with general-purpose inference frameworks turn deployment into a long-term maintenance burden.

This project takes a narrower, more thorough path: not a general-purpose framework, but a deliverable native product built for one purpose: running Qwen3.6-35B-A3B on AMD395 Linux.

Product positioning

AIMA AMD395 Qwen3.6 35B Linux Engine is a batch-1 BF16 inference engine for fixed hardware and a fixed model. Instead of claiming it "theoretically runs," it focuses on clear product boundaries, reproducible performance numbers, and auditable release packages.

• Target hardware: AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151)
• Target model: Qwen3.6-35B-A3B BF16
• Target system: Linux x86-64
• Service interface: OpenAI-compatible Chat Completions API
• Open source license: Apache License 2.0

Core architecture

Portable native runtime package

The release package contains a static launcher, the native inference engine, pinned versions of ROCm, AOTriton, and Composable Kernel userspace dependencies, and its own glibc loader. The target machine does not need pre-installed Python, PyTorch, vLLM, Triton, Transformers, or system-level ROCm userspace.

The full runtime package is ~366 MiB uncompressed and ~101 MiB compressed. The Linux kernel driver and GPU architecture compatibility remain the responsibility of the host. Model weights must also be obtained separately by the user.

Model-resident and OpenAI-compatible interface

The engine loads the model once at startup, keeping weights, execution plans, KV/recurrent state, and prefix cache resident. The HTTP service supports:

• /v1/chat/completions
• Per-token SSE streaming
• OpenAI function tools, tool_choice, and parallel tool calls
• Assistant tool-call history and tool responses
• systemd resident service and clean shutdown

262K context, with clear boundaries

The engine validates fixed context matrices from 1K to 131K, plus the 262,144-token window endpoint. Cold requests must hit a published fixed context; longer requests only use prefix extension when strictly continuing an already-cached token prefix.

Rather than hide limits behind a vague "supports up to 262K," it encodes every available window and corresponding output length in a machine-readable product contract.

Measured performance

The numbers below come from the release engine running on the target AMD395 machine. Units are tokens/s.

Release criteria also cover:

• Maximum full-vocabulary KLD of 0.002174 across 9 contexts, below the 0.005 threshold, with top-1 all matching
• 128-token outputs from frozen test samples match token-by-token
• 8K context command-to-ready median of 44.69 s
• 32K context exact-prefix TTFT speedup of 2612x
• Streaming and non-streaming outputs produce identical token/text hashes

Full measurement records are kept in the repository's native release validation results.

Quick start

tar --zstd -xf aima-engine-native-portable-*.tar.zst
cd aima-engine-native-portable-*
 
./bin/aima-engine serve \
  --model-dir /srv/models/Qwen3.6-35B-A3B \
  --context-tokens 8192 \
  --host 127.0.0.1 \
  --port 8000

Once running, query it directly:

curl -fsS http://127.0.0.1:8000/health
curl -fsS http://127.0.0.1:8000/v1/models

Usage boundaries

• The target machine needs 128 GB of memory and a 96 GiB GTT pool configured as documented
• Only validated on Radeon 8060S / gfx1151; not a general-purpose AMD GPU runtime
• Model checkpoints are not distributed with the repository or release package
• The HTTP service has no built-in authentication and exposes /shutdown; by default it should only listen on 127.0.0.1
• Performance numbers represent only the hardware, model, context, and release version stated in the repository

Author and repository relationship

The project was created and is maintained by 关嘉伟 / Jiawei Guan（@skyguan92）.

• Personal original upstream: skyguan92/AIMA-AMD395-Qwen36-35B-Linux-Engine
• Organization fork and official showcase version: Approaching-AI/AIMA-AMD395-Qwen36-35B-Linux-Engine

Both repositories remain public; GitHub commit history, Python package metadata, and CITATION.cff use the same skyguan92 author identity.