Powdered Metal

PMetal

The Apple Silicon ML platform for training, serving, quantizing, merging, and operating local models.

A Rust workspace with a zero-allocation MLX bridge, custom Metal and ANE paths, TurboQuant long-context serving, multi-Mac distributed training, CLI/TUI/GUI surfaces, and Rust plus Python SDKs.

Get Started Supported Models

0.5.0

current release

TUI workspaces

4-6x

KV cache compression

M1-M5

Apple Silicon

pmetal

$ pmetal serve --model Qwen/Qwen3-0.6B \

--continuous-batch --kv-turboquant-preset q3_5

ready OpenAI, Anthropic, embeddings, shared prefix cache

$ pmetal pretrain --arch qwen --shards data/*.bin --steps 10000

loss 2.183 · 41.8k tok/s · checkpoint ./pretrain-output

$ pmetal cluster status

Thunderbolt ring: yes · peers: 2 · fabric: thunderbolt

Runtime

Native Apple Silicon Stack

Zero-allocation MLX bridge, custom Metal kernels, ANE paths, Metal 4 / MPP dispatch on M5, and tier-aware tuning from M1 through M5.

Training

Fine-Tuning to Pretraining

SFT, LoRA, QLoRA, DoRA, GRPO/DAPO, RLKD, distillation, embedding training, and full-parameter pretraining share one job/event substrate.

Operations

Every Surface Stays in Sync

CLI commands, a 20-workspace terminal TUI, a 19-screen Tauri GUI, MCP tools, Rust builders, and Python bindings map to the same specs.

Every Interface You Need

A 19-screen desktop GUI, a 20-workspace terminal TUI, CLI commands, MCP tooling, and SDK APIs all backed by the same Rust job specs and event stream.

PMetal desktop GUI showing the training dashboard with live loss curves, model configuration, and dataset management

Desktop GUI — Tauri + Svelte screens for training, pretraining, serving, jobs, inference, merging, quantization, and evaluation.

PMetal terminal TUI showing live training metrics with braille sparklines, device info, and job history

Terminal TUI — 20 workspaces with device info, tokenization, training, DFlash, serving, benchmarking, and live job logs.

Ship from CLI, Rust, or Python

Run the full application suite from the terminal, embed the high-level builders, or drop to lower-level crates when you need full control.

CLI Rust Python

# Long-context OpenAI + Anthropic-compatible serving
pmetal serve   --model Qwen/Qwen3-0.6B   --continuous-batch   --kv-turboquant-preset q3_5   --cb-max-slots 8

# Multi-Mac training over the fastest discovered fabric
pmetal cluster status
pmetal train   --model Qwen/Qwen3-0.6B   --dataset train.jsonl   --distributed-auto   --compression-strategy fp16

use pmetal::data::Tokenizer;
use pmetal::data::chat_templates::{detect_chat_template, Message};
use pmetal::hub::resolve_model_path;
use pmetal::models::{generate_cached_async, DynamicModel, GenerationConfig};

let model_id = "Qwen/Qwen3-0.6B";
let model_path = resolve_model_path(model_id, None, None).await?;
let tokenizer = Tokenizer::from_model_dir(&model_path)?;
let template = detect_chat_template(&model_path, model_id);
let prompt = template.apply(&[Message::user("What is 2+2?")]).text;
let input_ids = tokenizer.encode_with_special_tokens(&prompt)?;

let mut model = DynamicModel::load(&model_path)?;
let mut cache = model.create_cache(input_ids.len() + 256);
let mut mamba_cache = model.create_mamba_cache();

let output = generate_cached_async(
    |input, cache| {
        model.forward_with_hybrid_cache(input, None, Some(cache), mamba_cache.as_mut())
    },
    &input_ids,
    GenerationConfig::sampling(256, 0.7),
    &mut cache,
)?;

import pmetal

# Fine-tune with sensible defaults
result = pmetal.finetune(
    "Qwen/Qwen3-0.6B",
    "train.jsonl",
    lora_r=16,
    learning_rate=2e-4,
    epochs=3,
)
print(f"Loss: {result['final_loss']}")

# Inference with LoRA adapter
text = pmetal.infer(
    "Qwen/Qwen3-0.6B",
    "Explain quantum entanglement",
    lora="./output/lora_weights.safetensors",
)
print(text)

Built for Performance

PMetal 0.5.0 spans native inference, long-context serving, training, distributed execution, model operations, and hardware-specific kernels.

Serving

TurboQuant KV Cache

Random rotation, Lloyd-Max quantization, QJL residuals, and direct decode attention for 4-6x long-context cache compression.

Serving

Continuous Batching

OpenAI-compatible serving with paged-KV-style admission, shared prefix cache, concurrent decode slots, and Anthropic messages support.

Training

Full Pretraining

Tokenize JSONL corpora into binary shards, train dense or MoE models from scratch, and resume checkpoints with optimizer state.

Training

SOTA Distillation

ULD, GKD, MiniLLM, skewed JSD, hidden-state transfer, offline teacher-logit caches, and TAID observability.

Cluster

Distributed Macs

mDNS discovery, Thunderbolt-first fabric selection, ring all-reduce, pipeline harnesses, and gradient compression.

Models

Modern Model Merging

SLERP, TIES, DARE, Fisher, RegMean, MoE expert alignment, dtype-aware saves, sidecar copy, and sanity checks.

Models

Broad Quantization

GGUF export across K-quants, legacy GGML types, Q1/TQ/MXFP4/NVFP4/BF16, plus MLX-format target-bpw quantization.

Hardware

Metal 4 / MPP Backend

Apple10/M5 dispatch, MPP-optimized GEMM, attention, MoE, distillation, LoRA, cross-entropy, and persisted backend tuning.

Model Support

Load from HuggingFace Hub or local safetensors. Architecture detection is automatic, and dropped legacy families are intentionally no longer listed.

Causal Language Inference

Family	Variants	model_type
Llama	2, 3, 3.1, 3.2, 3.3	`llama, llama3`
Llama 4	Scout, Maverick	`llama4`
Qwen 2	2, 2.5	`qwen2, qwen2_5`
Qwen 3	3	`qwen3`
Qwen 3 MoE	3-MoE	`qwen3_moe`
Qwen 3.5	3.5 (Next)	`qwen3_next, qwen3_5`
DeepSeek	V3, V3.2, V3.2-Speciale	`deepseek, deepseek_v3`
Mistral	7B, Mixtral 8×7B	`mistral, mixtral`
Gemma	2, 3	`gemma, gemma2, gemma3`
Gemma 4	4 text path	`gemma4, gemma4_text`
Phi 3	3, 3.5	`phi, phi3`
Phi 4	4	`phi4`
Cohere	Command R	`cohere, command_r`
Granite	3.0, 3.1, Hybrid MoE	`granite, granitehybrid`
NemotronH	Hybrid (Mamba+Attn)	`nemotron_h`
GPT-OSS	20B, 120B	`gpt_oss, gpt-oss`

Other Model Families

Family	Capability	Status
BERT	Embeddings / encoders	BERT, RoBERTa, DistilBERT, XLM-RoBERTa
Flux	Diffusion	1-dev and 1-schnell dispatcher + pipeline
Vision / speech modules	Rust modules	Pixtral, Qwen2-VL, MLlama, CLIP, Whisper, T5

LoRA / QLoRA Training

Architecture	LoRA	QLoRA	Notes
Llama	Yes	Yes	Covers Llama 2–3.3. Gradient checkpointing.
Llama 4	Yes	Yes	Scout/Maverick adapters.
Qwen 2	Yes	Yes	Uses Qwen3 LoRA implementation internally.
Qwen 3	Yes	Yes	Gradient checkpointing supported.
Qwen 3 MoE	Yes	Yes	Sparse MoE support.
Qwen 3.5	Yes	Yes	Hybrid Next architecture.
Gemma	Yes	Yes	GeGLU activation, special RMSNorm.
Gemma 4	Yes	Yes	Text path support.
Mistral	Yes	Yes	Sliding window attention support.
Phi 3/4	Yes	Yes	Partial RoPE, fused gate_up.
DeepSeek	Yes	Yes	V3-family support.
Cohere	Yes	Yes	Command R support.
Granite	Yes	Yes	Dense and hybrid variants.
NemotronH	Yes	Yes	Hybrid architecture support.
GPT-OSS	Yes	Yes	MoE variants.

Install in Seconds

Prebuilt signed binaries, crates.io, or source builds. Requires macOS on Apple Silicon; serving and MCP are opt-in Cargo features.

Binary Cargo Source

# Download the latest release
curl -fsSL https://github.com/Epistates/pmetal/releases/latest/download/pmetal-aarch64-apple-darwin.tar.gz | tar xz
sudo mv pmetal /usr/local/bin/

# Install from crates.io
cargo install pmetal

# Add optional HTTP serving and MCP tools
cargo install pmetal --features "serve,mcp"

# Build from source
git clone https://github.com/epistates/pmetal.git && cd pmetal
cargo build --release

# Build the GUI (requires bun)
cd crates/pmetal-gui && bun install && bun tauri build