pmetal train

Fine-tune a model with LoRA/QLoRA/DoRA. Supports SFT on any supported architecture with automatic hardware detection and kernel tuning.

Usage

pmetal train \
  --model <MODEL> \
  --dataset <DATASET> \
  --output <OUTPUT_DIR> \
  [OPTIONS]

Examples

# Basic LoRA fine-tuning
pmetal train \
  --model Qwen/Qwen3-0.6B \
  --dataset train.jsonl \
  --output ./output \
  --lora-r 16 --batch-size 4 --learning-rate 2e-4

# QLoRA with 4-bit quantization
pmetal train \
  --model meta-llama/Llama-3.2-1B \
  --dataset train.jsonl \
  --output ./output \
  --quantization nf4 --lora-r 16

# DoRA with custom schedule
pmetal train \
  --model Qwen/Qwen3-0.6B \
  --dataset train.jsonl \
  --dora --lr-schedule cosine_with_restarts

# From a config file
pmetal train --config training.yaml

Parameters

Parameter	Default	Description
--model	required	HuggingFace model ID or local path
--dataset	required	Path to training dataset (JSONL, Parquet, CSV)
--output	./output	Output directory for weights and logs
--lora-r	16	LoRA rank
--lora-alpha	32.0	LoRA scaling factor (2× rank)
--batch-size	1	Micro-batch size
--learning-rate	2e-4	Learning rate
--max-seq-len	0	Max sequence length (0 = auto-detect)
--epochs	1	Number of training epochs
--max-grad-norm	1.0	Gradient clipping
--quantization	none	QLoRA method: nf4, fp4, int8
--gradient-accumulation-steps	4	Gradient accumulation steps
--no-ane	false	Disable ANE training
--embedding-lr	None	Separate LR for embeddings
--no-metal-fused-optimizer	false	Disable Metal fused optimizer
--lr-schedule	cosine	constant, linear, cosine, cosine_with_restarts, polynomial, wsd
--no-gradient-checkpointing	false	Disable gradient checkpointing
--gradient-checkpointing-layers	4	Layers per checkpoint block
--warmup-steps	100	Learning rate warmup steps
--weight-decay	0.01	AdamW weight decay
--no-sequence-packing	false	Disable sequence packing
--dora	false	Enable DoRA (Weight-Decomposed LoRA)
--config	—	Path to YAML configuration file

Dataset Formats

Training data is auto-detected:

• ShareGPT: {"conversations": [{"from": "human", "value": "..."}, ...]}
• Alpaca: {"instruction": "...", "input": "...", "output": "..."}
• OpenAI/Messages: {"messages": [{"role": "user", "content": "..."}, ...]}
• Reasoning: {"problem": "...", "thinking": "...", "solution": "..."}
• Simple: {"text": "..."}
• Parquet: Standard text columns or reasoning formats

Output

Training produces:

• lora_weights.safetensors — LoRA adapter weights
• training_metrics.jsonl — Per-step metrics log
• checkpoint/ — Resumable checkpoints (if training is interrupted)

See Also

• Configuration File — YAML config format
• Training Overview — All training methods
• Supported Models — LoRA-compatible architectures