Introduction

mach is an inference engine for Apple Silicon. It is the reference way to run Syzygy Research compressed models (~2-bit expert packs) on M-series Macs: point it at a HuggingFace repo id, and it downloads the pack and serves it with fused Metal kernels that decode the quantized weights inside the matrix-vector product. No dequantized copy of the model ever exists in memory, so a 35B-parameter model serves in about 8 GB of weights and fits a 16 GB Mac.

The engine is standalone: the decode loop, samplers, tokenizer, KV caches, and transformer blocks live in-tree, with mlx as the only framework dependency. Source is at SyzygyResearch/mach-engine (Apache-2.0).

Requirements

Quick start

pip install mach-engine

# One-shot generation. Downloads the pack from HuggingFace on first use (~8.1 GB)
mach-generate SyzygyResearch/Mach-1-Additive-35B \
  --prompt "Write a binary search in Python."

# Serve an OpenAI-compatible endpoint
mach-serve SyzygyResearch/Mach-1-Additive-35B --port 8080

Then point any OpenAI-compatible client at http://127.0.0.1:8080/v1 with any API key string:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mach-1",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'

The server also speaks the Anthropic Messages API at /v1/messages. See Serving for endpoints, streaming, tool calling, and image input.

Supported models

These are the officially published and tested packs. Admission is based on the Mach-1 pack format, not a repo-name allowlist, but other architectures and stock HuggingFace checkpoints are not supported. Run mach-check <repo-or-path> to find out, without loading weights, whether a checkpoint is servable and fits your machine's memory.

Documentation