Huawei Open-Sources MACaQ: Quantizes 122B Model in 64 Min

Huawei's Cloud & AI lab released MACaQ, an Apache 2.0-licensed quantization framework that claims to handle a 122B-parameter model in 64 minutes using just 32.5GB of device memory. The project, published under the huawei-csl GitHub handle, targets text, image, video and audio models across llama.cpp, vLLM, Diffusers and PyTorch ecosystems. Officially launched on November 12, 2026, MACaQ arrives with pre-quantized model weights and performance benchmarks positioning it ahead of existing tools like MR-GPTQ and DuQuant++.

Multi-Modal Quantization in One Framework

MACaQ stands for Model Agnostic Calibrated Quantization, and the scope reflects that ambition. The framework supports 2/3/4-bit GGUF outputs for text models via llama.cpp, W4A4 and weight-only serving through vLLM, W4A4 quantization for image, video and audio generation in Diffusers, and W4A4 speech processing in PyTorch. The README lists Ascend CANN kernel integration shipping through active pull requests.

Supported models span multiple size classes and modalities. Text coverage includes Qwen3 variants at 4B, 27B, 30B, 35B and 80B parameters, plus OLMoE-1B-7B. Image and video generation targets include FLUX.2-klein-4B and Wan2.2-TI2V. Audio support covers Stable Audio 1.0. The project tagline reads "High-quality GGUF for local inference. Model-agnostic quantization beyond text."

Speed and Accuracy Claims

The core performance claim centers on a 122B-parameter model quantized in 64 minutes using 32.5GB of device memory. For comparison, the authors state MACaQ achieves "lower perplexity than MR-GPTQ and DuQuant++, or match their best Llama-3-8B perplexity with 11× less quantization time."

Benchmark data for Qwen3.8-27B under W4A4 quantization shows concrete speedups:

• 2.33x prefill speedup over BF16 baseline
• 2.58x decode speedup over BF16 baseline
• Perplexity drift from 2.8463 (BF16) to 2.8994 (W4A4)

A pre-quantized Qwen3.6-35B-A3B-MACaQ-R GGUF file ships alongside the framework code, allowing immediate testing without running the quantization pipeline.

Release Date and Availability

Officially released on November 12, 2026, MACaQ arrives under Apache 2.0 licensing. The project is available now on GitHub at the huawei-csl organization handle. An arXiv preprint accompanies the code, listing Muller, Bich, Boretti, Chang, Zhuang and Cavigelli as authors. The release coincides with a busy period for low-precision tooling development, landing days after llama.cpp merged FP8 KV-cache support for OCP-FP8 models.

Implementation Details

The framework architecture emphasizes calibration-based approaches across model types. For text models, the llama.cpp integration produces GGUF files at 2, 3 and 4-bit precision levels. The vLLM path supports both W4A4 (4-bit weights and activations) and weight-only quantization modes for production serving. Diffusers integration enables W4A4 quantization for image, video and audio generation pipelines. PyTorch support focuses on W4A4 speech processing workflows.

The project documents active work on Ascend CANN kernel integration, suggesting hardware-specific optimization paths beyond general CUDA or CPU targets. The 32.5GB memory footprint for 122B quantization implies efficient staging and streaming strategies during the quantization pass.

What This Means

MACaQ expands the open-source quantization toolkit with a multi-modal framework that claims speed and accuracy advantages over established tools. The 64-minute quantization time for a 122B model reduces iteration cycles for researchers and practitioners working with large models on memory-constrained hardware. The Apache 2.0 license removes adoption barriers for commercial use. For teams running local inference with llama.cpp or deploying production services through vLLM, the framework offers a path to 2-3x inference speedups with minimal quality loss. The pre-quantized model releases provide immediate benchmarking baselines without requiring quantization infrastructure setup.