Meta releases Muse Spark Open model with Apple Silicon optimization

Meta has introduced Muse Spark Open, an open-weight multimodal model optimized for complex software engineering and agentic workflows. The model features 816 billion total parameters, utilizing a mixture-of-experts architecture that activates 42 billion parameters per token. This design aims to provide deep reasoning capabilities while maintaining computational efficiency, supported by a 1-million-token context window capable of processing entire codebases and interleaved modalities including text, audio, video, and images.

To facilitate local execution on Apple hardware, Meta collaborated with Apple to leverage the MLX framework. The model is optimized to run across clusters of Apple M5 Ultra Mac Studios. By utilizing a pooled memory architecture, multiple Mac Studios can be linked via Thunderbolt to create a unified memory environment, allowing the large parameter weight file to reside on-device. The system relies on the M5 Ultra's high memory bandwidth to manage the routing of active tokens to specialized experts without creating bottlenecks.

Performance metrics indicate that a two-device cluster of M5 Ultra Mac Studios delivers a throughput of approximately 20 tokens per second, while a four-device configuration increases this to roughly 36 tokens per second. The implementation uses dynamic weight pulling to map model weights directly into Apple Silicon memory spaces, bypassing intensive local format conversions. The model requires a minimum of 359 GB of memory to function in this local configuration.

Beyond the Apple ecosystem, Muse Spark Open is designed for broader deployment. It utilizes custom Helion GPU kernels integrated into SGLang to improve performance on NVIDIA and AMD hardware, with reported speedups for various configurations. Developers can also utilize the PyTorch TorchTitan framework for distributed fine-tuning and structural customization. The model is additionally available for cloud-based deployment through Amazon SageMaker JumpStart.

The model's architecture includes a 1.8-billion-parameter perception encoder that natively processes multiple modalities in a single pass, enabling reasoning over screenshots, error logs, and video walkthroughs. Meta positions the model as a tool for long-horizon tasks, such as planning multi-file refactors and coordinating subagents, which require sustained, multi-step reasoning beyond the capabilities of standard coding models.