Apple integrates on-device foundation language and vision models into macOS, iOS, and iPadOS, offering privacy-focused local AI processing with minimal system memory overhead. On older Apple Silicon hardware such as M1 and M2 Macs, the standard Apple Foundation Model handles tasks like grammar proofreading, document summarization, and OCR without high RAM requirements. On M3 Macs and newer iPhones, the Apple Foundation Advanced model features a 20-billion-parameter Mixture-of-Experts architecture for enhanced capabilities. Users can access these models through built-in macOS writing tools, third-party App Store applications like Locally, or directly via the command line using the built-in fm terminal utility. Furthermore, Apple Shortcuts supports local model actions, enabling users to capture receipt images on an iPhone and extract merchant, date, and total amount details into a CSV spreadsheet stored locally or in iCloud. Developers can also use frontier cloud models like ChatGPT to generate custom Swift applications that interface directly with the local Apple Foundation Models, combining automated app creation with local, private execution.
Apple Foundation Models are integrated directly into macOS, iOS, and iPadOS operating systems for local AI processing. The built-in local AI models operate with low RAM overhead, allowing background tasks without closing active applications.
Devices with M3 chips or newer access the Apple Foundation Advanced model, which utilizes a 20-billion-parameter Mixture of Experts architecture. Users can interact with local Apple models using built-in system writing tools, the free third-party app Locally, or command-line commands via the terminal.
Apple Shortcuts integrates on-device AI actions, enabling automated photo receipt scanning and CSV data export without sending documents to cloud servers. Developers can prompt frontier cloud models like ChatGPT to code custom native Swift apps that run entirely on local Apple Foundation Models.
Chapter guide
Worth noting
- The video includes a paid sponsorship promotion for Gusto payroll services between 01:13 and 02:15.
- On-device local foundation models have limited context windows (around 4,000 tokens) compared to large cloud-based LLMs.
- Initial setup for custom Swift applications generated via ChatGPT requires external cloud AI access for code generation.