Apple Foundation Models: On-Device LLM in Your iOS App#

Why Apple put an LLM inside iOS at all#

A year ago, at WWDC 2025, Apple opened up access to its own on-device language model — the same one that powers Apple Intelligence — through a new framework called Foundation Models. No API key, no per-token bill, no user data leaving the device. The model already sits on the phone, and with iOS 26 / macOS 26 / iPadOS 26 / visionOS 26 you can call it from your own code in a few lines of Swift.

That's not marketing copy — it genuinely changed which features make sense to build into an app. I'm a working iOS developer, and this isn't a keynote recap: it's the APIs I've actually tested, the limits you actually hit, and an honest take on when on-device LLM is the right call and when it isn't.

By mid-2026 the framework has already gone through another cycle: at WWDC 2026 Apple introduced the third generation of Apple Foundation Models (AFM 3) and, separately, the ability to plug third-party LLM providers into the same Swift API — the subject of a dedicated session, "Bring an LLM provider to the Foundation Models framework." That's a meaningful signal: Apple is turning Foundation Models from a wrapper around its own model into a unified interface for on-device and hybrid inference.

The base on-device model is dense, around 3 billion parameters (Apple calls it AFM 3 Core in the current generation). It's tuned for a specific set of tasks: summarization, entity extraction, text understanding and refinement, short dialog, short creative generation. It is not a general-purpose chatbot and not a search engine replacement — Apple's own documentation is explicit that it isn't meant to be a source of general world knowledge.

On some devices — in the current generation, the ones with 12GB of RAM, i.e. the top-tier iPhones — an extended model is available: AFM 3 Core Advanced, roughly 20 billion parameters with a sparse architecture that activates only 1–4 billion parameters per request. For developers this means the same line of code can run on a noticeably different "brain" depending on the device, and you shouldn't expect identical output quality between an iPhone 15 Pro and an iPhone 17 Pro.

Working with the model revolves around two types:

• SystemLanguageModel — the entry point to the model, its availability, and specialized modes (useCase, e.g. .contentTagging for out-of-the-box tagging and entity extraction).
• LanguageModelSession — the conversation session: this is what you actually send requests through, with history (transcript), instructions, and tools.

Availability gating: three reasons it "doesn't work" — and they're not the same problem#

The most common mistake in Foundation Models write-ups is treating unavailability as a single state. In practice SystemLanguageModel.default.availability returns one of three genuinely different reasons, and conflating them is a guaranteed way to ship a feature that feels broken:

import FoundationModels
 
func checkFoundationModelsAvailability() -> String {
    switch SystemLanguageModel.default.availability {
    case .available:
        return "✅ Model is ready to use"
 
    case .unavailable(.deviceNotEligible):
        // A13 or older — this feature will never appear on this device
        return "❌ Device doesn't support Apple Intelligence"
 
    case .unavailable(.appleIntelligenceNotEnabled):
        // Eligible device, but Apple Intelligence is off in Settings
        return "⚠️ Enable Apple Intelligence in Settings"
 
    case .unavailable(.modelNotReady):
        // Model is still downloading — this is temporary
        return "⏳ Model is downloading, try again later"
 
    case .unavailable(let reason):
        return "❓ Unavailable: \(reason)"
    }
}

deviceNotEligible is permanent: old hardware, hide the feature and don't nag about it. appleIntelligenceNotEnabled is a user-controlled setting — one polite prompt to enable it is fine. modelNotReady is a temporary download state — retry, don't show an error. Merging these three scenarios into one generic "feature unavailable" screen is the single most common source of "the AI is broken" complaints.

Supported platforms as of this writing: iOS, iPadOS, macOS, visionOS. watchOS and tvOS are not part of the framework.

Structured output: @Generable instead of hand-parsing JSON#

Before Foundation Models, structured LLM output meant asking the model to return JSON, parsing it by hand, and hoping it wouldn't add stray text before or after the braces. The @Generable macro removes that entire class of bugs: it generates a schema at compile time, and the model is guaranteed to return a value of the actual Swift type.

import FoundationModels
 
@Generable
struct TripSummary {
    @Guide(description: "Short trip title, up to 6 words")
    var title: String
 
    @Guide(description: "Key highlights of the trip", .count(3))
    var highlights: [String]
 
    @Guide(description: "Overall mood score from 1 to 5")
    var moodScore: Int
}
 
func summarizeTrip(notes: String) async throws -> TripSummary {
    let session = LanguageModelSession(
        instructions: "You are an assistant that briefly summarizes travel notes."
    )
 
    let response = try await session.respond(
        to: "User notes: \(notes)",
        generating: TripSummary.self
    )
 
    return response.content
}

@Guide attaches to a field not just a natural-language description for the model, but programmatic constraints — .count(), .maximumCount(), and others that genuinely narrow the generation space rather than just "asking nicely."

Streaming and tool calling#

For a responsive UI, structured output can be streamed piece by piece — the type generated by @Generable automatically gets a "partial" version with optional fields:

func streamTripSummary(notes: String) async throws {
    let session = LanguageModelSession()
 
    let stream = session.streamResponse(
        to: "Summarize this trip from the notes: \(notes)",
        generating: TripSummary.self
    )
 
    for try await partial in stream {
        // partial: TripSummary.PartiallyGenerated — fields are optional,
        // filled in progressively, convenient for live UI updates
        print(partial)
    }
}

And if the model needs data that isn't in the prompt — say, the user's current city — it can call a tool you've described through the Tool protocol:

struct CurrentLocationTool: Tool {
    let name = "getCurrentLocation"
    let description = "Returns the user's current city"
 
    @Generable
    struct Arguments {
        @Guide(description: "Whether neighborhood-level precision is needed")
        var preciseArea: Bool
    }
 
    func call(arguments: Arguments) async throws -> ToolOutput {
        let city = arguments.preciseArea ? "Lisbon, Alfama" : "Lisbon"
        return ToolOutput(city)
    }
}
 
let session = LanguageModelSession(
    tools: [CurrentLocationTool()],
    instructions: "Use getCurrentLocation when you need the user's city."
)

The model itself decides whether to call the tool, based on its description — a fundamentally different mental model than the manual intent routing most people are used to from classic NLP pipelines.

Where the line is: context, tasks, and hardware#

The main practical constraint is the size of the session's context window. Per Apple's technote TN3193 on managing the on-device model's context window, a session's limit is roughly 4,096 tokens for the entire conversation, including instructions, history, and room for the response. That's orders of magnitude smaller than cloud models like GPT or Claude, and it means long documents, extensive chat history, or a hefty few-shot prompt simply won't fit in one session — you need to summarize, trim, or start a new session.

Get close to the limit and the model can start failing before you formally exceed it — it may physically lack room to generate a response. Practical rule: budget headroom for the response itself, don't spend the entire window on input.

The second constraint is quality. This isn't a general-purpose chatbot or a source of world facts: a 3-billion (or even sparse 20-billion) parameter model is good at narrowing and transforming text you already have, not at open-ended "tell me about..." questions. The third is hardware fragmentation: the extended AFM 3 Core Advanced model isn't available on every Apple Intelligence device — only where there's enough RAM. Design your UX so degrading to the base model doesn't break the flow.

On-device LLM or the cloud: a table for an honest choice#

The practical takeaway: on-device LLM is a tool for specific, bounded tasks with high privacy requirements and zero cost — not a universal replacement for a cloud API.

Not every task needs an LLM — and a checklist to start with#

Working on MeteoHealth (/projects/meteohealth/) — an app that connects weather conditions to how you feel — I deliberately chose a classical statistical engine running on-device instead of any ML or LLM model. Correlations between pressure, humidity, and a user's symptoms are computed with deterministic statistical methods: the result is reproducible, explainable, and doesn't cost a single token of generative text.

That's not a compromise born out of limited resources — it's a deliberate choice of tool for the task. An LLM, even on-device, adds unpredictability exactly where transparent logic is needed: "if pressure dropped by X hPa, migraine risk went up." Foundation Models is a great fit for tasks where the input is unstructured text and the output is either structured or also text: summarizing notes, extracting entities, generating titles, short in-app conversational assistants. For deterministic computation over numeric series, it's not needed, and bolting an LLM onto something plain arithmetic already handles is complexity without benefit.

Checklist before you add Foundation Models#

1. Check all three unavailable reasons separately — they call for different UX, not one error screen.
2. Use @Generable/@Guide instead of hand-parsing JSON — it eliminates an entire class of bugs.
3. Budget the context window (~4,096 tokens) up front, not once sessions start failing.
4. Don't expect encyclopedic knowledge from a 3B model — it's a text-transformation tool, not a general chatbot.
5. Before reaching for an LLM, ask whether plain code without a generative model already solves the problem. Sometimes statistics beats generation.

Foundation Models isn't hype for hype's sake: a free, private, offline-capable LLM in a few lines of Swift is something developers have wanted for years. Which is exactly why it matters to use it precisely — where it genuinely beats the alternatives, not because "we have AI too."

Useful links:

• Apple Developer: Foundation Models framework
• WWDC25 — Meet the Foundation Models framework
• TN3193 — Managing the on-device foundation model's context window