On-Device Speech Transcription in iOS: WhisperKit vs SFSpeechRecognizer vs SpeechAnalyzer#

When I was designing the speech recognition layer for Lanternly — a journaling app I'm currently building — the requirement was simple and uncompromising at the same time: a voice note turns into text right on the phone, and not a single byte of audio is allowed to leave the device. A journal is one of the most personal categories of data that exists, and "we encrypt everything on the server" doesn't sound convincing here — not to me, and not to the user. The only honest answer is not having a server for this operation at all.

Over the last couple of years, three realistic ways to do this have appeared on iOS: the good old SFSpeechRecognizer with its on-device flag, the third-party WhisperKit engine from Argmax built on top of Whisper models, and the brand-new SpeechAnalyzer from iOS 26. This article — no marketing, just numbers and code — breaks down how they differ and how I chose between them.

Why on-device transcription matters at all#

Cloud transcription is the easiest path: send the audio to an API, get text back, next line of code. But it has a cost developers like to gloss over. First, privacy: the voice reaches someone else's server, and no matter what the provider's privacy policy says, legally and technically it's no longer "on your device only." Second, network dependency: a voice memo recorded on the subway with no signal simply won't transcribe. Third, cost — every minute of audio through a cloud API is a bill that grows with your user base.

For an app like a journal, the first point isn't up for debate at all — it's not a technical preference, it's a condition for the product to exist. So the question wasn't "local or cloud," it was "which local engine."

Three engines on the table#

SFSpeechRecognizer — part of the Speech framework, available since iOS 10. It has a requiresOnDeviceRecognition flag that forces recognition to run entirely on-device, without contacting Apple's servers. The model is already bundled with the system for supported languages — developers don't need to download anything.

WhisperKit — an open-source Swift package from Argmax that ports OpenAI's Whisper models to Core ML and the Apple Neural Engine. In 2026, Argmax merged WhisperKit with SpeakerKit (diarization) and TTSKit into a single open-source SDK, but the transcription core stayed the same: you pick and download the model size you need, and WhisperKit runs it locally with ANE-specific optimization.

SpeechAnalyzer / SpeechTranscriber — a new modular API in the Speech framework, introduced in iOS 26. It isn't a cosmetic update to SFSpeechRecognizer, but a different architecture: an async API built on AsyncSequence, with separate modules — SpeechTranscriber (long-form speech), DictationTranscriber (short utterances, the old recognizer's equivalent), and SpeechDetector (voice activity detection).

WhisperKit: what's under the hood#

WhisperKit doesn't train its own model — it takes ready-made Whisper weights (tiny, base, small, medium, large-v3, and the compact large-v3-turbo) and converts them to Core ML, distributing computation across the CPU, GPU, and Neural Engine. The size difference between models is huge: tiny is about 40 MB and is mostly useful for debugging, while large-v3 is around 1.5 GB. The practical compromise typically used in production on iPhone 13 and newer is large-v3-turbo: near full large-v3 accuracy at roughly five times the throughput, at around 600 MB on disk.

WhisperKit's key engineering advantage isn't the Whisper model itself (it's open and anyone could plug it in) — it's the ANE optimization: independent benchmarks show an extra 1.3–1.8x speedup on top of a plain Metal run, especially noticeable on the M3/M4 chip lineup. On iPhone 13 and newer this makes it possible to stream transcription in real time instead of waiting for the recording to finish.

Whisper's biggest advantage as a model is multilingual support out of the box: it's trained on 99 languages simultaneously, including Russian, and handles mid-sentence language switching naturally — "the meeting was in Russian, with a term and a half in English" is business as usual for Whisper, not an edge case.

SpeechAnalyzer: Apple's new trump card#

SpeechAnalyzer isn't just another API — it's Apple's answer to the fact that general-purpose models like Whisper were outperforming the system recognizer on quality. And the answer is strong: on the clean slice of the LibriSpeech dataset, SpeechAnalyzer achieves a word error rate (WER) of around 2.12%, and around 4.56% on the noisy slice. For comparison, Whisper Small scores around 3.74% on the same clean slice — meaning on English, Apple's new system engine objectively beats an open model of comparable size.

But that win comes with an important limitation worth stating honestly: as of iOS 26, the languages supported by SpeechTranscriber cover mostly the languages Apple has long invested in for Siri — English, Spanish, French, German, Italian, Portuguese, Chinese, Japanese, Korean. Russian isn't on that list. For an app that needs transcription in Russian (like Lanternly), that takes SpeechAnalyzer off the table immediately, no matter how impressive its English accuracy is.

One more architectural detail: SpeechAnalyzer's language models aren't baked into your app binary — they're managed by the system through AssetInventory and fetched on demand. That shrinks your app bundle, but adds a dependency on the right language asset actually existing and being available on the user's device.

An honest comparison#

The table below isn't "which engine is best" — it's which engine is best for which job. All three earn their place in different scenarios.

In practice: what it looks like in code#

Below is a basic WhisperKit integration skeleton: initialization, picking a model based on device memory, transcribing an already-recorded file, and streaming during recording. This is illustrative but true-to-spirit code — exactly the seam where I designed Lanternly's engine to be swappable, without touching the call site above it.

import WhisperKit
 
final class LocalTranscriber {
    private var pipeline: WhisperKit?
 
    /// Initializes WhisperKit, downloading and warming up the model
    func setUp() async throws {
        let config = WhisperKitConfig(
            model: "large-v3-turbo",   // compact yet accurate option
            downloadBase: nil,         // model is pulled from the Hugging Face Hub
            verbose: false,
            logLevel: .none,
            prewarm: true,             // warm up the Neural Engine ahead of time
            load: true
        )
        pipeline = try await WhisperKit(config)
    }
}

extension LocalTranscriber {
    /// Picks a model based on the device's available memory
    static func recommendedModel() -> String {
        let physicalMemory = ProcessInfo.processInfo.physicalMemory
        let memoryGB = Double(physicalMemory) / 1_073_741_824
 
        switch memoryGB {
        case ..<4:
            return "tiny"            // ~40 MB, for older devices
        case 4..<6:
            return "base"            // ~150 MB, speed/accuracy balance
        case 6..<8:
            return "small"           // ~500 MB
        default:
            return "large-v3-turbo"  // ~600 MB, best balance on the Neural Engine
        }
    }
}

extension LocalTranscriber {
    /// Transcribes an already-recorded audio file (m4a/wav) entirely on-device
    func transcribe(fileAt url: URL) async throws -> String {
        guard let pipeline else {
            throw TranscriptionError.notReady
        }
 
        let options = DecodingOptions(
            language: "ru",           // a hint to the model, not a hard constraint
            temperature: 0.0,
            withoutTimestamps: true
        )
 
        let results = try await pipeline.transcribe(
            audioPath: url.path,
            decodeOptions: options
        )
 
        return results?.text ?? ""
    }
}
 
enum TranscriptionError: Error {
    case notReady
}

extension LocalTranscriber {
    /// Streaming transcription while a voice note is being recorded
    func startStreaming(onPartialResult: @escaping (String) -> Void) async throws {
        guard let pipeline else {
            throw TranscriptionError.notReady
        }
 
        let streamer = AudioStreamTranscriber(
            audioEncoder: pipeline.audioEncoder,
            featureExtractor: pipeline.featureExtractor,
            segmentSeeker: pipeline.segmentSeeker,
            textDecoder: pipeline.textDecoder,
            tokenizer: pipeline.tokenizer!,
            audioProcessor: pipeline.audioProcessor,
            decodingOptions: DecodingOptions(language: "ru")
        ) { _, result in
            onPartialResult(result.text)
        }
 
        try await streamer.startStreamTranscription()
    }
}

What I'd pick#

If Russian weren't a requirement and iOS 26 could be the minimum target, I'd take SpeechAnalyzer without hesitation — higher accuracy, no binary bloat, and an API designed around modern Swift structured concurrency. But the reality for most products is Russian-language support and a minimum iOS version below 26, which means the real choice is between SFSpeechRecognizer and WhisperKit.

SFSpeechRecognizer with requiresOnDeviceRecognition is an honest, nearly free option: it's already in the system, requires no model download, and works great as a baseline. That's exactly what powers transcription in Lanternly today — I deliberately designed the interface so the engine can be swapped at a single point in the code, without any changes further up the stack. WhisperKit is the next step up, for when you need better robustness to noise, accent, and mixed-language speech, and a few hundred extra megabytes is an acceptable price for that quality.

Checklist before picking an engine#

• Do you strictly need offline mode with zero server requests — if so, any "cloud with caching" option is out immediately
• Is your primary language supported by SpeechAnalyzer at your target minimum iOS version
• Are you willing to add 150–600 MB to your app for a WhisperKit model
• Do you need code-switching (mixed languages in one utterance) — then WhisperKit is the clear answer
• Is SFSpeechRecognizer's built-in accuracy good enough for your scenario, or are users complaining about recognition quality

Conclusion#

There's no universal answer to "which engine is best" — there are three honest points on the trade-off curve between accuracy, weight, and language coverage. SpeechAnalyzer wins on raw accuracy wherever its language support is sufficient. WhisperKit remains the most flexible choice for multilingual and accented scenarios, at the cost of model weight. And SFSpeechRecognizer is the workhorse that costs almost nothing and is often good enough. When designing a layer like this, the most important decision isn't which specific engine you pick — it's the architecture that lets you replace it later without rewriting the app.