Argmax Open-Source SDK (Swift)

Quick Verdict

Argmax Open-Source SDK (Swift) is a compelling local speech stack for native Apple developers. The project grew from WhisperKit into a package containing WhisperKit for speech recognition, SpeakerKit for diarization, TTSKit for speech generation, and an ArgmaxOSS umbrella product. Release 1.0.0 runs converted OpenAI Whisper, Pyannote v4 community-1, and Qwen3-TTS models with Core ML on Apple hardware. For teams building transcription, meeting, accessibility, or read-aloud features, keeping inference on device can reduce network latency, recurring API cost, and exposure of raw recordings.

It is not a universal speech backend. The README currently lists macOS 14 or later and Xcode 16 or later as package prerequisites, while individual kits have different deployment floors; TTSKit requires macOS 15 or iOS 18. The SDK is MIT-licensed, its vendored modified swift-transformers code carries Apache-2.0 obligations, and model weights retain separate terms. Benchmark the exact devices, languages, noise conditions, and speaker overlap you expect. “Runs locally” is an architecture property, not a guarantee of accuracy, privacy compliance, or production reliability.

Best For

• iOS and macOS developers embedding speech features directly into a native application.
• Privacy-sensitive products handling meetings, interviews, personal notes, or accessibility audio on device.
• Prototype teams that want ASR, diarization, and TTS in one Swift Package Manager dependency.
• Research engineers comparing model size, accuracy, real-time factor, first-audio latency, memory, and thermal behavior.
• Not ideal for Android-, Windows-, or Linux-first products, massive hosted concurrency, old Apple devices, or teams unable to manage large model artifacts.

Key Features

• WhisperKit: multilingual file and microphone transcription with several Core ML model variants, prompts, streaming, and word or segment timestamps.
• SpeakerKit: on-device Pyannote diarization with optional speaker count, clustering controls, RTTM output, and alignment with WhisperKit segments.
• TTSKit: Qwen3-TTS 0.6B and 1.7B paths, nine built-in voices, ten languages, sentence chunking, and streaming playback.
• Selective products: import ArgmaxOSS for all kits or choose WhisperKit, SpeakerKit, and TTSKit independently.
• Model management: recommended models can download automatically and cache locally; custom Whisper model repositories are also supported.
• CLI and local server: argmax-cli handles transcription, diarization, and TTS, with an optional OpenAI Audio API-compatible server build.
• No-cloud inference: once artifacts are cached, core inference can run offline without an Argmax API account.

Use Cases

• Transcribe a local meeting with timestamps, add SpeakerKit labels, and require a person to correct names, numbers, and speaker assignments.
• Build incremental captions from microphone audio while measuring battery use, thermal throttling, background behavior, and end-to-end delay.
• Stream long-form TTS on device and test pronunciation, pacing, and voice suitability in every target language.
• Point an OpenAI SDK client at the optional local transcription and translation endpoints during migration testing, while accounting for the documented response-format subset.
• Convert and host a fine-tuned Whisper model with the companion tooling, then review both the source model license and the converted artifact’s distribution terms.

Pricing

Argmax OSS is free under MIT, and GitHub identifies 1.0.0 as the current release. The repository’s NOTICES file explains that modified Hugging Face swift-transformers sources under ArgmaxCore/External use Apache-2.0 and require retained notices. Those software terms do not automatically clear Whisper, Pyannote, Qwen3-TTS, custom weights, training data, built-in voices, or user-provided audio. Commercial teams need a model-by-model and content-by-content review, especially for voice imitation and biometric or consent-sensitive use.

Argmax Pro SDK is a separate commercial path offering additional models, real-time transcription with speakers, custom vocabulary, a Local Server for non-native clients, and Android support. The open-source library has no per-request vendor fee, but device QA, model hosting, download bandwidth, app-storage design, observability, and engineering support remain costs.

Source access depends on GitHub and model downloads mainly depend on Hugging Face. Availability and speed vary by region and repository status. A custom or gated model repository may require its own token even though Argmax OSS does not require an API key. Once cached, local inference has no cloud-provider request quota. Its practical rate limit is the device: CPU, GPU or Neural Engine load, memory pressure, concurrency, battery, and heat.

Pros

• Combines ASR, speaker diarization, and TTS in a native Swift and Core ML stack.
• On-device processing reduces cloud round trips, usage billing, and routine raw-audio transfer.
• Selective library products let applications avoid importing capabilities they do not need.
• CLI, RTTM, timestamps, and an optional compatible local API improve integration and testing.
• Multiple model sizes and decoder modes expose meaningful quality, memory, throughput, and latency tradeoffs.

Cons

• Apple-only deployment and recent Xcode or OS requirements exclude many products and older devices.
• Initial artifacts are large: TTS documentation describes roughly 1 GB for 0.6B and 2.2 GB for 1.7B.
• The local server implements a subset of the OpenAI Audio API, so drop-in compatibility has limits.
• SDK code, vendored code, model weights, voices, inputs, and generated outputs do not share one universal license.
• Recognition, speaker labels, and synthesized speech quality vary by language, hardware, acoustics, and model choice without a hosted SLA.

Local processing still requires a privacy design. Obtain recording consent, protect cached models and audio, encrypt or minimize logs, and control exported transcripts. The optional local HTTP service is not a public authentication product; if bound beyond loopback, add a firewall, TLS, identity checks, body-size limits, and rate limits. Audio and transcripts are also untrusted input. Spoken instructions or transcription errors must not automatically trigger a privileged agent, shell command, message, or account action.

Alternatives

FAQ

Is Argmax OSS free?

Yes. The SDK uses MIT, while vendored notices and each model’s license still apply independently.

Does it require a network connection or API key?

Models need an initial download. Cached inference can run locally without an Argmax key; gated custom repositories may require their own authentication.

Which devices and models are supported?

It targets Apple platforms and provides Core ML paths for Whisper, Pyannote, and Qwen3-TTS. Deployment minimums differ by kit.

Does it have a cloud-style rate limit?

The local libraries have no provider quota. Throughput is limited by hardware and thermal conditions, and any exposed HTTP server should enforce its own quotas.

Does local inference guarantee privacy?

No. The app must still protect microphone permission, recordings, caches, logs, exports, model repositories, and listening network ports.

Can a transcript directly drive an agent?

That is risky. Audio and transcripts can contain mistakes or malicious spoken instructions. Require confirmation before any external or privileged action.

Bottom Line

Argmax Open-Source SDK is one of the more complete native Apple speech packages because WhisperKit, SpeakerKit, and TTSKit cover recognition, speaker attribution, and generation in one ecosystem. It is a strong candidate when privacy, latency, offline behavior, and Swift integration matter. Production adoption still requires device-level benchmarks, pinned models, separate license review for every artifact, resilient download and storage policies, and least-privilege handling of transcripts and local endpoints. Cross-platform products or teams needing a managed SLA should evaluate a hosted service or another framework instead.