Executive Summary
GenAI Protos deployed Supertonic-3 as a CPU-only text-to-speech API for always-available voice output without GPU contention. The service supports multilingual synthesis, preset voices, expressive tags, streaming output, and OpenAI-compatible speech access through the shared gateway.
The focus is practical deployment: model packaging, gateway integration, operational controls, and reusable API access for product and engineering teams.
The content stays close to the documented technical build and avoids unsupported adoption or ROI claims.
At a Glance
- Use case
- Always-on CPU-only text-to-speech API.
- Languages
- 31 supported languages plus auto-detect.
- Voices
- 10 preset voices.
- Compute profile
- CPU-only ONNX inference with zero GPU usage.
- Serving pattern
- FastAPI service with OpenAI-compatible speech endpoint.
- Streaming
- Sentence-level streaming route available for live workflows.
The Challenge
Not every voice workflow should occupy GPU memory. Always-on assistants, IVR, accessibility tools, and fallback paths need a lightweight TTS option that stays available.
Cloud TTS introduces dependency, latency, and privacy tradeoffs for sensitive environments.
The deployment needed to provide a familiar speech API while keeping runtime cost and resource usage low.
What GenAI Protos Built
- Packaged Supertonic-3 as an always-on CPU-based TTS service.
- Exposed native OpenAI-compatible speech generation through the private gateway.
- Added streaming, voices, languages, features, models, and health routes.
- Supported expressive inline tags for more natural speech output.
- Kept GPU usage at zero so other AI workloads can use the DGX Spark GPU memory pool.
Solution Architecture
The architecture separates the application interface, gateway layer, model service, runtime, and operations lane so the model can be consumed as a reusable private AI capability rather than a one-off local experiment.
- Application layer
- IVR, chatbot, accessibility, localization, or internal tool sends text for speech output.
- Gateway layer
- Provides authenticated speech access and model discovery.
- TTS API
- Receives text, voice, language, speed, and response-format parameters.
- CPU inference runtime
- Runs ONNX speech synthesis without GPU allocation.
- Streaming route
- Chunks longer output by sentence for lower perceived latency.
- Operations lane
- Health, languages, voices, and feature manifests support integration and monitoring.
Prompt-to-Output Workflow
The CPU TTS workflow shows how text, voice, language, output settings, CPU inference, sentence streaming, and application playback connect through the private speech service.
An application sends text, voice, language, and output settings to the gateway.
The gateway forwards the request to the CPU TTS service.
The ONNX runtime synthesizes audio using the selected voice and language settings.
For streaming requests, sentence chunks are produced progressively.
The application plays, stores, or forwards the generated audio.
Implementation Highlights
This section captures CPU-first serving, OpenAI-compatible speech access, expressive controls, manifest endpoints, and streaming support for always-available private voice output.
- CPU-first serving
- The service uses ONNX CPU inference so it can remain always available without GPU contention.
- OpenAI-compatible access
- Applications can integrate through a familiar speech API shape.
- Expressive control
- Inline tags such as laugh, breath, and sigh enable more natural dialogue experiences.
- Manifest endpoints
- Voices, languages, and feature endpoints make product integration easier.
- Streaming support
- A streaming route supports live transcription-to-speech and conversational workflows.
Measured Technical Details
The technical details below record model size, model file size, resident memory, GPU usage, performance profile, and configured rate limits.
Why This Matters
The value of this service is not only multilingual speech output. The stronger pattern is an always-available CPU TTS baseline that supports private voice experiences without consuming GPU capacity.
Results
GenAI Protos delivered an API-ready private AI service that can be discovered, called, monitored, and operated through the same model-serving pattern used across the wider private AI catalog.
| Outcome | What changed |
|---|---|
| Always-available TTS | Voice output can run continuously without blocking GPU-heavy workloads. |
| Private speech output | Teams can synthesize speech without sending text or audio to external TTS services. |
| Integration-ready API | OpenAI-compatible access lowers integration effort for existing AI applications. |
| Fallback pattern | The CPU service provides a reliable fallback beside premium GPU-based voice models. |
Reusable Pattern
This pattern can be reused when teams need always-on voice output that does not consume GPU capacity and can sit beside heavier private AI services.
- Use CPU models for always-on voice paths when GPU allocation is unnecessary.
- Expose voice, language, and feature manifests so product teams can build controls around the service.
- Pair CPU TTS with STT services for local voice I/O workflows.
- Reserve GPU TTS for premium use cases and keep CPU TTS as the reliable baseline.
Build Always-On Private Voice Output Without GPU Contention
GenAI Protos helps teams deploy CPU and GPU voice models behind stable APIs for private assistants, accessibility tools, and multilingual applications.
Get custom solutions