Private Multilingual Voice AI API Deployed on DGX Spark

GenAI Protos deployed a private multilingual voice AI service to show how an advanced text-to-speech model can be converted from a model runtime into a governed, application-ready API capabili

The build centered on OmniVoice, a massively multilingual zero-shot TTS model with voice cloning, voice design, and optional Whisper-large-v3-turbo auto-transcription. It was packaged behind a custom FastAPI server, containerized with Docker, and integrated with the GenAI Protos LiteLLM gateway for authenticated access and model discovery.

The result is a reusable private voice AI pattern for teams that need multilingual speech generation, voice experimentation, streaming audio, and controlled deployment on private infrastructure without treating the model as a one-off local demo.

Use case

Private multilingual text-to-speech, voice cloning, and voice design API.

Deployment style

Dockerized FastAPI service running on NVIDIA DGX Spark with LiteLLM gateway integration.

Core model capability

Zero-shot TTS across 646 languages with optional reference-audio cloning and instruction-based voice design.

Primary interfaces

OpenAI-compatible /v1/audio/speech plus custom pass-through endpoints for cloning, streaming, language discovery, and voice attributes.

Streaming mode

Sentence-chunked NDJSON streaming designed for responsive audio playback in voice applications.

Operational visibility

Health endpoint, model status, VRAM status, available voices, supported languages, and voice-attribute vocabulary.

Technical outcome

A private voice AI service pattern that can support multilingual assistants, dubbing workflows, localization demos, and voice UX prototyping.

Voice AI systems are difficult to evaluate safely when the model remains a local runtime, notebook, or experimental script. Product teams need reliable endpoints. Platform teams need predictable resource behavior. Application teams need a clean way to call speech, cloning, and streaming workflows without rebuilding around every model-specific API shape.

• Expose multilingual TTS through a developer-friendly API instead of a local-only model workflow.
• Support both standard speech generation and richer voice workflows such as cloning and voice design.
• Keep authentication and discovery consistent with the wider private model gateway pattern.
• Give operators visibility into health, available languages, voice attributes, and runtime memory behavior.
• Separate low-latency streaming needs from gateway-based sync calls where buffering behavior matters.

GenAI Protos built a private voice AI API around OmniVoice and made it accessible through the same gateway-oriented model-serving pattern used across the DGX Spark catalog. The implementation supports standard speech generation, custom multipart voice workflows, sentence-level streaming, and operational endpoints for service visibility.

• A Dockerized FastAPI service for OmniVoice running on DGX Spark.
• Two model registrations: genaiprotos/omnivoice and genaiprotos/omnivoice-tts-1 for compatibility with OpenAI-style speech calls.
• OpenAI-compatible speech generation for simple application integration.
• Custom pass-through routes for voice cloning, voice design, streaming TTS, supported languages, available voices, voice attributes, model listing, and health checks.
• Optional Whisper-large-v3-turbo auto-transcription for reference-audio workflows.
• Documented rate limits, cost fields, memory behavior, and latency observations for practical evaluation.

The architecture keeps the application-facing API stable while allowing the voice model to expose both OpenAI-compatible and custom endpoints. This is important because voice cloning, voice design, and streaming workflows need more than a basic text-to-speech endpoint.

Application or voice UI

Submits text, target language, voice selection, voice design instruction, or reference audio.

LiteLLM gateway

Provides unified gateway access, model discovery, authentication, and pass-through routing.

Voice API service

FastAPI service receives speech, clone, design, stream, language, voice, and health requests.

Model runtime

OmniVoice generates speech; optional Whisper-large-v3-turbo supports reference-audio transcription.

Streaming lane

Sentence-level streaming supports lower perceived latency for voice agents and chat interfaces.

Operations lane

Health and VRAM status endpoints help teams monitor service readiness and runtime behavior.

Model serving

OmniVoice was wrapped behind a custom FastAPI server and containerized for repeatable startup and deployment.

Gateway integration

The service was exposed through LiteLLM with native OpenAI-compatible speech support and custom pass-through routes for extended voice workflows.

Voice cloning

Multipart endpoints support reference-audio upload for clone workflows, with optional auto-transcription support.

Voice design

Instruction-based voice design lets applications request attributes such as accent, tone, pitch, age, or speaking style.

Streaming

A sentence-chunked streaming endpoint returns NDJSON audio chunks for conversational interfaces and chatbot audio playback.

Catalog readiness

The model was documented inside the broader DGX Spark catalog so developers can discover model names, endpoint shapes, limits, and use cases.

The important outcome is not only synthetic speech generation. The stronger pattern is that a complex voice model was converted into a governed private API service with multiple access modes: simple TTS, clone, design, stream, discovery, and health.

GenAI Protos delivered a working private multilingual voice AI service that can be discovered, called, tested, monitored, and integrated into product workflows through a consistent gateway-oriented model-serving approach.

This use case can be reused for other private voice and multimodal AI systems where the model is capable but not yet product-ready. The same engineering pattern can support multilingual assistants, voice agents, localization tools, AI dubbing workflows, and internal voice UX experimentation.

• Model wrapper: package the voice model behind a stable API service layer.
• Gateway route: expose standard calls through a governed access layer and keep custom workflows available where needed.
• Streaming design: handle latency-sensitive voice experiences separately from synchronous request/response flows.
• Discovery endpoints: expose available voices, languages, and voice attributes for product builders.
• Operational documentation: record endpoint shapes, limits, memory behavior, latency findings, and known routing behavior.