Private Multilingual Speech-to-Text API Deployed on DGX Spark

GenAI Protos deployed a private multilingual speech-to-text service to convert audio files into text through a governed API surface. The implementation uses Qwen3-ASR-1.7B, a lightweight multilingual ASR model, served with vLLM and registered behind the GenAI Protos LiteLLM gateway.

The build focused on practical model-to-service engineering: avoiding port collisions, exposing the model through a standard transcription endpoint, managing request limits, and validating transcription behavior within a wider private AI model catalog.

The result is an API-ready speech-to-text capability for multilingual transcription workflows where teams need private infrastructure, predictable serving behavior, and integration with OpenAI-compatible client patterns.

Use case

Private multilingual speech-to-text API for transcription and audio processing workflows.

Model

Qwen3-ASR-1.7B, an open-source multilingual ASR model.

Language coverage

52 supported languages, including Arabic, Persian, Urdu, and Chinese dialects.

Serving approach

vLLM OpenAI-compatible API served through Docker and registered with LiteLLM.

Primary endpoint

OpenAI-compatible audio transcription route.

Deployment environment

NVIDIA DGX Spark with shared unified memory and ARM64 Ubuntu.

Lifecycle

On-demand model serving, started when needed rather than always-on.

Technical outcome

A reusable private ASR service pattern that can sit alongside other speech, chat, image, video, and voice models.

Multilingual transcription often starts as a local model test, but production-style adoption needs more than a model checkpoint. Engineering teams need a stable API, gateway controls, documented request behavior, and a way to run speech models without blocking other AI workloads on the same machine.

The key challenge was to package Qwen3-ASR into a private service that application teams could call through a familiar transcription interface while keeping the deployment operationally manageable on DGX Spark.

• Expose a multilingual ASR model through a standard audio transcription API.
• Avoid service collisions with other models already running in the catalog.
• Keep the model lightweight enough to co-run with other speech workloads.
• Document startup, health checks, request limits, and endpoint behavior for developers and operators.
• Support private transcription workflows without routing audio through a third-party cloud API by default.

GenAI Protos built a private multilingual ASR service around Qwen3-ASR-1.7B and integrated it into the DGX Spark model gateway. The model is served as an OpenAI-compatible transcription endpoint, making it easier for applications and existing client libraries to call the service without a custom SDK.

• A Docker-based vLLM service for Qwen3-ASR-1.7B.
• Gateway registration under the GenAI Protos model catalog.
• A standard audio transcription endpoint for application integration.
• A dedicated runtime port selected to avoid collision with the Gemma 4 service.
• Rate-limit and token-limit settings for controlled usage.
• Health validation and startup documentation for repeatable operation.
• Operational notes for concurrent speech-to-text testing with Whisper-large-v3.

The architecture keeps the transcription model behind a gateway-managed access layer while preserving the standard OpenAI-style audio transcription shape. This lets developers use familiar request patterns while platform teams retain control over model routing and limits.

Application or developer

Uploads an audio file with model, language, temperature, and optional prompt parameters.

LiteLLM gateway

Authenticates the request, exposes the model catalog, and routes the transcription call to the correct backend service.

Qwen3-ASR service

Runs a vLLM OpenAI-compatible server in a Docker container for transcription inference.

Model runtime

Loads Qwen/Qwen3-ASR-1.7B with remote-code support and GPU memory controls.

Output layer

Returns transcription text and, when requested by the backend, structured segment metadata.

Operations lane

Covers startup, health check, port assignment, request limits, and coexistence with other speech models.

The audio-to-text workflow shows how uploaded audio moves through the private gateway, Qwen3-ASR service, vLLM transcription runtime, and downstream text-processing paths.

This section summarises the model serving, container runtime, port isolation, gateway registration, private deployment pattern, and operational validation work completed for the ASR service.

Model serving

Qwen3-ASR-1.7B was served through vLLM using an OpenAI-compatible API for transcription workloads.

Containerized runtime

The ASR service runs through Docker Compose so it can be started on demand and managed separately from always-on services.

Port isolation

The default compose port was changed because another model already occupied the default service port.

Gateway integration

The model was registered in the LiteLLM gateway so it can appear in the same catalog as chat, speech, image, video, and voice models.

Private deployment pattern

Audio stays within the private serving environment rather than depending on an external transcription API by default.

Operational validation

Startup behavior, health checks, request limits, and compatibility notes were captured for operators.

The technical details below record model scale, disk footprint, runtime memory, language coverage, serving engine, lifecycle behaviour, startup profile, configured limits, and catalog context.

During bring-up, GenAI Protos ran Qwen3-ASR alongside Whisper-large-v3 and right-sized the speech-to-text stack to control GPU usage. A Turkish round-trip test was verified with exact transcription of Turkish characters, showing that the private speech stack could process non-English audio reliably in that test scenario. The implementation also documented response-format behavior for operators, so integration teams can use a consistent transcription response shape across downstream applications.

The value of this build is not only multilingual transcription. The stronger pattern is that an ASR model was converted into a private, governed API service that can be discovered, called, limited, and operated alongside other AI workloads. That gives teams a practical path from local speech model testing to production-ready audio intelligence workflows.

GenAI Protos delivered a working private multilingual transcription service that can be started on demand, discovered in the model catalog, called through an OpenAI-compatible transcription route, and operated alongside other private AI services on DGX Spark.

This case study is useful for teams that need speech-to-text inside private AI systems rather than as an isolated model demo. The same pattern can support call-center transcription, accessibility captions, multilingual content indexing, meeting/audio archive processing, and speech pipelines that feed downstream LLM or RAG workflows.

• Model wrapper: package ASR behind a stable service layer.
• Gateway registration: expose the model through governed authentication and routing.
• Endpoint discipline: keep the API shape familiar for developers.
• Runtime control: document startup, health, port use, and memory behavior.
• Pipeline readiness: route transcripts into search, summarization, QA, analytics, or compliance workflows.