Private Multilingual Transcription API Deployed on DGX Spark

GenAI Protos built a private multilingual transcription service around Whisper-large-v3 to convert recorded audio into searchable text through a governed API interface. The build focused on model serving, gateway integration, on-demand operation, and reusable documentation rather than a standalone demo.

The service is deployed on NVIDIA DGX Spark using vLLM and exposed through the shared model gateway with the standard audio transcription shape. This gives application teams a familiar API surface while keeping audio processing inside private infrastructure.

The result is a reusable private speech-to-text capability for meetings, podcasts, interviews, localization workflows, accessibility captions, and searchable audio archives.

Use case

Private multilingual speech-to-text API for recorded audio, transcripts, captions, and searchable audio workflows.

Deployment style

On-demand vLLM transcription service on NVIDIA DGX Spark.

Core model capability

Whisper-large-v3 speech-to-text with 99-language coverage.

Primary interface

OpenAI-compatible audio transcription endpoint through the GenAI Protos gateway.

Resource profile

Approximately 3-4 GB GPU footprint on DGX Spark with FP8 serving notes.

Operational profile

Started when needed through the STT inference stack and stopped when not required.

Technical outcome

Deployed as part of the GenAI Protos DGX Spark model catalog and private gateway pattern.

Teams that work with multilingual audio often need transcription for meetings, interviews, content archives, compliance review, and accessibility. Public transcription APIs are simple to start with, but they can create concerns around data movement, governance, repeatability, and integration into private AI platforms.

The technical challenge was to expose a reliable transcription model as a private, API-ready service while preserving a familiar developer experience.

• Provide a private transcription endpoint instead of relying on a one-off model script.
• Support multilingual audio workflows with a broad language coverage model.
• Use a standard API shape so existing speech-to-text integrations can connect with minimal changes.
• Keep the service aligned with the same gateway and catalog pattern used across the GenAI Protos model fleet.
• Document rate limits, cost fields, and operating behavior clearly for engineering teams.

GenAI Protos packaged Whisper-large-v3 as a private transcription service and registered it inside the DGX Spark model catalog.

• OpenAI-compatible audio transcription route through the shared LiteLLM gateway.
• On-demand containerized deployment using the STT inference stack.
• Model catalog registration for discovery across the private model fleet.
• Documented rate limits, cost fields, response format behavior, and transcription use cases.

The architecture keeps the transcription API familiar while running the model inside private infrastructure.

Audio source

Recorded meetings, interviews, podcasts, lectures, customer support audio, or archived files.

Gateway layer

Provides governed access and routes standard transcription requests to the private model backend.

Model serving layer

Runs Whisper-large-v3 through vLLM as an audio transcription service.

Output layer

Returns transcript text and, when requested, segment-level transcription metadata.

Operations layer

Supports on-demand startup, documented rate limits, and gateway-level catalog discovery.

The audio-to-transcript workflow shows how recorded audio is uploaded, routed through the private gateway, transcribed by Whisper-large-v3, and made available for downstream workflows.

This section captures how GenAI Protos packaged Whisper-large-v3 into a private transcription service with standard API compatibility, gateway integration, resource awareness, and repeatable documentation.

Model serving

Whisper-large-v3 was deployed behind a vLLM service for private speech-to-text inference.

API compatibility

The service uses the standard audio transcription route, avoiding a custom integration surface.

Gateway integration

The model was added to the shared gateway and catalog so teams can discover and call it consistently.

Resource awareness

The service runs on demand, helping avoid unnecessary GPU occupancy when transcription is not active.

Documentation

Model behavior, request format, response examples, rate limits, and use cases were captured for repeatable operation.

The technical details below document the model, language coverage, serving engine, GPU footprint, on-demand availability, configured rate limit, cost field, and catalog context.

The value of this service is not only transcription. The stronger pattern is that multilingual audio becomes a private, reusable data source that applications can search, summarize, translate, caption, and archive through a governed API.

The implementation turned a speech-to-text model into a reusable private transcription service that application teams can call through a governed API.

This pattern can be reused for private transcription services where audio must stay under tighter operational control.

• Use a standard transcription API for developer adoption.
• Keep model serving inside a private inference environment.
• Document rate limits, cost fields, response formats, and model behavior.
• Add multiple STT models for fallback or benchmark comparison where needed.
• Connect transcripts into downstream search, analytics, summarization, or accessibility workflows.