Private Long-Form Audio Transcription API Deployed on DGX Spark

Long-form audio is difficult to operationalize when recordings contain multiple speakers, domain terms, timestamps, and multilingual or code-switched speech. Basic transcription alone is not enough for product teams that need searchable transcripts, meeting intelligence, speaker-aware segments, and downstream summarization workflows.

GenAI Protos packaged VibeVoice-ASR into a private audio intelligence service on DGX Spark. The model is served through vLLM, registered in the GenAI Protos model catalog, and exposed behind a LiteLLM gateway so applications can call it through a governed API surface.

The result is a reusable private transcription pattern for meetings, interviews, podcasts, lectures, research sessions, and other long-form recordings where teams need structured Who/When/What output without sending sensitive audio through an unmanaged public workflow

Use case

Private long-form audio transcription and structured transcript generation.

Core capability

Audio-to-structured transcript output with speaker IDs, timestamps, and transcript content.

Deployment style

On-demand model serving on NVIDIA DGX Spark with gateway-managed access.

Primary interface

OpenAI-compatible chat-completions style request with audio content blocks.

Model category

Qwen2.5-7B-based ASR model served via vLLM.

Operational focus

Long recordings, streaming response, continuous batching, model discovery, and governed access.

Technical outcome

A private audio intelligence API pattern for long-form speech workflows that need structured output, not just raw transcript text.

Many transcription workflows start with a speech-to-text model and then add separate tools for diarization, timestamp alignment, domain vocabulary handling, and post-processing. That stack can become brittle when audio is long, speaker turns are frequent, and product teams need clean structured output for downstream analysis.

The challenge was to expose a long-form ASR capability as a private, API-ready service while preserving the details needed by real applications: speaker labels, start/end timestamps, transcript segments, streaming output, and hotword/context support for domain-specific vocabulary.

• Package a long-form audio model as an application-callable service rather than a local experiment.
• Support structured transcript output suitable for meeting analysis, podcast processing, and research workflows.
• Use the same gateway and model catalog pattern as the wider GenAI Protos DGX Spark fleet.
• Keep deployment facts clear for developers and operators without exposing sensitive endpoints or keys.
• Avoid forcing the model into a custom transcription route when it is designed to work through chat-completions with audio input blocks.

GenAI Protos built a private long-form audio transcription API around VibeVoice-ASR. The service accepts audio input through an OpenAI-compatible chat request, streams transcript segments, and returns structured output that can be used by downstream summarization, search, analytics, and review workflows.

• A vLLM-based serving layer for a Qwen2.5-7B-based ASR model.
• Gateway-managed access using the same LiteLLM pattern used across the private DGX Spark model catalog.
• Structured transcript output with start time, end time, speaker ID, and content fields.
• Inference-time hotword/context prompting for product names, account names, technical terms, and domain vocabulary.
• Support for long-form recordings, including meeting, interview, lecture, podcast, and research-session formats.
• A documented operating profile covering model loading, memory requirements, rate limits, and endpoint behavior.

The architecture separates the application layer, gateway layer, vLLM serving layer, and audio transcript output. This gives product teams a stable API surface while allowing the underlying audio model to use a multimodal chat-completions pattern.

Application or workflow

Submits audio content and transcription instructions, including desired fields such as speaker ID, timestamps, and content.

LiteLLM gateway

Provides model discovery, controlled access, and a consistent API surface across the private model fleet.

VibeVoice-ASR service

Runs the Qwen2.5-7B-based ASR model through vLLM with streaming and continuous batching support.

Audio reasoning prompt

Guides the output format and can include hotwords or extra context for domain-specific vocabulary.

Structured output layer

Streams JSON-style transcript segments that include who spoke, when they spoke, and what was said.

Downstream systems

Can use the structured transcript for summaries, action items, chapters, search indexing, compliance review, or analytics.

The structured transcription workflow shows how long-form audio, transcript instructions, hotword context, gateway routing, streaming output, and downstream review workflows connect.

This section summarises the serving, gateway integration, streaming output, long-form handling, hotword support, and operational profile documented for the VibeVoice-ASR deployment.

Model serving

The ASR model was served with vLLM using an OpenAI-compatible chat-completions interface for audio input.

Gateway integration

The model was registered as genaiprotos/vibevoice-asr inside the private DGX Spark model catalog.

Streaming output

The response uses server-sent streaming so transcript segments can accumulate as the model generates output.

Long-form handling

The model is configured with a long context profile suitable for extended audio inputs.

Hotword support

Domain vocabulary can be supplied at inference time to bias output toward project names, account names, product names, and technical terms.

Operational profile

The deployment notes capture startup behavior, memory requirements, model loading behavior, and rate-limit configuration.

The technical details below record the model family, serving engine, audio length profile, output structure, language coverage, hotword handling, model weights, memory profile, cold-start behaviour, warm request behaviour, rate limits, and catalog context.

The value of this build is not only speech-to-text conversion. The more useful pattern is turning long-form audio into structured data that can be used by product, operations, research, and compliance workflows.

GenAI Protos delivered a working private long-form audio transcription service that can be discovered, called, streamed, and operated within the same gateway-oriented pattern used for the broader DGX Spark AI catalog.

This case study can be reused as a pattern for private audio intelligence systems where the model is useful but raw model access is not enough. The same structure can support transcription, speaker-aware summarization, meeting intelligence, call review, and multilingual content analysis workflows.

• Model wrapper: serve the model through a stable API layer rather than a local-only process.
• Gateway route: expose the model through a governed access layer with catalog discovery.
• Instruction pattern: use prompts to define transcript schema, speaker labels, timestamp fields, and domain context.
• Streaming design: stream partial output where long recordings would otherwise block user-facing workflows.
• Documentation: record model behavior, limits, memory needs, rate limits, and operational constraints for repeatable delivery.