Private AI Music Generation API Deployed on DGX Spark

GenAI Protos deployed a private AI music generation service to demonstrate how a creative generative model can move from a local model artifact into an API-ready capability that product and engineering teams can evaluate and integrate.

The service uses an open-source 3B-parameter music generation model with a decoder, wrapped in a custom FastAPI service and routed through the GenAI Protos LiteLLM gateway. Users can submit lyrics and style tags, receive 48 kHz MP3 output synchronously, or queue longer requests through an asynchronous job workflow.

Use case

Private text-to-music generation API for AI media experimentation and product prototyping.

Deployment style

On-demand model serving on NVIDIA DGX Spark with a gateway-managed API surface.

Core model capability

Lyrics and style tags to 48 kHz MP3 audio.

Primary interface

Custom FastAPI service exposed through LiteLLM pass-through routing.

Output modes

Synchronous MP3 generation and asynchronous job submission with polling.

Operational controls

Health endpoint, preload, unload, lazy loading, auto-unload, and request rate limiting.

Technical outcome

A reusable private AI service pattern for generative media models that do not fit a standard chat-completion interface.

Many generative media experiments begin as notebooks, local scripts, or isolated model repositories. That is useful for exploration, but not enough for teams that need repeatable access, guarded endpoints, predictable resource behavior, and clear operational documentation.

The task was to turn a music generation model into a service that applications could call while respecting private infrastructure constraints: GPU memory must be managed, longer generation jobs should not block front-end workflows, and custom model APIs should not be forced into the wrong endpoint shape.

• Create a practical text-to-music endpoint instead of leaving the model as a local-only artifact.
• Support both immediate generation and queued generation for longer jobs.
• Expose the model through the same gateway pattern used across the GenAI Protos model fleet.
• Control GPU usage through on-demand loading and idle auto-unload behavior.
• Document the service clearly enough for developers and operators to test, call, and maintain it.

GenAI Protos built a private music generation API that transforms structured prompts into MP3 audio. The service was registered inside the DGX Spark model catalog alongside other private AI services across chat, speech, image, video, 3D, segmentation, and voice workflows.

• A FastAPI service around HeartMuLa-oss-3B and the HeartCodec decoder.
• Gateway-routed access for music generation, asynchronous jobs, job polling, model listing, health checks, preload, and unload operations.
• Synchronous generation for direct MP3 responses or JSON responses with encoded audio.
• Asynchronous job submission for application workflows that should not keep a blocking request open.
• On-demand model loading with a documented 15-minute default idle auto-unload window.
• A controlled request profile for early product integration and repeatable experimentation.

The architecture separates the application interface, gateway layer, music API service, model runtime, output handling, and operations controls —keeping the service usable even though the model uses custom generation routes.

Application or developer

Submits lyrics, tags, duration, sampling settings, and output format.

LiteLLM gateway

Provides authenticated access and pass-through routing for the private model fleet.

Music API service

FastAPI service receives sync or async requests and manages job execution.

Model runtime

HeartMuLa-oss-3B generates the music representation and HeartCodec decodes the audio output.

Output layer

Returns MP3 bytes directly or a JSON result containing audio and job metadata.

Operations lane

Health, preload, unload, lazy load, and auto-unload controls support day-to-day operation.

How a request moves from lyrics and style tags through to delivered MP3 audio.

The model was wrapped in a Python FastAPI service so it behaves like an application endpoint, not a local script.The model was wrapped in a Python FastAPI service so it behaves like an application endpoint, not a local script.

Model serving

The model was wrapped in a Python FastAPI service so it could behave like an application endpoint rather than a local script.

Gateway integration

The service was registered as a GenAI Protos model and exposed through gateway pass-through routing because the endpoint is custom.

Async workflow

Longer music generation can be submitted as a job so web or product interfaces do not need to keep a blocking request open.

Resource control

The model lazy-loads on first request and auto-unloads after inactivity, reducing idle GPU memory pressure.

Operator controls

Preload and unload operations let operators warm the model before demos or release GPU memory after testing.

Documentation

The service was added to the model catalog with endpoint shape, lifecycle, rate-limit, and usage notes.

Open-source model, Apache-2.0 licensed.Open-source model, Apache-2.0 licensed.Open-source model, Apache-2.0 licensed.Open-source model, Apache-2.0 licensed.Open-source model, Apache-2.0 licensed.Open-source model, Apache-2.0 licensed.Open-source model, Apache-2.0 licensed.Open-source model, Apache-2.0 licensed.

Registered in a DGX Spark catalog of 30 model names across 19 services.

The value of this build is not only that it generates audio. The larger engineering pattern is that a non-standard generative model was converted into a private, governed service teams can call, test, and operate the gap many enterprise AI prototypes fail to cross. Application teams get an endpoint instead of a model folder. Platform teams get lifecycle controls instead of unmanaged GPU usage. Product teams get sync and async workflows suited to real UI behavior. Decision makers get a practical view of what it takes to move generative media from experiment to service.

GenAI Protos delivered a working private AI music generation service that can be discovered, called, warmed, unloaded, monitored, and documented within the same gateway-oriented operating model used for the wider DGX Spark AI catalog.

This use case can be reused as a pattern for private AI media systems where the model is valuable but the raw repository is not enough. The same structure supports image, video, 3D, speech, voice conversion, and segmentation services when each model has different runtime behavior and endpoint requirements.

• Model wrapper: package the model behind a stable service layer.
• Gateway route: expose the model through a governed access layer.
• Job design: use async workflows where generation time is longer than normal web request expectations.
• Lifecycle design: preload for planned use, unload after idle periods, and monitor health continuously.
• Documentation: record endpoint shapes, limits, operational behavior, and known constraints.