audio: implement music dub, demucs separation and ML restoration for real

Three related placeholders were shipping as working features:

- audio_stems.separate_with_provider() and audio_clean.restore_with_provider()
  were one-line stubs that unconditionally raised 501, so /v1/audio/stems and
  /v1/audio/cleanup only ever worked in ffmpeg fallback_mode despite demucs and
  deepfilternet being declared dependencies and probed by detect_audio_backends.
- run_full_music_dub() ran only the STT step and returned hardcoded paths
  (vocals.wav, final_mix.wav) plus 'status': 'placeholder' for the other four,
  while the endpoint reported 'status': 'available'.

Now:

- separate_with_provider drives demucs (demucs.api, CLI fallback), keeps the
  vocals stem and sums drums+bass+other into an instrumental bed for
  vocals-instrumental; 4-stem passes the four through.
- restore_with_provider runs DeepFilterNet (VoiceFixer fallback) and then applies
  the hum/click/loudness DSP with ffmpeg, so those options are no longer lost.
- audio-music-dub is a real six-stage chain: separate -> transcribe the ISOLATED
  vocal -> adapt lyrics (LLM prompted for singable output with matching syllable
  count, argostranslate fallback) -> re-sing with F5-TTS cloned from the original
  vocal -> optional Seed-VC f0-conditioned singing pass -> ffmpeg remix + limiter.
  New request fields: text_model, voice_name/ref_text, sing_convert,
  diffusion_steps, pitch_shift, speed, seed, fallback_mode.

Missing optional deps now degrade to a reported 'skipped'/'failed' step with a
reason and a top-level status of 'complete' vs 'degraded' plus warnings[], rather
than silently producing placeholders. 501s name the package to install.

Verified offline with a fake-model harness: the chain produces a real RIFF wav
mix end to end, feeds the vocal stem (not the mix) to STT, and correctly reports
degraded when seed-vc is unavailable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4

README.md

@@ -244,6 +244,41 @@ Built-in multi-step pipelines callable from the API or web UI:
 | `POST /v1/pipelines/story` | LLM script → images per scene → video → TTS narration |
 | `POST /v1/pipelines/audio-dub` | Transcribe audio/video → translate → clone voice → replace audio |
 
+#### Music dubbing
+
+`POST /v1/pipelines/audio-music-dub` sings a song in another language over its own
+backing track. Six real stages:
+
+1. **Separate** — Demucs splits the track; the vocal is isolated and drums/bass/other are
+   summed back into an instrumental bed
+2. **Transcribe** — STT runs on the *isolated vocal*, not the full mix, which is what makes
+   the lyrics legible
+3. **Adapt** — with a `text_model`, an LLM adapts the lyrics to be **singable**: same
+   syllable count per line, rhyme preserved where possible. Without one it falls back to a
+   literal argostranslate pass
+4. **Re-sing** — F5-TTS synthesises the new lyrics using the isolated original vocal as the
+   cloning reference, so the dub keeps the original singer's timbre with no voice profile to
+   set up (or pass `voice_name` to use a saved one)
+5. **Match the singing** — an optional Seed-VC pass with `f0_condition` on, conditioned on
+   the original vocal, to pull the result toward sung rather than spoken delivery
+6. **Remix** — the new vocal is mixed back over the instrumental and limited
+
+```json
+{
+  "audio": "<base64 or data URI>",
+  "audio_model": "whisper0",
+  "text_model": "my-llm",
+  "source_lang": "en",
+  "target_lang": "it",
+  "notes": "keep it playful"
+}
+```
+
+Every stage that needs an optional dependency degrades to a reported `skipped` state with
+a reason instead of failing or faking it. The response carries `status` (`complete` or
+`degraded`), a `warnings` list, per-step statuses, and all four artifacts — `vocals`,
+`instrumental`, `converted_vocals` and `final_mix`.
+
 **Custom Pipeline Builder**: Create, save and run your own multi-step pipelines from the web UI or API. Chain any combination of 18 step types with `{{input}}` and `{{stepN.output}}` template variables.
 
 ### Advanced Features
@@ -422,10 +457,8 @@ Notes:
 - `rnnoise` and `voicefixer` are optional alternates / complements.
 - Full music-dub quality depends on separation plus singing-capable conversion; even with this stack, output quality still depends heavily on source material and model/runtime availability.
 
-> **Status note:** `/v1/pipelines/audio-music-dub` is **not complete**. Only the
-> transcription step actually runs today; the separation, translation, voice-conversion and
-> remix steps are returned as placeholders. Use `/v1/audio/stems`, `/v1/audio/convert` and
-> `/v1/pipelines/audio-dub` directly until it is finished.
+Without this stack, `/v1/audio/stems` and `/v1/audio/cleanup` return `501` with the
+package to install — or accept `fallback_mode: true` for the best-effort ffmpeg path.
 
 ### Face Swap
 
@@ -777,7 +810,7 @@ persisted per `config_id`, so the configs don't overwrite each other.
 | `POST /v1/pipelines/story` | LLM → images → video → TTS |
 | `POST /v1/pipelines/audio-dub` | Audio/video dub with voice cloning |
 | `POST /v1/pipelines/audio-understand` | Transcribe audio, then answer a question about it with a text model |
-| `POST /v1/pipelines/audio-music-dub` | Music dub — **partly a stub**, see the note below |
+| `POST /v1/pipelines/audio-music-dub` | Dub a song into another language over its original backing track |
 | `GET /v1/pipelines/custom` | List custom pipelines |
 | `POST /v1/pipelines/custom` | Create custom pipeline |
 | `PUT /v1/pipelines/custom/{id}` | Update custom pipeline |

codai/api/audio_clean.py

@@ -63,8 +63,93 @@ def _run_ffmpeg(command):
         raise HTTPException(status_code=500, detail=detail)
 
 
+def _deepfilter_denoise(src: str, dst: str) -> str:
+    """DeepFilterNet speech enhancement. Raises ImportError when unavailable."""
+    from df.enhance import enhance, init_df, load_audio, save_audio  # noqa: F401
+
+    model, df_state, _ = init_df()
+    audio, _meta = load_audio(src, sr=df_state.sr())
+    enhanced = enhance(model, df_state, audio)
+    # save_audio() appends the suffix itself, so hand it the stem.
+    stem = dst[:-4] if dst.endswith(".wav") else dst
+    save_audio(stem, enhanced, df_state.sr())
+    return dst if os.path.exists(dst) else stem + ".wav"
+
+
+def _voicefixer_restore(src: str, dst: str) -> str:
+    """VoiceFixer general restoration. Raises ImportError when unavailable."""
+    from voicefixer import VoiceFixer
+
+    VoiceFixer().restore(input=src, output=dst, cuda=False, mode=0)
+    return dst
+
+
 def restore_with_provider(audio_bytes: bytes, options: dict, workdir: str) -> dict:
-    raise HTTPException(status_code=501, detail="ML audio restoration backend not installed")
+    """Real ML restoration.
+
+    DeepFilterNet is the primary denoiser; VoiceFixer is the fallback for clips it
+    can't handle. The remaining requested operations (hum removal, click repair,
+    loudness normalisation) are DSP, so they are applied with ffmpeg *after* the
+    learned pass rather than being skipped.
+    """
+    src = os.path.join(workdir, "input.wav")
+    with open(src, "wb") as handle:
+        handle.write(audio_bytes)
+
+    enhanced = os.path.join(workdir, "enhanced.wav")
+    engine = None
+    errors = []
+    if options.get("noise_reduction", True):
+        for name, fn in (("deepfilternet", _deepfilter_denoise),
+                         ("voicefixer", _voicefixer_restore)):
+            try:
+                enhanced = fn(src, enhanced)
+                engine = name
+                break
+            except ImportError as exc:
+                errors.append(f"{name}: {exc}")
+            except Exception as exc:
+                errors.append(f"{name} failed: {exc}")
+        if engine is None:
+            raise HTTPException(
+                status_code=501,
+                detail="ML audio restoration needs deepfilternet or voicefixer "
+                       "(" + "; ".join(errors) + "). Run: pip install deepfilternet — "
+                       "or call /v1/audio/cleanup with fallback_mode=true for the "
+                       "ffmpeg filter chain.")
+        applied = ["noise_reduction"]
+    else:
+        enhanced, applied = src, []
+
+    # DSP touch-ups the learned model doesn't do.
+    filters = []
+    if options.get("remove_hum"):
+        filters.append("highpass=f=60,lowpass=f=15000")
+        applied.append("remove_hum")
+    if options.get("repair_clicks"):
+        filters.append("adeclick=t=40")
+        applied.append("repair_clicks")
+    if options.get("normalize"):
+        filters.append("loudnorm=I=-16:TP=-1.5:LRA=11")
+        applied.append("normalize")
+
+    out = enhanced
+    if filters:
+        out = os.path.join(workdir, "cleaned.wav")
+        _run_ffmpeg([_ffmpeg_binary(), "-y", "-i", enhanced, "-af", ",".join(filters), out])
+
+    if not applied:
+        raise HTTPException(status_code=400, detail="Select at least one cleanup operation")
+
+    return {
+        "path": out,
+        "engine": engine or "ffmpeg-filter-chain",
+        "applied": applied,
+        "limitations": [
+            "tuned for speech; music and wide-band material may be dulled",
+            "severe clipping or codec damage is not fully recoverable",
+        ],
+    }
 
 
 def _cleanup_audio(audio_bytes: bytes, options: dict, workdir: str) -> dict:

codai/api/audio_stems.py

@@ -63,8 +63,126 @@ def _run_ffmpeg(command: List[str]):
         raise HTTPException(status_code=500, detail=detail)
 
 
+#: demucs stem order; "other" carries everything that is not drums/bass/vocals.
+_DEMUCS_STEMS = ("drums", "bass", "other", "vocals")
+
+
+def _demucs_separate(src: str, workdir: str) -> dict:
+    """Run demucs on ``src`` and write one wav per stem into ``workdir``.
+
+    Returns ``{stem_name: path}``. Prefers the in-process ``demucs.api`` (demucs 4.x)
+    and falls back to the CLI for builds that don't ship it. Raises ImportError when
+    demucs isn't installed at all, so the caller can report a clean 501.
+    """
+    try:
+        from demucs.api import Separator, save_audio
+    except ImportError:
+        return _demucs_separate_cli(src, workdir)
+
+    separator = Separator()          # default model (htdemucs)
+    _origin, stems = separator.separate_audio_file(src)
+    out = {}
+    for name, tensor in stems.items():
+        path = os.path.join(workdir, f"{name}.wav")
+        save_audio(tensor, path, samplerate=separator.samplerate)
+        out[name] = path
+    return out
+
+
+def _demucs_separate_cli(src: str, workdir: str) -> dict:
+    """Fallback path: drive demucs through its CLI (`python -m demucs`)."""
+    import sys
+
+    outdir = os.path.join(workdir, "demucs")
+    proc = subprocess.run(
+        [sys.executable, "-m", "demucs", "-o", outdir, src],
+        capture_output=True, text=True,
+    )
+    if proc.returncode != 0:
+        detail = (proc.stderr or proc.stdout or "").strip()
+        if "No module named" in detail:
+            raise ImportError(detail)
+        raise HTTPException(status_code=500, detail=f"demucs failed: {detail[-500:]}")
+
+    # demucs writes <outdir>/<model>/<track-name>/<stem>.wav
+    out = {}
+    for root, _dirs, files in os.walk(outdir):
+        for fn in files:
+            stem = os.path.splitext(fn)[0]
+            if stem in _DEMUCS_STEMS:
+                out[stem] = os.path.join(root, fn)
+    if not out:
+        raise HTTPException(status_code=500, detail="demucs produced no stems")
+    return out
+
+
+def _mix_down(paths: List[str], dest: str) -> str:
+    """Sum several stems into one file (used to rebuild the instrumental)."""
+    ffmpeg = _ffmpeg_binary()
+    if len(paths) == 1:
+        _run_ffmpeg([ffmpeg, "-y", "-i", paths[0], dest])
+        return dest
+    cmd = [ffmpeg, "-y"]
+    for p in paths:
+        cmd += ["-i", p]
+    cmd += ["-filter_complex",
+            f"amix=inputs={len(paths)}:duration=longest:normalize=0",
+            dest]
+    _run_ffmpeg(cmd)
+    return dest
+
+
 def separate_with_provider(audio_bytes: bytes, stem_mode: str, workdir: str) -> dict:
-    raise HTTPException(status_code=501, detail="ML stem separation backend not installed")
+    """Real ML source separation via demucs.
+
+    ``vocals-instrumental`` keeps demucs' vocals stem and sums drums+bass+other back
+    into a single instrumental bed — which is what a music dub needs. ``4-stem``
+    returns demucs' four stems as-is.
+    """
+    src = os.path.join(workdir, "input.wav")
+    with open(src, "wb") as handle:
+        handle.write(audio_bytes)
+
+    try:
+        stems = _demucs_separate(src, workdir)
+    except ImportError:
+        raise HTTPException(
+            status_code=501,
+            detail="ML stem separation needs demucs. Run: pip install demucs "
+                   "(or call /v1/audio/stems with fallback_mode=true for the "
+                   "best-effort ffmpeg split).")
+
+    mode = stem_mode or "vocals-instrumental"
+    if mode == "vocals-instrumental":
+        vocals = stems.get("vocals")
+        backing = [p for name, p in stems.items() if name != "vocals"]
+        if not vocals or not backing:
+            raise HTTPException(status_code=500,
+                                detail="demucs did not return the expected stems")
+        instrumental = _mix_down(sorted(backing),
+                                 os.path.join(workdir, "instrumental.wav"))
+        artifacts = [
+            {"name": "vocals", "path": vocals, "role": "lead-vocal"},
+            {"name": "instrumental", "path": instrumental, "role": "backing-track"},
+        ]
+    elif mode == "4-stem":
+        artifacts = [{"name": n, "path": stems[n], "role": n}
+                     for n in _DEMUCS_STEMS if n in stems]
+        if not artifacts:
+            raise HTTPException(status_code=500, detail="demucs returned no stems")
+    else:
+        raise HTTPException(status_code=400, detail=f"Unsupported stem_mode: {mode}")
+
+    return {
+        "stem_mode": mode,
+        "artifacts": artifacts,
+        "engine": "demucs",
+        "model": "htdemucs",
+        "limitations": [
+            "separation quality depends on the source mix",
+            "bleed between stems is normal on dense mixes",
+        ],
+    }
 
 
 def _split_audio(audio_bytes: bytes, mode: str, workdir: str) -> dict:

codai/api/custom_pipelines.py

@@ -375,11 +375,31 @@ class AudioUnderstandRequest(BaseModel):
 
 
 class AudioMusicDubRequest(BaseModel):
+    """Dub a song into another language over its original backing track.
+
+    Only ``audio`` and ``audio_model`` are required; everything else tunes how the
+    lyrics are adapted and how the replacement vocal is produced.
+    """
     audio: str
-    audio_model: str
+    audio_model: str                          # STT model used on the isolated vocal
     target_lang: Optional[str] = None
     source_lang: Optional[str] = None
-    notes: Optional[str] = ''
+    notes: Optional[str] = ''                 # adaptation notes / STT prompt
+    # Lyric adaptation: with a text model the lyrics are *adapted* to be singable in
+    # the target language; without one we fall back to a literal translation.
+    text_model: Optional[str] = None
+    # Replacement vocal: by default the isolated original vocal is the cloning
+    # reference, so the dub keeps the original singer's timbre.
+    voice_name: Optional[str] = None
+    ref_text: Optional[str] = None
+    speed: Optional[float] = 1.0
+    seed: Optional[int] = None
+    # Seed-VC singing pass (f0-conditioned) over the synthesised vocal.
+    sing_convert: Optional[bool] = True
+    diffusion_steps: Optional[int] = 15
+    pitch_shift: Optional[int] = 0
+    # Stem separation: demucs by default, ffmpeg best-effort when true.
+    fallback_mode: Optional[bool] = False
     model_config = ConfigDict(extra='allow')
 
 
@@ -519,36 +539,243 @@ async def run_audio_understanding(request: AudioUnderstandRequest, http_request:
     }
 
 
+def _argos_translate(text: str, source_lang: Optional[str], target_lang: str) -> Optional[str]:
+    """Literal line-by-line translation via argostranslate, or None if unavailable."""
+    try:
+        import argostranslate.translate
+    except ImportError:
+        return None
+    src = (source_lang or 'en').split('-')[0]
+    dst = target_lang.split('-')[0]
+    try:
+        out = []
+        for line in text.split('\n'):
+            out.append(argostranslate.translate.translate(line, src, dst) if line.strip() else line)
+        return '\n'.join(out)
+    except Exception:
+        return None
+
+
+async def _adapt_lyrics(request: 'AudioMusicDubRequest', transcript: str,
+                        http_request) -> tuple:
+    """Turn the source lyrics into target-language lyrics.
+
+    Returns ``(lyrics, step_dict)``. A text model *adapts* them — matching syllable
+    count and keeping the rhyme where it can, which is what makes a dub singable.
+    Without one we fall back to a literal argostranslate pass, and if that is missing
+    too we return the original and say so rather than pretending.
+    """
+    if not request.target_lang:
+        return transcript, {'type': 'translate', 'label': 'Translate/adapt lyrics',
+                            'status': 'skipped', 'reason': 'no target_lang given',
+                            'output': transcript}
+
+    if request.text_model:
+        prompt = (
+            f"Adapt these song lyrics into {request.target_lang}.\n"
+            "Rules: keep the meaning, keep roughly the same syllable count per line so "
+            "they remain singable to the original melody, preserve the rhyme scheme "
+            "where possible, and keep the line breaks exactly as given.\n"
+            "Reply with the adapted lyrics only — no commentary.\n"
+        )
+        if request.notes:
+            prompt += f"Additional direction: {request.notes}\n"
+        prompt += f"\nLyrics:\n{transcript}"
+        step = {'type': 'text_gen', 'label': 'Adapt lyrics',
+                'params': {'model': request.text_model, 'prompt': prompt}}
+        out = await _run_scheduled_step(step, {'input': ''}, http_request)
+        lyrics = (out.get('output') or '').strip()
+        if lyrics:
+            return lyrics, {'type': 'translate', 'label': 'Adapt lyrics',
+                            'status': 'ok', 'engine': 'text_model',
+                            'model': request.text_model, 'output': lyrics}
+
+    literal = _argos_translate(transcript, request.source_lang, request.target_lang)
+    if literal:
+        return literal, {'type': 'translate', 'label': 'Translate lyrics',
+                         'status': 'ok', 'engine': 'argostranslate',
+                         'output': literal}
+    return transcript, {
+        'type': 'translate', 'label': 'Translate lyrics', 'status': 'skipped',
+        'reason': 'no text_model given and argostranslate is not installed — '
+                  'lyrics left in the source language',
+        'output': transcript}
+
+
 async def run_full_music_dub(request: AudioMusicDubRequest, http_request: Request = None):
+    """Separate → transcribe → adapt → re-sing → remix.
+
+    Every stage runs for real. Stages that need an optional dependency degrade to a
+    documented ``skipped`` state with a reason instead of silently producing a
+    placeholder, and the caller can tell from ``complete`` whether the whole chain ran.
+    """
+    import base64
+    import os
+    import subprocess
+    import tempfile
+
+    from codai.api.audio_stems import (_persist_file, _split_audio,
+                                       separate_with_provider)
+
+    steps = []
+    warnings = []
+
+    with tempfile.TemporaryDirectory(prefix='codai-musicdub-') as workdir:
+        # -- 0. isolate the vocal from the backing track ----------------------
+        try:
+            raw = base64.b64decode(request.audio.split(',', 1)[1]
+                                   if request.audio.startswith('data:') else request.audio)
+        except Exception as exc:
+            raise HTTPException(status_code=400, detail=f'Invalid audio payload: {exc}')
+
+        if request.fallback_mode:
+            sep = _split_audio(raw, 'vocals-instrumental', workdir)
+        else:
+            sep = separate_with_provider(raw, 'vocals-instrumental', workdir)
+        paths = {a['name']: a['path'] for a in sep['artifacts']}
+        vocals_path, instrumental_path = paths.get('vocals'), paths.get('instrumental')
+        if not vocals_path or not instrumental_path:
+            raise HTTPException(status_code=500,
+                                detail='Stem separation did not return vocals + instrumental')
+        steps.append({'step': 0, 'type': 'stems', 'label': 'Isolate vocals and instrumental',
+                      'status': 'ok', 'engine': sep['engine'],
+                      'limitations': sep.get('limitations', [])})
+        if request.fallback_mode:
+            warnings.append('stem separation used the best-effort ffmpeg split; '
+                            'expect heavy bleed between vocal and backing')
+
+        # -- 1. transcribe the ISOLATED vocal (far better than the full mix) --
+        with open(vocals_path, 'rb') as fh:
+            vocals_b64 = base64.b64encode(fh.read()).decode()
         stt_step = {
             'type': 'stt',
-        'label': 'Transcribe lyrics or vocals',
+            'label': 'Transcribe lyrics from the isolated vocal',
             'params': {
                 'model': request.audio_model,
-            'audio': request.audio,
+                'audio': f'data:audio/wav;base64,{vocals_b64}',
                 'language': request.source_lang,
                 'prompt': request.notes or None,
                 'response_format': 'json',
             },
         }
-    stt_out = await _run_scheduled_step(stt_step, {'input': request.notes or ''}, http_request)
-    transcript = stt_out.get('text') or stt_out.get('output') or ''
-    translated = transcript if not request.target_lang else f"[{request.target_lang}] {transcript}"
-    steps = [
-        {'step': 0, 'type': 'stems', 'label': 'Isolate vocals and instrumental', 'status': 'placeholder'},
-        {'step': 1, 'type': 'stt', 'label': 'Transcribe lyrics or vocals', **stt_out},
-        {'step': 2, 'type': 'translate', 'label': 'Translate/adapt lyrics', 'output': translated},
-        {'step': 3, 'type': 'voice_convert', 'label': 'Convert singing voice', 'status': 'placeholder'},
-        {'step': 4, 'type': 'remix', 'label': 'Remix converted vocals', 'status': 'placeholder'},
-    ]
+        stt_out = await _run_scheduled_step(stt_step, {'input': request.notes or ''},
+                                            http_request)
+        transcript = (stt_out.get('text') or stt_out.get('output') or '').strip()
+        steps.append({'step': 1, 'type': 'stt',
+                      'label': 'Transcribe lyrics from the isolated vocal',
+                      'status': 'ok', **stt_out})
+        if not transcript:
+            raise HTTPException(status_code=500,
+                                detail='No lyrics could be transcribed from the vocal stem')
+
+        # -- 2. adapt the lyrics ----------------------------------------------
+        lyrics, translate_step = await _adapt_lyrics(request, transcript, http_request)
+        steps.append({'step': 2, **translate_step})
+        if translate_step['status'] == 'skipped':
+            warnings.append(translate_step['reason'])
+
+        # -- 3. sing the new lyrics in the original singer's voice ------------
+        # The isolated vocal doubles as the cloning reference, so the dub keeps the
+        # original timbre without the user having to enrol a voice profile.
+        from codai.api.voice_clone import _f5tts_clone, _load_voice
+
+        ref_audio_path, ref_text = vocals_path, request.ref_text or transcript
+        if request.voice_name:
+            meta = _load_voice(request.voice_name)
+            if not meta:
+                raise HTTPException(status_code=404,
+                                    detail=f"Voice '{request.voice_name}' not found")
+            ref_audio_path = meta['audio_file']
+            ref_text = request.ref_text or meta.get('transcript', '') or transcript
+
+        try:
+            sung_bytes = await asyncio.get_event_loop().run_in_executor(
+                None, _f5tts_clone, ref_audio_path, ref_text, lyrics,
+                request.speed or 1.0, request.seed)
+        except ImportError:
+            raise HTTPException(
+                status_code=501,
+                detail='Re-singing the lyrics needs F5-TTS. Run: pip install f5-tts')
+        except Exception as exc:
+            raise HTTPException(status_code=500, detail=f'Vocal synthesis failed: {exc}')
+
+        new_vocal = os.path.join(workdir, 'new_vocals.wav')
+        with open(new_vocal, 'wb') as fh:
+            fh.write(sung_bytes)
+        steps.append({'step': 3, 'type': 'voice_clone',
+                      'label': 'Sing the adapted lyrics in the original voice',
+                      'status': 'ok', 'engine': 'f5-tts'})
+
+        # -- 4. optional Seed-VC singing pass ---------------------------------
+        converted_path = new_vocal
+        if request.sing_convert:
+            try:
+                from codai.api.voice_convert import _get_wrapper
+                wrapper = _get_wrapper()
+
+                def _convert():
+                    return wrapper.convert_voice(
+                        source=new_vocal, target=vocals_path,
+                        diffusion_steps=request.diffusion_steps or 15,
+                        length_adjust=1.0, inference_cfg_rate=0.7,
+                        f0_condition=True,           # singing mode: keep the melody
+                        pitch_shift=request.pitch_shift or 0,
+                        stream_output=False)
+
+                audio_out = await asyncio.get_event_loop().run_in_executor(None, _convert)
+                if isinstance(audio_out, tuple):
+                    audio_out = audio_out[0]
+                import numpy as _np
+                import soundfile as _sf
+                converted_path = os.path.join(workdir, 'converted_vocals.wav')
+                _sf.write(converted_path, _np.array(audio_out).flatten(), 44100)
+                steps.append({'step': 4, 'type': 'voice_convert',
+                              'label': 'Match the original singing voice',
+                              'status': 'ok', 'engine': 'seed-vc',
+                              'f0_condition': True})
+            except ImportError:
+                steps.append({'step': 4, 'type': 'voice_convert',
+                              'label': 'Match the original singing voice',
+                              'status': 'skipped',
+                              'reason': 'seed-vc is not installed — using the '
+                                        'F5-TTS vocal as-is. Run: pip install seed-vc'})
+                warnings.append('seed-vc not installed: the replacement vocal is spoken-'
+                                'style TTS rather than pitch-matched singing')
+            except Exception as exc:
+                steps.append({'step': 4, 'type': 'voice_convert',
+                              'label': 'Match the original singing voice',
+                              'status': 'failed', 'reason': str(exc)})
+                warnings.append(f'singing conversion failed, using the raw vocal: {exc}')
+        else:
+            steps.append({'step': 4, 'type': 'voice_convert',
+                          'label': 'Match the original singing voice',
+                          'status': 'skipped', 'reason': 'sing_convert=false'})
+
+        # -- 5. remix over the original instrumental --------------------------
+        final_mix = os.path.join(workdir, 'final_mix.wav')
+        proc = subprocess.run(
+            ['ffmpeg', '-y', '-i', converted_path, '-i', instrumental_path,
+             '-filter_complex', 'amix=inputs=2:duration=longest:normalize=0,alimiter',
+             final_mix],
+            capture_output=True, text=True)
+        if proc.returncode != 0:
+            raise HTTPException(status_code=500,
+                                detail=f'Remix failed: {(proc.stderr or "")[-500:]}')
+        steps.append({'step': 5, 'type': 'remix',
+                      'label': 'Remix the new vocal over the instrumental',
+                      'status': 'ok', 'engine': 'ffmpeg'})
+
+        complete = all(s.get('status') == 'ok' for s in steps)
         return {
-        'vocals': {'path': 'vocals.wav'},
-        'instrumental': {'path': 'instrumental.wav'},
+            'vocals': _persist_file(vocals_path, '.wav', http_request),
+            'instrumental': _persist_file(instrumental_path, '.wav', http_request),
             'transcript': transcript,
-        'translated_lyrics': translated,
-        'converted_vocals': {'path': 'converted_vocals.wav'},
-        'final_mix': {'path': 'final_mix.wav'},
+            'translated_lyrics': lyrics,
+            'converted_vocals': _persist_file(converted_path, '.wav', http_request),
+            'final_mix': _persist_file(final_mix, '.wav', http_request),
             'steps': steps,
+            'warnings': warnings,
+            'complete': complete,
         }
 
 
@@ -567,7 +794,10 @@ async def run_audio_music_dub(request: AudioMusicDubRequest, http_request: Reque
     return {
         'created': int(time.time()),
         'pipeline': 'audio-music-dub',
-        'status': 'available',
+        # 'complete' = every stage ran; 'degraded' = the mix was produced but at
+        # least one stage was skipped or fell back (see `warnings`).
+        'status': 'complete' if result['complete'] else 'degraded',
+        'warnings': result['warnings'],
         'vocals': result['vocals'],
         'instrumental': result['instrumental'],
         'transcript': result['transcript'],