Skip to content

Native audio input support for Gemini and Muse models #1131

Open

Description

@jhooc77

Feature Description

Is native audio input support planned for Gemini and Muse models?

Sending WAV audio as input_audio to Command Code’s Provider API (POST /provider/v1/chat/completions) returns HTTP 400 (Invalid input, messages.0.content) with both google/gemini-4-argon and meta/muse-ember.

Could Command Code support native audio input or provide a compatible endpoint for these models? Is there an expected timeline?

Use Case

Direct audio transcription, translation, and timestamped subtitle generation using the model’s audio understanding.

Additional Context

For comparison, Gemini 4 Argon successfully transcribes audio through OpenCode’s Google-native endpoint using inlineData, with AUDIO input tokens confirmed in the response:

POST https://opencode.ai/inference/go/google/v1beta/models/gemini-4-argon:generateContent

The audio is passed as Base64 WAV data in contents[].parts[].inlineData with mimeType: "audio/wav".

Related: #906

How important is this to you?

Important for my workflow

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions