Table of Contents

Sending Voice

In this section, you'll learn how to send audio to Discord voice channels using NetCord. We'll cover the basics of audio streaming, encoding, and best practices for creating music bots and other voice applications.

Sending Audio

NetCord provides 2 ways of sending audio to voice channels. CreateVoiceStream method creates a stream that you can write Opus-encoded audio data to, and it will handle sending it to Discord. If you need more control over the streaming process, you can use SendVoice or SendVoiceAsync. They allow you to send Opus-encoded audio data directly as well as handling the timing of the packets.

For most music bots, using the stream returned by CreateVoiceStream is sufficient. It's easier to use and abstracts away the complexities of packet timing and management. It also integrates well with OpusEncodeStream, allowing you to easily encode and stream audio in one step, which you will learn about in the next section.

var voiceStream = voiceClient.CreateVoiceStream();

The voice stream also supports configuration options, such as disabling the speed normalization or using a custom frame duration, you can set these options using VoiceStreamConfiguration when creating the stream. Note that disabling speed normalization makes you responsible for ensuring that the audio is written at the correct speed. Frame duration is also important to get right, you must ensure that the Opus frames you write match the frame duration specified in the configuration, otherwise you may encounter issues with audio timing. Allowed frame durations in milliseconds are 2.5, 5, 10, 20, 40, 60, 80, 100, and 120 (120 ms is only supported for mono audio). The default frame duration is 20ms. Refer to the Opus Documentation for more details on frame duration and its impact.

var voiceStream = voiceClient.CreateVoiceStream(new VoiceStreamConfiguration
{
    NormalizeSpeed = false,
    FrameDuration = 60,
});

Opus Encoding

Discord uses the Opus codec for voice communication. To send audio to a voice channel, you need to encode your audio data in Opus format. NetCord provides OpusEncodeStream, which allows you to encode PCM audio data into Opus format that can be sent to Discord. It uses OpusEncoder internally, which can be used directly if you need more control over the encoding process.

When you create an OpusEncodeStream, you need to specify the PCM format, the number of channels, as well as the Opus application. The supported PCM formats are 16-bit signed integer (Short) and 32-bit floating point (Float). Float is generally preferred for high-quality audio. The number of channels can be either 1 (Mono) or 2 (Stereo). The Opus application can be either Voip, Audio, RestrictedLowdelay, RestrictedSilk, or RestrictedCelt, depending on your use case. For music bots, you should use Audio, as it is optimized for music streaming. Feel free to refer to OpusApplication for more details on the different Opus applications and their use cases.

OpusEncodeStream opusEncodeStream = new(voiceStream,
                                        PcmFormat.Float,
                                        VoiceChannels.Stereo,
                                        OpusApplication.Audio);

Aside of the required parameters, you can also specify whether to segment the audio and the frame duration to use when encoding. If you choose to segment the audio, the stream will buffer the PCM data until it has enough to encode a full Opus frame. If you choose not to segment, you need to ensure that you write PCM data in chunks that correspond to the Opus frame size. Segmentation is enabled by default. You can also specify the frame duration to use when encoding, which must match the frame duration used by the voice stream you are writing to. The default frame duration is 20ms and matches the default frame duration of the voice stream.

OpusEncodeStream opusEncodeStream = new(voiceStream,
                                        PcmFormat.Float,
                                        VoiceChannels.Stereo,
                                        OpusApplication.Audio,
                                        new OpusEncodeStreamConfiguration
                                        {
                                            Segment = false,
                                            FrameDuration = 2.5f,
                                        });

Audio Formats

Discord voice channels only support raw Opus-encoded audio. Since Opus is a codec and not a container format, you will need to convert your audio files to raw Opus format before sending them to a voice channel. This typically involves converting your audio data to PCM and then encoding it to Opus format using OpusEncodeStream or OpusEncoder.

You can use for example FFmpeg to convert various audio formats to PCM. The example below demonstrates how to use it to convert an input audio to 32-bit float PCM format and 2 channels and then stream the audio directly to an OpusEncodeStream, which will handle encoding it to Opus and pass it to the voice stream to be sent to Discord.

using var process = Process.Start(new ProcessStartInfo
{
    FileName = "ffmpeg",
    ArgumentList =
    {
        // Input file
        "-i", input,
        // Output format 32-bit float PCM with native endianness
        "-f", BitConverter.IsLittleEndian ? "f32le" : "f32be",
        // Sampling rate 48kHz
        "-ar", "48000",
        // 2 channels (stereo)
        "-ac", "2",
        // Output to stdout
        "pipe:1",
    },
    RedirectStandardOutput = true,
})!;

var ffmpegOutput = process.StandardOutput.BaseStream;

// Copy the FFmpeg output directly to the OpusEncodeStream
await ffmpegOutput.CopyToAsync(opusEncodeStream);

// Flush the OpusEncodeStream to ensure all data is sent,
// appended by the silence frames to prevent audio interpolation
await opusEncodeStream.FlushAsync();

Updating Our Bot

Now, it's time to finally... add a /play command to our bot! This command plays an audio file in the voice channel the bot is currently connected to.

host.AddSlashCommand("play", "Plays audio", async (ApplicationCommandContext context) =>
{
    if (context.Guild is not { } guild)
    {
        await context.Interaction.SendResponseAsync(InteractionCallback.Message(new InteractionMessageProperties()
            .WithContent("The guild is not available. Try again later.")
            .WithFlags(MessageFlags.Ephemeral)));
        return;
    }

    var guildId = guild.Id;

    if (!voiceInstances.TryGetValue(guildId, out var voiceInstance) || voiceInstance is null)
    {
        await context.Interaction.SendResponseAsync(InteractionCallback.Message(new InteractionMessageProperties()
            .WithContent("Not connected to a voice channel in this guild.")
            .WithFlags(MessageFlags.Ephemeral)));
        return;
    }

    using var job = voiceInstance.TryEnterJob(VoiceJobType.Playing);
    if (job is not { CancellationToken: var cancellationToken })
    {
        await context.Interaction.SendResponseAsync(InteractionCallback.Message(new InteractionMessageProperties()
            .WithContent("Already playing audio in this guild.")
            .WithFlags(MessageFlags.Ephemeral)));
        return;
    }

    await context.Interaction.SendResponseAsync(InteractionCallback.Message($"Playing..."));

    var voiceClient = voiceInstance.Client;

    await voiceClient.EnterSpeakingStateAsync(new(SpeakingFlags.Microphone));

    using var voiceStream = voiceClient.CreateVoiceStream();
    using OpusEncodeStream opusEncodeStream = new(voiceStream,
                                                  PcmFormat.Float,
                                                  VoiceChannels.Stereo,
                                                  OpusApplication.Audio);

    const string Input = "https://netcord.dev/sounds/sample.mp3";

    using var ffmpeg = Process.Start(new ProcessStartInfo
    {
        FileName = "ffmpeg",
        ArgumentList =
        {
            "-i", Input,
            "-f", BitConverter.IsLittleEndian ? "f32le" : "f32be",
            "-ar", "48000",
            "-ac", "2",
            "pipe:1",
        },
        RedirectStandardOutput = true,
    })!;

    var ffmpegOutput = ffmpeg.StandardOutput.BaseStream;

    try
    {
        await ffmpegOutput.CopyToAsync(opusEncodeStream, cancellationToken);
        await opusEncodeStream.FlushAsync(cancellationToken);
    }
    catch (Exception ex)
    {
        ffmpeg.Kill();

        if (ex is not OperationCanceledException and not AggregateException { InnerException: OperationCanceledException })
            throw;
    }
}).AddContexts(InteractionContextType.Guild);

Now, our bot can finally play audio files! You can test it out by using the /play command. Please note that you need to use the /join command to have the bot join a voice channel before you can use the /play command.

See Also

Edit this page