Sending another AudioSource together with the voice over the server.
I've made an NPC that generates AI responses in voice. Only one player can speak to the AI NPC, but I do want other players to hear the responses. It's rather out of the box, but most simple solution would be to mix the NPC's audiosource in with the local player's voice audiosource before sending it to the Realtime Voice component. Before I try and do this, is this even possible?
2 Replies
Hi @Water. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
Short answer: not directly via RealtimeAvatarVoice. It automatically creates one microphone input stream for the owner and one playback stream for remotes; it doesn’t expose a way to mix arbitrary AudioSources into that upstream mic stream (Voice Chat). If you need to send NPC audio so all clients hear it, use Normcore’s media stream API to create your own audio input stream on the client that “hosts” the NPC and broadcast it, while others subscribe to the corresponding output stream:
- Create an AudioInputStream on the host: CreateAudioInputStream(voice: true, sampleRate, channels).
- Send the NPC’s generated PCM frames via SendRawAudioData on that stream.
- On other clients, get the matching AudioOutputStream using the host clientID and streamID, then play it back (e.g., pipe data to an AudioSource). (Room media streams, AudioInputStream, AudioOutputStream, Client API).
Notes:
- RealtimeAvatarVoice is great for mic chat and supports spatialization via a standard Unity AudioSource on the receiver, but it’s not documented to accept mixed external audio as input (Voice Chat, Common questions: voice).
- If all you need is to mute/unmute or level visualization, RealtimeAvatarVoice has mute and voiceVolume, but that won’t help inject NPC audio into the uplink (Voice Chat). If you need code, I can outline how to capture your NPC’s PCM buffer and push it through CreateAudioInputStream/SendRawAudioData, and how to subscribe on listeners via GetAudioOutputStream. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------
- If all you need is to mute/unmute or level visualization, RealtimeAvatarVoice has mute and voiceVolume, but that won’t help inject NPC audio into the uplink (Voice Chat). If you need code, I can outline how to capture your NPC’s PCM buffer and push it through CreateAudioInputStream/SendRawAudioData, and how to subscribe on listeners via GetAudioOutputStream. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------
<@456917293310541830> Kapa.ai is still learning and improving, please let me know how I did by reacting below.