Intermittent issues with voice in our Apple Vision Pro app
We're trying to rule-out a few issues with voice not working for some of the players in our app (in development). The typical scenario is that two players can speak and hear each other, but one can hear but not be heard. First, we want to rule out whether or not this is a known issue with Testflight builds, since it doesn't appear to be happening with builds we install on our AVPs out of Xcode. We are in different geographical regions, but latency doesn't really seem to be a problem. For instance, when it's working, it's just a little slower than Zoom calls which we'll test against as we're talking. Avatars work and we can see each other move. It's just the voice (and mouth movements also don't work on the affected user). The voice issue occurs randomly, and to a random user, but usually 2 of the 3 of us testing are able to communicate. We also have a random issue where a room code won't work for one of us. but I'm not sure if it's related. One of us is roughly 1300 miles (by air) from the other two, but I'm not sure if that matters, since it's not always the furthest away who is blocked from being heard.
8 Replies
Hi @s_q_u_i_s_h. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
Based on the information provided, there could be several reasons why the voice chat is not working for some players in your app. Here are a few things you can check:
1. Microphone Permissions: Ensure that your app has the necessary microphone permissions. On platforms like iOS, you need to set your Microphone Usage Description correctly in Unity. If something fails, Normcore will print an error message to the console. It's recommended to check your player logs for more information if the audio is not working [source (https://normcore.io/documentation/realtime/common-questions#voice-chat-doesnt-work-why)].
2. Network Issues: Normcore uses WebRTC for transport, which supports both UDP and TCP. Many corporate networks block UDP traffic, and if that happens, WebRTC allows you to fall back to TCP. If WebRTC detects that packets are stalling over TCP, Normcore can reduce the amount of traffic sent to prevent packets from building up. This could be a potential issue if the players are on different networks or geographical locations [source (https://normcore.io/documentation/architecture/transport#webrtc-supports-udp-and-tcp)].
3. Audio Preprocessor Settings: RealtimeAvatarVoice includes an AudioPreprocessor that automatically sets the gain for the microphone and filters out background noise, reverb, and echo cancellation. If you need to make changes to these settings, you can modify the source code of RealtimeAvatarVoice [source (https://normcore.io/documentation/realtime/xr/voicechat#audiopreprocessor)].
4. RealtimeAvatarVoice Component: Ensure that the RealtimeAvatarVoice component is correctly set up. It should be owned locally for sending audio and owned by a remote client for receiving audio. Also, check if the
mute property is correctly set. If RealtimeAvatarVoice represents a local player, this property will stop sending audio, effectively muting this player for everyone in the room [source (https://normcore.io/documentation/realtime/xr/voicechat)].
If you're still encountering issues after checking these points, I recommend reaching out to the Normcore community on their Discord (https://normcore.io/documentation/essentials/common-questions#does-normcore-support-x-y-z) for further assistance.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------<@684849348752048169> kapa.ai is still learning and improving, please let us know how it did by reacting below
Is this only a problem specifically on AVP, or you haven't tried other platforms? It sounds like the same issue I had a couple of years ago in my quest game. I didn't fix the issue from occurring, but i wrote a solution to fix it after it occurred. I think i shared my fix with Max a while back, but i dont know if it was ever fixed in a later normcore version.
I copied and customised the included voice script and added this coroutine. Its started in Start for remotely owned voice instances, and when _rebuildAudioStream is set to true it calls ConnectAudioStream() again. I've never seen the issue again since adding this, but im not 100% sure its the same issue you're seeing.
Thanks @liamc . It is only on AVP. It's nice to see a solution. Apprieciate it. We'll give it a try.
does it happen on the first launch / room join? or only after leaving / rejoining a room without starting the app launch from scratch?
we’ve seen some issues with Apple platforms around not wanting to reinitialize the microphone device if it hasn’t fully shut down from a previous session
but it won’t be triggered the first time you launch the app and connect to a room
if we can determine if that’s the bug, there are a few options
Hi @Max. It was happening when we first launched, but we were also asking for mic permissions after joining a room for the first time. We moved asking the mic permissions earlier and it solved some of the problem. Now we're seeing the mic functionality drop for one user during a session in the game. We think this is related to network latency and are creating a workaround to initiate a refresh of the mic functionality when this happens.
@liamc We tried your approach and we can't run it continually because it appears to be eating up too many resources on the AVP and causing pretty serious latency issues with the stream.
That doesnt sound right. A single coroutine per voice instance with a couple of null checks once a second shouldn't have a measurable cpu cost.
Yeah, I wouldn't think so either, but something is happening. My dev thinks it's memory, not CPU, but we're still looking into it.
He can chime in here: @fgamez1020