Intermittent issues with voice in our Apple Vision Pro app
We're trying to rule-out a few issues with voice not working for some of the players in our app (in development). The typical scenario is that two players can speak and hear each other, but one can hear but not be heard. First, we want to rule out whether or not this is a known issue with Testflight builds, since it doesn't appear to be happening with builds we install on our AVPs out of Xcode. We are in different geographical regions, but latency doesn't really seem to be a problem. For instance, when it's working, it's just a little slower than Zoom calls which we'll test against as we're talking. Avatars work and we can see each other move. It's just the voice (and mouth movements also don't work on the affected user). The voice issue occurs randomly, and to a random user, but usually 2 of the 3 of us testing are able to communicate. We also have a random issue where a room code won't work for one of us. but I'm not sure if it's related. One of us is roughly 1300 miles (by air) from the other two, but I'm not sure if that matters, since it's not always the furthest away who is blocked from being heard.
15 Replies
Hi @s_q_u_i_s_h. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
Based on the information provided, there could be several reasons why the voice chat is not working for some players in your app. Here are a few things you can check:
1. Microphone Permissions: Ensure that your app has the necessary microphone permissions. On platforms like iOS, you need to set your Microphone Usage Description correctly in Unity. If something fails, Normcore will print an error message to the console. It's recommended to check your player logs for more information if the audio is not working [source (https://normcore.io/documentation/realtime/common-questions#voice-chat-doesnt-work-why)].
2. Network Issues: Normcore uses WebRTC for transport, which supports both UDP and TCP. Many corporate networks block UDP traffic, and if that happens, WebRTC allows you to fall back to TCP. If WebRTC detects that packets are stalling over TCP, Normcore can reduce the amount of traffic sent to prevent packets from building up. This could be a potential issue if the players are on different networks or geographical locations [source (https://normcore.io/documentation/architecture/transport#webrtc-supports-udp-and-tcp)].
3. Audio Preprocessor Settings: RealtimeAvatarVoice includes an AudioPreprocessor that automatically sets the gain for the microphone and filters out background noise, reverb, and echo cancellation. If you need to make changes to these settings, you can modify the source code of RealtimeAvatarVoice [source (https://normcore.io/documentation/realtime/xr/voicechat#audiopreprocessor)].
4. RealtimeAvatarVoice Component: Ensure that the RealtimeAvatarVoice component is correctly set up. It should be owned locally for sending audio and owned by a remote client for receiving audio. Also, check if the
mute property is correctly set. If RealtimeAvatarVoice represents a local player, this property will stop sending audio, effectively muting this player for everyone in the room [source (https://normcore.io/documentation/realtime/xr/voicechat)].
If you're still encountering issues after checking these points, I recommend reaching out to the Normcore community on their Discord (https://normcore.io/documentation/essentials/common-questions#does-normcore-support-x-y-z) for further assistance.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------<@684849348752048169> kapa.ai is still learning and improving, please let us know how it did by reacting below
Is this only a problem specifically on AVP, or you haven't tried other platforms? It sounds like the same issue I had a couple of years ago in my quest game. I didn't fix the issue from occurring, but i wrote a solution to fix it after it occurred. I think i shared my fix with Max a while back, but i dont know if it was ever fixed in a later normcore version.
I copied and customised the included voice script and added this coroutine. Its started in Start for remotely owned voice instances, and when _rebuildAudioStream is set to true it calls ConnectAudioStream() again. I've never seen the issue again since adding this, but im not 100% sure its the same issue you're seeing.
Thanks @liamc . It is only on AVP. It's nice to see a solution. Apprieciate it. We'll give it a try.
does it happen on the first launch / room join? or only after leaving / rejoining a room without starting the app launch from scratch?
we’ve seen some issues with Apple platforms around not wanting to reinitialize the microphone device if it hasn’t fully shut down from a previous session
but it won’t be triggered the first time you launch the app and connect to a room
if we can determine if that’s the bug, there are a few options
Hi @Max. It was happening when we first launched, but we were also asking for mic permissions after joining a room for the first time. We moved asking the mic permissions earlier and it solved some of the problem. Now we're seeing the mic functionality drop for one user during a session in the game. We think this is related to network latency and are creating a workaround to initiate a refresh of the mic functionality when this happens.
@liamc We tried your approach and we can't run it continually because it appears to be eating up too many resources on the AVP and causing pretty serious latency issues with the stream.
That doesnt sound right. A single coroutine per voice instance with a couple of null checks once a second shouldn't have a measurable cpu cost.
Yeah, I wouldn't think so either, but something is happening. My dev thinks it's memory, not CPU, but we're still looking into it.
He can chime in here: @fgamez1020
Hi, was a fix found for this? We are sometimes experiencing odd muting of players, where some specific players aren't able to hear other specific players. It happens on a per-room basis, meaning if we reconnect to another room the issue sometimes resolves. This is on a Quest 3.
We did identify two bugs that we have fixes for:
1. Voice works but then stops working almost immediately or drops later on in this session.
- We found Meta had made some changes to the OVR microphone API and how samples are delivered. The fix has been implemented, but I need to triple check if it's been released or if it will be in our next release.
2. If the system microphone dialog comes up mic permissions work, but later on the mic is silent
- This one can have two causes, one was fixed a little over a year ago, so please make sure you're on the latest Normcore and if you've forked RealtimeAvatarVoice, you've updated to the later
- The other cause can come from trying to make any Oculus Platform API calls that lead to the Oculus Platform Initialize() or InitializeAsync() method being called at the same time. It turns out that if Initialize is called while another initialize operation is in progress, one of them will fail silently and never recover. This ends up with the OVR Microphone API never getting initialized. We see a lot of folks do this on accident when implementing name tags and trying to request the Oculus username from the platform API on prefab instantiation. The best move here is to initialize the oculus platform before you connect to a room server.
Voice works for us usually, but we do move to different rooms during sessions, and in some of those rooms we experience this issue, only for it to change in other rooms.
We've been using the stock RealtimeAvatarVoice, and most recently on Normcore 2.13.1, though we did switch to the above fork yesterday, but haven't tested that yet in a multiplayer session yet.
We do use the
PlatformInit.GetEntitlementInformation(OnEntitlementFinished); call for both nametags and our backend solution. The first call we make to this occurs in a title screen before connecting to a room, but the nametag calls do occur in rooms.
We did find also this post from a few years ago where you say:
Typically VR players with voice chat, you can get 4-8 players per room on Normcore Public before the server hits CPU limits we have in place.We have been testing with anywhere between 1-6 players, but are curious if increasing the room size might also be a way to alleviate this (or other) issue(s). Thank you for looking into this!
If it's only when switching rooms, sometimes the issue can be the microphone device re-initialization and it can be worth modifying RealtimeAvatarVoice to hold the reference to the mic between instances
what effects are you seeing?
The way I'd describe it technically is that, of each directional player->player audio stream, some random subset fails per room session. So, some certain players end up not able to hear other certain players, but it isn't always reciprocal nor consistent across players, though it is sometimes.
I'm going to try having the nametag logic run before the room connection starts in case that call is making another Initialize() somehow like you've suggested.
yeah that’s what it sounds like to me
The fix for #1 has shipped in Normcore v2.13.2
@fgamez1020 @rMadrid13 👆