juanje89
juanje893mo ago

Syncing music

Is it possible to execute an audiosource clip and that all the clients hear it? There is a recipe in the documentation (Recipe: Timers + Synchronized Playback) that says that this sync method can be done to sync audio, however, I could not achieve it. I'm trying to sync a float with the playback time but no success.
Solution:
We're still refining it. But here's the current state of RealtimeTimer if you want to throw this in your project. Maybe throw it in a different namespace so it doesn't conflict when we ship this in Normcore.
Jump to solution
41 Replies
kapa.ai
kapa.ai3mo ago
Hi @juanje89. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Yes, it is possible to synchronize audio playback across all clients in Normcore. The key is to use a synchronized clock to drive the playback. This clock is set to match the room server's clock exactly with no added latency. Once connected to a room, the room time is available via the realtime.room.time property. Here is a basic example of how you can use this property to create a synchronized stopwatch:
using Normal.Realtime;

public class Stopwatch : RealtimeComponent<StopwatchModel> {
public float time {
get {
// Return 0 if we're not connected to the room yet.
if (model == null) return 0.0f;

// Make sure the stopwatch is running
if (model.startTime == 0.0) return 0.0f;

// Calculate how much time has passed
return (float)(realtime.room.time - model.startTime);
}
}

public void StartStopwatch() {
model.startTime = realtime.room.time;
}
}
using Normal.Realtime;

public class Stopwatch : RealtimeComponent<StopwatchModel> {
public float time {
get {
// Return 0 if we're not connected to the room yet.
if (model == null) return 0.0f;

// Make sure the stopwatch is running
if (model.startTime == 0.0) return 0.0f;

// Calculate how much time has passed
return (float)(realtime.room.time - model.startTime);
}
}

public void StartStopwatch() {
model.startTime = realtime.room.time;
}
}
In this example, realtime.room.time is used to calculate the elapsed time since the stopwatch was started. You can use a similar approach to synchronize the playback time of your audio source. However, please note that Normcore uses metadata on the prefab in order to connect RealtimeComponents on all clients. Therefore, you cannot add RealtimeComponents at runtime. If you're still having trouble, it might be helpful to review the Normcore documentation (https://normcore.io/documentation/guides/recipes/timers-and-synchronized-playback) on this topic. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------
kapa.ai
kapa.ai3mo ago
<@759006240646103041> kapa.ai is still learning and improving, please let us know how it did by reacting below
maxweisel
maxweisel3mo ago
@kapa.ai how would you use this to synchronize playback of an audio file in Unity? @juanje89 we’ll see if kapa responds here, but in short, I believe you can set an AudioClip/AudioSource playback position. You’d use the time property here to set it. We’re also about to ship a proper timer component that will help make this easier. @Tom any idea why kapa isn’t responding to mentions?
Towellie
Towellie3mo ago
@Normcore Assistant Hello?
kapa.ai
kapa.ai3mo ago
Let me check the knowledge base for you. One moment! :books: ---------------------- Hello! How can I assist you today? ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------
kapa.ai
kapa.ai3mo ago
<@497453196797804544> kapa.ai is still learning and improving, please let us know how it did by reacting below
maxweisel
maxweisel3mo ago
oh is there two kapa users in here? can we remove the one that doesn’t work? @Normcore Assistant how would you use this to synchronize playback of an audio file in Unity?
kapa.ai
kapa.ai3mo ago
Let me check the knowledge base for you. One moment! :books: ---------------------- To synchronize the playback of an audio file in Unity using Normcore, you can use a similar approach to the stopwatch example provided in the documentation. Here's a basic outline of how you might do it: 1. Create a new RealtimeModel to hold the start time of the audio playback. This could be similar to the StopwatchModel in the example.
using Normal.Realtime;
using Normal.Realtime.Serialization;

[RealtimeModel]
public class AudioSyncModel {
[RealtimeProperty(1, true)] private double _startTime;
}
using Normal.Realtime;
using Normal.Realtime.Serialization;

[RealtimeModel]
public class AudioSyncModel {
[RealtimeProperty(1, true)] private double _startTime;
}
2. Create a new RealtimeComponent that uses this model. This component will start the audio playback and calculate the current playback time.
using Normal.Realtime;
using UnityEngine;

public class AudioSync : RealtimeComponent<AudioSyncModel> {
private AudioSource _audioSource;

private void Awake() {
_audioSource = GetComponent<AudioSource>();
}

public void StartPlayback() {
model.startTime = realtime.room.time;
_audioSource.Play();
}

public float PlaybackTime {
get {
if (model == null || model.startTime == 0.0) return 0.0f;
return (float)(realtime.room.time - model.startTime);
}
}
}
using Normal.Realtime;
using UnityEngine;

public class AudioSync : RealtimeComponent<AudioSyncModel> {
private AudioSource _audioSource;

private void Awake() {
_audioSource = GetComponent<AudioSource>();
}

public void StartPlayback() {
model.startTime = realtime.room.time;
_audioSource.Play();
}

public float PlaybackTime {
get {
if (model == null || model.startTime == 0.0) return 0.0f;
return (float)(realtime.room.time - model.startTime);
}
}
}
3. Attach this component to the GameObject that has the AudioSource you want to synchronize. Call StartPlayback() to start the audio and synchronize it across all clients. Please note that this is a simplified example and might not cover all edge cases. For example, it doesn't handle pausing the audio, seeking to a different time, or synchronizing the audio state (playing/paused) across clients. You would need to extend this example to handle those cases. Also, remember that Normcore uses Unity's audio engine for playback, so all of the Unity audio APIs should work correctly. However, voice chat spatialization does not work on the web because Unity's audio engine, FMOD, does not run on the web. All voice-chat audio is played directly through the browser rather than through Unity (source (https://normcore.io/documentation/platforms/webgl#voice-chat-spatialization-does-not-work-on-the-web)). For more information, you can refer to the Normcore documentation (https://normcore.io/documentation/guides/recipes/timers-and-synchronized-playback). ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------
kapa.ai
kapa.ai3mo ago
<@353762082451095552> kapa.ai is still learning and improving, please let us know how it did by reacting below
juanje89
juanje893mo ago
Oh, great, I'll try this!! Though the response looks the same as the recipe
maxweisel
maxweisel3mo ago
This response is good, but PlaybackTime is off. You don't need much on top of the recipe. @Normcore Assistant How do you set the audio source playback time to match the model? Assume playback was started by another client and the model already has a startTime set.
kapa.ai
kapa.ai3mo ago
Let me check the knowledge base for you. One moment! :books: ---------------------- To set the audio source playback time to match the model, you can use the AudioSource.time property in Unity. This property gets or sets the current position of the audio clip that the audio source is playing, in seconds. You can calculate the desired playback time based on the startTime in the model and the current room time. If the audio source is not already playing when the model is replaced (i.e., when another client has started playback), you should start it and set the time accordingly. Here's how you might modify the AudioSync class to handle this:
using Normal.Realtime;
using UnityEngine;

public class AudioSync : RealtimeComponent<AudioSyncModel> {
private AudioSource _audioSource;

private void Awake() {
_audioSource = GetComponent<AudioSource>();
}

protected override void OnRealtimeModelReplaced(AudioSyncModel previousModel, AudioSyncModel currentModel) {
if (model != null && model.startTime != 0.0) {
if (!_audioSource.isPlaying) {
_audioSource.time = PlaybackTime;
_audioSource.Play();
}
}
}

public void StartPlayback() {
model.startTime = realtime.room.time;
_audioSource.Play();
}

public float PlaybackTime {
get {
if (model == null || model.startTime == 0.0) return 0.0f;
return (float)(realtime.room.time - model.startTime);
}
}
}
using Normal.Realtime;
using UnityEngine;

public class AudioSync : RealtimeComponent<AudioSyncModel> {
private AudioSource _audioSource;

private void Awake() {
_audioSource = GetComponent<AudioSource>();
}

protected override void OnRealtimeModelReplaced(AudioSyncModel previousModel, AudioSyncModel currentModel) {
if (model != null && model.startTime != 0.0) {
if (!_audioSource.isPlaying) {
_audioSource.time = PlaybackTime;
_audioSource.Play();
}
}
}

public void StartPlayback() {
model.startTime = realtime.room.time;
_audioSource.Play();
}

public float PlaybackTime {
get {
if (model == null || model.startTime == 0.0) return 0.0f;
return (float)(realtime.room.time - model.startTime);
}
}
}
In this example, OnRealtimeModelReplaced is overridden to check if the audio should be playing when the model is replaced. If the startTime in the model is not zero (indicating that playback has started), and the audio source is not already playing, it starts the audio source and sets the playback time to match the model. Please note that this is a simplified example and might not cover all edge cases. For example, it doesn't handle pausing the audio, seeking to a different time, or synchronizing the audio state (playing/paused) across clients. You would need to extend this example to handle those cases. For more information, you can refer to the Normcore documentation (https://normcore.io/documentation/guides/recipes/timers-and-synchronized-playback). ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------
kapa.ai
kapa.ai3mo ago
<@353762082451095552> kapa.ai is still learning and improving, please let us know how it did by reacting below
maxweisel
maxweisel3mo ago
This looks good
juanje89
juanje893mo ago
Great, I'll give it a try!
cryptomax
cryptomax3mo ago
a proper timer with a start and stop event would be fantastic, i'm about to make one atm
Solution
maxweisel
maxweisel3mo ago
We're still refining it. But here's the current state of RealtimeTimer if you want to throw this in your project. Maybe throw it in a different namespace so it doesn't conflict when we ship this in Normcore.
cryptomax
cryptomax3mo ago
THANK YOU!
maxweisel
maxweisel3mo ago
@Tom reputation bot should give me more points because it was written in all caps ;P
cryptomax
cryptomax3mo ago
lol... double xp
maxweisel
maxweisel3mo ago
lmk how it works for you. would love any feedback so we can incorporate it before the feature ships
cryptomax
cryptomax3mo ago
I have multiple use cases but is it a bad idea to use something like this for say hundreds of objects?
maxweisel
maxweisel3mo ago
depends on when you start / stop them. This only incurs a network cost when you start, stop, or pause. If you start 100+ of these on the same frame you may end up with a big packet, but I doubt it will be too too bad worth trying!
cryptomax
cryptomax3mo ago
should be offset sweet sounds good! I get RealtimeTimer.cs(3,18): error CS0311: The type 'Normal.Realtime.RealtimeTimerModel' cannot be used as type parameter 'TModel' in the generic type or method 'RealtimeComponent<TModel>'. There is no implicit reference conversion from 'Normal.Realtime.RealtimeTimerModel' to 'Normal.Realtime.RealtimeModel'. i'm running 2.7 would I need to update probably?
maxweisel
maxweisel3mo ago
hmm no you should be good does the model have any other errors? this looks like the autogenerated code isn’t being generated here and so the model doesn’t have the base class of RealtimeModel
cryptomax
cryptomax3mo ago
I don't think i've made a new model since updating to 2.7, is there something you need to do to make models compile? I mean generate
maxweisel
maxweisel3mo ago
nope, it should be automatic unless you’re on an old version of Unity
cryptomax
cryptomax3mo ago
2020.3 so a bit
maxweisel
maxweisel3mo ago
in which case you’ll need to compile it but the project needs to be in a compiling state for the button to appear let me check ah yeah roslyn generator support is 2021+
cryptomax
cryptomax3mo ago
crap... 2021 hard crashes my current game on start 😦
maxweisel
maxweisel3mo ago
all good. start by adding just the model. then hit the compile button then add the timer and timer editor files to your project should work just fine
cryptomax
cryptomax3mo ago
copy we're back in business!
maxweisel
maxweisel3mo ago
💪🏻💪🏻
cryptomax
cryptomax3mo ago
beautiful! implemented and working like a charm! I haven't actually tested it multiplayer but since it's activating from the timer change event should be working.
maxweisel
maxweisel3mo ago
awesome!
Towellie
Towellie3mo ago
LMAO agreed
juanje89
juanje893mo ago
I have tried the suggested script and it works when a client plays the AudioSource and then, another client connects to the same room; however, existing players in the room do not hear the audio. I believe this happens becauce OnRealtimeModelReplaced is executed when the client connects @Normcore Assistant
Mechabit
Mechabit3mo ago
just add the didchange event parameter to the model
juanje89
juanje893mo ago
Do I need to subscribe to such event, or does OnRealtimeModelReplaced gets called automatically after adding the third parameter to the model?
Mechabit
Mechabit3mo ago
sub the event. check the docs for how to sub an event
juanje89
juanje893mo ago
Thanks!