Can I spawn a prefab with RealtimeView on it, and then immediately remove the RealtimeView?

I basically have a prefab that I want to mostly spawn with Normal.Realtime.Instantiate. Except for ONE case where I'd like to just spawn them locally with GameObject.Instantiate. There are a TON of variants, so I really do NOT want to make a separate set of prefabs without the RealtimeView on it.

Is this allowed? Just instantly removing the components after GameObject.Instantiate?