☁️Network Integration

Introduction

KuteEngine™ can be used in a networking environment, and is totally agnostic to any network solution. For demo purposes, we provide sample code using Fusion, but you can use whichever framework you want.

All the integration is done by implementing a single interface, VirtualBeings.Network.NetworkInterface.

As the result of a tradeoff between the amount of computations per client, the amount of data to exchange per frame and the risk of differences of behavior between the clients for a same input, all high-level behavior decisions are only made on the host. This decision making by the host results in calls done to a low-level layer of the beings (called the actuator), which is what gets synchronized between the clients. Most of those calls are very punctual in nature and can be thought of as RPC calls.

Some specific properties need to be synchronized on a per-frame basis (like the position and orientation of the being), and should then get special treatment.

NetworkInterface

The interface that should be implemented and used is the following :

/// <summary>
/// This interface is the entry point for all network related calls from and to 
/// the Beings.
/// It is agnostic of any network solution, and the synchronization of the data sent 
/// through the interface has to be implemented by the user.
///
/// Each function can be seen as a host / client duet, with the host calling the 
/// "Send" or "Update" function, that then gets sent over the network, and the 
/// counterpart "OnReceived" or "OnUpdated" then gets called by the client upon.
///
/// For most of the calls, SendData() is called by the host (and must be forwarded 
/// to the clients via OnDataReceived()).
/// It can be thought of (and should probably be treated as) an RPC call.
///
/// Some properties need a per frame synchronization (root transform or look targets 
/// for example), and are hence separated to allow special treatment / optimizations 
/// (e.g. only sending if they change).
///
/// Upon initialization of a client being, retrieve the current state of the host by 
/// querying it through GetBeingState(), and forward the result to the client using 
/// SetBeingState().
/// </summary>
public abstract class NetworkInterface
{
  /// <summary>
  /// Called by the host. The given (opaque) buffer has to be transmitted to all 
  /// the clients.
  /// Sending this buffer is the responsibility of the user, using the network 
  /// solution of its choice.
  /// This call can be thought of (and should probably be implemented as) an RPC 
  /// call.
  /// </summary>
  public abstract void SendData(byte[] buffer);
	
  /// <summary>
  /// When the client receives a message coming from the host (via a call to 
  /// SendData()), it has to be forwarded to KuteEngine using this method.
  /// </summary>
  public void OnDataReceived(byte[] buffer)
  {
    _networkSerializer.DeserializeAndCall(buffer);
  }
	
  /// <summary>
  /// Called by the host on a per frame basis. Send data is then forwarded to the 
  /// client using OnLookDataUpdated.
  /// </summary>
  public abstract void UpdateLookData(Vector3? headTarget, Vector3? eyesTarget);
  public void OnLookDataUpdated(Vector3? headTarget, Vector3? eyesTarget)
  {
    _networkSerializer.UpdateLookTargets(headTarget, eyesTarget);
  }
	
  /// <summary>
  /// Called by the host on a per frame basis. Sent data is then forwarded to the 
  /// client using OnRootTranformUpdated.
  /// </summary>
  public abstract void UpdateRootTransform(Vector3 position, Quaternion rotation);
  public void OnRootTransformUpdated(Vector3 position, Quaternion rotation)
  {
    _networkSerializer.RootPosition = position;
    _networkSerializer.RootRotation = rotation;
  }
	
  /// <summary>
  /// Should be called by the host after the connection of a new client.
  /// The result is then sent to the client, and it has to call SetBeingState with 
  /// the given buffer.
  /// This can also be used to recover the state of the host after losing too much 
  /// information and the beings get desynchronized from the host state.
  ///
  /// Warning: This buffer is quite big compared to the one in SendData (a bit less 
  /// that 3kB), so it might require special treatment given the limitations of the 
  /// networking solution.
  /// </summary>
  public byte[] GetBeingState()
  {
    return _networkSerializer.GetActuatorState();
  }
	
  /// <summary>
  /// Called by the client upon reception of the result of GetBeingState sent by the 
  /// host.
  /// </summary>
  /// <param name="state"></param>
  public void SetBeingState(byte[] state)
  {
    _networkSerializer.SetActuatorState(state);
  }
	
  /// <summary>
  /// Called by the host each time it has to send a call that involves an 
  /// interactable (that should be networked and managed by the user). 
  /// It is used to make the link between the local (managed by KuteEngine) 
  /// IInteractable.ID, associated after the IInteractable got initialized on 
  /// KuteEngine's side.
  ///
  /// NOTE: The returned value MUST be strictly greater that 0.
  ///
  /// NOTE: No caching is done on KuteEngine's side, the function gets called every 
  /// time the information is needed for an interactable.
  /// </summary>
  public abstract int GetNetworkInteractableId(int localId);

  /// <summary>
  /// Called by the client after a call involving an interactable has been received. 
  /// This allows the client being to retrieve the correct local interactable.
  ///
  /// See GetNetworkInteractableId.
  ///
  /// </summary>
  public abstract int GetLocalInteractableId(int networkId);
}

State initialization (Get / Set Being State)

To be able to start the being with the correct state, or the resync it after packet losses, the NetworkInterface class provides the GetBeingState() and SetBeingState() functions.

GetBeingState() is supposed to be called on the host side. It retrieves a (large) opaque buffer, which then get sent to the client who requested it. SetBeingState() is called with the given buffer as input to force the client state to this.

The buffer here is quite large (3 KB at the time of writing), so one might have to write custom code to be able to deal with it (an example with fusion is provided later in this documentation).

SendData / OnDataReceived

Most of the magic happens here. The SendData() (which you have to implement) is called by the host being. It is a punctual call, that can be viewed as an RPC call. Upon receiving the message that has been sent, the clients forward it to the being using OnDataReceived(), with the given buffer as input.

The fusion sample implementation uses RPC calls to implement that.

UpdateProperty / OnPropertyUpdated

Some properties need to be synchronized on a per frame basis. To allow custom handling for such properties, they are separated and their values and meaning are not opaque.

Similarly to SendData() and OnDataReceived(), the host calls UpdateProperty() with the given values, and it should be forwarded to the clients using the OnUpdateProperty() method.

It is ok to do optimization such as not sending the call if the values have not changed, it is not mandatory to call it on the clients side in that case (and that will always hold true for those properties).

The fusion sample implementation uses networked properties with an associated callback to handle those function calls.

N.B.: The list of properties that will be synchronized using this method will probably grow a bit bigger in the future.

Interactable identifiers

Some objects (toys, food, etc.) with which the beings can interact and are spawned dynamically need some special treatment. Indeed, KuteEngine uses an ID (accessible through IInteractable.ID property), which is managed internally and makes no assumption regarding the fact it can be used over the network or not, and so it could end up being different between the host and the clients (depending on the order of spawning for example).

In that regard, when a message that will get sent over the network contains info about an interactable object, KuteEngine first asks the user for the “Networked” ID of this interactable. The assumption here is that the network solution used should provide some ID for all the networked objects, which will by design be the same for all the clients.

The local ID is assigned inside the OnEnable() Unity callback, so it’s available right after that.

N.B.: The GetNetworkInteractableId and GetLocalInteractableIdfunctions are called every time the ID of the interactable is needed, there is no caching done inside KuteEngine to avoid getting in the way of the user’s implementation of those IDs.

Fusion example implementation

The Fusion implementation is decoupled into two classes: FusionNetworkInterface (the implementation of the NetworkInterface) and the NetworkBeing (the fusion network object).

FusionNetworkInterface

The network interface implementation is very basic, and just forwards the calls to the actual network behavior (NetworkBeing)

public class FusionNetworkInterface : NetworkInterface
{
  private NetworkBeing _being;

  public FusionNetworkInterface(NetworkBeing being)
  {
    _being = being;
  }
  
  public override int GetLocalInteractableId(int networkId)
  {
      return NetworkManager.GetLocalInteractableId(networkId);
  }

  public override int GetNetworkInteractableId(int localId)
  {
      return NetworkManager.GetNetworkInteractableId(localId);
  }
  
  public override void SendData(byte[] bytes)
  {
    // Misc.Log("FUSION Send Data " + bytes.Length, Color.yellow);
    _being.SendData(bytes);
  }

  public override void UpdateLookData(Vector3? headTarget, Vector3? eyeTarget)
  {
    if (headTarget.HasValue)
    {
      _being.HeadTargetIsNull = false;
      _being.HeadTarget       = headTarget.Value;
    }
    else
    {
      _being.HeadTargetIsNull = true;
    }

    if (eyeTarget.HasValue)
    {
      _being.EyeTargetIsNull = false;
      _being.EyeTarget       = eyeTarget.Value;
    }
    else
    {
      _being.EyeTargetIsNull = true;
    }
  }
    
  public override void UpdateRootTransform(Vector3 position, Quaternion rotation)
  {
    _being.Position = position;
    _being.Rotation = rotation;
  }
}

NetworkBeing

Most of the magic happens here.

The code has been cleaned up to only keep relevant bits.

The interesting parts are:

  • The different properties, synced per frame (HeadTarget, Position, etc.) and their associated callback

  • The SendData() function (which in turn calls RPC_SendData() which is then using an RPC call to transmit the info and forward the result to Interface.OnDataReceived

  • The RPC_RequestState (called by the client to the host) and the RPC_SendCurrentStateChunk which is the answer by the host, which show a possible implementation of splitting the actuator state into multiple chunks of data which can then be sent over the network.

public class NetworkBeing : NetworkBehaviour
{
  private Being                   _being;
  private ANetworkInterface       _networkInterface;
  private bool                    _ready;

  [Networked(OnChanged = nameof(OnHeadTargetsChanged), 
   OnChangedTargets = OnChangedTargets.Proxies)]
  public Vector3 HeadTarget { get; set; }

  [Networked(OnChanged = nameof(OnHeadTargetsChanged), 
   OnChangedTargets = OnChangedTargets.Proxies)]
  public bool HeadTargetIsNull { get; set; }

  [Networked(OnChanged = nameof(OnHeadTargetsChanged), 
   OnChangedTargets = OnChangedTargets.Proxies)]
  public Vector3 EyeTarget { get; set; }

  [Networked(OnChanged = nameof(OnHeadTargetsChanged), 
   OnChangedTargets = OnChangedTargets.Proxies)]
  public bool EyeTargetIsNull { get; set; }

  [Networked(OnChanged = nameof(OnRootTransformChanged), 
   OnChangedTargets = OnChangedTargets.Proxies)]
  public Vector3 Position { get; set; }

  [Networked(OnChanged = nameof(OnRootTransformChanged), 
   OnChangedTargets = OnChangedTargets.Proxies)]
  public Quaternion Rotation { get; set; } 

  private bool   _waitingForNewState = false;
  private int    _nChunks            = 0;
  private int    _readChunks         = 0;
  private int    _bufferOffset       = 0;
  private int    _dataLength         = 0;
  private byte[] _stateBuffer;

  private void Start()
  {
    Being.NetworkMode mode = Runner.IsServer 
      ? Being.NetworkMode.Host 
      : Being.NetworkMode.Client;

    _networkInterface = new FusionNetworkInterface(this);
    
    BeingData beingData = new BeingData();
    beingData.AssignID(id);
    
    BeingInfo beingInfo = new(
      _being, 
      beingData, 
      beingToSpawn.BeingSettings, 
      sharedSettings, 
      beingToSpawn.Type,
      networkMode: mode, 
      networkInterface: _networkInterface);
            
    _being.InitializeAndStartBeing(beingInfo);

    if (Runner.IsClient)
    {
      _stateBuffer        = null;
      _waitingForNewState = true;
      RPC_RequestState();
    }
  }

  public void SendData(byte[] bytes)
  {
    RPC_SendData(bytes);
  }

  [Rpc(InvokeLocal = false)] // Sent by server, is executed on clients
  private void RPC_SendData(byte[] bytes)
  {
    if (_ready)
    {
      _networkInterface.OnDataReceived(bytes);
    }
  }

  private static void ZeroBuffer(byte[] bytes)
  {
    for (int i = 0; i < bytes.Length; i++)
    {
      bytes[i] = 0;
    }
  }

  [Rpc(sources: RpcSources.All, targets: RpcTargets.StateAuthority)]
  private void RPC_RequestState(RpcInfo info = default)
  {
    // This function requests the state of the being and divides it in multiple 
    // chunks for proper sending over the network (the data sent here is too large 
    // to be sent in one call)
    byte[] state = _networkInterface.GetBeingState();

    int chunkSize   = 1024;
    int bufferStart = 1;
    int inSize      = chunkSize - bufferStart;

    int nChunks = state.Length / inSize + (state.Length % inSize == 0 ? 0 : 1);

    byte[] bytes = new byte[chunkSize];

    int remainingLength = state.Length;

    RPC_SendStateInfos(
      info.Source, 
      nChunks * chunkSize, 
      nChunks, 
      bufferStart, 
      inSize
    );

    for (int i = 0; i < nChunks; ++i)
    {
      ZeroBuffer(bytes);

      bytes[0] = (byte)i;

      Buffer.BlockCopy(
        state,
        i * inSize,
        bytes,
        bufferStart,
        Math.Min(remainingLength, inSize)
      );
      remainingLength -= inSize;

      RPC_SendCurrentStateChunk(info.Source, bytes);
    }
  }

  // https://doc.photonengine.com/en-us/fusion/current/manual/rpc#targeted_rpc
  [Rpc(sources: RpcSources.StateAuthority, RpcTargets.All)]
  private void RPC_SendStateInfos(
    [RpcTarget] PlayerRef playerRef, 
    int totalSize, 
    int nChunks, 
    int offset, 
    int dataLength
  )
  {
    _stateBuffer  = new byte[totalSize];
    _nChunks      = nChunks;
    _bufferOffset = offset;
    _dataLength   = dataLength;
  }

  [Rpc(sources: RpcSources.StateAuthority, targets: RpcTargets.All)]
  private void RPC_SendCurrentStateChunk(
    [RpcTarget] PlayerRef playerRef, 
    byte[] chunk
  )
  {
    if (_stateBuffer == null)
    {
      throw new Exception("Did not receive StateInfos call");
    }

    int chunkIndex = chunk[0];

    Buffer.BlockCopy(chunk, 1, _stateBuffer, _dataLength * chunkIndex, _dataLength);
    _readChunks += 1;
    
    // Are we receiving the last chunk ? 
    if (_readChunks == _nChunks)
    {
      // Yup. We can now set the complete being state.  
      _networkInterface.SetBeingState(_stateBuffer);
      
      // Synchronize initial transform and look target,
      // which might have been lost because of 
      // the fact that we were not "ready"
      _networkInterface.OnRootTransformUpdated(Position, Rotation);
      
      Vector3? headTarget = HeadTargetIsNull ? null : HeadTarget;
      Vector3? eyeTarget  = EyeTargetIsNull ? null : EyeTarget;
      _networkInterface.OnLookDataUpdated(headTarget, eyeTarget);
      
      _ready = true;
    }
  }

  private static void OnHeadTargetsChanged(Changed<NetworkBeing> changed)
  {
    if (changed.Behaviour._ready)
    {
      Vector3? headTarget = changed.Behaviour.HeadTargetIsNull 
        ? null 
        : changed.Behaviour.HeadTarget;
      Vector3? eyeTarget  = changed.Behaviour.EyeTargetIsNull 
        ? null 
        : changed.Behaviour.EyeTarget;

      changed.Behaviour._networkInterface.OnLookDataUpdated(headTarget, eyeTarget);
    }
  }

  private static void OnRootTransformChanged(Changed<NetworkBeing> changed)
  {
    if (changed.Behaviour._ready && changed.Behaviour.Runner.IsClient)
    {
      changed.Behaviour._networkInterface.OnRootTransformUpdated(
        changed.Behaviour.Position, 
        changed.Behaviour.Rotation
      );
    }
  }
}

FusionInteractableID

This code showcases how to synchronize an interactable through a network session, with Fusion. Attach it to every interactable object.

public class FusionInteractableID : NetworkBehaviour, IAfterSpawned
{ 
    NetworkManager NetworkManager => NetworkManager.Instance;
    
    // Called when this object is spawned in the network.
    public void AfterSpawned()
    {
        IInteractable interactable = GetComponent<IInteractable>();
        NetworkManager.RegisterInteractable(interactable.InteractableID, 
            (int)Id.Object.Raw);
    }
}

NetworkManager

The FusionNetworkInterface and FusionInteractableID mention a NetworkManager for the interactable synchronization. The relevant bits are shown below:

public class NetworkManager : MonoBehaviour
{
    private static NetworkManager _instance;
    public static NetworkManager Instance
    {
        get
        {
            // Instance requiered for the first time, we look for it
            if (_instance == null)
            {
                _instance = FindObjectOfType<NetworkManager>();

                // Object not found, we create a temporary one
                if (_instance == null)
                {
                    throw new Exception("No instance of NetworkManager.");
                }
            }
            return _instance;
        }
    }
    
    // The following code shows how to link local <-> network ID
    // LocalId is the interactable ID of the local scene
    // NetworkId is the id shared through the network
    Dictionary<int, int> networkToLocalId = new Dictionary<int, int>();
    Dictionary<int, int> localToNetworkId = new Dictionary<int, int>();
    
    public void RegisterInteractable(int localId, int networkId)
    {
        networkToLocalId.Add(networkId, localId);
        localToNetworkId.Add(localId, networkId);
    }
    
    // Called by FusionNetworkInterface.GetLocalInteractableId()
    public int GetLocalInteractableId(int networkId)
    {
        return networkToLocalId[networkId];
    }
    
    // Called by FusionNetworkInterface.GetNetworkInteractableId()
    public int GetNetworkInteractableId(int networkId)
    {
        return localToNetworkId[networkId];
    }
}

Last updated