UI in 3D Space

While most UI Is locked to your screen like a frame around the camera’s view, some UI is required to be visible within the 3D world itself. I’m going to be using Lamina Island’s Interaction System for reference to explain how to do these.

You can do this in unity easily enough by creating your UI element (in our case, text), & then setting the Render mode to World Space

The tricky part comes in when you need to get it to always face the player, be locked to a position, or work for local co-op.

In our interaction system, rather than making everything you can interact with define their own panel. We use a single panel for each player, that holds the interaction text.

Now, the biggest problem we’ll have to solve with our 3D UI is going to be the local co-op problem. We get around this by actually making two separate floating UIs & setting their layer to a a layer per camera that is unique to our floating UI. For Lamina Island, we call this P1_UI & P2_UI.

Creating a second camera specifically for the floating UI (per player).

And finally editing the culling masks of the layers.

This allows the two cameras to be able to record the UI without stepping on each others toes. You can technically accomplish this without the use of the second camera, you just make sure to set your culling masks accordingly, but we chose to use two cameras so that the UI camera could display it’s UI without real world characters and environments clipping it.

Now that the cameras will be able to pick up and display the UI, we have to make sure it maintains a useful position & rotation. And set it’s text.

We do that with these two Monobehaviors, the first of which we will cover being FaceCamera.

First let’s define our serialized properties, of which we only need two. One being the camera we are going to face. This is important because in a co-op game, you can’t just assume the main camera is our target. This allows us to have player 1’s UI face their camera & player 2’s UI face their camera. And the other to give us the ability to flip the panel 180 degrees. This just helps for when you mix something up & face it the wrong way and you need to inverse the rotation easily.

using UnityEngine;

public class FaceCamera : MonoBehaviour
{
    [Tooltip("Camera we are going to be facing")]
    [SerializeField]
    private Camera _targetCamera;
    public Camera TargetCamera
    {
        get => _targetCamera;
        set => _targetCamera = value;
    }


    [Tooltip("Flip it 180 degrees. " +
             "This makes it face the same direction as the camera rather than face the camera.")]
    [SerializeField]
    private bool rotate180Degrees;     

}

And the only thing left to cover for this part is the update method. What we need to do is take the inverse of the camera’s direction by negating it’s transform.forward. And then we use the Quaternion.LookRotation to get the rotation in that direction.

   private void Update()
    {
        //make sure we have a target to face
        if (TargetCamera == null)
        { return; }


        //negative because we want to be facing the camera 
        var forwardVector = -TargetCamera.transform.forward;

        //if they are rotating it 180 degrees then they want to face the same direction as the camera
        if (rotate180Degrees)
        {
            forwardVector *= -1;
        }

        Quaternion targetRotiation;
        targetRotiation = Quaternion.LookRotation(forwardVector, TargetCamera.transform.up);

        transform.rotation = targetRotiation;
    }

And that’s everything for our super simple “FaceCamera”. This has a lot of uses outside just the 3D UI system. One example is anything that you billboard such as 2D loot, which we cover here.

So with this in place, we now can display the text & face the proper rotation. If the text was hardcoded, this would be everything we’d need to handle for our 3D UI. In our scenario however, we want to be able to integrate our intereaction system with this floating UI. So we have to cover the AdvertismentUIPanel.

So the first thing we are going to need to do is grab some components in our awake function.

public class AdvertisementUiPanel : UnityPanel
{   
   private Canvas floatingUI;
   private TextMeshProUGUI floatingText;

  private void Awake()
  {
    floatingUI = GetComponent<Canvas>();
    floatingText = GetComponentInChildren<TextMeshProUGUI>();
  }
}

Then we just need to give ourselves a target and then position our UI to follow said target.

  [SerializeField]
  private float cameraOffset;
  [SerializeField]
  private float yOffset;
  [SerializeField]
  [Range(.1f, 1f)]
  private float textSmoothTime = .1f;   

  private Transform target;
  private bool running = false;


private void Update()
  {
    if (!running)
    { return; }

    //if something happens to the game object BEFORE we are notified,
    // we need to check against that as well
    if (target == null)
    {
      return;
    }

    Vector3 targetPos = target.position + Vector3.up * yOffset;
    targetPos += transform.forward * cameraOffset;

    floatingUI.transform.position = Vector3.Lerp(floatingUI.transform.position, targetPos, textSmoothTime);
  }

We make sure to give the user a ‘camera offset’ and a vertical offset to allow for slight tweaking And we use a lerp instead of a hard position set, so that minor changes, don’t feel like constant refreshes, but rather animations.

And lastly. We have to give our interaction system something to lock into.

 public void DisplayAdvertisement(ActionAdvertisement input)
  {
    target = input.transform;
    floatingUI.enabled = true;
    floatingText.text = input.InteractText;
    running = true;
  }

  public void HideInteractions()
  {
    running = false;
    target = null;
    floatingUI.enabled = false;
  }

And that’s it. Our 3d UI should be completely usable with our interaction system. Now, if your curious what the Action Advertisment is. That’s the main element in Lamina Island’s interaction system, which you can read more about here.