Around Yangjae Station, Seoul, South Korea

Gross Area: Approx. 79,659 m²

MAXSTVPS Unity Project for Smart Phones

MAXSTVPS Unity Project for Nreal Glasses

VPS Map

VPS Map includes the following information of the locations: Mesh, Reference Camera, Simulation Data

Placement by Raycast

This is a sample of placing 3D objects on to the surface of buildings with the point of contact. Using mouse or touch panel, ray is casted that connects to camera's focal point, and thus calculate code based on transactional points with building mesh.

public void AttachLogo()
{
    Vector2 vTouchPos = Input.mousePosition;

    Ray ray = Camera.main.ScreenPointToRay(vTouchPos);

    RaycastHit vHit;
    if (Physics.Raycast(ray.origin, ray.direction, out vHit))
    {
        maxstLogObject.transform.position = vHit.point;
        maxstLogObject.transform.rotation = Quaternion.FromToRotation(Vector3.forward, vHit.normal) * Quaternion.Euler(-90.0f ,0.0f, 0.0f);
    }
}

AR Navigation

The code to determine the point of departure and arrival to find the navigation path is as follows. If you wish to get a navigation path from the point of user's location, set arCamera.tranform.position as a point of departure. This will only be successful if the localization is practiced correctly.

public void OnClickNavigation()
{
    NavigationController navigationController = vPSStudioController.GetComponent<NavigationController>();
    navigationController.MakePath(arCamera.transform.position, new Vector3(77.975977f, 0, 71.859565f), serverName);
}

NavigationController is collected from VPSStudio GameObject. Through MakePath() from Instance of NavigationController, you will receive the path from the point of departure to the point of arrival.

Requirements & Supports

Requirements

• In order to use MAXSTVPS SDK for Unity, you need basic knowledge of Unity developer tools and developer resources.

Install MAXST VPS Unity SDK

1. Download MAXSTVPS SDK

2. Create or Open Unity Project

3. Newly created project, must download MAXSTVPSSDK_0.8.0.unitypackage and install on Unity

4. Download VPSData and install VPSMap in accordance to location

Unity Setting

Description of File/Build Settings/Player Settings for Unity

Refer to the following Github links for Manifest and Gradle

Install Editor Coroutines Package

MAXSTVPS SDK requires Editor Coroutines Package in Unity

The type or namespace name 'EditorCoroutines' does not exist in the namespace ‘Unity'.  

Install VPS Map

VPS Map includes mesh, reference images and simulation data of the location. As suggested below, VPSData folder needs to be created at same hierarchy as Unity Project folder (MAXSTVPSUnity), and VPSMap folder needs to be saved under VPSData.

Once you unzip VPSData_yangjae_indoor.zip by MAXST, it automatically opens under VPSData folder as VPSMap and VPSSimulationData. yangjae_indoor folder will be generated automatically under these two folders.

Load VPS Map

Load VPS Map on VPSStudio GameObject via VPSStudioController

Once VPS Map is loaded, it is presented in UnitySceneView as depicted below.

Contents Placement

Placing content on Map is based on the location of the mesh. Place contents around mesh and confirm through ReferenceCamera

Check the status of Content Generation via Simulation Data

Upon completion of content generation, you will be able to confirm through VPS Simulation Data on VPSStudio. The Simulation Data of the particular regions will be available on OSX or Windows.

Mesh Settings for Occlusion

The features of occlusion (Hidden features of building meshes) is specified on the page below

Nreal Unity SDK Download

MAXST VPS Unity SDK for Nreal Glasses Download

Unity Scene Components

Unity Scene components for Nreal Glasses are as follows. Detailed information regarding Unity Scene components can be found in the link below. ARCamera from the former Unity Scene is replaced with NRCameraRig.

SceneManager Components

ARCamera from SceneManager is replaced with NRCameraRig Object

Nreal Camera Start/ VPS Tracker Start

‌Play() of Instance from NRCollect YUV starts Nreal Camera.

VPSTracker Start can be found in the page below.

nRCollectYUV = new NRCollectYUV();
nRCollectYUV.Play();
TrackerManager.GetInstance().StartTracker();

if (serverName != "")
{
    string vpsquery = "{\"vps_server\":\"" + serverName + "\"}";
    TrackerManager.GetInstance().AddTrackerData(vpsquery);
}

Get result for VPS Tracker

This is a code for ARFrame Instance from Unity Update(). This is a code to obtain the posture of 6Degrees of Freedom on VPS Map at the time of update and reflect it on the virtual camera for rendering.

void Update()
{
    TrackerManager.GetInstance().UpdateFrame();
    nRCollectYUV.UpdateFrame();
    var eyePoseFromHead = NRFrame.EyePoseFromHead;
    Matrix4x4 Mhe = MatrixUtils.ConvertPoseToMatrix4x4(eyePoseFromHead.RGBEyePos);

    ARFrame arFrame = TrackerManager.GetInstance().GetARFrame();

    if (arFrame.GetARLocationRecognizeState() == ARLocationRecognizeState.ARLocationRecognizeStateNormal)
    {
        Matrix4x4 targetPose = arFrame.GetTransform(Mhe);

        arCamera.transform.position = MatrixUtils.PositionFromMatrix(targetPose);
        arCamera.transform.rotation = MatrixUtils.QuaternionFromMatrix(targetPose);
        arCamera.transform.localScale = MatrixUtils.ScaleFromMatrix(targetPose);

        rootTrackable.SetActive(true);
    }
    else
    {
        rootTrackable.SetActive(false);
    }
}

Required components of SceneManager

The following is code for required components of SceneManager

ARManager arManagr = FindObjectOfType<ARManager>();
if (arManagr == null)
{
    Debug.LogError("Can't find ARManager. You need to add ARManager prefab in scene.");
    return;
}
else
{
    arCamera = arManagr.gameObject;
}

vPSStudioController = FindObjectOfType<VPSStudioController>();
if (vPSStudioController == null)
{
    Debug.LogError("Can't find VPSStudioController. You need to add VPSStudio prefab in scene.");
    return;
}
else
{
    string serverName = vPSStudioController.vpsServerName;
    vPSStudioController.gameObject.SetActive(false);
}

cameraBackgroundBehaviour = FindObjectOfType<CameraBackgroundBehaviour>();
if (cameraBackgroundBehaviour == null)
{
    Debug.LogError("Can't find CameraBackgroundBehaviour.");
    return;
}

if(rootTrackable == null)
{
    Debug.LogError("You need to add RootTrackable.");
}

Occlusion Culling

When rendered 3D Objects are visibly hidden from the building mesh is called Occlusion Culling. Occlusion is applied when runtimeBuildingMaterial is rendered.

foreach (GameObject eachGameObject in occlusionObjects)
{
    Renderer[] cullingRenderer = eachGameObject.GetComponentsInChildren<Renderer>();
    foreach (Renderer eachRenderer in cullingRenderer)
    {
        eachRenderer.material.renderQueue = 1900;
        eachRenderer.material = runtimeBuildingMaterial;
    }
}

카메라 시작 / 시뮬레이션 시작

스마트폰과 스마트 글래스와 같은 모바일 기기 환경에서는 아래 코드를 통해 하드웨어 카메라가 시작됩니다. MAC OS X와 Windows 환경에서는 VPSStudioController를 통해 선택한 시뮬레이션 데이터가 시작됩니다. 이를 통해 현장에 직접 나가지 않아도 앱 개발이 가능합니다.

if (Application.platform == RuntimePlatform.OSXEditor || Application.platform == RuntimePlatform.WindowsEditor)
{
    string simulatePath = vPSStudioController.vpsSimulatePath;
    if (Directory.Exists(simulatePath))
    {
        CameraDevice.GetInstance().Start(simulatePath);
        MaxstAR.SetScreenOrientation((int)ScreenOrientation.Portrait);
    }
}
else
{
    if (CameraDevice.GetInstance().IsFusionSupported(CameraDevice.FusionType.ARCamera))
    {
        CameraDevice.GetInstance().Start();
    }
    else
    {
        TrackerManager.GetInstance().RequestARCoreApk();
    }
}

VPS Tracker 시작

VPS Tracker를 시작하기 위해서는 해당 VPS 공간지도에 접근할 수 있도록 서버 이름을 입력해야 합니다.

TrackerManager.GetInstance().StartTracker();

if (serverName != "")
{
    string vpsquery = "{\"vps_server\":\"" + serverName + "\"}";
    TrackerManager.GetInstance().AddTrackerData(vpsquery);
}

VPS 트래킹 결과 얻기

트래킹 결과는 UpdateFrame()과 GetARFrame()을 통해서 얻을 수 있습니다. GetARFrame()을 통해 얻은 ARFrame의 인스턴스에는 현재 트래킹 상태, 이미지, 6자유도 자세를 포함되어 있습니다.

ARFrame의 GetARLocationRecognitionState()를 통해서 현재 위치 인식 상태를 얻을 수 있습니다. 이 상태 정보에 맞춰 증강시킬 3D Object의 활성화 여부를 결정합니다.

void Update()
{
    TrackerManager.GetInstance().UpdateFrame();

    ARFrame arFrame = TrackerManager.GetInstance().GetARFrame();
    
    TrackedImage trackedImage = arFrame.GetTrackedImage();


    if (trackedImage.IsTextureId())
    {
        IntPtr[] cameraTextureIds = trackedImage.GetTextureIds();
        cameraBackgroundBehaviour.UpdateCameraBackgroundImage(cameraTextureIds);
    }
    else
    {
        cameraBackgroundBehaviour.UpdateCameraBackgroundImage(trackedImage);
    }

    if(arFrame.GetARLocationRecognizeState() == ARLocationRecognizeState.ARLocationRecognizeStateNormal)
    {
        Matrix4x4 targetPose = arFrame.GetTransform();

        arCamera.transform.position = MatrixUtils.PositionFromMatrix(targetPose);
        arCamera.transform.rotation = MatrixUtils.QuaternionFromMatrix(targetPose);
        arCamera.transform.localScale = MatrixUtils.ScaleFromMatrix(targetPose);

        rootTrackable.SetActive(true);
    }
    else
    {
        rootTrackable.SetActive(false);
    }
}

Scene Components

The Unity Scene Hierarchy for MAXSTVPS is as follows.

ARCamera GameObject

CameraBackground GameObject

CameraBackground display images extracted from hardware camera

VPSStudio GameObject

VPSStudio manages VPS Map data

VPSStudio Controller menu is as follows:

• Using VPS Map combo box, check the VPS Map list under VPSData Folder and select the desired location

• Using VPS Map combo box, check the simulation data list under VPSData/VPSSimulationData/your_vps_map and select the desired simulation data

• The mesh information & reference camera of the selected location will be loaded when you press Load VPS Map button

• The mesh information and reference camera will be deleted when you press clear button

Navigation Controller will offer navigation function within the selected location

• Position Distance indicates the interval between rendering arrow objects in alignment to navigation path. The unit is in meters.

• Arrow prefab is 3D arrow objects in the navigation path.

• Root Trackable is Root GameObject to render 3D Object on the navigation path based on tracking position

Scene Manager

SceneManager manages VPS controls

• Disable Objects are GameObjects that needs to be disabled from the perspective of Unity Play. (VPSStudio Game Object includes camera eye, thus needs to be disabled).

• Root Trackable is GameObject that includes all tracking contents

• Occlusion Objects are Mesh that needs to add occlusion from the perspective of Unity Play. If there are additional mesh that needs occlusion culling, you can simply add the following element.