❮ Background Fading (Built-In Render Pipeline)

Vehicle Camera Pass-Through (AR)

The Vehicle Camera Pass-Through extension package (com.holoride.elasticsdk.camera) displays external physical camera feeds (e.g., from bumper or external cameras) inside the virtual environment for Mixed Reality (MR) and Augmented Reality (AR) applications.

To prevent motion sickness, the package uses calibration parameters (intrinsics and extrinsics) to align the virtual camera's perspective with the physical vehicle's orientation. The external video stream is projected onto the background skybox using an equirectangular shader, matching the physical camera's real-world field of view.

Installation

Open the Unity Package Manager.
Search for and install the com.holoride.elasticsdk.camera package.
Ensure that the core com.holoride.elasticsdk package is installed and active in your project.

Usage

Integrating the camera pass-through system into your scene requires two steps:

Instantiate the Prefab: Drag the pre-configured CameraVehicle (or AugmentedRealityVehicle) prefab directly into your active scene hierarchy.
Configure the Camera: Ensure your scene's Main Camera is childed under the prefab's CameraRigParent child object and has the ARCameraSync script attached.
At runtime, the active StateReceiver will automatically bind the live camera stream and calibration parameters to your camera setup.

The `AugmentedRealityVehicle` Prefab Hierarchy

This prefab contains the structural components required for physical-to-virtual alignment:

Root Vehicle: Handles positional tracking.
- CameraRigParent: Isolates head-tracking reference offsets.
  - Main Camera: Equipped with the ARCameraSync script.

The AR system relies on several individual, decoupled setter scripts under this hierarchy to synchronize the virtual world with the real-world camera feed:

ARSensorToCameraOffsetSetter: Sets the local position offset between the physical IMU sensor and the lens center.
ARSensorToCameraRotationSetter: Sets the local rotation offset between the physical IMU sensor and the lens center.
ARSensorHeightFromGroundSetter: Sets the vertical offset of the IMU sensor from the ground (used as a fallback if the State Receiver/State stream provides no data).

`AugmentedRealityModel` (Singleton ScriptableObject Settings)

The AugmentedRealityModel is a core Singleton ScriptableObject (saved under Resources/ElasticSDK/AugmentedRealityModel.asset) responsible for managing the entire Augmented Reality camera state and its dependencies. It implements INotifyBindablePropertyChanged and serves as the central data source.

Warning

SPE Calibration Precedence: These values are specific to the car model and not to the game. The Spatial Perception Engine (SPE) will overwrite these values dynamically to match the physical properties of each car model. These values should never be used to tune procedural distances and visual sizes within the game; those adjustments belong inside your procedural spawning rules in the Elastic Graph or the dimensions settings of the Elastic Scene Generator.

Homography and Intrinsic Parameter Correction

The system employs a homography correction process using four UV corner coordinates derived from the CameraIntrinsic struct. This ensures the physical camera image is accurately aligned with the scene rendering camera.

Important

Horizontal Field of View Baseline:
When configured correctly, the Spatial Perception Engine (SPE) streams exactly 100 degrees of horizontal FOV (PredeterminedHorizontalFieldOfView = 100f) as the standard calibration baseline. This represents a highly reliable alignment reference that experience developers can always rely on out-of-the-box. However, this is not a rigid or immutable engine restriction; while the low-level UV homography scaling parameters are dynamically calculated and set automatically by the SPE based on sensor calibrations, developers still retain full, high-level flexibility to adapt, stretch, or customize this final perspective utilizing Aspect Ratio Modifications!

The following table details how the four UV corners are manipulated to achieve specific image corrections:

Correction Type	Effect on Image Geometry	UV Corner Adjustment Logic
Undistortion	Corrects remaining perspective aberrations and minor distortions after initial pre-processing.	Requires setting the four UV coordinates to map the distorted input texture onto a rectified output plane. These values are derived from a camera calibration process that calculates the necessary warp based on the camera's intrinsic parameters (like focal length and center point).
Field of View (FOV) Adjustment	Ensures the visual perspective of the camera feed matches the virtual scene camera's frustum.	The Spatial Perception Engine (SPE) streams exactly 100 degrees of horizontal FOV as the standard calibration baseline (`PredeterminedHorizontalFieldOfView = 100f`) used to calculate the required Vertical FOV for the scene camera based on the viewport aspect ratio. To align with this baseline, the SPE automatically calculates and sets the UV corner coordinates to scale the texture until the displayed image geometry matches the 100-degree virtual frustum, which the developer can then customize via high-level aspect ratio modifications.
Image Rotation	Rotates the image by a few degrees (like 2 deg) to match the horizon (rolling the image slightly) or rotates the physical camera feed by large angles (e.g., 90 or 180 degrees) if the camera sensor is mounted in a non-standard orientation.	Achieved by cyclically permuting the four UV corner pairs. For example, a 90-degree clockwise rotation remaps the default corners `(0,0)`, `(1,0)`, `(1,1)`, `(0,1)` to `(0,1)`, `(0,0)`, `(1,0)`, `(1,1)`.
Aspect Ratio Modifications	Stretches or compresses the image to precisely match the target rendering viewport's aspect ratio.	Involves scaling the X or Y components of the UV coordinates individually. The aspect ratio is determined by the relative scaling between these axes.
Image Mirroring	Flips the physical camera feed horizontally or vertically.	Achieved by swapping the UV coordinates. Horizontal mirroring swaps the X-coordinates (e.g., `[0,X]` becomes `[1,X]`), and vertical mirroring swaps the Y-coordinates (e.g., `[X,0]` becomes `[X,1]`).