Unit
- Augmented reality (AR): A technology that superimposes digital information - such as images, text, or 3D models - onto the user's view of the real world. Unlike virtual reality, AR does not replace the environment but enhances it.
- Key characteristics: AR systems combine real and virtual content, operate interactively in real time, and register 3D content accurately with the physical environment. This alignment is called registration.
- Display technologies: AR content can be viewed through head-mounted displays (HMDs), handheld devices like smartphones and tablets, or spatial projectors that cast light onto surfaces.
- Tracking systems: To overlay content correctly, AR must track the user's position and orientation. Common methods include visual SLAM (simultaneous localization and mapping), inertial sensors, and fiducial markers.
- Illumination consistency: Convincing AR requires virtual objects to match real-world lighting. Techniques like environment mapping and light estimation help achieve realistic shading and shadows.
- Interaction modalities: Users interact with AR through touch gestures, voice commands, gaze-based selection, and hand tracking. Each modality has trade-offs in precision and learnability.
- Performance constraints: AR applications demand low latency (under 20 ms) to maintain illusion. High frame rates, efficient rendering, and sensor fusion are critical to prevent motion sickness.
- Development ecosystem: AR solutions are built using SDKs like ARKit, ARCore, and WebXR, which provide tracking, rendering, and device integration. Cross-platform frameworks simplify deployment.
- Occlusion handling: Virtual objects must be correctly occluded by real world surfaces to appear believable. Depth sensors or machine learning based depth estimation allow AR to hide portions of a digital object that lie behind physical objects in the scene.
- Multi user experiences: Shared AR allows multiple participants to see the same virtual content anchored to a common physical space. This requires network synchronization, shared coordinate systems, and persistent anchors that survive across sessions.
- Privacy and ethics: AR applications often capture camera and sensor data, raising concerns about surveillance and data security. Developers must implement transparent consent, local processing where possible, and anonymization to protect user privacy.
- Future trends: Advances in wearable optics, 5G connectivity, and spatial computing are driving lightweight, always on AR glasses. Cloud assisted rendering and AI driven scene understanding will enable richer, context aware experiences.
System Architecture
Pattern: Layered AR System Architecture
Components:
- Sensor Layer: Cameras, IMUs, depth sensors, and GPS provide raw data about the environment and user motion.
- Tracking Layer: Algorithms fuse sensor data to estimate camera pose and map the environment in real time.
- Rendering Layer: 3D graphics engine composites virtual objects onto video frames using the estimated pose and lighting.
- Application Layer: Business logic, user interface, and interaction handlers define the AR experience.
- Cloud Layer: Backend services for content delivery, multi-user synchronization, and persistent anchors.
Code Example
JavaScript (WebXR)
navigator.xr.requestSession('immersive-ar', {
requiredFeatures: ['local']
}).then((session) => {
let viewerSpace = session.requestReferenceSpace('viewer');
console.log('AR session started');
});Requesting an immersive AR session in WebXR. The feature 'local' enables orientation tracking without a floor plane.
Quick Revision
- AR overlays digital content on the real world with real-time registration.
- Tracking uses visual and inertial sensors to determine pose.
- Rendering must respect real-world lighting and occlusion.
- Low latency and high frame rate are essential for immersion.