Using the GPU to Create a Seamless Display from Multiple Projectors

Authors: Steve Nash (NVIDIA), Paul Green (Scalable Display Technologies). Published in: 2011. Download pdf.

The Problem

Increases in pixel density and total pixels have not kept pace with increases in CPU and GPU power. Some of the solutions to adding more pixels:

  • LCDs – Obtrusive bezels in the way

seamless-display

  • Nearly bezel-less

  • Projectors overlap the edges to hide the seam. This creates a hotspot since the overlap region gets twice the light:

projector-edge-blend-hotspot

  • How to project on dome and curved screens?

edge-blending-dome-projection  edge-blending-curved-screen

Currently Nvidia Mosaic allows you to specify overlap. But to be truly seamless, there is a need to account for geometry and brightness differences between projectors.

The Solution: Warp and Blend

  • Warp: Geometry corrections
  • Blend: Intensity adjustments

Up until now, warp and blend for projection displays has been done in one of the following two ways:

  1. Hardware appliance for warp and intensity adjustment
    • Expensive
    • Extra-performance delay tax on the display pipeline
    • Additional complexity
  2. Software warp and intensity adjustment
    • Applications need to be written to manage the display for every use-case
    • There has not been an easy way to implement this for any application, until now…

NVIDIA’s Solution

NVIDIA can do this on the GPU, which has several important advantages:

  • GPUs are fast and already have the pixel information
  • GPUs perform the transformation in the display pipeline before the pixels get scanned out
  • By doing this on the GPU we have more flexibility: high quality filtering, integration with NVIDIA Mosaic, etc.
  • Works on Quadro 5000, 6000, and Quadro Plex 7000
  • Use it with G-sync to get synchronization between displays

How Is It Done: Workflow

  1. Take a picture of calibration pattern
  2. Create warping mesh based on picture of the calibration pattern relative to the true image
  3. Typical warping mesh contains 1K to 10K vertices

How Is It Done: NVAPI

  • NVAPI is Nvidia’s programmatic interface to configure and control the GPU’s – http://developer.nvidia.com/nvapi.
  • New interfaces are added in the 275+ NDA version to allow warping and intensity adjustment before the final scanout.
  • Currently it works with single screen. Multiple screen capability coming with R285 driver (mid September)

How Is It Done: Warping

  1. Initialize NVIDIA
  2. Get GPU handle
  3. Get display ID
  4. Define warping mesh
    • Use NvAPI_GPU_GetScanoutConfiguration() to get current desktop
    • Each vertex has:
      1. x,y: 2D vertex coordinates
      2. u,v: 2D Texture coordinates
      3. r,q: Perspective correction
  5. Call NvAPI_GPU_SetScanoutWarping

How Is It Done: Blending (Intensity Adjustment)

  1. Initialize NVAPI
  2. Get GPU handle
  3. Get display ID
  4. Define Intensity Map: GL_RGB value for each pixel
  5. Call NVAPU_GPU_SetScanoutIntensity

Scalable Display Technologies

Leading provider of auto-calibration software for seamless multiple-projectors displays. Founded in 2004, based on P.h.D. work of Rajeev Surati at MIT. Scalable has over 100 customers in the following markets:

  • Command and Control
  • Business Collaboration
  • Visualization
  • Simulation and Training
  • Pro A/V
  • Projector OEMs: 70,000+ licenses

Core Auto-Calibration Technology

Camera-feedback to automatically warp and blend many projectors into one seamless display:

before-warp-and-blend          warp-and-blend

Benefits of the NVIDIA Warp and Blend API

Scalable Display Manager – Military & Simulation, Command & Control

  • No longer need of SDK integration
  • No need for expensive warping hardware
  • Application independent
  • Quadro support for Gen-lock

Fewer requirements, barriers and restrictions mean broader market and stronger growth.

ScalableDesktop (warped Win7 Desktop)

  • Full integrations with Windows
  • Eliminates mouse pointer issues
  • Performance benefits
  • Expands potential outputs to 24

Case Studies

AFRL – 100 MegaPixel Space Command and Control

afrl-dome-projection  afrl-dome-projection-warp-and-blend

25 Computers with Quadro cards running 50 projectors. Custom SDK integration that can run a single application; application independence with NVAPI warp and blend API.

Navy Combat System Display

Aegis Combat System Display: 22 Cruisers have been delivered and there are 62 Destroyers to follow with systems in U.S, Japan, Spain, Norway, Korea and Australia. Required Gen-Lock and SDK integration.

MIT Lincoln Labs – 20MPixel Touch Wall

20-megapixel-touch-wall12 Projectors systems using dual Quadroplexes, it is a prototype design for a virtual shopping aisle system. It will be the first install to use NVIDIA warp and blend API.

AFRL – 48 MPixel F-16 Simulator

projector-warp-and-blend-simulatorThis “Low Cost” simulator ($500K) by Immersive Display Solutions and using Scalable software, requires no warping box hardware which helps keep costs down. The application independence is essential to make it a “turn key” solution.

Summary

  • Scalable Display Technologies is leader in auto-calibration for multi-projector displays.
  • NVAPI Warp and Blend API makes our technology application independent.
  • Fewer requirements, barriers and restrictions means broader market.
  • Full Windows integration.

Further Information