RGB-W: When Vision Meets Wireless

Alexandre Alahi
Stanford University

Albert Haque
Stanford University

Fei-Fei Li
Stanford University


Abstract

Inspired by the recent success of RGB-D cameras, we propose the enrichment of RGB data with an additional quasi-free modality, namely, the wireless signal emitted by individuals' cell phones, referred to as RGB-W. The received signal strength acts as a rough proxy for depth and a reliable cue on a person's identity. Although the measured signals are noisy, we demonstrate that the combination of visual and wireless data significantly improves the localization accuracy. We introduce a novel image-driven representation of wireless data which embeds all received signals onto a single image. We then evaluate the ability of this additional data to (i) locate persons within a sparsity-driven framework and to (ii) track individuals with a new confidence measure on the data association problem. Our solution outperforms existing localization methods. It can be applied to the millions of currently installed RGB cameras to better analyze human behavior and offer the next generation of high-accuracy location-based services.

Paper

Full Paper: CVF Open Access [PDF]


RGB-W Dataset

conference-1

conference-2

patio-1

patio-2

Note: All sequences contain RGB, depth, and wireless (W) modalities.
Sequence Name Length (mm:ss) # Frames # People # W Devices Download
conference-1 01:53 1,697 5 5 zip (116 M)
conference-2 05:18 4,782 12 12 zip (379 M)
conference-3 23:31 21,165 1 2 zip (1.32 G)
conference-4 06:27 4,832 1 2 zip (357 M)
conference-5 06:03 4,525 2 2 zip (290 M)
patio-1 07:22 6,636 4 4 zip (474 M)
patio-2 04:36 4,144 2 2 zip (258 M)
Full Dataset 55:10 47,781 -- -- zip (3.23 G)

Citation

If you would like to cite our work, please use the following.

Alahi A, Haque A, Fei-Fei L. (2015). RGB-W: When Vision Meets Wireless. International Conference on Computer Vision (ICCV). Santiago, Chile. IEEE.

@inproceedings{alahi2015rgb,
  title={RGB-W: When vision meets wireless},
  author={Alahi, Alexandre and Haque, Albert and Fei-Fei, Li},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2015}
}