A History of Silicon ValleyTable of Contents | Timeline of Silicon Valley | A photographic tour
History pages | Editor | Correspondence
Purchase the book
These are excerpts from Piero Scaruffi's book
The Selfies (2011-16)click here for the other sections of this chapter
Virtual and Augmented Reality
The idea of the head-mounted display harkened back to the early days of virtual reality. In 2013 Google Glass, manufactured by Foxconn, captured the headlines: it coupled that kind of display with a "wearable" computer and provided the kind of functionalities offered by Internet-enabled consumer electronics. Thad Starner, one of the pioneers of wearable computing, was the mastermind behind Glass, a project largely implemented by Greg Priest-Dorman at Georgia Tech. Google Glass, which was basically a wearable smartphone, failed because it was socially disruptive, eventually killed by social stigma. Autodesk adapted the concept of the Google Glass to the CAD market and created Autodesk Virtual Reality (AVR) Glass, introduced in 2014. Google Glass failed just like its closest competitor, Atheer, founded in 2011 in Mountain View by Lebanese-born entrepreneur Soulaiman Itani and Allen Yang, whose glasses offered a wider 65-degree field of view (as opposed to Google Glass' 12 degrees) but required a physical cable to an Android smartphone.
insert photo of Priest-Dorman at the Computer History Museum
insert photo of Google Glass prototypes (2011-12)
The vanguard in this field was perhaps Lumus, founded in 2000 in Israel by Yaakov Amitai, that in 2012 introduced see-through wearable displays.
Virtual Reality had failed in the 1990s, but between 1995 and 2015 three factors changed the conditions for adoption: 1. the cost of LCD screens had declined (thanks mainly to Hitachi's In Plane Switching design and Samsung's Multi-domain design) so that in 2007 for the first time LCD TV sets surpassed CRT TV sets in worldwide sales; 2. the cost of 3D motion capture had declined (Microsoft Kinect had come out in 2010); and, last but not least, 3. movies such as "The Matrix" (1999), a Hollywood remake of Rainer Fassbinder's "World on a Wire" (1973), had popularized the idea of life in a simulated world and had therefore inspired a new generation to live inside virtual worlds.
In 2014 Facebook acquired Oculus XE "Oculus" , a manufacturer of virtual reality headsets for games founded in 2012 in Irvine by Palmer Luckey XE "Palmer Luckey" and Jack McCauley , and originally funded through a Kickstarter campaign.Jaunt VR, founded in 2013 in Palo Alto by Jens Christensen and others, introduced a 360-degree camera that allowed users to create a virtual-reality video. In 2015 Matterport (founded in 2010 in Mountain View by Matt Bell and by David Gausebeck, the engineer who designed the first commercial CAPTCHA security system at PayPal) introduced a $4,500 camera to turn interior spaces into virtual worlds. Lucid VR, founded by Han Jin and Adam Rowell in Fremont in 2014 with the aim of becoming the GoPro of virtual reality, introduced the LucidCam, a stereoscopic 180-degree 3D camera. 360 degree cameras soon flooded the market. In 2015 the big camera brands introduced their models, such as Ricoh's Theta, Kodak's SP360, Lytro's Immerge, as well as the Jump built by Google and GoPro (marketed as Odyssey in 2016), and (in 2016) Nokia's Ozo. In 2016 two French independents responded with the Orah 41 by VideoStitch (founded in 2012 by Nicolas Burtey in France), and the 360cam by Giroptic (also founded in France in 2008 by Richard Ollier). The alternative to these cameras was the light-field technology that could create a 3D representation similar to a hologram. Refocus Imaging (later Lytro), founded in 2006 in Mountain View by Ren Ng, introduced the first light-field camera for consumers in 2012.
In 2016 GoPro indeed entered the virtual-reality market with an "end-to-end platform": the six-camera Omni VR coupled with the LiveVR wireless streaming tool and the GoPro VR video channel, plus partnerships with 100+ developers.
In 2015 Samsung Gear VR, powered by Oculus and strapped to a Samsung Galaxy (Android) phone, further lowered the threshold to play with virtual-reality games, but Google already offered a cheap virtual-reality viewer, Cardboard, developed by David Coz and Damien Henry in Google's Paris office, that used the smartphone as a screen.
By 2016 the viewers could be divided in three groups:
In 2016 Intel announced its stand-alone headset (neither computer nor smartphone required), Project Alloy .
Meanwhile, NextVR, founded by filmmaker DJ Roller and David Cole in 2009 in Los Angeles promised live VR experience for live broadcast.
In 2015 Apple acquired Swiss-based Faceshift, a spinoff from Lausanne's Polytechnique Federale, whose technology was capable of capturing the user's facial expressions in real time and creating animated avatars of the user (the base for the feature Animoji introduced in 2017). Then Apple acquired the pioneering, ten-year-old German augmented-reality startup Metaio (2015), which became the base for Apple's ARkit of 2018; UC San Diego A.I. spinoff Emotient (2016), which had developed an emotion-sensing app for Google Glass; augmented-reality startup Flyby Media (2016), a former partner of Google's Project Tango, founded by Oriel Bergig in New York in 2013, acquired right after Apple hired Doug Bowman, director of the Center for Human-Computer Interaction at Virginia Tech; the German maker of eye-tracking technology SensoMotoric Instruments (2017); Vrvana, a Canadian maker of head-mounted displays (2017); and Colorado-based Akonia Holographics, a maker of lenses for augmented-reality glasses (2018).
In 2013 graphics-processing specialist Nvidia demonstrated a head-mounted display that used light fields. A near-eye light field display was also being developed by Florida-based startup Magic Leap, founded by Rony Abovitz in 2010.
Virtual Reality required a new generation of user interfaces, beyond voice and touch. Portland's OnTheGo, founded in 2012 by Ryan Fink, introduced a purely software system to track a user's gestures, a gesture-recognition system that could work on any Android-based smart glass with a standard camera. (It was acquired by Atheer in 2015).
Augmented Reality systems allowed the user to mix virtual and reality objects. Meta, founded in 2012 in Redwood City by Meron Gribetz, introduced the first augmented reality system in 2014, see-through glasses that allowed wearers to move and manipulate 3D content using hand gestures.
Coincidentally, Google shut down Glass the very same week of 2015 in which Microsoft announced HoloLens, a cordless, self-contained smart-glasses headset with an embedded Windows computer, whose user interface replaced the mouse with the gaze of the user and the mouse click with a motion of her finger. It allowed the user to walk around three-dimensional virtual objects and employed gaze, gesture and voice to modify them. To carry out the same functions, Oculus' Rift needed to be supplemented with gesture recognition (Leap Motion or Kinect) and stereoscopic cameras; and Rift was plugged into a host computer, whereas HoloLens was stand-alone. Microsoft leveraged technology acquired from Osterhout Design Group, a company founded in 1999 in San Francisco by Ralph Osterhout that introduced its first consumer glasses in 2015. If Facebook/Oculus targeted the gaming community, Microsoft aimed at the enterprise, viewing the HoloLens as a productivity tool.
However, what truly made augmented reality popular was a mobile game introduced by Niantic and Nintendo in 2016: Pokemon Go. Niantic, founded in 2010 in San Francisco by John Hanke and initially incubated within Google, had already tested the market for augmented-reality mobile games with Ingress in 2012. Pokemon Go became a worldwide phenomenon: within one month of its introduction, it was downloaded more than 100 million times.
A number of startups wanted to give virtual reality a social life, improving over the original Second Life model. In 2013 David Gudmundson, Eric Romo and Gavan Wilhite launched Altspace in Redwood City. In the same year Second Life's original inventor Philip Rosedale started High Fidelity in San Francisco.
The media were mostly emphasizing the devices to "consume" virtual reality (either for fun or for work), not the tools that helped designers to create their own virtual world. The main platforms for virtual-reality creators were sold by veterans of the gaming world who had been offering engines to create 3D games for a decade or more: Unreal Engine 4 (North Carolina, 1998), Razer OSVR or Open Source Virtual Reality (San Diego, 1998), EON Reality (Sweden, 1998), Worldviz (Santa Barbara, 2002), Unity 5 (San Francisco, 2005), as well as Autodesk's Stingray (2015), built around the Bitsquid technology that Autodesk acquired in 2014.
Virtual Reality was one of the factors triggering a revolution in human-machine interfaces. Israeli-based Lumus offered a see-through display for augmented reality. Survios, founded in Los Angeles in 2013, an offshoot of the University of Southern California's Mixed Reality Lab, offered an immersive headset ΞαΞιΞι la Oculus that was also capable of tracking the user's physical movements ΞαΞιΞι la Kinect. Various versions of "optical touch" turned every object into an input device. Zhen Liu, a Chinese-born graduate from Harbin's Institute of Technology, introduced in 2013 in Singapore Touchjet Pond, that turned every surface into a touch-screen. Israeli-based Lumio turned any surface into a keyboard by tracking the movement of the fingers on a projection. Measuring neural activity became more affordable and led to wearables that could determine one's state of mind, from South Korea's SOSO (tested in schools to determine children's concentration) to Israel's ElMindA.
By 2018 there was a feeling that the Oculus generation of virtual reality had run its course and left little behind other than a few games. There was a need for new applications. For example, San Francisco-based Vivid Vision (founded in 2014 by Manish Gupta, Tuan Tran and James Blaha) was using VR for repairing eye conditions, and Palo Alto-based Moodru (founded in 2013 by Tony Burton) was using VR to control anxiety and phobias. This was a general trend: in 2016 the Swiss startup MindMaze, that was developing virtual reality hardware and software to provide clinical treatments for stroke victims, raised enough funding to become a unicorn. There was also a feeling that augmented reality made more (commercial) sense than virtual reality, and that bridging the virtual and physical world was the real challenge. Occipital's headset Bridge for the iPhone enabled both VR and AR applications.
The 3D capture system HoloPortal by San Jose-based DoubleMe, founded in 2014 by South Korean machine-vision expert Albert Kim, converted pictures and videos taken from multiple angles with regular cameras into 3D models for both VR and AR applications (an immediate hit with the porn industry).
Education remained high in the list of applications. Paul Kellenberger's Sunnyvale-based Infinite Z (later renamed zSpace), originally incubated in 2007 by the CIA's incubator In-Q-Tel to develop a platform for 3D software and later employed by NASA for simulation in virtual environments, introduced its virtual-reality platform for education in 2014 and in 2016 a virtual-reality browser.
Then there were the studios, the dreamers who pioneered a new form of film-making. For example, Baobab, founded in 2015 in Redwood City by Maureen Fan, produced VR movies.
Hardware innovation for virtual reality slowed down dramatically, but Avegant, founded in 2012 in Belmont by Allan Evans and Edward Tang, first introduced the total-immersion wearable display Glyph (a sort of personal movie theater) and then a light-field display that allowed users to view virtual objects at multiple focal planes at the same time.
In 2018 a number of new products used augmented reality to provide new kinds of communications. Magic Leap introduced an avatar-based chat on Twilio, founded in 2007 in San Francisco by Jeff Lawson to add text, voice and video messaging to apps that already claimed high-profile users such as Uber, Airbnb and Home Depot. Spatial, founded in 2016 in New York by Anand Agarawala and Jinha Lee (both formerly founders of BumpTop, acquired by Google in 2010), introduced a similar app for virtual interaction via Magic Leap's device or Microsoft's HoloLens.
click here for the other sections of the chapter "The Selfies (2011-16)"