Smartglasses

Using the touch pad built on the side of the 2013 Google Glass to communicate with the user's phone using Bluetooth, device.

Man wearing a 1998 EyeTap, Digital Eye Glass.^[1]

Man with BuBBles glasses

Smartglasses or smart glasses are wearable computer glasses that add information alongside or to what the wearer sees.^{[1][2][3][4][5][6][7][8]} Typically this is achieved through an optical head-mounted display (OHMD) or embedded wireless glasses with transparent heads-up display (HUD) or augmented reality (AR) overlay that has the capability of reflecting projected digital images as well as allowing the user to see through it, or see better with it. While early models can perform basic tasks, such as just serve as a front end display for a remote system, as in the case of smartglasses utilizing cellular technology or Wi-Fi, modern smart glasses are effectively wearable computers which can run self-contained mobile apps. Some are handsfree that can communicate with the Internet via natural language voice commands, while other use touch buttons.^{[9][10][11][12][13][14][15]}

Like other computers, smartglasses may collect information from internal or external sensors. It may control or retrieve data from other instruments or computers. It may support wireless technologies like Bluetooth, Wi-Fi, and GPS. While a smaller number of models run a mobile operating system and function as portable media players to send audio and video files to the user via a Bluetooth or WiFi headset.^[16][17] Some smartglasses models, also feature full lifelogging and activity tracker capability.^{[18][19][20][21]}

Such smartglasses devices may also have all the features of a smartphone.^[22][23] Some also have activity tracker functionality features (also known as "fitness tracker") as seen in some GPS watches.^[8][24]

Features and applications

As with other lifelogging and activity tracking devices, the GPS tracking unit and digital camera of some smartglasses can be used to record historical data. For example, after the completion of a workout, data can be uploaded onto a computer or online to create a log of exercise activities for analysis. Some smart watches can serve as full GPS navigation devices, displaying maps and current coordinates. Users can "mark" their current location and then edit the entry's name and coordinates, which enables navigation to those new coordinates.^[25][26]

Although some smartglasses models manufactured in the 21st century are completely functional as standalone products, most manufacturers recommend or even require that consumers purchase mobile phone handsets that run the same operating system so that the two devices can be synchronized for additional and enhanced functionality. The smartglasses can work as an extension, for head-up display (HUD) or remote control of the phone and alert the user to communication data such as calls, SMS messages, emails, and calendar invites.^[27]

Healthcare applications

Several proof of concepts for Google Glasses have been proposed in healthcare. In July 2013, Lucien Engelen started research on the usability and impact of Google Glass in health care. As of August 2013, Engelen, who is based at Singularity University and in Europe at Radboud University Medical Center,^[28] is the first healthcare professional in Europe to participate in the Glass Explorer program.^[29] His research on Google Glass (starting August 9, 2013) was conducted in operating rooms, ambulances, a trauma helicopter, general practice, and home care as well as the use in public transportation for visually or physically impaired. His research consisted of taking pictures, streaming videos to other locations, dictating operative log, and tele-consultation through Hangout. Engelen documented his findings in blogs,^[30] videos,^[31] pictures, on Twitter,^[32] and on Google+.^[33] and is still ongoing.

Key findings of Engelen's research included:

1. The quality of pictures and video are usable for healthcare education, reference, and remote consultation.The camera needs to be tilted to different angle^[34] for most of the operative procedures
2. Tele-consultation is possible—depending on the available bandwidth—during operative procedures.^[35]
3. A stabilizer should be added to the video function to prevent choppy transmission when a surgeon looks to screens or colleagues.
4. Battery life can be easily extended with the use of an external battery.
5. Controlling the device and/or programs from another device is needed for some features because of sterile environment.
6. Text-to-speech ("Take a Note" to Evernote) exhibited a correction rate of 60 percent, without the addition of a medical thesaurus.
7. A protocol or checklist displayed on the screen of Google Glass can be helpful during procedures.

Dr. Phil Haslam and Dr. Sebastian Mafeld demonstrated the first concept for Google Glass in the field of interventional radiology. They demonstrated the manner in which the concept of Google Glass could assist a liver biopsy and fistulaplasty, and the pair stated that Google Glass has the potential to improve patient safety, operator comfort, and procedure efficiency in the field of interventional radiology.^[36] In June 2013, surgeon Dr. Rafael Grossmann was the first person to integrate Google Glass into the operating theater, when he wore the device during a PEG (percutaneous endoscopic gastrostomy) procedure.^[37] In August 2013, Google Glass was also used at Wexner Medical Center at Ohio State University. Surgeon Dr. Christopher Kaeding used Google Glass to consult with a colleague in a distant part of Columbus, Ohio. A group of students at The Ohio State University College of Medicine also observed the operation on their laptop computers. Following the procedure, Kaeding stated, "To be honest, once we got into the surgery, I often forgot the device was there. It just seemed very intuitive and fit seamlessly."^[38]

The November 16, 2013, in Santiago de Chile, the maxillofacial team led by Dr.gn Antonio Marino conducted the first orthognathic surgery assisted with Google Glass in Latin America, interacting with them and working with simultaneous three-dimensional navigation. The surgical team was interviewed by the ADN radio medium and the LUN newspaper. In January 2014, Indian Orthopedic Surgeon Selene G. Parekh conducted the foot and ankle surgery using Google Glass in Jaipur, which was broadcast live on Google website via the internet. The surgery was held during a three-day annual Indo-US conference attended by a team of experts from the US, and co-organized by Dr Ashish Sharma. Sharma said Google Glass allows looking at an X-Ray or MRI without taking the eye off of the patient, and allows a doctor to communicate with a patient's family or friends during a procedure. "The image which the doctor sees through Google Glass will be broadcast on the internet. It's an amazing technology. Earlier, during surgeries, to show something to another doctor, we had to keep moving and the cameraman had to move as well to take different angles. During this, there are chances of infection. So in this technology, the image seen by the doctor using Google Glass will be seen by everyone throughout the world," he said.

Baby Eve with Georgia for the Breastfeeding Support Project

In Australia, during January 2014, Melbourne tech startup Small World Social collaborated with the Australian Breastfeeding Association to create the first hands-free breastfeeding Google Glass application for new mothers.^[39] The application, named Google Glass Breastfeeding app trial, allows mothers to nurse their baby while viewing instructions about common breastfeeding issues (latching on, posture etc.) or call a lactation consultant via a secure Google Hangout, who can view the issue through the mother's Google Glass camera.^[40] The trial was successfully concluded in Melbourne in April 2014, and 100% of participants were breastfeeding confidently.^[41][42] Small World Social Breasfteeding Support Project

Display types

Various techniques have existed for see-through HMDs. Most of these techniques can be summarized into two main families: “Curved Mirror” (or Curved Combiner) based and “Waveguide” or "Light-guide" based. The curved mirror technique has been used by Vuzix in their Star 1200 product, by Olympus, and by Laster Technologies. Various waveguide techniques have existed for some time. These techniques include diffraction optics, holographic optics, polarized optics, reflective optics, and projection:

• Diffractive waveguide – slanted diffraction grating elements (nanometric 10E-9). Nokia technique now licensed to Vuzix.
• Holographic waveguide – 3 holographic optical elements (HOE) sandwiched together (RGB). Used by Sony and Konica Minolta.
• Polarized waveguide – 6 multilayer coated (25-35) polarized reflectors in glass sandwich. Developed by Lumus.
• Reflective waveguide – thick light guide with single semi reflective mirror. This technique is used by Epson in their Moverio product.
• "Clear-Vu" reflective waveguide – thin monolithic molded plastic w/ surface reflectors and conventional coatings developed by Optinvent and used in their ORA product.
• Switchable waveguide – developed by SBG Labs.
• Virtual retinal display (VRD) – Also known as a retinal scan display (RSD) or retinal projector (RP), is a display technology that draws a raster display (like a television) directly onto the retina of the eye.

The Technical Illusions castAR uses a different technique with clear glass. The glasses have a projector, and the image is returned to the eye by a reflective surface.

Human Computer Interface (HCI) control input

Head-mounted displays are not designed to be workstations, and traditional input devices such as keyboard and mouse do not support the concept of smartglasses. Instead Human Computer Interface (HCI) control input needs to be methods lend themselves to mobility and/or hands-free use are good candidates, for example:

• Touchpad or buttons
• Compatible devices (e.g. smartphones or control unit) for remote control
• Speech recognition
• Gesture recognition^[43]
• Eye tracking
• Brain–computer interface

Products

In development

• AiR (Augmented interactive Reality) Platform by Atheer Labs – gesture-controlled mobile AR smartglasses for industrial applications
• AMA Experteye - Advanced Mobile Applications (AMA Studios) software for off the shelf customizable smart glasses interface^[44]
• castAR by Technical Illusions – wearable AR device for gaming
• Google Glass (consumer) – optical head-mounted display
• HoloLens by Microsoft – Windows 10 based AR unit, with high-definition 3D optical head-mounted display and spatial sound
• Mirama by Brilliantservice Co., Ltd.– gesture controlled augmented reality smartglasses
• Meta Company "spaceglasses"
• Vuzix "Vuzix M300 and Vuzix M3000, expected summer 2016"^[45]

Epson Moverio eyewear

Current

• Airscouter, a virtual retinal display made by Brother Industries^[46][47]
• BuBBles glasses – augmented reality glasses by BuBBles lab^[48]
• Epiphany Eyewear - smartglasses developed by Vergence Labs, a subsidiary of Snap Inc.
• Epson Moverio BT-200 – augmented reality smartglasses by Epson.^[49]
• EyeTap – eye-mounted camera and head-up display (HUD)
• Golden-i – head-mounted computer
• Google Glass – optical head-mounted display
• Microsoft HoloLens - a pair of mixed reality smartglasses developed and manufactured by Microsoft, using the Windows Holographic platform.
• Optinvent ORA-1 – eye-mounted camera and heads-up display (HUD) wearable computing platform^[50]
• Pivothead SMART – "Simple Modular Application-Ready Technology", released in October 2014^[51]
• Recon Snow 2 – eye-mounted camera and head-up display (HUD) snow goggles^[52]
• Recon Jet – rugged eye-mounted camera and head-up display (HUD) for sporting^[53]
• SixthSense – wearable AR device
• Spectacles -
• Vuzix – Augmented reality glasses for 3D gaming, manufacturing training, and military applications.

Discontinued

• Looxcie – ear-mounted streaming video camera^[54]

2010s

2012

• On 17 April 2012, Oakley's CEO Colin Baden stated that the company has been working on a way to project information directly onto lenses since 1997, and has 600 patents related to the technology, many of which apply to optical specifications.^[55]
• On 18 June 2012, Canon announced the MR (Mixed Reality) System which simultaneously merges virtual objects with the real world at full scale and in 3D. Unlike the Google Glass, the MR System is aimed for professional use with a price tag for the headset and accompanying system is $125,000, with $25,000 in expected annual maintenance.^[56]

2013

• At MWC 2013, the Japanese company Brilliant Service introduced the Viking OS, an operating system for HMD's which was written in Objective-C and relies on gesture control as a primary form of input. It includes a facial recognition system and was demonstrated on a revamp version of Vuzix STAR 1200XL glasses ($4,999) which combined a generic RGB camera and a PMD CamBoard nano depth camera.^[57]
• At Maker Faire 2013, the startup company Technical Illusions unveiled CastAR augmented reality glasses which are well equipped for an AR experience: infrared LEDs on the surface detect the motion of an interactive infrared wand, and a set of coils at its base are used to detect RFID chip loaded objects placed on top of it; it uses dual projectors at a frame rate of 120 Hz and a retro reflective screen providing a 3D image that can be seen from all directions by the user; a camera sitting on top of the prototype glasses is incorporated for position detection, thus the virtual image changes accordingly as a user walks around the CastAR surface.^[58]
• At D11 Conference 2013, the startup company Atheer Labs unveild its 3D augmented reality glasses prototype. The prototype includes binicular lens, 3D images support, a rechargeable battery, WiFi, Bluetooth 4.0, accelerometer, gyro and an IR. User can interact with the device by voice commands and the mounted camera allows the users to interact naturally with the device with gestures.^[59]

2014

• The Orlando Magic, Indiana Pacers, and other NBA teams used Google Glass on the CrowdOptic platform to enhance the in-game experience for fans.^[60]
• Rhode Island Hospital's Emergency Department became the first emergency department to experiment with Google Glass applications.^[61]
• 18 January 2014, BuBBles Lab's CEO Dragos GHETU announced first design of BuBBles Glasses, the first commercial smartglasses dedicated to augmented reality, working with animations, movies, 3D models on a full view device under Aurasma augmented reality engine: https://www.youtube.com/watch?v=QPp0Mg6Rb1k

Market structure

Analytics company IHS has estimated that the shipments of smart glasses may rise from just 50,000 units in 2012 to as high as 6.6 million units in 2016.^[62] According to a survey of more than 4,600 U.S. adults conducted by Forrester Research, around 12 percent of respondents are willing to wear Google Glass or other similar device if it offers a service that piques their interest.^[63] Business Insider's BI Intelligence expects an annual sales of 21 million Google Glass units by 2018.^[64][65][66] Samsung and Microsoft are expected to develop their own version of Google Glass within six months with a price range of $200 to $500. Samsung has reportedly bought lenses from Lumus, a company based in Israel. Another source says Microsoft is negotiating with Vuzix.^[67] In 2006, Apple filed patent for its own HMD device.^[68] In July 2013, APX Labs founder and CEO Brian Ballard stated that he knows of 25 to 30 hardware companies which are working on their own versions of smartglasses, some of which APX is working with.^[69]

Public reception for commercial usage

Critical reception

In November 2012, Google Glass received recognition by Time Magazine as one of the "Best Inventions of the Year 2012", alongside inventions such as the Curiosity Rover.^[70] After a visit to the University of Cambridge by Google's chairman Eric Schmidt in February 2013, Wolfson College professor^[71] John Naughton praised the Google Glass and compared it with the achievements of hardware and networking pioneer Douglas Engelbart. Naughton wrote that Engelbart believed that machines "should do what machines do best, thereby freeing up humans to do what they do best".^[72] Lisa A. Goldstein, a freelance journalist who was born profoundly deaf, tested the product on behalf of people with disabilities and published a review on August 6, 2013. In her review, Goldstein states that Google Glass does not accommodate hearing aids and is not suitable for people who cannot understand speech. Goldstein also explained the limited options for customer support, as telephone contact was her only means of communication.^[73]

In December 2013, David Datuna became the first artist to incorporate Google Glass into a contemporary work of art.^[74][75] The artwork debuted at a private event at The New World Symphony in Miami Beach, Florida, US and was moved to the Miami Design District for the public debut.^[76] Over 1500 people used Google Glass to experience Datuna's American flag from his "Viewpoint of Billions" series.^[15]

Privacy concerns

The EyeTap's functionality and minimalist appearance have been compared to Steve Mann's EyeTap,^[77] also known as "Glass" or "Digital Eye Glass", although Google Glass is a "Generation-1 Glass" compared to EyeTap, which is a "Generation-4 Glass".^[78] According to Mann, both devices affect both privacy and secrecy by introducing a two-sided surveillance and sousveillance.^[79] Concerns have been raised by various sources regarding the intrusion of privacy, and the etiquette and ethics of using the device in public and recording people without their permission.^[80][81][82] There is controversy that Google Glass would violate privacy rights due to security problems and others.^[83][84][85]

Privacy advocates are concerned that people wearing such eyewear may be able to identify strangers in public using facial recognition, or surreptitiously record and broadcast private conversations.^[12] Some companies in the U.S. have posted anti-Google Glass signs in their establishments.^[86][87] In July 2013, prior to the official release of the product, Stephen Balaban, co-founder of software company Lambda Labs, circumvented Google’s facial recognition app block by building his own, non-Google-approved operating system. Balaban then installed face-scanning Glassware that creates a summary of commonalities shared by the scanned person and the Glass wearer, such as mutual friends and interests.^[88] Additionally, Michael DiGiovanni created Winky, a program that allows a Google Glass user to take a photo with a wink of an eye, while Marc Rogers, a principal security researcher at Lookout, discovered that Glass can be hijacked if a user could be tricked into taking a picture of a malicious QR code.^[89][90]

Other concerns have been raised regarding legality of Google Glass in a number of countries, particularly in Russia, Ukraine, and other post-USSR countries. In February 2013, a Google+ user noticed legal issues with Google Glass and posted in the Google Glass community about the issues, stating that the device may be illegal to use according to the current legislation in Russia and Ukraine, which prohibits use of spy gadgets that can record video, audio or take photographs in an inconspicuous manner.^[91] Concerns were also raised in regard to the privacy and security of Google Glass users in the event that the device is stolen or lost, an issue that was raised by a US congressional committee. As part of its response to the governmental committee, Google stated in early July that is working on a locking system and raised awareness of the ability of users to remotely reset Google Glass from the web interface in the event of loss. Several facilities have banned the use of Google Glass before its release to the general public, citing concerns over potential privacy-violating capabilities. Other facilities, such as Las Vegas casinos, banned Google Glass, citing their desire to comply with Nevada state law and common gaming regulations which ban the use of recording devices near gambling areas.^[92]

Safety considerations

Concerns have also been raised on operating motor vehicles while wearing the device. On 31 July 2013 it was reported that driving while wearing Google Glass is likely to be banned in the UK, being deemed careless driving, therefore a fixed penalty offense, following a decision by the Department for Transport.^[93] In the U.S., West Virginia state representative Gary G. Howell introduced an amendment in March 2013 to the state's law against texting while driving that would include bans against "using a wearable computer with head mounted display." In an interview, Howell stated, "The primary thing is a safety concern, it [the glass headset] could project text or video into your field of vision. I think there's a lot of potential for distraction."^[94]

In October 2013, a driver in California was ticketed for "driving with monitor visible to driver (Google Glass)" after being pulled over for speeding by a San Diego Police Department officer. The driver was reportedly the first to be ticketed for driving while wearing a Google Glass.^[95] While the judge noted that 'Google Glass fell under "the purview and intent" of the ban on driving with a monitor', the case was thrown out of court due to lack of proof the device was on at the time.^[96] In November 2013, a Canadian company, Vandrico, released a study that highlighted the fact that the bone conduction transducer's audibility is improved while wearing foam ear plugs, which could encourage workers to wear hearing protection in loud work environments.^[97]

See also

Wikimedia Commons has media related to Smartglasses.

• Head-mounted display
• Wearable technology
• Quantified Self logging

References

Further reading

• 3D VIS Lab, University of Arizona – "Head-Mounted Display Systems" by Jannick Rolland and Hong Hua
• Optinvent – "Key Challenges to Affordable See Through Wearable Displays: The Missing Link for Mobile AR Mass Deployment" by Kayvan Mirza and Khaled Sarayeddine
• Optics & Photonics News – "A review of head-mounted displays (HMD) technologies and applications for consumer electronics" by Jannick Rolland and Ozan Cakmacki
• Google Inc. – "A review of head-mounted displays (HMD) technologies and applications for consumer electronics" by Bernard Kress & Thad Starner (SPIE proc. # 8720, May 31, 2013)
• SPIE Newsroom – Bernard Kress plenary: Designing the next generation of wearable displays (31 August 2015, SPIE Newsroom)