How to Control Your iPhone Using Your Eyes

During the keynote at WWDC earlier this year, Apple introduced the latest iteration of its mobile operating system, iOS 12. Despite a thorough demo on stage, Apple glossed over new eye tracking features that use ARKit 2. Developers can now use the TrueDepth camera on the iPhone X, XS, XS Max, and XR to determine where your eyes are looking, opening up incredible new possibilities for new apps.

Hawkeye Access for iOS takes advantage of Apple's latest eye tracking technology by allowing you to use eye and face movements to control your device hands-free. Although the application is geared towards people with motor impairments, those with busy hands can also make use of Hawkeye Access. Let's take an in-depth look at how it works.

Step 1: Download Hawkeye Access

To get started, you'll need to install the Hawkeye Access app from the App Store. The app only works on devices with a TrueDepth camera, including the iPhone X, XR, XS, and XS Max. There's a dedicated iPad App for the iPad Pro as well.

App Store Link: Hawkeye Access (free)

How to Control Your iPhone Using Your Eyes

Step 2: Calibrate Your Eyes

Next, open Hawkeye Access and you'll be prompted to calibrate your eye movement. Ensure that your face is fully visible and that there is adequate light — without good lighting, the app does not work. You'll be asked to stare at various points across the screen for a few seconds. Once a checkmark appears, move on to the next point until calibration is complete.

Step 3: Choose a Selection Mode

Once you've finished calibrating, the app will ask you to choose a selection mode. You can select "Smile," "Blink," or "Hold Gaze" to make selections in the app. If you're unsatisfied with your chosen selection mode, you can later go into the app's settings and choose a new selection mode to try out.

Step 4: Control Your eyePhone

Hawkeye Access is essentially a web browser which allows you easily access bookmarked websites through the home page of the app. A set of popular websites and apps are provided, including Google, Twitter, YouTube, Instagram, Facebook, Amazon, and Wikipedia.

So when you'd like to control your iPhone with only your eyes, start by opening the Hawkeye Access app. Move your eyes around to move the cursor. Each website appears in a sizable block — six to a page — making it easier to make a selection. Depending on which selection mode you chose, you'll either have to blink, smile, or hold your gaze in order to "tap" an icon. Out of the three, I preferred the blink selection mode.

If you go into "Web," you can point your cursor to the top (where the URL is) and select it. You can then use speech-to-text to search something as you would in the Google search bar in Safari.

In order to move across a page, use your eyes to look either all the way up or down until a circle with an arrow appears. Use your selection mode and the page will immediately scroll up or down. You can also go forward or back by looking either right or left until a cursor with an arrow appears. If you want to head directly home, look all the way down at the bottom-right corner of your screen until the home button appears, then select it.

Take advantage of the bookmarked websites so that you don't have to use speech-to-text to find them. A few of the easier-to-use apps in "Access" include Google, YouTube, Amazon, and Wikipedia.

Conclusion

Hawkeye Access can be tricky to use. It requires you to type in emails and passwords, because the speech-to-text feature doesn't work for that. The eye tracking can also finicky; at times it works with a charm, but if a website has smaller icons, it's almost impossible to make a selection (you can tinker with the sensitivity, which actually does improve the app). And there are features missing that would be useful, including the ability to bookmark your own websites.

Even with its flaws, Hawkeye Access works as a viable option for when you want to look up a recipe on your phone while your hands are covered in grease, or you want to change a song while busy cooking. Basically, it's great to use while in the kitchen, but it still needs some work before it's marketed towards those with motor impairments.