Introducing Core Haptics

I am Michael Diu from the Interactive Haptics Team.

And I am forward to sharing the many advances in haptics in iOS 13 with you.

Let's take a look at our agenda.

First, we'll find out where we can use Core Haptics.

How it fits in with other audio and haptic APIs.

And we'll talk about the two groups of classes in the API.

And the basic dimensions and descriptors that we will use to describe our haptics and audio content.

We're going to walk through the basic recipe to start playing out that content.

And then we're going to move on to introducing dynamic parameters. And dynamic parameters are a way that you can customize your haptic patterns at playback time in response to your user or your apps behavior.

And we're going to explore a new way to express, store, and share your audio haptics content. A new file format we're calling the Apple Haptic Audio Pattern, or AHAP.

So let's get to it.

First, what is Core Haptics? We can think of it as an event-based audio and haptic rendering API. Or synthesizer for iPhone.

We can continue to use our other audio and haptics and feedback API's like AVAudioPlayer and UIKit's UIFeedbackGenerator in parallel with Core Haptics.

You might be wondering, "Which iPhones can I use this on?" With one, just one API and one file format, we will be able to access hundreds of millions of haptic engine-equipped iPhones starting from iPhone 8 onward.

And we've taken care that your haptic patterns will have the same feel across all of these products. So much so that you're going to be able to prototype and release just using one product.

And these iPhones aren't equipped with just any old commodity actuator. They all have the Apple-designed taptic engine. Which offers you that unique combination of power, a wide, expressive range, and an unmatched precision and control and subtlety.

Next, I'd like to talk about those of you who have already started adopting haptics on iPhone with UIKit's feedback-generated API's.

Now Core Haptics is not a replacement for this API.

In most cases, you want to keep on using FeedbackGenerator, especially for UIKit controls and adding haptics to that.

With that API, you indicate the design intent for your event. Whether that's a selection, an impact, or a notification. And you let someone else, Apple, worry about developing a vocabulary to express that and mixing the right modalities like audio haptics animation to communicate that message.

Now, this API is also being improved in iOS 13. So please check out its documentation for more details.

In contrast, Core Haptics is good when you want to be your own sound and haptic designer.

With it, you develop your own patterns.

And you can have a lot more control over exactly what time it gets played. So you can synchronize with other API's like an animation from Core Animation. Or a sound event from AVAudioEngine.

You have a much richer set of playback and modulation controls.

Now, UIKit is built on top of Core Haptics. So both API's share the same low-latency performance characteristics.

Now designing your own haptic patterns is going to take more time. But when it allows you to do something that you couldn't otherwise do and when it allows you to differentiate your app, then it's worth thinking about.

Now next, I'd like to talk a bit more about those audio capabilities.

So Core Haptics is also an Audio API.

And so that allows you to play short, synthesized or custom waveform audio in synchronization, tight sync with your haptics.

This type of audio haptic duality has been crucial to many of Apple's own haptics experiences. Like the haptic home button in iPhone 7, the haptic crown in Series 4 Watch, and the UIDatePicker, those scrolling wheels that you use to select dates and times and alarms and calendars.

And you may not have realized that. You may not even have noticed that there was audio in these experiences. But if you were to cover up that audio once you take it away, you'll realize that it's an inseparable an integral part of that experience. So now, you can do the same with Core Haptics in your own apps. And I want to talk about some categories of apps, one app -- a huge category in particular where you might want to think about Core Haptics, and that's games.

So imagine we are at the race track. We want to go into turbo mode. Let's imagine.

When you've got that brute force message to deliver, think about using synchronized haptics and audio in your app to generate those visceral explosions and rumbles.

Now another very nice application is for to simulate physical contact to make your applications feel more realistic.

Think about a tennis game.

You could have audio and haptic components where the pitch of your audio, the intensity of your haptics are proportional to how fast your swing is or how centered the ball lands in the middle of the racket. And you can even control how long the strings your racket will resonate for after the impact.

So another great area to think about using Core Haptics is in augmented reality apps.

And there, if you're working in this space, you're already familiar with the benefits of having high visual fidelity paired with 3D audio working in concert. Now we can reach for that next level of immersion by considering how custom haptic feedback can ground our user gestures, or respond to app, device, and AR object events. For example, moving your device around or moving your entire users around.

As an inspiration this year, we've enhanced the Swift Shot sample code by using haptics that are modulated based on how fast you pull back the sling.

How fast you pull back your phone. You're going to feel the tension building up as you stretch it back. And then a very satisfying thunk as you release.

I'd like to show you a video of this and I'm going to use audio to represent just the haptics that you're going to feel.

They're going to sound like this.

Now, we're going to see the whole thing together.

Visuals and haptics, no regular audio.

So that was an example of how we can use haptics, sound, and visuals all synchronized together to enhance our AR experience. Now these are just a few categories of apps, games, and AR that are ripe for creative explorations, with haptics and the corresponding sounds.

I'm sure you're going to think of many, many more.

So now let's get into how we can start expressing our content with Core Haptics.

There are just two groups of classes in Core Haptics. There's those to represent your content.

And those to playback that content.

Let's take a closer look at the content side first.

The basic indivisible content elements is in Core Haptics is called a CHHapticEvent.

Now, each event has a type and a time. And optionally, parameters that will customize its feel.

These events can overlap each other and when they do, they mix.

And all events are grouped into a pattern.

Next, I'd like to talk about those types of events that we can have.

Our first type is called the HapticTransient.

The HapticTransient, I think of it as a gavel. It's a striking motion. It's momentary and instantaneous.

And then we have two continuous types.

We have HapticContinuous and AudioContinuous. And there I think of, for example, bowing a stringed instrument.

It is longer than a transient. It can be, for example, used as a background texture. And you have a much richer set of knobs that you can use. For example, to modulate the resonance of it.