Designing Fluid Interfaces

Hey, guys. Thank you.

Thanks for coming by, guys.

Welcome to Designing Fluid Interfaces. My name is Chan. And, I work on the human interface team here at Apple. And, most recently, I worked on this, the fluid gestural interface for iPhone 10.

So, me, Marcos, and Nathan, we want to share a little bit about what we learned working on this, and other projects like this in the past.

So, the question we ask ourselves a lot is, what actually makes an interface feel fluid? And, we've noticed that a lot of people actually describe it differently. You know, sometimes, when people actually try this stuff, when we show them a demo, and they try it, and they hold it in their hands, they sometimes say it feels fast. Or, other people sometimes say it feels smooth. And, when it's feeling really good, sometimes people even say it feels natural, or magical.

But, when it comes down to it, it really feels like, it's one of those things where you just know it when you feel it. It just feels right.

And, you can have a gestural UI, and we've seen lots of gestural UI's out there, but if it's not done right, something just feels off about it. And, it's oftentimes hard to put your finger on why.

And, it's more than just about frame rates, you know. You can have something chugging along at a nice 60 frames per second, but it just feels off. So, what gives us this feeling? Well, we think it boils down to when the tool feels like an extension of your mind. An extension of your mind.

So, why is this important? Well, if you look at it, the iPhone is a tool, right? It's a hand tool for information and communication. And, it works by marrying our tactile senses with our sense of vision.

But, if you think about it, it's actually part of a long line of hand tools extending back thousands of years.

The tool on the left here was used to extract bone marrow 150,000 years ago, extending the sharpness of what our fingers could do. So, we've been making hand tools for some time now. And, the most amazing thing is that our hands have actually evolved and adapted alongside our tools. We've evolved a huge concentration of muscles, nerves, blood vessels that can perform the most delicate gestures, and sense the lightest of touches.

So, we're extremely adapted to this tactile world we all live in.

But, if you look at the history of computers, we started in a place where there was a lot of layers of extraction between you and the interface. There was so much you had to know just to operate it. And, that made it out of reach for a lot of people.

But, over the last few decades or so, you've sort of been stripping those layers back you know, starting with indirect manipulation, where things were a little bit more one-to-one. A little bit more direct all the way to now, where we're finally stripping away all those layers back, to where you're directly interacting with the content. This to us is the magical element. It's when it stops feeling like a computer, and starts feeling more like an extension of the natural world. This means the interface is now communicating with us at a much more ancient level than interfaces have ever done. And, we have really high standards for it. You know, if the slightest thing feels wrong, boom, the illusion is just shattered. But, when it feels right, it feels like an extension of yourself, an extension of your physical body. It's a tool that's in sync with your thought. It feels delightful to use, and it feels really low-friction, and even playful.

So, what gives us this feeling? And, when it feels off, how do we make it feel right? That's what this presentation's all about.

We're going to talk about four things today. And, we're going to start with designing some principles, talking about how we build interfaces that feel like an extension of us.

How to design motion that feels in tune with the motion of our own bodies, and the world around us.

And, also designing gestures that feel elegant and intelligent. We're also going to talk about that, now that we've built this kind of stuff, how do we build interactions on top of it that feel native to the medium of touch, as a medium? So, let's get started. How do we design an interface that actually extends our mind? How do we do this? Well, we think the way to do it, is to align the interface to the way we think and the way we move.

So, the most important part of that is that our minds are constantly responding to changes and stimulus and thought, you know? Our minds and bodies are constantly in a state of dynamic change. So, it's not that our interfaces should be fluid, it's that we're fluid, and our interfaces need to be able to respond to that.

So, that begins with response.

You know, our tools depend on the latency. Think about how hard it would be to use any tool, or play an instrument, or do anything in the physical world, if there was a delay to using it? And, we found that people are really, really sensitive to latency. You know? If you introduce any amount of lag, things all of a sudden just kind of fall off a cliff in terms of how they respond to you. There's all this additional mental burden. It feels super disconnected. It doesn't feel like an extension of you anymore.

So, we work so hard to reduce latency. Where, we actually engineered the latest iPhone to respond quicker to your finger, so we can detect all the nuances of your gestures as instantly as possible.

So, we really care about this stuff, and we think you should too. And, that means look for delays everywhere. It's not just swipes. It's taps, it's presses, it's every interaction with the object. Everything needs to respond instantly.

And, during the process of designing this stuff, you know, oftentimes the delays kind of tend to seep in a little bit. You know? So, it's really important to keep an eye out for delays. Be vigilant and mindful of all the latencies or timers that we could introduce into the interface so that it always feels responsive.

So, that's the topic of response. It's really simple, but it makes an interface feel lively and dynamic. Next, we want to allow for constant redirection and interruption. This one's big.

So, our bodies and minds are constantly in a state of redirecting in response to change in thought, like we talked about.

So, if I was walking to the end of this stage here, and I realize I forgot something back there, I could just turn back immediately. And, I wouldn't have to wait for my body to reach the end before I did that, right? So, it's important for our interfaces to be able to reflect that ability of constantly redirecting. And, it makes it feel connected to you.

That's why for iPhone 10, we built a fully redirectable interface.

So, what's that? So, the iPhone 10's an actually-- it's pretty simple two-axis gesture. You go horizontally between apps. And, you go vertically to go home. But you can also mix the two axes, so you can be on your way home, and peek at multitasking and decide whether or not to go there. Or, you can go to multitasking and decide, actually, no, I want to go home.

So, this might not seem that important, but what if we didn't do this? What if it wasn't redirectable? So, what if the only gestures you could do was this horizontal gesture between apps, and then a vertical gesture to go home, and that's it. You couldn't do any of that in-between stuff I just mentioned.

Well, what would happen is that