Build for the iPadOS pointer

Hello and welcome to WWDC.

Hi. I'm Mohammed, a UIKit engineer. Later, I'll be joined by my colleague Joey from the iOS System UI team, and this is "How to Build for the iPadOS Pointer." In 13.4, iPadOS added general pointing device support to the iPad. This is a brand-new input method for an OS that has been primarily touch-based since its inception. Rather than just bring over the exact interaction model that exists on a Mac, we thought about what value a pointer can add to iPadOS and how it can coexist with a touch-based interface. The result is a system built on a fluid adaptive pointer that morphs into controls when hovering over them to adapt its accuracy and provide a clear indication that you're about to interact with them. In other contexts, the pointer changes its shape and motion characteristics to provide contextual hints. When hovering over text, for example, the pointer changes into a beam and snaps to lines to make interacting with text easier. In this video, we'll discuss the strategy we followed to update iPadOS for pointer and how you can use the same strategy when updating your app. In the course of the discussion, we'll talk about the pointer customization APIs introduced in iOS 13.4, and we'll talk about some of the best practices to keep in mind while updating your app. We'll also touch on some of the basic design principles behind the pointer UI. For a more detailed discussion of these principles and some of the thought processes behind the pointer design, check out the session on "Design for the iPadOS Pointer." When you use an iPad with a pointing device connected, you'll notice that a lot of things work with the pointer automatically without any additional adoption from apps. This is because a lot of system components have pointer support built in. Controls like UIBarButtonItem, UISegmentedControl and UIMenuController, to name a few, have built-in pointer effects and behaviors. Scroll views respond to scrolling with two fingers as well as mouse wheels and pinching to zoom on the trackpad. And in addition to these scroll view basics, collection and table view now support two-finger panning for swipe actions. UITextView and other components that use UITextInteraction support a suite of new quick text selection and editing gestures that should be familiar to anyone who's used a Mac. UIDragInteraction now allows you to drag quickly via a click-and-drag instead of requiring a long press as it does with touch. And UIContextMenuInteraction lets you invoke its menu in a new, compact appearance via a secondary click. So, how do we go about updating an app for pointer? We found that it's best to take a top-down approach. Start with the higher-level APIs. They'll offer you the most polished experience with tuned behaviors and visuals that are consistent with similar UI in the rest of the system. There are a number of pointer-related APIs available to you at different levels of the stack. Many controls have built-in pointer effects. Some, like bar buttons and segmented controls, have their effects enabled by default, while others, like UIButton, offer API that allows you to enable and customize their effects. With UIPointerInteraction, you can make your custom UI react to and interact with the pointer in a way that feels consistent with the rest of the system. Using the interaction, you can choose from one of a collection of system-vended effects to apply to your views. Or you can change the pointer's shape within an area of your app. UIHoverGestureRecognizer lets you respond to the pointer's motion more directly. This is great for any custom behavior that doesn't involve applying hover or highlight effects or modifying the pointer's appearance. For more details on this gesture and other pointer-related additions to gestures and events, check out the session on handling trackpad and mouse input. When updating your app, you want to aim for an experience that makes it feel consistent with the rest of the OS. When deciding where to add pointer support, take your cues from the Human Interface Guidelines and built-in apps. To that end, a good starting point is ensuring that controls in your app's chrome have the appropriate effects. By "chrome," we mean the application's top and bottom bars, so start with any bar buttons in the app. UIBarButtonItems created using the system item, image or title APIs get appropriate pointer effects automatically. If you're using the custom view API, you'll have to implement the pointer behavior yourself, either using the view's convenience API, if it has one, or by installing a pointer interaction and managing the effect yourself.

UIButtons that have been placed in bars via the custom view API have their built-in interactions enabled by default and are given an effect that the system has deemed appropriate for that button in its containing bar. You'll be able to tweak this effect using UIButton's convenience APIs. To make this really easy, UIButton has a two-stage convenience API. To enable the automatic effect, simply set the button's isPointer- InteractionEnabled property to "true." After enabling the effect, you may customize it using the button's pointerStyleProvider. In this closure, the system offers you a proposed effect and shape that have been determined based on the appearance, size and contents of the button.

Here, you can customize either of these or replace them entirely and construct a new style. Before we look at some examples, let's take a brief detour and talk about pointer styles. All APIs that modify the appearance of the pointer use UIPointerStyle to describe the modifications they apply. Styles fit into two categories.

The first is what we call a content effect. Content effects usually cause the pointer to morph into a view in the app and apply some visual treatment to it. A common example of this