Empower your intents

Hello, and welcome to WWDC.

Hello, and thanks for joining us. My name is Roman, and I'm a shortcuts engineer at Apple.

Today I'm going to be sharing with you some techniques and some strategies for efficiently implementing Siri and Shortcuts support in your applications.

First, we are going to take a tour of the new APIs in SiriKit.

Then, we are going to learn all about how you can fine-tune custom intents in your app.

So let's start with a brief overview.

SiriKit encompasses the Intents and Intents UI frameworks, which you use to integrate your services with Siri, Shortcuts, and Maps. And in iOS 14, you can use Intents to add configuration and intelligence to widgets.

An Intents UI app extension displays custom UI in the Siri, Shortcuts, or Maps interface after your Intents app extension fulfills a user request. An Intents app extension receives user requests from SiriKit and turns them into app-specific actions, such as sending a message, playing some music, getting current weather conditions, or ordering a soup.

SiriKit defines a large number of system intents that represent common tasks people take.

For system intents, Siri defines the conversational flow while your app provides the data to complete the interaction. If your app lets people perform an everyday task that doesn't fit into any of the SiriKit domains, you can create a custom intent to represent it.

The job of your Intents extension is to provide SiriKit with the handler objects that you use to handle specific intents.

You provide these objects from the handlerForIntent: method of your extension. Let's take a look at the life cycle of handling an intent. Every time the user interacts with an intent, whether at the resolve, confirm, or handle phases, your intent handler has ten seconds to complete the request.

The ten-second timeout starts as soon as the user request initiates a connection to your extension.

When this happens, if your extension is not yet running, it will be launched by the system. The amount of time it takes to launch your extension depends on how long it takes to load all of the frameworks your process links, and how much time it takes to run the +loads and static initializers included in your process or any frameworks it links.

Your extension also needs some time to perform your business logic.

So it's important to optimize for launch time by making sure that your extension only links the frameworks that it really needs, because the time spent loading all of the link frameworks counts towards the ten-second timeout. The amount of time it takes to handle a request also affects how people perceive interactions with your app.

Siri interactions are intended to be quick and lightweight, so you should avoid making the user wait while your intent handler is working.

Ten seconds seems like a lot. Since in most of the cases your Intents extension doesn't really need to link all of the frameworks that your app links, you have the opportunity to minimize the number of symbols that you import.

Another noteworthy characteristic of Intents extensions is that they are modular, independently run processes with a lower memory footprint than an app.

However, sometimes it's not always possible or convenient to use extensions.

In iOS 14, we are introducing in-app intent handling. Now, you have an option to add an intent handler to your app where you can handle SiriKit requests such as Resolve, Confirm, Handle.

Let's talk about a few good use cases that should help you decide when to handle your intents in your app versus your extension.

Starting or controlling media playback, or starting a workout, previously required you to perform resolve and confirm in your Intents extension, and then handle in your app.

It is more efficient to do this entirely in your app process now.

In another scenario, if handling your intent affects your app's user interface live on-screen, it's also a good candidate for in-app intent handling.

In-app intent handling also opens up an opportunity for new use cases that weren't possible before due to memory constraints of extensions, such as photo and video processing.

And, well, let's be honest, in some cases your app's structure doesn't currently allow you to factor out code into an intents extension or a shared framework. But, of course, be mindful about the launch time of your app because it will eat into the ten-second timeout we talked about earlier in this session.

When you're designing your intent handlers, you should evaluate which intents need to be handled in your intents extension, and which can be moved to your app.

Let's see how you can implement support for in-app intent handling in your app.

The first thing you need to do is to make sure that your app supports multiple windows and has adopted the UIScene life cycle. When your app is launched in a response to a SiriKit request, it will be launched without any UIScene objects connected to your app.

Then, you need to list all of the intents that you would like to handle inside of your application in the Supported Intents section of your app's target.

And finally, you need to implement the handlerForIntent: method in your app's delegate.

This method acts as a dispatcher mapping incoming intents to the object capable of handling them. In your implementation, check the type of the intent parameter and return an object capable of handling that type of intent.

The object you return must adopt the protocol used to handle that intent.

For example, when handling a ProcessPhotoIntent object, return an object that adopts the ProcessPhotoIntentHandling protocol.

If handling an intent updates your application's user interface, and requires your users to be looking at something in the app before using the intent, your intent handler may need to make sure that the relevant UI is on-screen in the handle phase.

You can do this by checking to make sure the app is not in the background. And if it is, asking the system to open it by responding with continueInApp response code. Let's see how to add in-app intent handling to an app in Xcode.

I got into cooking at home recently, so I've been working on this app called Recipe Assistant that allows me to browse my favorite recipes.

When I tap on a recipe, I can see all of the ingredients needed to make this recipe.

I can tap on the "Directions" button to view step-by-step directions. When I tap on the "Next Step" button I can advance to the next step. However, sometimes it's not always convenient to tap buttons on-screen while you're preparing the meal.

So I would like users of my app to be able to advance to the next step by using a shortcut that they invoke with their voice using Siri.

I'm going to implement in-app intent handling in my app since my users will be interacting with the content on-screen.

Here in Xcode, I have my intent-definition file where I define my custom intent called "ShowDirections." I also added the "Add to Siri" button to my recipe directions view so users can easily set up a shortcut here. This is what it looks like in the app.

Now, back to Xcode. I have already adopted multiple-window support in my app. Now, I need to add ShowDirectionsIntent to the list of intents eligible for in-app intent handling. To do that, I'm going to click on the "Plus" button and add my intent.

Now, what I want to do is to make every view controller in my app to be able to respond to the "Next Step" command.

To do that, I'm going to define a common intent handler that accepts an object conforming to the NextStepProviding protocol.

A view controller that adopts the NextStepProviding protocol will need to return an instance of the IntentHandler class. It would also need to implement the nextStep function that will take the user to the next step in the app. Let's conform our intent handler to the showDirectionsIntentHandling protocol that was code-generated for us, and implement the resolve method.

In the resolve method for the recipe parameter, we are going to check if we have a recipe. If we don't have a recipe, we will ask for disambiguation. Otherwise, we will return success.

In the handle method, we are going to tell the next-step provider to go to the next step. We need to make sure that our app is in the foreground. And if it's not, that we are going to launch the app using the continueInApp response code. We will need to handle launching with a user activity in our SceneDelegate, and I will come back to this in a moment. Now, I will create a new singleton class that will hold a weak reference to the current intent handler.

We are going to assign the current intent handler in viewDidAppear of each view controller in our app.

In my app delegate, I need to implement the new handlerForIntent: method, and return the current IntentHandler instance.

In my SceneDelegate object, I need to implement both willConnectToSession and continueUserActivity methods to continue user activity when we respond with continueInApp response code in our handle method.

willConnectToSession will be invoked when the app does not have a UIScene object connected to it.

Now, we need to conform each view controller to the NextStepProviding protocol.

And finally, we need to make sure to assign the current intent handler in viewDidAppear.

Let's give it a try.

First, I need to add my shortcut to Siri.

Now, let's see what the experience looks like. Hey Siri, next step.

Which recipe would you like to see? Spicy tomato sauce, chickpea curry, or cinnamon apple cake? The first one. Okay. Viewing. Here are the ingredients for spicy tomato sauce.

Hey Siri, next step.