Introducing RealityKit and Reality Composer

Hi, everybody.

My name is Cody and I'd like to welcome you to the introduction to RealityKit and Reality Composer.

So today, the App Store is full of many different AR experiences that utilize the power of ARKit. And developers of these apps have a lot of common needs, for example, rendering, physics and animation. And ARKit makes building AR experiences pretty simple but now we're going to make it even easier.

The developing applications for AR, provides very unique challenges that really don't exist anywhere else, and it mostly centers around the fact that your virtual content will now interact with the real world and vice versa. So if you put like a virtual lamp into a real living room, it should probably light up the surrounding objects whether they're real or virtual. And additionally, any content that gets placed in the real world should look like it belongs there. And this can be very important in a case like online shopping, where you want to see how the product will look in your home. So this gives us a very strong requirement for a very realistic render, otherwise, you'll lose the illusion of augmented reality.

So enter RealityKit, which is a brand new Swift framework designed to help you build your AR applications and easily exploit the power of ARKit. RealityKit is an AR first framework, which means that it's been completely designed from the ground up with an emphasis on AR application development.

So the framework heavily emphasizes highly realistic physically-based rendering and accurate object simulation with the real life environment.

We also take full advantage of the power of Swift to deliver a high-quality framework that has a very simple API.

In addition to RealityKit, we're also introducing Reality Composer, which is a Mac and iOS tool that enables simple AR-based content creation. And its intuitive design is really targeted at anybody who wants to make their own content exist in the world around them, even allowing you to lay out your scenes directly in AR.

But before we get into that, let's focus on RealityKit and see it in action. So here we have a real living room with a couch and a table and some small objects that are on the table.

It's purposely a bit blurry because the focus of the camera is very shallow here. So with RealityKit, we can add virtual objects to this world and believe that they're really in our environment.

So notice here how very accurate lighting and shadows and camera effects, really helped to make the object feel like it belongs, even something fantastical like this. RealityKit helps with the heavy lifting and making your content fit in the world automatically. All you have to do is tell the framework where you want to show up and then supply any custom logic that's specific to your app.

And it's really easy to get up and running with your very first app. And in fact for that app you just saw, I only needed to write four lines of code.

So here, I'm placing a horizontal anchor in the world. I'm loading a model that's called flyer from my asset bundle. And I'm attaching that model to the anchor. So over the course of this talk, I'll dive deeper into what each of these concepts mean.

But now that we've seen this framework in action, let's go into some of the systems that make up RealityKit and the basics to help you get started right away.

So to help build your apps, RealityKit takes care of a lot for you to all of its built-in systems.

All of which are integrated with other Apple frameworks such as ARKit and Metal.

So first off, the rendering system has a job of making your content look amazing and realistic in a real environment. And it does this with a physically-based shading system that accurately simulates lighting and material interactions. It's all built on top of Metal, it means that it's highly optimized for Apple devices.

The system takes advantage of all the Metal has to offer, such as multithreaded rendering and other low level functionality. And since RealityKit is designed for AR applications, the feature set of the render is entirely focused around making your content look great and in a real environment.

On top of rendering, animations really helped to breathe life to your content and really enrich your scene. And RealityKit's animation system achieves this through supporting both skeletal and transform animations, both of which can be imported directly from USDZ. And you can even animate your objects procedurally through ARKit's motion capture technology.

The physics system is responsible for simulating complex interactions between content, including real world objects. And it provides a collision detection system that supports