Networking with NSURLSession

LUKE CASE: Good morning! Thanks for coming out and welcome to "Networking with NSURL Session." I'm Luke Case and I'm an engineer on the CF Network team. We have got some great stuff lined up for you guys this morning.

First, we're going to talk a little bit about app transport security and following that we will talk about some new protocol support we've added in NSURL Session.

Now, we all heard about watchOS on Monday. We'll talk about the NSURL Session features we've added for watchOS. Following that, we will wrap things up with some API changes and some new additions we've added to the NSURL Session.

Now, to begin, I would like to start with a little bit of background on NSURL Session and the HTTP protocol.

NSURL Session is a networking API that is mainly used to download content, HTTP content, from the web.

It has a rich set of delegate methods that can be used for authentication and other important event handling.

And one of the greatest features about NSURL Session is that it allows you to do networking even while your app is not running via the background download APIs. Now, for those of you that aren't familiar with NSURL Session, it's recommended that you go back a few years to the previous WWDC sessions on the introduction of NSURL Session. So now let's talk a little bit about the hypertext transfer protocol. It's a very well-known protocol and many of you x probably are familiar with it.

Essentially at its core, what you are doing is you are making requests to the server and you are pulling in data in response.

Now, HTTP in itself, as some of you may know, is sent out over cleartext. So it's inherently insecure.

And in today's environment, there's many hostile parties going after the data that's leaving your app and going out onto the network.

Now, there was a time when using cleartext HTTP alone was perfectly reasonable. But that time has passed.

Now, fortunately for all of us, this problem has been solved for many years and there's a tried and true solution known as HTTPS. Now, HTTPS is essentially HTTP layered on top of another protocol known as transport layer security.

And transport layer security performs a multi-leg handshake using public key cryptography and, when complete, creates a secure connection. Now this connection is considered secure because of three properties.

The first being that the data that's leaving your app goes over the network encrypted. So it can't be read.

The second that it provides message integrity so the message can't be altered without detection.

And finally, the third, it provides authentication so you can actually prove the identity of exactly who you are talking to. Now, NSURL Session has excellent HTTPS support embedded within in it, and so often in your client code, it's as easy as using HTTPS instead of using HTTP.

Now, do note that some additional server support is required, but HTTPS is supported by most server vendors today. Also keep in mind that most data should be considered sensitive, and this is because even if you might not think that the data that's leaving your app going out onto the network is not sensitive, your customer may, in fact, think it is. For example, if you have a TV streaming app, you may think, well, it's just TV. You know? He's just watching TV. He or she is just watching TV, but to them, it may mean that they don't want people to know what they are watching, essentially. Let's put it that way.

So -- now, why is it important to use HTTPS? Well, it's essentially, it comes down to this: your customers trust you with their data and their privacy.

And we at Apple, we really want to work together with you guys to make sure that we build upon and maintain that trust. And today, I'm proud to introduce App Transport Security.

Now, App Transport Security is a new feature from Apple in iOS 9 and OS X, El Capitan. And essentially, at its core it helps prevent accidental disclosure of sensitive customer data.

Now, ATS also strengthens the default policy of NSURL Session.

And perhaps the most important aspect of that strengthening is that now NSURL Session, by default, will disallow cleartext HTTP loads. It'll only use HTTPS connections.

Now, these connections that it does make uses today's best practices.

So ATS places restrictions on TLS versions, cipher suites used, certificate trusts, and certificate key sizes that are used in that transaction.

Now ATS is simply configured via your apps Info.plist.

And you can see an example here.

Essentially what we are asking you to do is to declare your intended network behavior within your app.

So at its core, ATS wants you to really describe what you intend to do with the network, and we really want you to worry less about the security of your app and rely more heavily on the system to do the right thing.

So describing your network transaction is, of course, easiest when it's all secure.

So if your app exclusively uses secure connections, and they only use best practice properties, then there's nothing else you have to do to configure ATS within your app.

So if you are writing a new app, this is exactly where we want you to start, and if you have an existing app or you are supporting legacy code, this is where you should aim to be. Now, we understand that this may not be always the case that you have a server that can comply with these new restrictions on TLS versions or cipher suites, so we allowed that to -- through the use of exceptions. And essentially, with ATS you can declare exceptions and let us know what versions of TLS you want to use, or if you want to opt out of forward secrecy, or other options.

Now, we do understand that existing apps may have different constraints.

Let's say for the most part you do use secure connections, but in some cases, you have servers that just don't support HTTPS or the best practices.

So, for example, if you have a media server that doesn't support HTTPS and only HTTP, you can easily describe this behavior using an exception.

Essentially all you have to do is declare which domain needs to load over HTTP and NSURL Session will still allow that cleartext load.

Now we also know that some of you may be supporting a general purpose web browser or another app that loads URLs based off dynamic user input, and we've allowed for that case. In this case here, you can't possibly describe what secure connections you need ahead of time, and so you can simply opt out by declaring an allow-all policy.

Now the allow-all policy can be used to allow NSURL Session to load any HTTP or HTTPS resource.

Now, do note that this is a useful tool for debugging that I believe is already out on the Internet. So we ask, if you do use it for debugging, to do it temporarily for obvious reasons.

Now, even if you are in this state, you can still protect specific servers and domains.

So you are in this state where you are allowing everything, but you have your own servers, say, that you pull the configuration data from or metadata from, and to do this, again, is just an exception within ATS.

You simply declare this domain that you want to protect as an exception to the allow-all policy.

And this will allow NSURL Session to continue to protect this resource by only loading over HTTPS and with best practices.

Now, the SDK has been out since Monday, and there's probably a few of you that woke up at this awful hour, just to find out what ATS is all about and why your network loads are failing.

Well, we understand that and we want to work together with you to get you up and running.

So, do note that ATS is only active if you build against a current SDK. If you are targeting the previous release, ATS rules do not apply. NSURL Session will transfer all HTTP resources or loads URLs to HTTPS automatically.

And, again, as I said earlier, some of you already know about the allow-all key, and we ask you to use this temporarily to find out if the new network failures in your app are because of ATS or not. And this will kind of help you narrow things down. So the next step, if you allow-all ATS loads, or allow-all loads, and you've determined that the problem is ATS, the next step would be to log the NSURL Session errors that you are seeing, so you can try to determine and narrow down which load is actually failing, and what the underlying error is.

There is also a great debugging tool that's an environment variable known as CF Network Diagnostics.

If you set CF Network Diagnostics to level 1, all the failing URL loads -- so, all the loads that failed will log the URLs and the underlying TLS error.

And so what you can do in the next step is take that underlying TLS error, and look up and secure transport.h, and see exactly what the underlying issue is.

Usually it's a failure in negotiation between client and server at the TLS layer.

So now, do understand that this is a new API, and it's still under development.

So we strongly encourage you guys to pay attention to the seed notes and release notes, and also, please file radars so we can track any issues that you guys are hitting and help you guys get up and running.

So I hope it's clear that the time for secure networking is now.

And we really want to work with you in order to protect your customers' data.

So, again, if you are writing a new app, start with HTTPS and try to get your servers up and running with the best practices.

Now, for those of you with existing apps, we ask that you first start by moving what you can to HTTPS, and for the places where you can't, go ahead and use the exceptions that are available through your Info.plist.

Now, do keep in mind that your customers trust you with their data, and it is truly sensitive in all cases.

And so Apple wants to work together with you to provide a more secure environment for your customers.

So, again, please give us feedback via radar, come see us down at the lab after this session and tomorrow.

We really want to work with you to protect our customers.

Thank you all for being here and I will be followed up by Andreas who will talk about new protocol support and NSURL Session. Have a great conference.

ANDREAS GARKUSCHA: Thanks, Luke.

Good morning, everyone! My name is Andreas and I'm going to talk about new protocol support in NSURL Session.

Yes, NSURL Session supports HTTP/2 protocol.

Your apps are ready to communicate using HTTP/2 and it's very easy to adopt. Let me play it one more time.

All right! If you are already using NSURL Session in your code, you are automatically a part of all of that.

Future of the web, major milestone in the web's evolution, your apps are running faster. You don't need to change your source code. Everything works automatically.

So today, we are going to talk about three things.

Why do we need a new protocol and what are the common problems of the current HTTP/1.1 protocol? We will learn about the most important HTTP/2 features.

And finally, we will talk about HTTP/2 protocol adoption in your apps.

So, why another new protocol? We already have a lot of protocols for every kind of communication.

The reason is that most of the protocols were designed many years ago for the needs of their time.

Look at one of the first Apple websites.

Today's needs look significantly different than those 15 years ago.

So it is time for an update.

HTTP has been around since the beginning of the web, and it is not a secret that it has many issues.

The most famous HTTP issue is the problem of only one outstanding request per TCP connection.

The solution for this was HTTP pipelining, but HTTP pipelining is not available on all servers or networks.

In fact, it's disabled on most popular desktop web browsers.

Another solution was to open multiple connections to a host.

This could help to fetch multiple resources faster, but together with other things like textual protocol overhead, the lack of header compression, it just adds up to higher system requirements and lower performance on both the client and the server. Last year, we introduced SPDY Support in NSURL Session.

SPDY was an attempt to make the web faster.

It was an experimental protocol, however, it was chosen as a basis for a new version of HTTP.

The specification for the new protocol version went through the IETF standardization, and officially got an RFC number assigned last month.

So today, as you already know, NSURL Session is extended to support HTTP/2 protocol.

Let's take a look at the key differences between HTTP/1.1 and HTTP/2.

As opposed to HTTP/1.1, HTTP/2 opens only one TCP connection to a host.

It's network friendly and requires less system resources on both the client and the server.

HTTP/2 is fully multiplexed.

That means that a new request does not need to wait until the server sends the response for the previous request.

HTTP/2 has request priorities, so that more important resources can be delivered at a higher priority to the client.

Let's take a look at how HTTP/2 multiplexing resolves the Head-of-Line Blocking problem.

We have three requests for resources on a web server.

We sent out the first request and we get a response.

Only after this, we can send out the second request and get the second response.

Same happens with the third request. Now, with pipelining enabled, we can send out all the requests, one after another, without waiting for the previous responses to arrive.

But we still get the responses in the same order.

And you see that the first response for the image in blue, blocks the following two responses.

With HTTP/2, we have the same three requests, with different priorities.

We still can send out all the requests at the beginning, but we are receiving all the responses at the same time. Moreover, the requests with the higher priority, I get and deliver faster to the client.

You see that the second response was a medium priority and the third response for request was a high priority, arrived prior to the first response even though they were scheduled later.

The image does not block them anymore and this is great. Great for your applications and the performance.

Let's continue with the comparison.

HTTP/2 is a binary protocol.

That makes data processing and parsing faster.

HTTP/1.1 does not use header compression.

SPDY also could not use header compression because of a security exploit.

HTTP/2 uses HPACK, a more secure mechanism for header compression.

Let's talk about HPACK.

HPACK header compression is based on two tables, a static table and a dynamic one.

The static table contains the most used HTTP headers and is unchangeable.

The headers, which are not included in the static table, can be added to the dynamic table.

The headers from the tables can be referenced by index.

As an example, you see a simple HTTP/1.1 request.

Highlighted is the data which is going to be sent to the server. And here's HTTP/2 representation of the same request. So let's encode this request.

The pseudo headers, method, scheme, and path, can be referenced using the static table.

The authority header is included in the static table, but without its value.

So to encode this request, we need three bytes for the first three headers, plus an additional byte, which tells that we want to add the authority header to the dynamic table and the value of the authority with its length.

And this is what is going to be sent to the server plus additional overhead for the header frame.

Now with the second request, and you see that the authority header goes in the dynamic table.

So with the second request, HTTP/1.1 would send the same headers over and over again.

In HTTP/2 case, in that particular case, we can reference all the headers using the static and the dynamic table. We are using only one byte for each header.

It is a huge savings of the bandwidth and it's remarkable how few bytes are needed to encode a request or response header in HTTP/2.

So let's talk about what you need to adopt HTTP/2 protocol in your apps.

There's not much work to do. HTTP/2 protocol is seamlessly integrated into NSURL Session API.

If you are already using NSURL Session in your code, your apps and OS X programs will get this functionality automatically.

You will not need to write any new code or provide any additional configuration to turn it on.

Let's take a look at the source code example.

This source code looks exactly like the code you guys already have in your apps.

You see, there's no difference, no new configuration flags. It just works.

Yes, you only need an HTTP/2 server.

But it's okay.

Your apps are ready to communicate using HTTP/2 protocol. If you do not deploy an HTTP/2 server yet, then your apps will use HTTP/1.1 directly, or the best available protocol will be selected automatically for the network communication.

Once you start using a web server which supports HTTP/2, there is no additional work needed.

Your apps will use HTTP/2 protocol automatically.

Please keep in mind that NSURL Session supports HTTP/2 protocol only over an encrypted connection.

And that your HTTP/2 server requires to support ALPN or NPN for protocol negotiation.

Currently at Apple, HomeKit remote access via iCloud is using HTTP/2 protocol for communication between HomeKit accessories and iCloud.

Many big companies are already using HTTP/2 protocol.

Google provides its services in HTTP/2.

Twitter is using HTTP/2 as well.

There are many HTTP/2 open source web servers out there, and finally, some CDN service providers are working on the HTTP/2 protocol support today.

We worked hard to provide HTTP/2 protocol support in NSURL Session so that you guys can adopt and use it as easy as possible. HTTP/2 is available today in WWDC seed.

It's seamlessly integrated into NSURL Session API and it's enabled in Safari, on OS X, 10.11, and iOS 9.

Thank you.

And now I would like to invite Dan up to the stage. Dan? DAN VINEGRAD: Good morning, everyone. My name is Dan. I'm a software engineer on the CF Network team at Apple, and the first thing I would like to talk about today is using NSURL Session on watchOS.

With the WatchKit SDK that was released alongside watchOS 2 in beta earlier this week, I'm pleased to say that HTTPS loads are fully supported on watchOS.

And this means that everything we've already talked about today, like App Transport Security and HTTP/2, are built into this support.

One major difference between using NSURL Session on watchOS and using it on other platforms is that underneath the hood, we actually will use the best connectivity mechanism that's available to us at the time. And this means that in most circumstances, if the users' watch is nearby their paired iPhone device, then we will actually leverage the Bluetooth connection between them, perform the HTTP loads on the phone itself, and deliver the results back to the watch over Bluetooth.

If the user happens to be out and away from their phone, but with their watch, and the watch is connected to a known Wi-Fi network, then we can actually use that Wi-Fi network directly.

But the good news for you is that this is all abstracted away from the API itself. You can use the API just as you have been on other platforms. You don't need to worry about how we are connecting. It just works like magic. So, with that said, there are a few best practices and things to keep in mind when using NSURL Session on watchOS.

The first is that you should really try to just download the minimal size assets that are actually required for your app to function.

Keep in mind that the watch has a very small screen. So if you are downloading images you don't really need to download the full resolution images that you would want to display on an iPhone 6 Plus or a Mac with a retina display. The screen's a lot smaller; you can download smaller images.

And also keep in mind that the watch has a lot less processing power than a phone or a Mac, and additionally will often be limited by the bandwidth and latency constraints of the Bluetooth connection to your phone. So we're not really going to be able to get you bytes as quickly to the watch as we would on other platforms. So keep that in mind as well. Another thing to note is that apps on watch will generally run for a much shorter period of time than iPhone apps or definitely Mac apps. You will mostly be limited by how long the user wants to sit there standing with his wrist raised staring at the watch and interacting with it.

So if you are using a default session configuration or an ephemeral session configuration, keep in mind that these networking transfers will happen only while your app is actually running. So this is totally fine if you are sending small amounts of data like fetching stock quotes or weather data or social network status updates, but for any larger content like videos, for instance, you 'll want to use background uploads or downloads, which can continue out of process while your app is no longer running. And for more information on background uploads and downloads, I highly encourage you to check out the WWDC sessions on Foundation Networking from previous years. Next up, I would like to talk about some API changes that we have made in this year's releases.

The first thing I would like to talk about is NSURL Connection, something we have not talked about today so far.

So, this year, I'm announcing that NSURL Connection is deprecated on OS X, El Capitan, and iOS 9.

And let's just take a moment to think about what that means. Deprecation does not mean that NSURL Connection is going away entirely. We know that there are a lot of apps out there that are using NSURL Connection, and we're not just going to break them. So it will still work. Those transfers will still work, but keep in mind that new features are really only going to be added to NSURL Session at this point. We highly encourage you to switch over existing code from NSURL Connection to NSURL Session if you haven't already, and if you are writing new code, we would really hope that you would only use NSURL Session and not Connection.

Another thing to keep in mind is that NSURL Connection is not supported at all on watchOS. So if you need to load HTTP content from a WatchKit extension, you have to use NSURL Session.