December 14th, 2016
This is the second in a two-part series about the future of links. Part one recapped the history of links on the web and how things became so broken on mobile. In this post, we will cover the solution, and what we have learned at Branch as we built it.
We need to take a fresh look at how we handle linking on the internet. A new approach to linking infrastructure is needed, adopting the best of what has worked in the past, but designed from the start to accommodate both websites and mobile apps, and flexible enough to adapt as future technologies come on the scene. “Deep linking” will be part of it, but what we need now goes far beyond that.
Here are the eight critical aspects that this system must fulfill:
In other words, a remote link service. An API for link routing. A dedicated link redirection server in the cloud. Hosted deep links. Whatever name you prefer, the reason for it is the same: as the mobile ecosystem becomes more fragmented, link routing is becoming too complex to handle locally on the user’s device. We are going to need something more powerful.
Working on Branch for the last two years, we have realized that a system like this actually does not need to replace existing link standards. Instead, it can mesh parts from all of them together, filling in the holes and adding glue between the pieces to complete the user experience. There are two basic components necessary to make this happen:
On the web, we never needed anything like this. Every web browser could handle its own routing based only on the data contained in the link URL, and there was no risk that it might be unable to load the experience. But that happens all the time on mobile — if the app isn’t installed, for example — and the result is frequently an error message. Not ideal. The primary purpose of this remote service is to take everything we know about incoming visitors (device type, app installation status, data from link opened, etc.), use that to determine the best possible user experience (website, App/Play Store, directly to app), and then send them there.
Aside from the simplified routing process, there are several other advantages to implementing this routing logic remotely instead of on each device:
The remote service is also responsible for creating links. This involves accepting a set of data keys — the content locations on each platform, behavior control parameters, tracking and display configuration, plus other custom data — and returning a link address to reference them. That link is then valid for use anywhere, and if you pass up a system-wide unique ID for each piece of content, it’s even possible to deduplicate for data integrity when multiple links all point to the same thing.
Once the remote service has determined which experience is best for a visitor, we need to communicate what should happen next. The app or website needs link data to know what content the visitor requested, and there are tracking and analytics housekeeping tasks to be done. All of this is handled by the local modules, which are packaged as platform-specific SDKs.
Each SDK is optimized to best fit its programing language and environment, and is broadly responsible for the following tasks:
Links on the web have conditioned us to expect an immediate response, so our new links should be no different: all of this must take place instantly, which sounds easy, but can sometimes require careful choreography so as to remain invisible to the user. And of course the SDKs also facilitate easy link creation by wrapping convenient programming interfaces around calls to the remote service.
This is the platform we began building at Branch in 2014. What is the result, and what have we learned so far?
We’re far past “deep linking” now. This is something much bigger.
SDKs for app linking make perfect sense. There has never been a broadly-accepted, universal approach to app deep linking, so this is brand new territory and Branch now has an SDK for every major platform and development toolkit. On the other hand, web links already “just work,” so while the Branch web SDK technically can be used as a standalone method for routing into websites, it’s almost always better to use a hybrid approach together with standard redirect rules. Any data returned from the remote link service is then used for added value rather than primary navigation.
As for the remote link service, Branch does use a remote backend for much of the heavy lifting, but in practice, we often have to lean on local web browsers for “last mile” functionality. Even if support for the remote service is eventually built directly into device operating systems, routing via web browser will always be necessary for full backwards compatibility.
The end result is a linking infrastructure that does indeed fulfill all eight of the requirements above, and works better than anything else available, in almost every situation. The Branch platform has now been widely adopted by over 11,000 active apps, including most of the top names in the industry.
But the most surprising thing we have learned is just how many ways things can go wrong. Apple and Google both try to pretend that mobile deep linking is a simple thing. A check-the-box, “nice to have” feature that can be implemented by following a few guides buried deep in the developer documentation for each platform. Unfortunately, this couldn’t possibly be further from the truth, and more importantly, it misses the true scale of the problem: if links are to be used everywhere, they need to work everywhere. The true complexity of how difficult this is to accomplish only becomes clear with much painful experimentation.
First of all, there are numerous competing mobile deep link standards, all with different functionality and requirements for implementation:
Note: iOS and Android together control 99.3% of the mobile market. Because of this, Branch has very limited support for other platforms, and they are omitted here for simplicity.
Of course, none of these standards is supported on every platform or OS version:
Facebook deserves special recognition for inventing an open-source deep linking standard that actually showed promise, and then completely abandoning it:
Even within each platform, every web browser does things differently. This matters, because the majority of links clicks still happen in browsers, and every link routing system relies on browsers for at least some core functionality:
Things get even messier outside the web browser category. So many other apps are capable of opening links that attempting to generate a comprehensive list would be an impossible task, but here are a few notable highlights:
First of all, building a linking system that can accommodate all these edge cases is a lot of effort. At Branch, our team spends every waking moment exclusively working on this, and we have learned that the devil is in the details.
For example, what if you add app deep linking to all your emails, only to realize your unsubscribe link needs to go to the web on every platform? Now your emails are broken even worse than they were before. And even once the infrastructure is built, ecosystem changes happen so frequently that nothing can be relied upon to “just work” in the future. Keeping up to date with ever-changing standards is a full time job you don’t want, because you already have more exciting things to do.
But more importantly, there is a critical final piece we haven’t discussed yet: search. App content doesn’t show up in web searches, and the best workaround Google has developed involves (yet again) the hack of tying apps and websites more closely together. As the app ecosystem continues to develop, this cannot be our best answer to the problem of content discoverability. Just like the vast majority of us use Google for web searches without a second thought (how many other search engines have become their own verb?), you will want your app’s links to be part of the largest mobile content index in the world. This index is Branch, and it is growing every day.
For better or worse, the age of the native mobile app is here. But no advancement comes without costs, and the move to apps means that links — a core concept of the internet — have taken a big step backward. It’s up to us as developers to find a solution that brings back the openness of the web, otherwise apps will be doomed to live inside walled gardens forever.
This is a pretty fundamental adjustment to the way we’ve grown up handling links on the web, so of course there are some valid objections to address.
This is probably the biggest risk of using a centralized system for link routing. But in reality, we already rely on remote services like DNS and CDNs for much of the modern internet. The benefits of these systems outweigh the risks, and link success rates over 99.9% (meaning less than one failure per 1000 attempts) are well within reach.
When this happens, whether due to network congestion or simply because an extra network call is unwanted, all the link data can be encoded into a long-form URL and then passed up to the remote service the first time that link is loaded. The result is a much longer link address, but the same behavior.
One of the great things about web links is that the addresses seen by users often follow a path/to/content structure. In fact, this is so common that it is taken as a best practice. Structuring things this way was originally reflective of the underlying filesystem of the server where the site was hosted, but has already started to break down with the rise of URL shortener services like Bit.ly and Goo.gl. The Branch platform (along with knockoff attempts like Firebase Dynamic Links) have used a similar short URL approach, because there is no guarantee that content inside a mobile app will be structured the same way as the corresponding website. This frees links from being tied to the technical implementation of any single platform, but nothing prevents building a structure into the addresses of links, and the Branch system already allows customization of these addresses if desired.
Even though Apple and Google intended these standards to more closely tie apps and websites — the opposite goal from a comprehensive linking infrastructure focused on flexibility — it’s easy to incorporate them into the system. For our purposes, they function as pre-authorized shortcuts to speed up the user experience.