Engineering Zoom: The finer details.

In the previous post, A Closer Look at Zoom, we explored the design inspiration behind the loupe tool. We hope that after experiencing the design for the first time, you shared our delight in its combination of simplicity and utility. As a team, upon seeing what the loupe could be, we immediately knew that it […]

In the previous post, A Closer Look at Zoom, we explored the design inspiration behind the loupe tool. We hope that after experiencing the design for the first time, you shared our delight in its combination of simplicity and utility. As a team, upon seeing what the loupe could be, we immediately knew that it had to be built.

Alas, the road from a compelling design to a compelling implementation is often long, windy, and paved with discarded prototypes. While our team prides itself in not shying away from such engineering adventures, we must always weigh the short- and long-term technical complexities of designs against the value that they bring to the experience. In this post, we’ll share some of the engineering challenges that the loupe presented and how we approached them.

Custom two-touch gestures recognizers

In early prototypes of the loupe, it became immediately clear that the pinch gesture that is used to activate the tool would conflict with another key gesture in Paper: Rewind. Since both gestures are based on two touches and the early stages of their execution are often indistinguishable, we realized that we needed a more sophisticated method of disambiguating them.

The problem was that we could not continue building on top of UIPinchGestureRecognizer and UIPanGestureRecgonizer as we had been doing. They simply did not provide the fine-grained control we needed around activation thresholds, mutually exclusivity, and gesture-specific constraints.

Our only option was to reverse-engineer the built-in UIGestureRecognizers in order to build custom recognizers for two-touch pan and pinch from scratch that are designed to work in tandem. Not only did this approach solve the problem of disambiguating pinch-to-zoom from Rewind, it also made both gestures more robust in isolation and set us up for a more customizable and nuanced gesture system down the road.

Simultaneous gesture recognition

A key motivation for the loupe-based approach to zooming is being able to dive in to add detail and return to the original context quickly. As a result, design was adamant that you should be able to work in the loupe while manipulating it — you most certainly should not have to position the loupe, then edit, then dismiss the loupe with three discrete actions.

This implies an implementation where you can can pinch to open and resize the loupe with one hand while simultaneously using the other to draw in the loupe, activate the Rewind gesture, or work with the tool tray. In gesture recognition parlance, this means allowing gestures to be recognized simultaneously.

Simultaneous gesture recognition poses many challenges, primarily of the edge case variety. For example, take the simple case of starting a pinch to summon the loupe, then lifting one of the touches (which leaves the pinch gesture active and allows panning the loupe). Now, placing a touch inside the loupe should draw, placing one on the loupe’s edge should continue to resize it, and touching outside the loupe should do nothing. However, if you place two touches at any of these locations and activate the Rewind gesture, none of those other actions should take place and Rewind should proceed.

Ultimately, the solution to these many edge cases involved the UIGestureRecognizerDelegate delegating to the various controllers involved, so they could make localized decisions about which touches the gestures should receive, whether the gestures could activate given the current state, and what to do when the gestures did activate.

Offscreen gestures

We love taking full advantage of the iPad’s multi-touch system to really give the sense that you’re directly manipulating the objects in Paper. Much of the interaction model for the loupe was inspired by simply treating it like a physical object. Unfortunately, in experimenting with early versions, we quickly learned that abruptly ending a manipulation when the touch moves onto the bevel can often shatter the illusion of physicality.

We’re not expecting Apple to extend the touch sensor onto the bevel any time soon, so in the meantime, if your touch exits and returns to screen while manipulating the loupe, our implementation guesses at the path that it took and allow it to continue manipulating the loupe instead of, say, drawing a stroke.

We find that many times little engineering challenges like this pop up whenever you try to push the boundaries of multi-touch. Without tackling them, the solution simply loses its magic. Go try it yourself by opening the loupe, releasing, your pinch, then drag it with a single touch that quickly leaves and re-enters the screen.

Page to circle clipping

As opposed to other zoom implementations that simply magnify the canvas but continue to clip it to the bounds of the screen or viewport, the loupe clips the magnified contents to a circle.

Since much of Paper is rendered in OpenGL, the only efficient means of rendering the loupe is with OpenGL. Unfortunately, OpenGL has no built-in functionality for anti-aliased circle rendering. Luckily, we can solve that problem fairly simply by constructing circle geometry with a little 1-pixel border on the edge to fade the edges and simulate the effect of anti-aliasing.

However, a larger challenge presents itself: one of my favorite features of the loupe is that it now allows you to work on the edge of the page without drawing at the edge of the screen. (Simply move the center of the loupe past the edge.) Unfortunately, clipping a page with rounded corners to a circle (technically this shape is a superellipse, for the curious) is no easy task.

We ended up settling on a solution where we segment the background from the page using a pixel shader. The best aspect of this approach is that it gives a nice crisp boundary to the page no matter what zoom level you’re viewing.

Unfortunately, on some devices the shader proved too costly. Luckily, observing that we could break the pixel shader out into three increasingly costly shaders that are responsible for decreasing areas of the screen did the trick: a shader for when no edge is shown, a shader for when only a horizontal or vertical edge is shown, and a shader for when a corner is shown.

In future posts, we’d love to share more details on these topics, so let us know if you’re particularly curious about any of them. You can now follow this blog and ask any questions at @FiftyThreeTeam.

Source: Fiftythree