Apple Clips update uses iPhone X's TrueDepth sensors for 360-degree selfie scenes

Apple is releasing an update to its Clips movie-making app that adds iCloud support and a new “Selfie Scenes” mode that uses the iPhone X’s TrueDepth camera sensors to augment the user onto a 360-degree digital stage. It’s the first big update to the app since it added Disney and Pixar stickers back in July as additional ways to decorate your photos and video clips.

Clips has offered Instagram and Prisma-like filters since its launch, but Selfie Scenes allow users to move around and interact with an augmented landscape. Using the iPhone X’s TrueDepth camera, Clips uses proximity sensors to bring in the person or object in front of the camera and swaps the background with the digital “scenes.” It essentially functions like a green screen in cinematography, but the effect happens in real time and stylizes the object or person to match the background.

When I tried this out in a briefing, Clips was able to pick up the foreground subject quickly. I was seated on a couch, and sometimes Clips will augment the couch cushion into the shot, but a second later that often goes away. A slight movement will also erase background items like the couch out of the scene. Since it uses the front-facing camera, you’ll need to be pretty close to the phone to get into the shot. (This is especially true if you attempt to have multiple people in your frame.) Once you have your shot set up, you can move around and explore the scene in 360 degrees. A Star Wars-themed scene, for example, lets you look around the spacecraft and catch BB-8 rolling in through a corridor, if you happen upon it at the right time.

Apple says the idea behind Selfie Scenes is to give creators more options to make videos, even if their filming locations aren’t the most ideal — such as a messy bedroom or kitchen. The app is aimed at casual movie editors on the go, not ones with studios or a full editing setup. Though Apple declined to share user numbers, it said that adoption has been “very successful.” It cites education as one of its biggest growth markets, with teachers encouraging kids to film things like science experiments and language courses in classrooms around the world. Selfies Scenes is fun to use, and is clearly yet another kitschy way for Apple to push more exclusive features optimized for the iPhone X, as with Animoji. Still, I wouldn’t be surprised if Snapchat and Instagram follow Apple soon with a similar feature on their respective apps.

another way for Apple to push more exclusive features optimized for the iPhone X

The Clips update today also brings iCloud support so you can shoot on an iPhone and continue editing on an iPad. There are new interface and content updates as well, such as labeled editing buttons, more Pixar and Disney filters and stickers, original, royalty free songs to use as a soundtrack, and a way to select a Live Title language directly from the app without needing to change your phone’s system language. (It’s part of another move to cater to the education market, Apple says.)

Clips 2.0 is available as a free update to the iPhone 5S and later, and the iPad Air, Pro, and Mini 2 and later running iOS 11.1.