iPhone Face Mocap

  Accurate, Accessible and Portable

How to Use the iPhone for Face Mocap?

The iPhone facial motion capture works with iClone Motion LIVE by using the exclusive TrueDepth camera system powered by sophisticated high-res sensors. The iPhone tracks faces with a depth map and analyzes subtle muscle movements for live character animation. The Reallusion LIVE FACE App enables the iPhone to live stream captured facial mocap data directly to a PC or Mac, transforming the iPhone into a powerful 3D biometric mocap camera.

*LIVE FACE Profile only works with devices with TrueDepth Camera (Face ID).
Please refer to the iPhone and iPad models that support Face ID.
iphone facial mocap app

Features

  • The TrueDepth Camera projects and analyzes more than 30,000 invisible dots to create a precise depth map of the face.
  • Analyzes more than 50 different muscle movements to mirror expressions.
  • Highly sensitive to the details, like rotating eyeballs and tongues.
  • Works with different skin colors and extreme light conditions.
  • Runtime Calibration: on-the-fly calibration without pausing the face tracking process.
  • Supports WiFi, USB, and Ethernet connection.
    • Ethernet connection can eliminate all potential lag for the optimal facial mocap performance.
    • Ethernet connecter sold in Apple Store, Amazon.
      *Requires LIVE FACE Profile v1.03 & LIVE FACE App v1.0.7 above
    • How to connect: USB & Wireless  |  Ethernet (highly recommended)

Visit apple.com to know more

iphone facial mocap app - features and workflow

iPhone can live stream analyzed facial data instead of full image frames to free up CPU loading.

High Precision and Environment Tolerance

Combine the LIVE FACE app on the iPhone with Motion LIVE to create incredible 3D facial animation for performance capture, live TV, and virtual hosts for YouTube and web streaming.

Note: To enable the

iPhone facial motion capture

process, please install the LIVE FACE Profile in iClone Motion LIVE and LIVE FACE App on the iPhone.   Workflow >    Quick Guide >    Required Tools >

LIVE FACE Profile

iPhone Gear Profile For Motion LIVE Plug-in

LIVE FACE App

iPhone Face Motion Capture App

*Compatible with any iPhone/iPad enabled by TrueDepth camera for facial recognition: iPhone X, iPhone XS, iPhone XS Max, iPhone XR, iPhone 11, iPhone 11 Pro, iPhone 11 Pro Max, iPad Pro 11-in. (1st and 2nd generation) and iPad Pro 12.9-in. (3rd and 4th generation). More >
FREE

Up Your Game with Better Facial Mocap

Designed by animators, for animators - Reallusion's latest facial mocap tools go further, addressing core issues in the facial mocap pipeline to produce better animation. Starting from all new ARKit expressions, to easily adjusting raw mocap data as well as retargeting, to multipass facial recording and essential mocap cleanup - we're covering all the bases to provide you with the most powerful, flexible and user friendly facial mocap approach yet.

iphone facial mocap workflow - the most powerful and flexible mocap

The biggest issue with all live mocap is that it's noisy, twitchy and difficult to work with, so smoothing the live data in realtime is a big necessity; because if the mocap data isn't right, or the the model's expressions aren't what you want, and the current mocap recording isn't quite accurate then you need to have access to the backend tools to adjust all of this. Finally, while you can smooth the data in realtime from the start, you can also cleanup existing recorded clips using proven smoothing methods which can be applied at the push of a button.

1  Highest Tracking Precision (Free App)

  • Unmatched tracking accuracy and stability.
  • Alleviate unwanted lips or neck jitters.

*update required: LIVE FACE App version 1.0.8 (Feb 22, 2021)

iphone facial mocap tutorial and workflow - highest tracking precission free app

ARKit 1-to-1 Blendshapes

  • 52 ARKit Blendshapes derived from human scans.
  • Additional 11 tongue animation morphs.
  • Fully compatible with existing CC3+ facial expressions.

    • ExPlus for ARKit 1to1 >

iphone facial mocap tutorial and workflow -ARKit 1 to 1 blendshapes

2  Tracking Data Multiplier

  • No need to adjust expressions to compensate for bad data - simply adjust the mocap data itself with the multiplier.
  • Be able to Increase or decrease tracking data strength for the ideal performance.
  • Make your own multiplier profiles for your iPhone setup, along with individual subject's faces and individual characters.
iphone facial mocap tutorial and workflow - tracking data multiplier

3  Multipass Recording

  • Supports 3 recording modes: New, Blend, Replace.
  • No more redo, just adding up, or replacing the part you don't like; it's a super manageable method for professional facial production
  • Record just the brows, the eyes, the cheeks, mouth or jaw to blend/replace parts on an existing recording.
iphone facial mocap tutorial and workflow - multipass recording

4  Live Smoothing

  • Take out the noise and twitches to get more natural results from the very start of a mocap - save time and effort in production.
  • Instant cleanup to minimize the need for manual editing.
  • Smooth live data in realtime for easier setup, and get controlled performances over head, eyeball, eyelid, and facial muscle.
iphone facial mocap tutorial and workflow - live smoothing

5  Expression Mapping

  • Retargeting (Tracking Data to Blendshape) made easy with the new, flexible mapping UI.
  • Default and ARKit 1-to-1 presets provide instant retargeting for CC3+ avatars.
  • Live adjustment over the triggered blendshapes to get your preferred expression style.
  • Customize expressions easily by blending.
iphone facial mocap tutorial and workflow - expression mapping

6  Clip Smoothing

  • Smooth recorded data on clips using either Smooth (moving average) or Downsample (key reduction) methods.
  • Fine-tune the amount of smoothing on live, or recorded, facial animations, as well as blinks, eyeball and head rotations.

More about intutive iClone facial animation editing.

iphone facial mocap tutorial and workflow - clip smoothing
Basic Features
iphone facial mocap - Deal with Low Light and Skin Tone

Deal with Low Light and Skin Tone

  • In a real capture environment, such as in live event or using the iPhone for head mount, it’s hard to control the light condition.
  • Highly accurate with various skin tones.
  • Work under low light, backlight, or dynamic lighting condition.
iphone facial mocap - Flexible Capture from Different Angles

Flexible Capture from Different Angles

  • Ideally, accurate mocap is done with the iPhone right in front of the face, but that's not always possible. For this we provide the Zero Pose function for the best possible capturing results.
  • No matter where you place your iPhone, the unique Zero Pose design will quickly recalibrate the angle offset of the face, and accurately standardize the facial tracking in one-click!   More >
iphone facial mocap - Custom Capture Profiles

Custom Capture Profiles

  • Define your own expression style with facial muscle sliders and save it as a Custom Capture Profile.    More >
  • A default capture profile with 58 facial muscle sliders optimized for iPhone Face IDs.
  • Take advantage of the Search function to quickly locate morph sliders.
iphone facial mocap - Synchronous Audio Recording during Facial Capture

Synchronous Audio Recording during Facial Capture

  • Optionally record audio with the face capture session. *   More >
  • Recorded audio generates lip synch data.
  • Blend lip-synch and

    facial motion capture

    to create the best result.

* An extra microphone is needed

WORKFLOW QUICK VIEW

iphone facial mocap tutorial and workflow
Try
Buy