Face — tdlidar_face

Make a face at the front camera and drive a 3D character’s expressions, eyebrows, jaw and gaze live.

Category: Camera & Vision · Tier: Free · Needs: front TrueDepth camera (iPhone X and later)

What it does

Uses the front TrueDepth camera and ARKit to read your face in real time and break it into 52 expression sliders (blendshapes) — each eye blink, brow raise, jaw drop, smile, cheek puff and more, every one a smooth 0–1 value. Alongside the blendshapes it reports where your head is pointed (a full transform) and where your eyes are looking (gaze). Wire it straight into a rigged face mesh to puppet a digital character, or pick out single channels to drive any parameter with an eyebrow or a jaw.

OSC in

| address | type | range | rate | |—|—|—|—| | /tdlidar/face/* | float | 52 ARKit blendshapes (0–1), head transform, gaze / look-at | per frame |

The * expands to one channel per blendshape (e.g. …/jawOpen, …/browInnerUp, …/eyeBlinkLeft, …/mouthSmileLeft) plus the head-pose and gaze channels — all numeric, so an OSC In CHOP reads them all.

Outputs

  • out1 (CHOP) — every face channel: the 52 named blendshapes (0–1), the head transform, and the gaze/look-at values, named by their address.
  • out_top (TOP) — optional readout overlay of the strongest blendshapes for quick sanity-checking on stage.

Parameters

| par | default | what it does | |—|—|—| | OSC Port | 9000 | UDP port to listen on (match the app) | | Smooth | 0.0 | built-in lag on all channels (0 = raw, higher = calmer expressions) |

Quick start (beginner)

  1. In the app, enable Face and let the front camera see your face (it switches to TrueDepth automatically).
  2. Drop the Face op. Blink, smile, raise a brow — the matching channels in out1 move.
  3. Find a single channel by name (e.g. tdlidar/face/jawOpen) with a Select CHOP and wire it to any parameter to drive it with your jaw.
  4. To puppet a mesh, export the same-named blendshapes onto a face geometry (below).

Advanced patterns

  • Drive a face mesh’s blendshapes: name your character’s morph targets to match ARKit’s 52 blendshape names, then map out1 channel-by-channel onto the geometry’s blend weights (a Rename / Select CHOP to line names up, into the SOP/COMP that holds the morphs). Now your face puppets the model 1:1.
  • Eyebrow / jaw to params: pull browInnerUp, browOuterUpLeft/Right or jawOpen out with a Select CHOP, run through Math/Range CHOP to taste, and ride a filter cutoff, a particle rate, an exposure.
  • Gaze to aim: take the look-at / gaze channels into a Math CHOP and feed a Geometry COMP Look At or a light’s direction, so wherever you glance, an object or beam follows.
  • Smooth expressions: the Smooth par (or an external Lag CHOP on out1) removes micro-jitter so a driven mesh doesn’t quiver between frames.

Gotchas

  • Front camera only, and exclusive of the rear depth modes. Face takes the TrueDepth sensor, so it can’t run at the same time as the rear LiDAR depth pipeline — switching to Face pauses that.
  • Blendshapes are 0–1, not symmetric. Resting face ≈ 0 on most channels; many come in left/right pairs (eyeBlinkLeft / eyeBlinkRight) — don’t assume a single channel covers both sides.
  • Lighting matters. TrueDepth needs your face reasonably lit and roughly facing the phone; extreme angles or a turned-away head make channels drift or hold.
  • All face data is numeric — read it with an OSC In CHOP, not a DAT.