How Face Shape Detection Works: 478 Landmarks, Four Ratios, Six Shapes

import { FaceLandmarker, FilesetResolver } from "@mediapipe/tasks-vision";

const files = await FilesetResolver.forVisionTasks(
  "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.32/wasm"
);
const landmarker = await FaceLandmarker.createFromOptions(files, {
  baseOptions: {
    modelAssetPath:
      "https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task",
    delegate: "GPU",
  },
  runningMode: "IMAGE",
  numFaces: 1,
});

const mesh = landmarker.detect(imageElement).faceLandmarks[0];

mesh holds 478 points. Their coordinates are normalized to the image width and height separately, so convert them to pixels before measuring anything. On a portrait photo, distances taken from the raw values are wrong.

const px = (i) => ({ x: mesh[i].x * width, y: mesh[i].y * height });
const dist = (a, b) => Math.hypot(a.x - b.x, a.y - b.y);

// Angle at vertex v between points a and b, in degrees.
function angleAt(v, a, b) {
  const [ax, ay, bx, by] = [a.x - v.x, a.y - v.y, b.x - v.x, b.y - v.y];
  const cos = (ax * bx + ay * by) / (Math.hypot(ax, ay) * Math.hypot(bx, by));
  return Math.acos(Math.max(-1, Math.min(1, cos))) * (180 / Math.PI);
}

const [top, chin] = [px(10), px(152)];
const [foreheadL, foreheadR] = [px(54), px(284)];
const [cheekL, cheekR] = [px(234), px(454)];
const [jawL, jawR] = [px(172), px(397)];
const cheekWidth = dist(cheekL, cheekR);

const ratios = {
  heightToWidth: dist(top, chin) / cheekWidth,
  foreheadToFace: dist(foreheadL, foreheadR) / cheekWidth,
  cheekToJaw: cheekWidth / dist(jawL, jawR),
  jawAngle: (angleAt(jawL, chin, cheekL) + angleAt(jawR, chin, cheekR)) / 2,
};

There is no forehead-to-jaw ratio. It equals foreheadToFace * cheekToJaw, so adding it would count the same geometry twice.

Each shape is a set of ideal values for those four numbers. The detector measures how far a face is from each set, in units of tolerance, and picks the nearest. These are the values the live detector uses.

const WEIGHT = { heightToWidth: 5 / 17, foreheadToFace: 4 / 17, cheekToJaw: 5 / 17, jawAngle: 3 / 17 };
const TOLERANCE = { heightToWidth: 0.1, foreheadToFace: 0.07, cheekToJaw: 0.1, jawAngle: 10 };
const PROFILES = {
  oval:    { heightToWidth: 1.24, foreheadToFace: 0.88, cheekToJaw: 1.26, jawAngle: 137 },
  round:   { heightToWidth: 1.13, foreheadToFace: 0.88, cheekToJaw: 1.2, jawAngle: 141 },
  square:  { heightToWidth: 1.16, foreheadToFace: 0.9, cheekToJaw: 1.2, jawAngle: 130 },
  heart:   { heightToWidth: 1.24, foreheadToFace: 0.92, cheekToJaw: 1.37, jawAngle: 134 },
  oblong:  { heightToWidth: 1.34, foreheadToFace: 0.88, cheekToJaw: 1.24, jawAngle: 137 },
  diamond: { heightToWidth: 1.24, foreheadToFace: 0.84, cheekToJaw: 1.31, jawAngle: 134 },
};

function classify(ratios) {
  const ranked = Object.entries(PROFILES)
    .map(([shape, ideal]) => {
      let sum = 0;
      for (const key of Object.keys(WEIGHT)) {
        sum += WEIGHT[key] * ((ratios[key] - ideal[key]) / TOLERANCE[key]) ** 2;
      }
      return { shape, distance: Math.sqrt(sum) };
    })
    .sort((a, b) => a.distance - b.distance);

  const [best, next] = ranked;
  return {
    shape: best.shape,
    runnerUp: next.shape,
    matchStrength: Math.round(100 / (1 + best.distance)),
    borderline: next.distance - best.distance < 0.25,
  };
}

classify({ heightToWidth: 1.22, foreheadToFace: 0.88, cheekToJaw: 100 / 78, jawAngle: 135 });
// { shape: 'oval', runnerUp: 'diamond', matchStrength: 85, borderline: true }

That call is the portrait from the top of this guide: cheekbones 100, forehead 88, jaw 78, height 1.22 times the width. It comes out Oval, with Diamond close enough to be named.

The match strength is not a probability. It is 100 for an exact match with a profile and falls as the distance grows. Read any percentage on a tool like this the same way unless the tool says how it was measured.

Textbook ratios did not fit the mesh. Textbooks measure from the hairline, and the top of the mesh sits below it, so height reads short. The textbook values sat outside the 5th to 95th percentile of real scans on most measures: square had an ideal jaw angle of 100 degrees when the 5th percentile of real scans was 128. Most faces came out oval by default, so the profiles were re-centred on 2,217 real scans, keeping each shape's position relative to the others.

That is calibration to a distribution, not an accuracy benchmark. Without a labelled set of faces nobody can say how often the label is right. Oval is still the largest group: 49.2% of 15,429 scored scans on 19 August 2026, in the face analysis study.

Clipped scores lose direction. An early version scored each ratio inside a band and stopped changing outside it, so a very long face could land nearer round than oblong. Unclipped distance keeps growing, so the direction survives.

Tolerances have to match across shapes. If one profile has a wider tolerance on a measure than the others, it wins at extreme values by default. Each measure has one tolerance shared by all six shapes.

A single label hides close calls. When the runner-up is within 0.25 distance units, the result says so and names both shapes.

On Android, in about 1% of scans, the GPU's texture upload failed without an error. The model ran on a blank frame and returned one off-scale mesh, usually with coordinates in the hundreds and the same pose on every photo. Nothing threw, so the numbers were measured and reported. The fix is a range check on every point and one retry on the CPU:

const plausible = (points) =>
  points.every((p) => [p.x, p.y].every((v) => Number.isFinite(v) && v >= -2 && v <= 3));

Landmarks can fall a little outside 0 to 1 when a face is cut off at the edge of the frame, so the range is wider than that. Across 48,000 scans, healthy meshes peaked at 1.07 (99.9th percentile) and the bad mesh sat at 5.6 or beyond. If the CPU pass also fails, the detector shows no result. The landmarker also crops the face and runs at 192 px internally, so photos are capped at 2,048 px on the long edge, which moved points by at most 2.9 px RMS in tests.

For the face shape detector, the photo and the 478-point mesh stay on your device. The result does not: the shape, the ratios and the scores are saved on our server, without the mesh, so you can reopen your result. Free results are deleted after 2 days, and copies of the scores and labels go to usage analytics. You can see the request in your browser's network tab.

Other tools on this site, such as the face age estimate, use a vision model and do send the photo, after a consent step. "On-device" is a claim about one tool, so check it tool by tool.