iTemplate2 McMaster Baby Lab logo: a hatching eaglet
Open-source · Python · Desktop GUI

Eye movements on moving faces, in one common space.

iTemplate2 maps eye-tracking gaze onto a canonical face template, frame by frame. Define your areas of interest once, then apply them across faces, expressions, stimuli and participants.

In pressXiao, Román-Caballero, Fang, Ripley, Han & Trainor

Demo · Visualization exportLeft: gaze samples over the original stimulus video. Right: the same samples after transformation onto the template face. Dots are gaze samples; colours distinguish participants.
01 · The problem

A fixed AOI stops fitting the moment a face moves

In a video of a moving face, the pixel coordinates of facial features change in every frame. A gaze point at coordinate (350, 200) might fall on the left eye in Frame 1, the nose in Frame 10, and the background in Frame 20.From the paper
The same pixel lands on different facial features Three video frames in which one gaze pixel falls on the left eye, the nose and the background, mapped through facial landmarks onto one template face with a single set of AOIs. VIDEO FRAMES · GAZE AT PIXEL (350, 200) TEMPLATE FACE FRAME 1 1 → left eye FRAME 10 2 → nose FRAME 20 3 → background STEP 3 Landmark map piecewise affine TEMPLATE 1 2 3 dashed AOI, defined once One pixel, three different answers. One template, one set of AOIs.

Template-based registration answers this by projecting every gaze sample onto one canonical face, so a single set of AOIs is enough. The original iTemplate (Xiao & Lee, 2018) did this for static images, with semi-manual landmarks and a MATLAB dependency. iTemplate2 extends the idea to video, wearable recordings and several faces at once.

02 · What's new

Four advances over the original iTemplate

01

Dynamic stimuli

Frame-wise coordinate transformation, with normalisation for wearable and scene-based eye-tracking data and temporal preprocessing for trial segmentation.

02

Automatic landmarks, many faces

478 facial landmarks per frame from MediaPipe's Face Landmarker, with multi-face tracking and identity persistence that reduce manual annotation.

03

Open and cross-platform

A graphical interface with no commercial dependencies, Apache Parquet storage for video-scale data and saved settings for reproducible analyses.

04

Richer visualisation

Interactive heatmaps, snap-to-landmark AOI definition, and facial blendshape parameters for concurrent expression analysis.

The iTemplate2 processing pipeline Stimuli and gaze data flow through face extraction, gaze preparation, template transformation, visualisation, AOI analysis and export. Stimuli Gaze data STEP 0–1 Extract faces 478 landmarks STEP 2–2.5 Prepare gaze trials · timing STEP 3 Transform gaze → template STEP 4 Visualise playback · heatmaps STEP 5 Analyse AOIs template or stimulus STEP 6 Export CSV LEGEND DATA YOU PROVIDE PIPELINE STEP CORE TRANSFORMATION

The workflow. The process begins with data import and automated, high-density facial landmark extraction (Steps 0–1). Data review and temporal and spatial preprocessing segment trials and normalise gaze coordinates for screen-based and wearable data (Steps 2–2.5b). The core transformation maps gaze onto a standard face template (Step 3). Researchers then visualise the registered data (Step 4), run AOI analysis (Step 5) and export for statistics (Step 6). Redrawn from paper figure 1.

03 · Inside the app

One desktop app for the whole pipeline

Each step is its own page in a sidebar, so you can work from top to bottom without hunting through menus. Click any screenshot to enlarge it.

Project Manager screen with a file browser and a colour-coded data table
Paper figure 2 · Project ManagerCentralises file organisation and initial import. An integrated file browser structures the workflow into standard folders, and a colour-coded table previews the continuous eye-tracking data. Column remapping and stimulus-name standardisation accommodate diverse eye-tracker outputs.
Face Extraction screen showing a 478-point mesh on a video frame and a Face ID Management table
Paper figure 3 · Face ExtractionThe left panel shows the automated 478-point landmark mesh, tessellated by Delaunay triangulation. The right panel holds Face ID Management, used to resolve identity fragmentation or ID swaps caused by occlusion in dynamic video.
Visualization screen with raw gaze on the video, transformed gaze on the template, and heatmaps
Paper figure 4 · VisualizationSynchronised playback compares raw gaze on the original dynamic stimulus (left) with transformed gaze registered to the template face (right). The lower panel builds aggregate heatmaps by two-dimensional Gaussian kernel density estimation, grouped by categorical variables and normalised per participant.
AOI Analysis screen with eye and mouth polygons drawn on the template face
AOI AnalysisDraw polygon AOIs on the template face, snapped to the 478-point mesh so boundaries follow anatomy. AOIs can be exported as JSON to share identical definitions across studies and sites, or drawn on the original stimulus to study gaze beyond the face.
04 · How the mapping works

Triangles in, template coordinates out

Each face is divided into small triangles whose corners are facial landmarks (a Delaunay triangulation). For every frame, the pipeline finds the triangle that contains a gaze sample and describes the sample as weights on that triangle's three corners, its barycentric coordinates. Applying the same weights to the matching template triangle gives the mapped location.

Barycentric mapping from a stimulus triangle to a template triangle A gaze point inside a triangle of facial landmarks is described by three weights; applying the same weights to the matching template triangle gives its template location. STIMULUS FRAME f TEMPLATE FACE SAME WEIGHTS λ1 + λ2 + λ3 = 1 0 ≤ λi ≤ 1 V1 V2 V3 G T1 T2 T3 G′ The same weights, applied to the matching template triangle, give the mapped location.

Because the mapping is piecewise affine, it follows local facial deformation, such as the stretching of a mouth corner during a smile, to the extent that the landmarks and the gaze are accurately estimated. The paper uses "deformation invariance" only in this limited sense.

Which face did the gaze land on?

Many paradigms show several faces at once. Selecting faces for transformation decides which detected meshes are candidates. Every gaze sample is then labelled on_face, off_face or unmapped in the exported gaze_region column. The Visualization page can detect sections where facial meshes overlap and lets you review each ambiguous sample, assign it to a matching face or exclude it, without inferring which face is physically in front.

How a gaze sample is classified A gaze sample is first tested against the facial meshes of all selected faces, then against peripheral triangles, and is labelled on_face, off_face or unmapped. YES NO YES NO Gaze sample in frame f Inside a selected face mesh? Peripheral mapping usable? on_face template coordinates + face_id off_face display coordinates · no face_id unmapped invalid or no usable mapping ASSIGNMENT RULES 1 Facial triangles of all selected faces come first. 2 If several faces match, the first in ID order wins. This is not a depth order. 3 Peripheral triangles come later and are for display. 4 Template AOIs count on_face samples only.

Off-face gaze can carry task-relevant information. Examine it with stimulus-space AOIs in the original video coordinates, for example the region between two interlocutors. These AOIs are currently static, so they may not follow people who move.

Four video frames of an infant in a car seat showing correct, shifted, oversized and duplicate face detections
Paper figure 6Face tracking outcomes in discontinuity scenarios. (A) Expected detection. (B) Misaligned detection after occlusion or rapid movement. (C) Oversized detection capturing non-facial areas. (D) A spurious second face (orange) while the true face is still detected (green).
05 · Quality control

Automatic detection, with a human in the loop

A face can be missed or detected in the wrong place during occlusion, rapid head movement or extreme rotation, and tracking can swap or fragment IDs when a face reappears. Left uncorrected, gaze may be assigned to the wrong face or to incorrect locations.

iTemplate2 lets you inspect the affected frames, keep gaps as missing rather than interpolating across them, correct face IDs, remove unreliable detections, or re-run detection on a selected frame range. Corrected IDs can carry labels such as "Caregiver" and "Infant".

A face contour margin handles another source of error: because eye trackers have limited accuracy, gaze at the edge of a face can be recorded just outside the outline. The margin enlarges each outline by a chosen percentage during transformation, and is saved with the output so you can report it.

06 · Example applications

Four developmental eye-tracking projects

The paper uses these examples to show scope and feasibility. They are not benchmarks of accuracy against a ground truth.

1

Toddlers and emotional expressions

Fang and Xiao (2026) showed 12- to 24-month-old toddlers videos of actresses expressing emotions toward novel objects. Facial features moved continuously and varied across actresses, so fixed AOIs would likely have misclassified gaze. The 478-point mesh follows each expression, and snap-to-landmark AOIs give one consistent set of regions across actresses and emotions. The hand gesture in the figure below is classified as off-face.

Gaze samples on a video frame of an actress (left) and the same samples after transformation onto the template face (right); off-face samples are magenta.
Paper figure 5Gaze samples before (left, screen space) and after (right, template space) coordinate transformation. On-face samples are mapped using the estimated landmarks of the selected face. Off-face samples, shown as magenta dots, appear in peripheral template space for display only and are excluded from facial AOIs.
2

Live caregiver–infant interaction

Ripley et al. (2025) used dual eye tracking to capture the gaze of caregivers and 4- to 6-month-old infants during face-to-face communication. Each partner's face continuously shifts in scale, orientation and position, and caregivers lean in, turn away or are briefly occluded. iTemplate2's face ID controls and identity timeline let researchers merge tracks, swap IDs or mark gaps before transformation (see the tracking figure above).

3

Infant gaze to infant-directed speech and singing

Román-Caballero et al. (2025) followed infants at 4, 6 and 12 months. Manual frame-by-frame AOIs were estimated at about 240 hours for this study. Mapping all 40 videos onto one template put gaze in a common space, even though infants contributed valid data for different videos at different ages. AOI classifications differed from four trained manual coders in about 7% of frames, and the transformed data showed the reported shift from eye to mouth preference by 12 months. The 7% figure describes agreement for this example only, not general accuracy. Other automated AOI methods could give the same comparison; the template additionally allowed common heatmaps.

Bar charts comparing manual and iTemplate2 AOI classification at 4, 6 and 12 months, and template heatmaps at each age
Paper figure 7(A) Distribution of infants' visual attention across AOIs from manual coding and from iTemplate2 at each age; classifications differed in about 7% of frames. (B) Heatmaps of infants' gaze at each age, with warmer colours showing higher gaze density.
4

Face looking during free play at home

Fully mobile infants (11 to 20 months) and their caregivers wore eye trackers during unconstrained play. Scene cameras captured cluttered rooms in which faces constantly change scale, orientation and position. Automatic detection tracks the faces, with optional manual checks, and gaze on adult and infant faces is summarised on the same template. This shared registration does not show that landmarks are detected equally well on adult and infant faces, and feature-level reading depends on whether wearable gaze accuracy is sufficient for the size of the faces in the scene video.

Egocentric views from a caregiver and an infant during free play, with gaze mapped onto the template face and density heatmaps
Paper figure 8Caregivers' and infants' face looking during natural free play at home. Green dots show original gaze on the egocentric views, red dots show transformed gaze on the template face, and heatmaps show gaze density on the template faces.
07 · Positioning

A complement to existing AOI methods

Automatic and dynamic AOI tools already handle moving faces well. iTemplate2 is a different way of working, and is not meant to replace them.

KEYFRAME

Keyframe AOIs

Tobii Pro Lab and EyeLink Data Viewer interpolate 2D translation and scaling between keyframes you define. Between keyframes they do not follow non-rigid deformation or 3D head rotation.

LANDMARK

Landmark-based AOIs

Hessels et al. (2018) and FaceTrack-AOI (Zhou et al., 2026) build AOIs from landmarks detected in each frame, in the coordinates of the original video.

TEMPLATE

Template space

iTemplate2 registers gaze to a common template, so AOIs are defined once, including regions landmarks do not delineate directly, such as both eyes combined, the forehead or the cheeks. It also supports heatmaps and spatial analysis, and pairs with pixel-wise statistics such as iMap4.

Template registration does not remove the decisions in AOI analysis. It moves some of them to the choice of template, the definition of AOIs on it, and the inspection of facial detections. Validity still depends on your research question, your gaze data quality and the accuracy of the facial mapping.

08 · Limits

Interpret results with care

The transformation changes how gaze is represented. It does not change the gaze itself.

Gaze quality still limits you

Transformation does not improve accuracy or precision. In wearable and infant data, gaze error can match the distance between facial features, so be cautious near AOI boundaries and check whether conclusions change with the boundaries.

Landmarks are estimates

Detection degrades with occlusion, head rotation beyond about ±70°, low resolution and poor light. Infant faces are under-represented in the model's training data, so allow extra time for quality control.

The template shapes the output

The built-in template averages White female faces from the Chicago Face Database. It places faces of different shapes in one space but does not establish equal accuracy across ages or demographic groups. Custom templates are supported.

Off-face is not a face location

Peripheral template coordinates are for display only. Use stimulus-space AOIs for gaze between or beyond faces; these are currently static, and fully automatic identity tracking in crowded scenes is unresolved.

More in Interpreting Transformed Gaze.

09 · Get started

Install and run

Requires Python 3.10 to 3.13 on macOS, Windows or Linux. The face model and template are included in the repository.

# 1. Get the code git clone https://github.com/McMaster-Baby-Lab/iTemplate2-public.git cd iTemplate2-public # 2. Create an environment python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate python -m pip install -r requirements.txt # 3. Launch python start_gui.py

Prefer conda? Run conda env create -f environment.yml and conda activate iTemplate2 instead of step 2.

Linux: install xcb-util-cursor first (a PyQt6 requirement).

Also on OSF. The software is archived on the Open Science Framework, the project cited in the paper.

Free software. iTemplate2 is released under the GNU General Public License v3 or later. Use and modify it freely, including for research and commercial work. Your data and results are not affected. If you distribute a modified version, share it under the same license with its source.

New to the app? Read Before You Begin to check that you know your data's columns, timing and stimulus names, then follow Steps 0 to 6 in the sidebar.

Read the user guide

10 · Cite and credits

Cite iTemplate2

Xiao, N. G., Román-Caballero, R., Fang, W., Ripley, S., Han, D., & Trainor, L. J. (in press). iTemplate2: An open-source Python toolbox for automatic eye-movement analysis of dynamic facial stimuli.Paper · in press

On GitHub, the Cite this repository button in the sidebar gives ready-made APA and BibTeX entries from CITATION.cff.

Xiao, N. G., & Lee, K. (2018). iTemplate: A template-based eye movement data analysis approach. Behavior Research Methods, 50(6), 2388–2398. doi.org/10.3758/s13428-018-1015-xOriginal method

The team

Naiqi G. Xiao 1Corresponding

Conceived the software framework, led software development and wrote the manuscript.

Rafael Román-Caballero 1, 2

Contributed to software development, testing and manuscript preparation.

Wei Fang 1

Contributed to software testing and validation, and manuscript preparation.

Sara Ripley 1

Contributed to application testing and manuscript preparation.

Danyang Han 3

Contributed to software validation and manuscript preparation.

Laurel J. Trainor 1

Contributed to conceptual development and revised the manuscript.

  • 1Department of Psychology, Neuroscience & Behaviour, McMaster University, Canada
  • 2Department of Experimental Psychology, University of Granada, Spain
  • 3Department of Psychology and Behavioral Sciences, Zhejiang University, China
Corresponding author

Naiqi G. Xiao, Ph.D.
Department of Psychology, Neuroscience & Behaviour, McMaster University, 1280 Main St W, Hamilton, ON, Canada L8S 4L8
nx@mcmaster.ca

Funding

This work was supported by the following. Initials show the grant holder.

  • Social Sciences and Humanities Research Council of Canada (SSHRC): grants 435-2022-0838 (N.G.X.) and 435-2020-0442 (L.J.T.)
  • Natural Sciences and Engineering Research Council of Canada (NSERC): grants RGPIN-2020-07129 (N.G.X.) and RGPIN-2019-05416 (L.J.T.)
  • Canadian Institute for Advanced Research (CIFAR): awarded to L.J.T.
  • National Natural Science Foundation of China: grant 32400886 (D.H.)
  • European Union's Horizon 2020 research and innovation programme, Marie Skłodowska-Curie Actions Global Postdoctoral Fellowship: project 101149355 (R.R.-C.)

Acknowledgements

We thank the research students and volunteers who helped test the software and create figures: Lorena Morcone, Emma Whittier, Grace Kakarelis, Muhan (Joanne) Liu, Maya Krieger, Selena Li, Emma Resendes, Meg Lyu, Maya Mammon, Olive Rinaldi, Leqi Feng and Fei Yang.

Developed at

The McMaster Baby Lab

iTemplate2 is developed by researchers at the McMaster Baby Lab, McMaster University. Learn more about the lab and its work on the lab website.

Visit babylab.mcmaster.ca