What Fylgja does.
Fylgja measures behaviour from video without asking your lab to label a single frame. Upload a recording, mark the apparatus, and the neural network pipeline locates the animal’s landmarks, follows it through the session and scores the behaviour against definitions you set. Every result carries the configuration, data and code that produced it.
The numbers on this page come from our own research record. Accuracy, time saved and agreement with expert scoring are each reported from a named study with its own reference; the first of those, with Sahlgrenska, is described below.
Our research.
Each study states its question, its measure and its threshold for success before any measurement is taken. Results are recorded with the code and data that produced them, and a held-back test set is read once.
Since July 2026 we have recorded over a thousand experiments. Results are recorded on the same terms whichever way they fall, and both kinds appear below.
Progress.
Four measures of where the work stands. The figures behind each are beneath it.
Finding the animal
96%
of frames now show the animal, up from 24%, after the detector learnt from 392 single taps on the video. No landmarks drawn by hand.
Figures
| Measure | Before | After |
|---|---|---|
| Animal found on held-back tapped frames | 24% | 96% |
| Landmarks placed, stranger phase | 48% | 83% |
Steady tracking
0.2%
of frames drift onto the apparatus, down from 28%, after one tracking pass that trains nothing. Flicker between frames fell from 3.7% to 0.1%, and the effect held on five clips across three rigs.
Figures
| Measure | Before | After |
|---|---|---|
| Frames drifting onto the apparatus | 28.1% | 0.22% |
| Flicker between frames | 3.73% | 0.09% |
Without labels
+7.3 points
of landmark accuracy from adapting the pipeline to new recordings with no labels at all, measured on 2,724 frames from scenes never seen in training. The animal was found on 56% of frames rather than 30%.
Figures
| Measure | Before | After |
|---|---|---|
| Landmark accuracy, change | +7.3 points (5.6 to 9.1) | |
| Frames with the animal found | 30.2% | 55.6% |
Our research
1,000+
experiments recorded since July 2026, each kept with the code and data that produced it. Results are recorded whichever way they fall.
Figures
| Month | Entries | |
|---|---|---|
| July 2026 | 165 | |
| August 2026 | 295 | |
| September 2026 | 529 | |
| October 2026 | 35 | |
| Dated entries | 1,024 |
What the measurements show.
New recordings, no labels
Adapting without labels raises accuracy on recordings the pipeline has never seen.
On a public dataset of 5,460 frames annotated by people, split by scene so that no recording arrangement appears on both sides, adapting the pipeline to the new recordings with no labels raised landmark accuracy by 7.3 points (95% range 5.6 to 9.1) and found the animal on 56% of frames rather than 30%.
Finding the animal
Accuracy follows where the detector looks.
In the stranger phase of the three-chamber test, the detector often framed the cup rather than the mouse. Pointed at the animal instead, the same pipeline placed landmarks on 83% of frames rather than 48%. Retrained from 392 single taps on the video, with no landmarks drawn, it now finds the animal on 96% of held-back frames, up from 24%, and landmark accuracy rose by 10 points with the rest of the pipeline untouched.
Steady tracking
One tracking pass, no training.
A tracking pass that trains nothing cut drift onto the apparatus from 28% of frames to 0.2% and flicker between frames from 3.7% to 0.1% on one clip, and held on five clips across three rigs.
Where the work is going.
Two programmes run side by side: adapting the pipeline to a new laboratory with no hand labelling, and a convergence study begun on 13 October. Early results are marked as such.
- Keep the animal in view first. The largest gains so far come from measures that keep the animal in view: the tap-trained detector, a tracker that carries it through the stranger phase, and landmarks filled in from their tracked neighbours. The work concentrates there.
- Masks decide which predictions to trust. A segmentation mask filters the pipeline’s own predictions before they are used to adapt it, with no labels. Confidence alone would keep about half of the misplaced points on paws and tail; the mask removes them. On the home rig this adds some 25 points of landmark accuracy.
- Residual error is structured. Early result. Roughly 80% of what remains is a consistent offset along the spine rather than scatter, so it can be corrected per rig rather than averaged away.
Validation with Sahlgrenska.
A protocol is in preparation with colleagues at Sahlgrenska to establish whether Fylgja’s behavioural measures agree with expert scoring as closely as experts agree with one another. The design:
- Shared definitions. Entry, exit and contact are written down before scoring begins, so that observer and machine measure the same thing.
- Blinded observers. Trained observers score the same recordings without sight of the machine output; a subset is scored twice, by different observers.
- A human benchmark. Agreement is judged against agreement between observers, not against an arbitrary threshold.
- Fixed in advance. Endpoints, analyses and the criteria for success are set before the comparison is made.
The hand-scored record so far covers six animals; on it, observer and machine differed on where the middle chamber ends, not on where the animal was. Agreement is reported when the protocol does.
Assays that become possible.
Scoring by eye confines measurement to what an observer can see and sustain. Inference from video removes several of those limits. Each measure below is validated within its own assay before it is reported as a result.
- Continuous measurement
- Every frame of every session, rather than sampled intervals or the opening minutes.
- Kinematics and posture
- Gait, speed profiles, curvature of the body axis and head direction: quantities no observer can score reliably by eye.
- Long observation
- Hours or days in the home cage, including the dark phase under infrared light, where manual scoring is impractical.
- Retrospective analysis
- Archived recordings re-measured under a revised definition, without scoring them again by hand.
- Behavioural structure
- Movement decomposed into recurring motifs without predefined categories, so that new phenotypes may be found rather than presupposed.
- Comparison between laboratories
- Definitions held as versioned code, so that a measure means the same thing wherever it is applied.
Fine-grained acts such as grooming and close investigation need more than a single overhead camera resolves; the chamber measures are validated first.
Background: Wiltschko et al. (2015), Mapping sub-second structure in mouse behavior, Neuron 88, 1121–1135; Weinreb et al. (2024), Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics, Nature Methods 21, 1329–1339.
How to read our numbers.
Three kinds of checks stand behind a result. Run records show what ran and with which settings. Agreement between trackers flags frames for review. Human reference annotations and blinded assessment establish accuracy, and only they support a claim about behaviour.
- Recording conditions. Occlusion, reflections, apparatus edges and camera changes alter what the pipeline sees, so each recording arrangement is validated on its own footage.
- Definitions. A geometric or kinematic rule is a definition you set, versioned with the result, so a measure means the same thing wherever it is applied.
- Development figures. Training curves and agreement with the pipeline’s own earlier output, such as a 2.59-pixel error on a 409-frame development set, are kept as development figures, apart from human benchmarks.
The question a result answers is not whether a trajectory exists, but whether it supports the inference your study intends to make.