Applying a JAABA Classifier to a New Video

After you have trained a behavior classifier, there are several ways to apply it to automatically classify the behaviors of all animals in novel videos. This can be done within JAABA, with JAABAPlot, or from the MATLAB command-line using JAABADetect.

The classifier's automatic predictions for a given experiment are output to a scores .mat file within the experiment directory.

Applying a Behavior Classifier
Scores File Structure
Loading Predictions

Applying a Behavior Classifier

Using JAABA

Within JAABA, you can apply the classifier associated with the current JAABA project using the Classifier->Predict menu:

->Current Fly: The entire trajectory of a single animal. The predictions are not saved to a scores file, and are only viewable within the current JAABA session.
->Current Experiment: All animals in the current experiment. Results will be store to a scores file.
->All Experiments: All experiments in the current project. Results will be store to scores files.

Using JAABAPlot

You can use JAABAPlot to apply behavior classifiers in batch to a set of experiments. As described in Plotting JAABA's Output, this can be done by

Adding the experiment(s) of interest to JAABAPlot, see Loading Experiments for more information.
Adding the classifier(s) of interest to JAABAPlot, see for more information.

Pushing the Classify button. The automatic classifications will be stored to the scores files within the experiment directory.

Using `JAABADetect`

From the MATLAB command-line, you can run the function JAABADetect to classify input experiment(s) with input classifier(s). To do this:

Change into the directory JAABA/perframe within MATLAB.
Run SetUpJAABAPath to add necessary directories to the MATLAB path.
Run JAABADetect.

This will result in the scores files being saved to the experiment directories.

JAABADetect usage:

JAABADetect(expdirs,'jabfiles',jabfiles,...) JAABADetect(expdirs,'jabfilelist',jabfilelist,...) Inputs:

expdirs: Path(s) to experiment directories on which to run the classifier(s). This should either be a string (one experiment) or a cell array of strings (multiple experiments).
jabfiles: Path(s) to JAABA project(s) corresponding to the classifier(s) to apply. This should be either a string (one classifier) or a cell array of strings (multiple classifiers).
jabfilelist: Path to a file containing a list of the JAABA project(s) corresponding to the classifier(s) to apply. This file should have one JAABA project per line.

Optional inputs are given using the form 'name',value:

'forcecompute': Whether to run the classifier even if the scores files already exist. Default value: false.
'debug': Whether to just test running the classifier and not actually save any files. Default value: false.

Example usage: JAABADetect('/home/bransonk/JAABA/demo/SampleChaseExperiment','jabfiles','/home/bransonk/JAABA/demo/SampleChaseJAABAProject.jab')

Scores File Structure

The output of the JAABA classifier associated with a given JAABA project is stored to a scores .mat file within the experiment directory. This is a MATLAB mat file containing the following variables:

allScores: Main output struct with the following fields:
- scores: Raw, continuous output of the classifier. This is a cell array with one entry for each animal. allScores.scores{i}(t) is the output of the classifier for animal i at frame t. This should be NaN for frames for which there is no data (if the animal's trajectory does not last the entire length of the movie). This will be a positive number if the animal is predicted to be performing the behavior in this frame and a negative number if it is not. The magnitude of this score reflects the confidence of the classifier.
- tStart: The first frame on which each animal is tracked. This is an array with an element for each animal trajectory in the video.
- tEnd: The last frame on which each animal is tracked. This is an array with an element for each animal trajectory in the video.
- postprocessed: The thresholded classifications for each animal and each frame. This is a cell array with an element for each animal. allScores.postprocessed{i}(t) is the behavior classification for animal i at frame t. It is NaN for frames for which there is no data (if the animal's trajectory does not last the entire length of the movie). It is 1 if the animal is predicted to be performing the behavior in this frame and 0 if it is not.
- postprocessparams: Parameters for postprocessing the continuous scores into binary classififications.
- t0s: Frames on which bouts of the behavior start. This is a cell array with an element for each animal trajectory. allScores.t0s{i}(j) is the frame on which bout j starts for animal trajectory i.
- t1s: Frames on which bouts of the behavior end. This is a cell array with an element for each animal trajectory. allScores.t1s{i}(j) is the frame on which bout j ends for animal trajectory i. Because we use the convention that the animal is performing the behavior from frame t to t+1 if it is labeled/classified as performing the behavior at frame t, allScores.postprocessed{i}(allScores.t1s{i}(j)) will be 0 and allScores.postprocessed{i}(allScores.t1s{i}(j)) will be 1.
- scoreNorm: Normalization factor for moving scores into the range of -1 to 1, after thresholding.
timestamp: Timestamp of the classifier used to create the scores file.
version: Version of JAABA used when creating the scores tils.
jabFileNameAbs: Path to the JAABA project corresponding to the behavior classifier used to create the scores file.

Loading Predictions

Previously saved predictions can be loaded from file into the current JAABA session. These loaded scores will be shown in the bottom portion of the Automatic Timeline if Loaded is selected from the pop-up menu on the left. Predictions can be loaded by selecting any of the options under the menu File -> Import Scores:

File -> Import Scores -> For current experiment from default location: Loads the scores for the current experiment from the default scores file within its corresponding experiment directory. The name of the scores file within the experiment directory is the same for all experiments, and is specified during project setup as Score/Output File Name.
File -> Import Scores -> For current experiment from different location: Prompts the user for a file location, and loads the scores for the current experiment from that file.
File -> Import Scores -> For all experiments from default location: Loads the scores for all experiments from the default scores file within each corresponding experiment directory. The name of the scores file within the experiment directory is the same for all experiments, and is specified during project setup as Score/Output File Name.