Preparing Data for JAABA

On this page, we describe the format of the data JAABA uses. We provide a program called PrepareJAABAData for converting data output from several trackers to JAABA's format.

Experiment Directory Structure
PrepareJAABAData GUI

Usage
Input Data Type
Input Files
Output Files
Conversion
Options
Save and Load

Tracks File Structure
Per-Frame Directory Structure

Experiment Directory Structure

The inputs to JAABA are the video and the animals' trajectories in the videos. JAABA terms all data associated with a given video an experiment. All data associated with a single video (the video, the trajectories file, and other data used within JAABA) must be stored in a single, separate directory, called an experiment directory. JAABA uses the directory's name to refer to this experiment. Within this directory, all the files must be named consistently. For example, the filename for the video for different experiments should always be the same (e.g., movie.avi). JAABA can open avi, seq, mov, ufmf, fmf, mmf, and sbfmf video formats. The PrepareJAABAData program can be used to convert the outputs from several existing trackers to the experiment directory structure used by JAABA. Users familiar with MATLAB programming can refer to the tracks file structure below.

The screenshots shows 2 JAABA experiments: pBDPGAL4U_TrpA_Rig1Plate15BowlB_20110922T145928 and pBDPGAL4U_TrpA_Rig2Plate14BowlD_20110615T164545. Each contains the following:

movie.ufmf: The input video file.
trx.mat: Input matlab mat file containing the trajectories of the animals. This file is called the tracks file.
perframe: Directory containing per-frame data that is generated and used by JAABA. This directory is called the per-frame directory.

labels file

The files movie.ufmf and trx.mat are inputs to JAABA, while the directory perframe and the file labeledChases.mat are created by JAABA.

Screenshot of two JAABA experiment directories.

PrepareJAABAData GUI

With JAABA, we have included a standalone program called PrepareJAABAData which allows users to create experiment directories compatible with JAABA from the outputs of several existing tracking systems.

Screenshot of PrepareJAABAData GUI

PrepareJAABAData Usage

Under Input Data Type, select the program used to track your animals.
Under Input Files, choose the input files specific to this tracker. Most tracking systems require a video file and file(s) containing the trajectories. Some trackers require additional files. Details of the types of input files required for each tracking system are given below.
Under Output Files, Experiment directory choose the output experiment directory location. This is the directory in which the converted data will be created. There should be one directory per video, see above.
Under Output Files, set the names of the video and trx files, and the per-frame directory. When creating a new JAABA project, the names of these files should correspond.
Under Conversion, set the conversion between frames and seconds and between pixels and millimeters. This is necessary because behavior definitions created using JAABA should be somewhat independent of the video's resolution. Thus, JAABA's per-frame features are all defined in real units, e.g. the speed of an animal is measured in mm/s. The "Conversion" panel also allows the user to set the type of arena the animals are contained within. Options include a circular arena, a rectangular arena, and no arena. The location of the arena wall is used when computing JAABA's landmark-based per-frame features such as the distance to the wall. The unit conversion factors and the arena parameters can be set in any of three ways:
1. Numbers can be entered into the text boxes.
2. After pushing the "Compute arena parameters..." button, the user can draw on a frame of video and label known arena parameters (circular arena diameter, rectangular arena width, or distance between two landmark points if arena type is set to "None").
3. For several of the tracking systems, the conversions and/or landmark parameters can be read from the input files. Pushing "Read arena parameters..." attempts to read the arena parameters and/or spatial conversion factor from the input files. Similarly, pushing "Read FPS..." attempts to read the frame to second conversion factor from either the video or the input files.
Under Options, set any optional parameters.
Push the Convert button to perform the conversion and create an experiment directory that can be input into JAABA. This potentially will take some time. After conversion is done, you will be prompted with a message indicating whether the process was successful and a summary of what was done.

Input Data Type

PrepareJAABAData has currently been extended to work with the following tracking systems:

Ctrax (flies)
Marta Rivera-Alba's larva tracker
Motr (mice)
Qtrax (part of CADABRA, flies)
MAGATAnalyzer (larvae)
Multi-Worm Tracker (C. elegans, larvae)
Reid's and Louis' larva data (larvae)
Ctrax + wing tracker (flies)
Kristin's Simple Two-Fly Tracker (flies)

Ctrax, Marta Rivera-Alba's larva tracker, and Motr are the trackers used in the original JAABA paper. Please email jaaba@googlegroups.com to discuss the possibility of extending PrepareJAABAData to work with the output of your tracking system.

Input Files

The required input files depend on the tracking system used. Almost all tracking systems require a video file and file(s) containing the trajectories. The input files for each of the input data types are listed below.

Ctrax

Video file
Trx mat file: MATLAB mat file containing trajectories that can be exported using Ctrax, see Ctrax documentation.
Ann file: Annotation file output by Ctrax, see Ctrax documentation.

Marta Rivera-Alba's larva tracker
- Video file
- Trx mat file: MATLAB mat file containing trajectories output by Marta Rivera-Alba's larva tracker.
Motr
- Video file
- Seq index file: Currently, Motr tracks .seq video files which require a separate index MATLAB mat file.
- Tracks file: File containing trajectories output by Motr, either a Motr *_tracks.mat file or a Catalytic *.ctc file.
Qtrax (CADABRA):

Video file
Trx mat file: MATLAB mat file containing trajectories output by Qtrax.

MAGATAnalyzer:

Video file
Experiment mat file: MATLAB experiment mat file containing trajectories output by MAGATAnalyzer. To read in the experiment mat file, the MAGATAnalyzer code (not included with JAABA) must be on your MATLAB path.

Multi-Worm Tracker:
- Video file (optional)
- Blobs file(s): Text file(s) output by the MWT, each containing the real-time tracked positions of groups of up to 1,000 animals (note that this is the "blobs" file, not the "blob" file).
- Spine file (optional): Text file output by the MWT's Choreography containing the spines fit to each animal in each frame.
- Dat file(s) (optional): Text files output by the MWT's Choreography containing derived statistics.
Reid's Larva Data:
- Video file (optional)
- Blobs file(s): Text file(s) output by the MWT, each containing the real-time tracked positions of groups of up to 1,000 animals (note that this is the "blobs" file, not the "blob" file).
- kinData mat file output by Reid's analysis program.
- eventData mat file output by Reid's analysis program (optional).
Louis' Larva Data:
- Video file (optional)
- Data txt file output by Louis' larva tracker.
- kinData mat file output by Reid's analysis program.
Ctrax plus wing tracker

Video file
Trx mat file: MATLAB mat file containing trajectories including the wing positions, output by first running Ctrax then the simple wing tracking program.
Ann file: Annotation file output by Ctrax, see Ctrax documentation.
Per-frame directory: Directory containing per-frame data output by the wing tracker.

Kristin's Simple Two-Fly Tracker

Video file
Trx mat file: MATLAB mat file containing trajectories including the wing positions, output by the simple two fly tracker.
Per-frame directory: Directory containing per-frame data output by the wing tracker.

Output Files

Experiment directory: The directory to write the converted data to. After conversion, this experiment directory can be input into JAABA.
Video file: The name of the file to save the video to within the experiment directory.
Trx file: The name of the tracks file within the experiment directory.
Per-frame directory: The name of the per-frame directory within the experiment directory.

Conversion

Some or all of the following controls are found in the Conversion panel, depending on the type of tracking input. Some types of tracking systems are required to contain some or all of the Conversion parameters (e.g. the output of the Qtrax tracker is in real units), in which case these conversion parameters cannot be changed.

Frames per second: Conversion factor between frames and seconds.
Over-ride timestamps: Select this to over-ride any timing information contained within the input files with Frames per second above.
Pixels per mm: Conversion factor between pixels and millimeters.
Over-ride arena: Select this to over-ride any scaling or arena information contained within the input files with the Pixels per mm and arena information entered in the Conversion panel.
Arena type: Shape of the arena confining the animals, if any. The arena parameters are used in computing the landmark-based per-frame features such as the distance to the wall. Choices include Circle, Rectangle, and None.
Center x (px), Center y (px): If Arena type is Circle or Rectangle, this is the centroid of the arena, in pixels.
Arena radius (px): If Arena type is Circle, this is the radius of the arena, in pixels.
Arena width (px), Arena height (px): If Arena type is Rectangle, this is the width and height of the arena, in pixels.
Read arena parameters...: Attempt to read the arena parameters and pixel-to-mm conversion factor from the input files.
Read FPS...: Attempt to read the frames-per-second conversion factor from the video or other input files.

Pushing the Compute arena parameters... button brings up a dialog allowing the user to annotate the arena location and set known lengths in millimeters.

If the Arena type is Circle, then the user is prompted to click points on the circular arena wall. A circle is fit to the points entered. The user is then prompted for the diameter of this circle in millimeters, and the pixels-per-millimeter factor is computed from this.
If the Arena type is Rectangle, then the user is prompted to click and drag to outline the rectangular arena wall. The user is then prompted for the width of this rectangle in millimeters, and the pixels-per-mm factor is computed from this.
If the Arena type is None, then the user is prompted to draw a line of known length, e.g. between two landmark points. Then, the user is prompted for the length of this line in millimeters, and the pixels-per-millimeter factor is computed from this.

Dialog for labeling circular arena wall after pushing "Compute Arena Parameters...".

Options

The following conversion options can be set:

Soft-link files: If this is checked, instead of copying the video and other relevant input files, only links to the files are put in the experiment directory. This saves disk space, but the experiment directory will break if the locations of the original input files are changed at a later date. This is implemented with shortcut files in Windows and symbolic (soft) links in Linux and Mac.
Flip Left-right, Up-down, Transpose: Whether to flip or transpose the trajectories when converting. This is necessary if for some reason the trajectories appear mirrored compared to the video.
First frame, End frame: Use these options to specify that only a portion of the video should be included, e.g. if the original video is extremely long. Enter Inf for End frame to specify that the video and trajectory ends should not be clipped.

Save and Load

The configuration parameters can be saved to a MATLAB mat file by pushing the Save button. These configuration parameters can be loaded in at a later time by pushing the Load button.

Tracks File Structure

The trajectories of the animals computed by automatic tracking systems such as Ctrax are stored in the tracks file, which is a MATLAB .mat file. The tracks file contains the variable trx. trx is a MATLAB structure array where each element corresponds to an animal. The details of the structure fields are listed in the table below.

Structure Field	Description
`x`	x-coordinate of the animal in pixels (`1 x nframes`).
`y`	y-coordinate of the animal in pixels (`1 x nframes`).
`theta`	Orientation of the animal (head) (`1 x nframes`).
`a`	1/4 of the major-axis length in pixels (`1 x nframes`).
`b`	1/4 of the minor-axis length in pixels (`1 x nframes`).
`nframes`	Number of frames in the trajectory of the current animal (scalar).
`firstframe`	First frame of the animal's trajectory (scalar).
`endframe`	Last frame of the animal's trajectory (scalar).
`off`	Offset for computing index into `x`, `y`, etc. Always equal to `1 - firstframe` (scalar).
`id`	Identity number of the trajectory (scalar).
`x_mm`	x-coordinate of the animal in mm (`1 x nframes`).
`y_mm`	y-coordinate of the animal in mm (`1 x nframes`).
`theta_mm`	Orientation of the animal in real coordinates. This is often the same as `theta`, if no transformation other than translation and scaling is performed between pixels and real coordinates (`1 x nframes`).
`a_mm`	1/4 of the major-axis length in mm (`1 x nframes`).
`b_mm`	1/4 of the major-axis length in mm (`1 x nframes`).
`sex`	Sex of the animal. Can be just one value ('M' or 'F' or '?') or a cell array of 'M' and 'F' giving the sex for each frame. The size of the cell array should be nframes.
`dt`	Difference in timestamps of the current frame and next frame, in seconds (`1 x nframes-1`).
`fps`	Average frames-per-second (scalar).
`timestamps`	Timestamp of each frame (optional), in days (`1 x nframes`).
`area`	Larvae only. Area of the larva in pixels (`1 x nframes`).
`xcontour`	Larvae only. x-coordinates of the contour of the larva in pixels (`1 x nframes` cell array).
`ycontour`	Larvae only. y-coordinates of the contour of the larva in pixels (`1 x nframes` cell array).
`xspine`	Larvae only. x-coordinates of the spine fit to the larva in pixels (`nspinepts=11 x nframes`).
`yspine`	Larvae only. y-coordinates of the spine fit to the larva in pixels(`nspinepts=11 x nframes`).
`area_mm`	Larvae only. Area of the larva in mm² (`1 x nframes`).
`xspine_mm`	Larvae only. x-coordinates of the spine fit to the larva in mm (`nspinepts=11 x nframes`).
`yspine_mm`	Larvae only. y-coordinates of the spine fit to the larva in mm(`nspinepts=11 x nframes`).

Below is a screenshot of the MATLAB command window showing the trx variable.

Example trx variable

The optional variable timestamp in the trxfile gives timestamps for each frame of the movie, represented as a serial date number (described in the table above).

Per-Frame Directory Structure

The per-frame features for a given experiment directory are pre-computed and stored in the per-frame directory. Within this directory, there is a MATLAB mat file for each per-frame feature.