BEHAVE Interactions Test Case Scenarios

Introduction

The dataset comprises of two views of various scenario's of people acting out various interactions.

The data is captured at 25 frames per second. The resolution is 640x480. The videos are available either as AVI's or as a numbered set of JPEG single image files.

Researchers can freely use the dataset. If results based on the dataset appear in a publication, please include a citation to: S. J. Blunsden, R. B. Fisher, "The BEHAVE video dataset: ground truthed video for multi-person behavior classification" , Annals of the BMVA, Vol 2010(4), pp 1-12.

A lot (but not all) of the video sequences have ground truth bounding boxes of the pedestrians in the scene. The bounding boxes are given in the VIPER XML format. The site will be updated when more of the ground truth becomes available.

Scenarios

Ten basic scenarios were acted out by some of the Vision Group team members. These were called InGroup (IG), Approach (A), WalkTogether (WT), Split (S), Ignore (I), Following (FO), Chase (C), Fight (FI), RunTogether (RT), and Meet (M). Many of the interactions in the video sequence (Clip 1) are labelled accordingly. These are given in a seperate file called markup.txt. The annotation in markup.txt is as follows:

ID1 - This is the first group which is interacting. The numbers within the square brackets [] are the members of the group.
ID2 - This is the second group which is interacting with the first (in ID1). This entry is optional.
Start - The starting frame number.
End - The ending frame number.
Label - This is the class label which describes the activity taking place (the scenarios being enacted). A short description of each scenario follows:

InGroup - The people are in a group and not moving very much
Approach - Two people or groups with one (or both) approaching the other
WalkTogether - People walking together
Meet - Two or more people meeting one another
Split - Two or more people splitting from one another
Ignore - Ignoring of one another
Chase - one group chasing another
Fight - Two or more groups fighting
RunTogether - The group is running together
Following - Being followed

All entries are delimited by a semicolon (;)

Examples

This example is interpreted as persons with ids 3 and 4 are in a group (represented as [3,4]) which has been labelled as "WalkTogether" between frames 5826 and 5926.

This example shows that person 2 is being approached by persons 0 and 1 ([0,1]) between frames 60296 and 60349. Persons 0 and 1 are in the approaching group.

Corresponding pixel positions in image:

For people interested in the ground plane homography, the mapping can be computed from this information. The image pixel positions will depend on your image scaling. Here the image sizes are 640x480. For the image just below the corresponding pixel positions are:

Point (Col,Row)(pixels) (X,Y)(cm) A (453,250) (913,-358) B (416,317) (926,-454) C (63,327) (348,-876) D (44,155) (-148,-290) E (125,158) (0,-253) F (259,152) (346,-120) G (249,130) (270,0) H (158,113) (0,0)

Clips

There are 4 clips that can be downloaded as .wmv files.

• Clip 1 (93MB).
• Clip 2 (27MB).
• Clip 3 (134MB).
• Clip 4 (48MB).

Subclips

Clip 1 has been devided into 8 sequences. The AVI files are named N-M.avi and contain video frames N through M. Similar naming conventions apply for the other files. Some segments of video with little or no action have been removed (eg. from 11201 through 11499). The table below contains information about the kinds of scenarios enacted in each clip along with links to each clips' video, ground truth file, archived JPEGs, and the .info file used by VIPER. The .xgtf extension is used by VIPER but these are essentially text files. Please note that the ground truth files are not complete yet. In order to run the .info files the .xgtf files need to be edited in order to reflect the location in which the .info files are saved. So for example if file 1-11200.info is saved in drive D:\ then 1-11200.xgtf should reflect this as:

<sourcefile filename="file:/D:/1-11200.info">

Moreover, the .info file has to be in the same directory as the unzipped archived JPEGs for each sequence. Please also note that the .info files contain information about the whole sequence, therefore, when the .xgtf files are run using VIPER the screen will appear blank until the frame range of the ground truth data is reached.

Back to the BEHAVE homepage

There have been accesses since October 2007.

© 2007 Aroosha Laghaee