Downloadable Datasets
This page provides a subset of the full Fish4Knowledge video and
fish analysis dataset, which is about 200 Tb in size. See
http://gad240.nchc.org.tw/tai/video_query/
for downloading options from the full video data. Use of the data should include this citation:
Bastiaan J. Boom, Jiyin He, Simone Palazzo, Phoenix X. Huang, Hsiu-Mei Chou,
Fang-Pang Lin, Concetto Spampinato, Robert B. Fisher;
A research tool for long-term and continuous analysis of fish
assemblage in coral-reefs using underwater camera footage.
Ecological Informatics, 2014, DOI: dx.doi.org/10.1016/j.ecoinf.2013.10.006.
The SQL files are stored as CSV files. Each line in the file
records the detection of a fish in a given frame in the
corresponding video. The structure of each record is described
here.
There are two interesting subsets of the data that are provided here:
- 1) All Years: a 10 minute video clip from all working cameras taken at 08:00
every day in the project Oct 1, 2010 - July 10, 2013, giving
5824 video clips (of 7-30 Mb). Note, some dates are missing. This data allows analysis of fish
patterns over annual cycles and comparison between sites.
- 2) Full Day: 690 video clips from the 9 cameras taken from 06:00 - 19:00 on April 22, 2011 (out of 702 possible).
This data allows analysis of fish patterns over a full day period and comparison between sites.
There are 3 sites with different numbers of cameras:
- 1 - NPP-3: 4 cameras
- 2 - HoBiHu harbour: 3 cameras
- 3 - Lanyu/Orchid Island - 2 cameras
We also supply 2 example Matlab script files to demonstrate how to use the data files:
- species_histogram.m - gives a histogram of the number of fish in a given species over time.
To specify which data to access, you need to edit the script file.
Specifying the start and end date and time means editing start_date_vector and end_date_vector
according to the example shown in the file.
To specify the Site and Camera at the Site, use the variables site and camera.
To specify the desired species, use the species variable. The list of allowable
species is in the matlab file.
The program (using species_extraction.m) loads the selected CSV files (constrained by
the supplied dates) and extracts the specified species instances and counts them.
Two examples are shown, one for extracting data from the 8am subset, and the
other example for the April 22, 2011 dataset.
- species_extraction.m - called by species_histogram to download the CSV files from the Fish4Knowledge web site and extract the data.
With the interface below, you can also download the videos and CSV files directly
into your own filespace. You can then adapt the supplied matlab files to
analyse the locally stored data.
The "All Years" dataset consists of video from all cameras at the 3 sites,
for the time period 8:00-8:10. (Except for some videos, where there
were technical problems.) This data allows analysis of fish
patterns over annual cycles and comparison between sites.
There are 5824 video files each about 25 Mb long
(Oct 1, 2010 - July 10, 2013 - 1035 days * 9 cameras).
Select the start and stop days desired by clicking in the 'Day'
boxes to pop-up a calendar. Select the camera(s) and sites.
Select multiples sites/cameras by using the CTRL-click on
the desired selections.
When you click 'Download', the system will produce 2 or more 'zip' files,
which will appear as clickable links, along with the file sizes.
One zip file will contain all selected SQL files.
The other zip files will contain up to 5GB of video files in each download.
Note: download.php script will timeout after 120 seconds.
Compressing very large sets of data (e.g. more than 10 videos, e.g. 1 day of all
9 cameras from the All-Days dataset at once, or
10 minutes of all 9 cameras on the Full Day dataset) takes
more than this time.
The "Full Day" dataset consists of videos from all cameras at the 3 sites,
for all time periods (6:00 - 19:00, the daylight hours), but for only
a single date: April 22, 2011. (Except for some videos, where there
were technical problems.) This data allows analysis of fish
patterns over a full day period and comparison between sites.
There are 690 videos (out of 702 cameras * 13 hours * 6 10 minute clips).
Select the start and stop times desired, as well as which camera(s)
and sites. Select multiples sites/cameras by using the CTRL-click on
the desired selections.
When you click 'Download', the system will produce 2 'zip' files,
which will appear as 2 clickable links below, along with the file sizes.
The zip files will contain all selected SQL and video files in one
download.
Note: download.php script will timeout after 120 seconds.
Compressing very large sets of data (e.g. over 10 videos at once) takes
more than this time.
|
