AMI Corpus - Data Problems

As indicated in the table below, some of the recordings are affected by signal discontinuities. In most cases, this was due to a sudden failure by the audio recording equipment. For the purpose of generating transcriptions and annotated data for these files, the signals were concatenated with zero padding (performed on 48kHz files) in correspondence with the length of the discontinuity. Frame dropping also occurred with some of the video signals collected in Edinburgh and Idiap when encoding them from DV tape to DivX. These have undergone additional processing and are now synchronized with the audio signals. Due to hardware issues during the recording stage, video signals collected at TNO are not perfectly synchronized. However, data for these have undergone an additional processing step (see Meeting Rooms), and now exhibit acceptable levels of synchronization.

Other data collection issues relate to seat swapping by meeting participants. In a few cases, this has resulted in the mislabeling of participant IDs and thumbnails. A very small number of meetings also feature an inappropriate positioning of headset microphones. In one meeting, a participant removes his headset altogether. These and other data collection and pre-processing issues are presented in the following table, along with details of whether and, where relevant, how the issue was resolved.

Other general things to note are that some Edinburgh non-scenario meetings last for longer than 1 hour, even though the video files for those meetings are exactly 1 hour long as that was the length of the tapes used (see table for details). Also, TNO recorded scenario meetings 3 and 4 as one meeting, together with the discussion between the two designers that occured between the two meetings; in the corpus as distributed the recording has been split into its component parts. Finally, for all Edinburgh meetings collected prior to May 2005 note that the Logitech pens were not synchronized before the start of recording. This means that we will need human intervention to establish the relationship between pen time and the standardized timecode.

More Information on Synchronization and drop-outs in TNO data

Due to a faulty hardware configuration, acquisition on video devices was not properly synchronised and the timecode insertion did not work correctly. TNO have worked hard to improve the data as much as possible - by laboriously recovering synchronisation between each video file after detailed analysis of skew and drift. Having done this, the video-video and video-audio synchronisation (there is no problem with audio-audio synch) is now within reasonable bounds for a human observer looking for lip synch. Multiple people have checked a portion of the data and human agreement has been found to be within a bound of ~100ms (~5 frames). In addition, the hardware setup has been rectified and tested ensuring that the problem will not occur in any future data recorded at TNO.

We have therefore approved this data for inclusion in the AMI Hub Corpus, as we feel that this level of synchronisation will in practice be sufficient for AMI research tasks. This and any other problems with data in the AMI corpus (e.g. location of dropped frames) will be noted in metadata in the eventual corpus.

While all research tasks will be encouraged to use AMI data from the 3 sites (TNO, Idiap, UEDIN), if certain researchers really find they require stricter synchronisation, such as maybe emotion recognition or audio-visual speech recognition, then they can define evaluation protocols using only the UEDIN & Idiap portion of the data (~65 hours).

If any researcher foresees a major problem in using the TNO data due to the looser synchronisation, please email the WP2 list with details of your task and your rationale.