Workshop and Challenge: Automated Analysis of Marine Video for Environmental Monitoring


This workshop features a data challenge in this unique domain. Large amounts of underwater image and video data have been collected and annotated by the National Oceanic and Atmospheric Administration (NOAA) from a variety of image and video sensors deployed underwater in a variety of oceanic environments and climates. The data shows many species of fish, shellfish, underwater plants, habitat and other biota. All submissions will be ranked, with the top few invited to present their approaches at the workshop.

Data and data release plan

We have released a majority of the data so that submitters can start experimentation. All of this data can be used for training and testing by submitters. For evaluation, another data subset will be released, without annotations. Performers can compute results on this data and submit them for scoring once the submission system is available. Performers may not train on or tune or annotate this data. For determining the challenge winners to present at the workshop, a final test subset without annotations will be released shortly before the deadline.

Dataset details and download instructions are available on the data information page.


The challenge will evaluate accuracy in detection and classification, following the methodology in the MSCOCO Detection Challenge, for bounding box output (not pixel-level segmentation masks). The annotations for scoring are bounding boxes around every animal, with a species classification label for each.

Kitware’s online challenge platform will be used to perform submission scoring in an automated way, with participants uploading their annotations (in the MSCOCO format) to the challenge website. Kitware hosts a number of challenges in the biomedical domain, which can be found here.

Please see the challenge submission guidelines page for more detailed information about submission.

Important Dates

  • April 2 (11:59 PM EDT): Paper submissions due for standard track (non data-challenge) papers
  • February 23: Data challenge initial training data preview (phase 0)
  • April 20: Data challenge updated training data released. Download.
  • May 7: Final challenge data released (no annotations). Download.
  • May 11: Data challenge submission server opens.
  • May 23: Data challenge final submissions due
  • May 30: Competition results announced

Future Work

The challenge will continue indefinitely after the workshop, with more data being added. We anticipate including fish tracking, fish behavior detection, habitat classification and other problems over the next year or more.