In July, the Marine Institute (MI) hosted Marco Francescangeli, who is a PhD candidate at Universitat Politécnica de Catalunya (UPC) working on ecological monitoring of fishes using footage from the OBSEA camera.
Hosting Marco for a period of one month facilitated an invaluable information and expertise exchange and was made possible through collaborating with EMSO partners Jacopo Aguzzi at ICM CSIC and Joaquin Del Rio at UPC, both of whom co-tutored Marco.
While at the Marine Institute, Marco worked on artificial intelligence techniques with the high resolution video data from the SmartBay observatory. Marco analysed the different tools for video and imagery annotations and trained MI staff on how to use the recommended programmes.
His expertise in image annotations will allow the team at MI to introduce AI learning for automating detection of marine activity in the large volume of EMSO SmartBay video footage.
Marco’s technology review identified the “Roboflow” image annotation tool as the best open source image labelling tool because it allows the user to easily annotate videos frame by frame, giving the option to decide the frequency of the frames required (1 frame per second is recommended). This tool also permits users to divide their image dataset in the three main datasets for machine learning (training, validation and testing sets), and at the end of labelling also to train your own model with YOLOv5 (Roboflow + YOLO partnership). The EMSO Obsea and Smartbay Observatories are also participating as EMSO affiliate partners in the iMagine Horizon Europe project – iMagine project -.. iMagine provides a portfolio of image datasets, high performance image analysis tools empowered with Artificial Intelligence (AI), and Best Practice documents for scientific image analysis. EMSO observatories have developed a number of marine video observation related use cases for the application of the iMagine tools and Marco also worked on maturing and developing these use cases during his time at EMSO SmartBay.
Photo 1: An example of image annotation on SmartBay Observatory footage using Label Studio tool. The red boxes identify starfish and pouting in the image.
Photo 2: Marco Francescangeli. Universitat Politécnica de Catalunya (UPC)
Authors: Paul Gaughan, Christine Loughlin (Marine Institute)