Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analysis of Hybrid Compositions in Animation Film with Weakly Supervised Learning

Mónica Apellaniz Portos, Roberto Labadie-Tamayo, Claudius Stemmler, Erwin Feyersinger, Andreas Babic, Franziska Bruckner, Vrääth Öhner, Matthias Zeppelzauer

TL;DR

This work combines ideas from semi-supervised and weakly supervised learning to train a model that can segment hybrid compositions without requiring pre-labeled segmentation masks, and shows a performance close to a fully supervised baseline.

Abstract

We present an approach for the analysis of hybrid visual compositions in animation in the domain of ephemeral film. We combine ideas from semi-supervised and weakly supervised learning to train a model that can segment hybrid compositions without requiring pre-labeled segmentation masks. We evaluate our approach on a set of ephemeral films from 13 film archives. Results demonstrate that the proposed learning strategy yields a performance close to a fully supervised baseline. On a qualitative level the performed analysis provides interesting insights on hybrid compositions in animation film.

Analysis of Hybrid Compositions in Animation Film with Weakly Supervised Learning

TL;DR

This work combines ideas from semi-supervised and weakly supervised learning to train a model that can segment hybrid compositions without requiring pre-labeled segmentation masks, and shows a performance close to a fully supervised baseline.

Abstract

We present an approach for the analysis of hybrid visual compositions in animation in the domain of ephemeral film. We combine ideas from semi-supervised and weakly supervised learning to train a model that can segment hybrid compositions without requiring pre-labeled segmentation masks. We evaluate our approach on a set of ephemeral films from 13 film archives. Results demonstrate that the proposed learning strategy yields a performance close to a fully supervised baseline. On a qualitative level the performed analysis provides interesting insights on hybrid compositions in animation film.
Paper Structure (26 sections, 2 equations, 5 figures, 4 tables)

This paper contains 26 sections, 2 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Terminology
  5. Method Overview
  6. Stage 1: Image Classification (Proxy Task)
  7. Stage 2: Mask Generation
  8. Stage 3: Image Segmentation
  9. Data Corpus and Experimental Setup
  10. Data Corpus
  11. Dataset Compilation
  12. Experimental Setup
  13. Preprocessing.
  14. Proxy classification (RQ1)
  15. Boundary cases (RQ1.1)
  16. ...and 11 more sections

Figures (5)

  • Figure 1: Four sample frames from our database (left to right): a frame from a fully animated sequence Fig1Animation, two hybrid compositions combining photographic and graphic content tibav:14363tibav:10786, and a purely live-action frame Fig1Live.
  • Figure 2: Proposed overall approach including proxy task learning (stage 1, yellow), segmentation mask generation (stage 2, blue) and image segmentation (stage 3, grey).
  • Figure 3: Boundary cases in the proxy classification task and GradCAM explanations (red high, blue low attribution for predicted class). For each image, we provide the true label (P or NP) and the model's prediction with its likelihood in brackets. Sources: (a)Neithardt, (b)Hai, (c)Polen, (d)tibav:10606, (e)Quallen
  • Figure 4: Hybrid frames together with GradCAM explanations (for the class P), the ground truth (black NP, white P), and proxy segmentation masks.
  • Figure 5: Segmentation results on hybrid frames (P: green; NP: blue).