Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ideal Observer for Segmentation of Dead Leaves Images

Swantje Mahncke, Malte Ott

TL;DR

This work provides a rigorous, image-computable Bayesian ideal observer for segmentation in dead leaves images, outlining how to generate occluding scenes and how to compute posterior probabilities over pixel partitions. It decomposes the prior into leaf-level contributions, derives constructive methods for leaf probabilities via area-bounded position integrals and boundary geometry, and formulates the likelihood using discrete or continuous color-texture models. Although computationally intensive, the framework yields an principled upper bound on segmentation performance that can benchmark humans and algorithms on small, controlled pixel sets; it also clarifies where approximations are necessary and how to extend the approach to other leaf shapes and distributions. Overall, the paper bridges a generative occlusion model with an image-computable ideal observer, offering a tractable path to principled segmentation benchmarks in constrained visual stimuli.

Abstract

The human visual environment is comprised of different surfaces that are distributed in space. The parts of a scene that are visible at any one time are governed by the occlusion of overlapping objects. In this work we consider "dead leaves" models, which replicate these occlusions when generating images by layering objects on top of each other. A dead leaves model is a generative model comprised of distributions for object position, shape, color and texture. An image is generated from a dead leaves model by sampling objects ("leaves") from these distributions until a stopping criterion is reached, usually when the image is fully covered or until a given number of leaves was sampled. Here, we describe a theoretical approach, based on previous work, to derive a Bayesian ideal observer for the partition of a given set of pixels based on independent dead leaves model distributions. Extending previous work, we provide step-by-step explanations for the computation of the posterior probability as well as describe factors that determine the feasibility of practically applying this computation. The dead leaves image model and the associated ideal observer can be applied to study segmentation decisions in a limited number of pixels, providing a principled upper-bound on performance, to which humans and vision algorithms could be compared.

Ideal Observer for Segmentation of Dead Leaves Images

TL;DR

This work provides a rigorous, image-computable Bayesian ideal observer for segmentation in dead leaves images, outlining how to generate occluding scenes and how to compute posterior probabilities over pixel partitions. It decomposes the prior into leaf-level contributions, derives constructive methods for leaf probabilities via area-bounded position integrals and boundary geometry, and formulates the likelihood using discrete or continuous color-texture models. Although computationally intensive, the framework yields an principled upper bound on segmentation performance that can benchmark humans and algorithms on small, controlled pixel sets; it also clarifies where approximations are necessary and how to extend the approach to other leaf shapes and distributions. Overall, the paper bridges a generative occlusion model with an image-computable ideal observer, offering a tractable path to principled segmentation benchmarks in constrained visual stimuli.

Abstract

The human visual environment is comprised of different surfaces that are distributed in space. The parts of a scene that are visible at any one time are governed by the occlusion of overlapping objects. In this work we consider "dead leaves" models, which replicate these occlusions when generating images by layering objects on top of each other. A dead leaves model is a generative model comprised of distributions for object position, shape, color and texture. An image is generated from a dead leaves model by sampling objects ("leaves") from these distributions until a stopping criterion is reached, usually when the image is fully covered or until a given number of leaves was sampled. Here, we describe a theoretical approach, based on previous work, to derive a Bayesian ideal observer for the partition of a given set of pixels based on independent dead leaves model distributions. Extending previous work, we provide step-by-step explanations for the computation of the posterior probability as well as describe factors that determine the feasibility of practically applying this computation. The dead leaves image model and the associated ideal observer can be applied to study segmentation decisions in a limited number of pixels, providing a principled upper-bound on performance, to which humans and vision algorithms could be compared.

Paper Structure

This paper contains 20 sections, 10 theorems, 124 equations, 17 figures, 3 algorithms.

Table of Contents

  1. Dead leaves model
  2. The model
  3. Environment
  4. Projection
  5. Color & texture
  6. Model
  7. Scene statistics
  8. Implementation
  9. Ideal observer
  10. Prior
  11. Decomposition of prior probability
  12. Leaf probabilities
  13. Area of possible positions
  14. Changing radii
  15. Implementation
  16. ...and 5 more sections

Key Result

Theorem 1

Let $a$ be a finite set of visible points in $\mathop{\mathrm{\mathbb{R}}}\nolimits^2$ and $\mathop{\mathrm{\mathbf{m}}}\nolimits_a = \{\mathop{\mathrm{\mathbf{v}}}\nolimits_{a,i}\}_{i\in[n]}$ an ordered partition of $a$. Then the prior probability of $\mathop{\mathrm{\mathbf{m}}}\nolimits_a$ based where $a_{\setminus i} \coloneq a \setminus \mathop{\mathrm{\mathbf{v}}}\nolimits_{a,1},\dots,\math

Figures (17)

  • Figure 1: Dead leaves by pixabay user 7631258 (https://pixabay.com/photos/dry-leaves-fallen-dry-dead-leaves-4364822/, CC BY).
  • Figure 2: Example of a random environment using random circles as objects.
  • Figure 3:
  • Figure 6: Dead leaves scene with (\ref{['img:dlm random color']}) random RGB color and (\ref{['img:dlm gaussian noise']}) Gaussian texture or with (\ref{['img:dlm sample color']}) color sampled from the histogram of Figure \ref{['img:dead_leaves']} and (\ref{['img:dlm Brodatz texture']}) random Brodatz textures Brodatz1999 blended onto leaves.
  • Figure 7: Random dead leaves image generated with algorithm \ref{['alg:dlm']} with normally distributed colors and mean-zero Gaussian texture in HSV ranging from $0$ to $1$ ($s=500$ px, $r_{\min}=5$ px, $r_{\max}=50$ px, $\mu_c=(0.5,0.5,0.5)$, $\sigma_c=(0.1,0.1,0.1)$, $\sigma_t=(0.05,0.05,0.05)$).
  • ...and 12 more figures

Theorems & Definitions (41)

  • Definition 1: Environment
  • Example 1. a
  • Definition 2: Visible parts of leaves
  • Remark 1
  • Definition 3: Random Dead leaves partition
  • Remark : Discretization
  • Definition 4: Membership function
  • Example 1. b
  • Definition 5: Dead leaves image
  • Remark
  • ...and 31 more