Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vertical Federated Image Segmentation

Paul K. Mandal, Cole Leo

TL;DR

Privacy concerns with dispersed data motivate vertical federated learning for image segmentation. The authors implement a vertical FCN split across two federates, exchanging heavily compressed intermediate features through an interactive layer while the top model handles labels on a central server. On CamVid, the approach achieves high pixel accuracy and IOU in a binary road task, demonstrating the viability of VFL for segmentation under data privacy constraints, albeit with limitations in validation and reliance on feature compression. They discuss limitations of FATE interoperability and propose future work including multiclass expansion, larger datasets like Cityscapes, and exploring transformer-based architectures to improve efficiency and scalability. Overall, the paper provides a practical blueprint for privacy-preserving, vertically partitioned image segmentation.

Abstract

With the popularization of AI solutions for image based problems, there has been a growing concern for both data privacy and acquisition. In a large number of cases, information is located on separate data silos and it can be difficult for a developer to consolidate all of it in a fashion that is appropriate for machine learning model development. Alongside this, a portion of these localized data regions may not have access to a labelled ground truth. This indicates that they have the capacity to reach conclusions numerically, but are not able to assign classifications amid a lack of pertinent information. Such a determination is often negligible, especially when attempting to develop image based solutions that often necessitate this capability. With this being the case, we propose an innovative vertical federated learning (VFL) model architecture that can operate under this common set of conditions. This is the first (and currently the only) implementation of a system that can work under the constraints of a VFL environment and perform image segmentation while maintaining nominal accuracies. We achieved this by utilizing an FCN that boasts the ability to operate on federates that lack labelled data and privately share the respective weights with a central server, that of which hosts the necessary features for classification. Tests were conducted on the CamVid dataset in order to determine the impact of heavy feature compression required for the transfer of information between federates, as well as to reach nominal conclusions about the overall performance metrics when working under such constraints.

Vertical Federated Image Segmentation

TL;DR

Privacy concerns with dispersed data motivate vertical federated learning for image segmentation. The authors implement a vertical FCN split across two federates, exchanging heavily compressed intermediate features through an interactive layer while the top model handles labels on a central server. On CamVid, the approach achieves high pixel accuracy and IOU in a binary road task, demonstrating the viability of VFL for segmentation under data privacy constraints, albeit with limitations in validation and reliance on feature compression. They discuss limitations of FATE interoperability and propose future work including multiclass expansion, larger datasets like Cityscapes, and exploring transformer-based architectures to improve efficiency and scalability. Overall, the paper provides a practical blueprint for privacy-preserving, vertically partitioned image segmentation.

Abstract

With the popularization of AI solutions for image based problems, there has been a growing concern for both data privacy and acquisition. In a large number of cases, information is located on separate data silos and it can be difficult for a developer to consolidate all of it in a fashion that is appropriate for machine learning model development. Alongside this, a portion of these localized data regions may not have access to a labelled ground truth. This indicates that they have the capacity to reach conclusions numerically, but are not able to assign classifications amid a lack of pertinent information. Such a determination is often negligible, especially when attempting to develop image based solutions that often necessitate this capability. With this being the case, we propose an innovative vertical federated learning (VFL) model architecture that can operate under this common set of conditions. This is the first (and currently the only) implementation of a system that can work under the constraints of a VFL environment and perform image segmentation while maintaining nominal accuracies. We achieved this by utilizing an FCN that boasts the ability to operate on federates that lack labelled data and privately share the respective weights with a central server, that of which hosts the necessary features for classification. Tests were conducted on the CamVid dataset in order to determine the impact of heavy feature compression required for the transfer of information between federates, as well as to reach nominal conclusions about the overall performance metrics when working under such constraints.
Paper Structure (12 sections, 5 figures)

This paper contains 12 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Image Segmentation & Fully Convolutional Networks
  4. Vertical Federated Learning
  5. Experimental Setup
  6. CamVid Dataset
  7. Bottom Model for Convolutions
  8. Dimensionality in the Interactive Layer
  9. Top Model for Classification
  10. Modifications to FATE
  11. Results
  12. Discussion & Future Work

Figures (5)

  • Figure 1: Vertical Federated Learning Architecture as proposed in 10.1145/3298981
  • Figure 2: Architecture of our bottom model.
  • Figure 3: Architecture of our top model.
  • Figure 4: Pixel Accuracy, IOU, and loss on the training set.
  • Figure 5: Left - The original image; Center - The segmentation map; Right - Our model's output