A Practical Exercise in Adapting SIFT Using FHE Primitives

TL;DR

The paper assesses the practicality of implementing the SIFT algorithm under CKKS-based Fully Homomorphic Encryption, highlighting key limitations such as the absence of a standard encrypted comparison operator and the need for costly polynomial approximations for non-linear tasks. It demonstrates how regular algorithms can be adapted to the FHE setting by reworking operations (e.g., division and magnitude) and introducing a deferred computation paradigm that outsources expensive comparisons to the client. The study analyzes histogram binning, branchless control flow, and novel max-finding approaches to reduce multiplicative depth, illustrating how interactive or deferred strategies can yield more practical results. The authors conclude that current FHE libraries lack sufficient abstractions for general algorithms and advocate for an intermediate framework that balances usability with security and performance constraints.

Abstract

An exercise in implementing Scale Invariant Feature Transform using CKKS Fully Homomorphic encryption quickly reveals some glaring limitations in the current FHE paradigm. These limitations include the lack of a standard comparison operator and certain operations that depend on it (like array max, histogram binning etc). We also observe that the existing solutions are either too low level or do not have proper abstractions to implement algorithms like SIFT. In this work, we demonstrate: 1. Methods of adapting regular code to the FHE setting. 2. Alternate implementations of standard algorithms (like array max, histogram binning, etc.) to reduce the multiplicative depth. 3. A novel method of using deferred computations to avoid performing expensive operations such as comparisons in the encrypted domain. Through this exercise, we hope this work acts as a practical guide on how one can adapt algorithms to FHE