LFFR: Logistic Function For (multi-output) Regression

TL;DR

This work extends the LFFR framework to multi-output regression in the encrypted domain, leveraging a Simplified Fixed Hessian to enable efficient training under CKKS-based fully homomorphic encryption. By deriving a diagonal surrogate Hessian and adopting a logistic-function–like regression approach, the method supports non-linear relationships while avoiding explicit sigmoid computations in the encrypted state. A key contribution is the improved LFFR variant, which normalizes both inputs and predictions and introduces a gamma-based transformation to stabilize and linearize the learning problem, enhancing practicality for secure multi-output tasks. The approach is validated on real datasets, showing privacy-preserving training with competitive predictive performance and a feasible pipeline for secure applications in environments where data sharing is restricted. The work thus offers a scalable, secure pathway for multi-output regression in sensitive domains, supported by open-source implementation and concrete guidance on parameter choices and pipeline design.

Abstract

In this manuscript, we extend our previous work on privacy-preserving regression to address multi-output regression problems using data encrypted under a fully homomorphic encryption scheme. We build upon the simplified fixed Hessian approach for linear and ridge regression and adapt our novel LFFR algorithm, initially designed for single-output logistic regression, to handle multiple outputs. We further refine the constant simplified Hessian method for the multi-output context, ensuring computational efficiency and robustness. Evaluations on multiple real-world datasets demonstrate the effectiveness of our multi-output LFFR algorithm, highlighting its capability to maintain privacy while achieving high predictive accuracy. Normalizing both data and target predictions remains essential for optimizing homomorphic encryption parameters, confirming the practicality of our approach for secure and efficient multi-output regression tasks.