Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning Approach for Knee Point Detection on Noisy Data

Ting Yan Fok, Nong Ye

TL;DR

A deep-learning approach is presented and a Convolutional Neural Network with a U-Net-like architecture is employed, to accurately detect the knee point(s) of the underlying true distribution and the proposed model outperforms existing methods in all synthetic datasets, regardless of whether the samples have single or multiple knee points.

Abstract

A knee point on a curve is the one where the curve levels off after an increase. In a computer system, it marks the point at which the system's performance is no longer improving significantly despite adding extra resources. Thus a knee point often represents an optimal point for decision. However, identifying knee points in noisy data is a challenging task. All previous works defined knee points based on the data in the original scale. However, in this work, we define knee points based on normalized data and provide a mathematical definition of curvature for normalized discrete data points, based on the mathematical definition of curvature for continuous functions. The impact of normalization exerted on curvature and the location of knee points are also discussed. Nevertheless, assessing the effectiveness of methods is difficult in the absence of ground truth data and benchmark datasets, which makes comparing existing methods challenging. In view of this, we create synthetic data that simulate real-world scenarios. We achieve this by selecting a set of functions that possess the required characteristics in this research and then introducing noise that satisfies the underlying distribution. In addition, we present a deep-learning approach and employ a Convolutional Neural Network (CNN) with a U-Net-like architecture, to accurately detect the knee point(s) of the underlying true distribution. The proposed model is evaluated against state-of-the-art methods. Experiments show that our network outperforms existing methods in all synthetic datasets, regardless of whether the samples have single or multiple knee points. In fact, our model achieves the best $F_{1}$ scores among all existing methods in all the test sets.

Deep Learning Approach for Knee Point Detection on Noisy Data

TL;DR

A deep-learning approach is presented and a Convolutional Neural Network with a U-Net-like architecture is employed, to accurately detect the knee point(s) of the underlying true distribution and the proposed model outperforms existing methods in all synthetic datasets, regardless of whether the samples have single or multiple knee points.

Abstract

A knee point on a curve is the one where the curve levels off after an increase. In a computer system, it marks the point at which the system's performance is no longer improving significantly despite adding extra resources. Thus a knee point often represents an optimal point for decision. However, identifying knee points in noisy data is a challenging task. All previous works defined knee points based on the data in the original scale. However, in this work, we define knee points based on normalized data and provide a mathematical definition of curvature for normalized discrete data points, based on the mathematical definition of curvature for continuous functions. The impact of normalization exerted on curvature and the location of knee points are also discussed. Nevertheless, assessing the effectiveness of methods is difficult in the absence of ground truth data and benchmark datasets, which makes comparing existing methods challenging. In view of this, we create synthetic data that simulate real-world scenarios. We achieve this by selecting a set of functions that possess the required characteristics in this research and then introducing noise that satisfies the underlying distribution. In addition, we present a deep-learning approach and employ a Convolutional Neural Network (CNN) with a U-Net-like architecture, to accurately detect the knee point(s) of the underlying true distribution. The proposed model is evaluated against state-of-the-art methods. Experiments show that our network outperforms existing methods in all synthetic datasets, regardless of whether the samples have single or multiple knee points. In fact, our model achieves the best scores among all existing methods in all the test sets.
Paper Structure (20 sections, 6 equations, 5 figures, 2 tables)

This paper contains 20 sections, 6 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. L-Method
  4. Dynamic First Derivative Threshold
  5. AL-Method
  6. S-Method
  7. Kneedle
  8. U-Net
  9. Knee Point Definition
  10. Proposed Approach
  11. Model Architecture
  12. Soft $F_{1}$ score
  13. Non-maximal Suppression
  14. Experiments
  15. Synthetic Data
  16. ...and 5 more sections

Figures (5)

  • Figure 1: An example showing data normalization changes the curvature shape and knee position. (a) The curve of $y = 5 \times \frac{1}{ 1 + e^{-10x + 5}}$ generated by 1000 evenly-spaced $x$ values in $[0, 1]$. The normalized values are plotted as $\tilde{y}$ in the figure; (b) Curvatures and the corresponding knee point indices of the curves. The normalization operation applies a squeezing effect to the curve of ${y}$, resulting in a smaller rate of change as observed in $\tilde{y}$. This reduces the range of values of $K_{\tilde{y}}(\tilde{x})$ and causes a shift in the position of the knee point.
  • Figure 2: An illustration of the architecture of our proposed method, UNetConv. The model is comprised of two main components: a U-Net model and a sequence of convolutional layers. The U-Net model component part passes the input through the encoding path, followed by a bottleneck layer and then to the decoding path. Both the encoding path and decoding path contain four levels of blocks. The numbers beneath and in the bottom right corner of each block respectively indicate the number of channels and size of the resulting feature map passed through that specific layer.
  • Figure 3: A graphical representation illustrates a FT12 multi-knee sample, created by summing the graphs of individual single-knee function. This composite sample is created in the sequence of FT8, FT1, and FT6.
  • Figure 4: An example showing varying the $x$ interval can generate samples with a variety of curve shapes, different ranges of curvature values, and thus different positions of knee point(s). (a) A graph showing the logistic function, $y = \frac{1}{1 + e^{-x}}$, for $x \in [-40, 40]$. (b) The curve shape of $\tilde{y}_{1}$ is noticeably different from $\tilde{y}_{2}$, even though both are produced by the same function. The figure also shows the flipped curve of $y_{1}$. Unlike the logit function, the knee point of flipped$\tilde{y}_{1}$ occurs at the very beginning of the curve.
  • Figure 5: $F_{1}$ scores of (a) UNetConv, DFDT, AL, S and Kneedle methods for varying allowable index error on the $sknee$ data set; (b)UNetConv and Kneedle methods for varying allowable index error on the $mknee$ data set; (c) UNetConv, DFDT, AL, S and Kneedle methods for varying allowable index error on the $ng$ data set.