TCProF: Time-Complexity Prediction SSL Framework

TL;DR

TCProF introduces a time-Complexity Prediction SSL framework that excels in low-resource settings by integrating augmentation (BT and LC), a symbolic module for pseudo-labeling, and a co-training scheme. It demonstrates substantial gains over self-training baselines (e.g., over 60% in accuracy/F1) and yields competitive results against commercial LLMs on CorCoD and CodeComplex datasets, while operating under few labeled examples. The framework combines model-based and symbolic pseudo-labels, with function extraction, regex/AST-based analysis, and diverse augmentation strategies to robustly classify code snippets into discrete time-complexity classes such as $O(1)$, $O( log N)$, $O(N)$, $O(N log N)$, $O(N^2)$, $O(N^3)$, and $O(2^N)$. These results highlight the practical potential of SSL for program analysis tasks under data scarcity and suggest avenues for zero-shot extension and broader language generalization.

Abstract

Time complexity is a theoretic measure to determine the amount of time the algorithm needs for its execution. In reality, developers write algorithms into code snippets within limited resources, making the calculation of a code's time complexity a fundamental task. However, determining the precise time complexity of a code is theoretically undecidable. In response, recent advancements have leaned toward deploying datasets for code time complexity prediction and initiating preliminary experiments for this challenge. We investigate the challenge in low-resource scenarios where only a few labeled instances are given for training. Remarkably, we are the first to introduce TCProF: a Time-Complexity Prediction SSL Framework as an effective solution for code time complexity prediction in low-resource settings. TCProF significantly boosts performance by integrating our augmentation, symbolic modules, and a co-training mechanism, achieving a more than 60% improvement over self-training approaches. We further provide an extensive comparative analysis between TCProF, ChatGPT, and Gemini-Pro, offering a detailed evaluation of our approach. Our code is at https://github.com/peer0/few-shot-tc.

Figures (13)

• Figure 1: The overall framework of TCProF.
• Figure 2: A procedural illustration of our symbolic module $\mathbb{Sym}$.
• Figure 3: Confusion matrix of ours for CodeComplex (Python) 10-shot.
• Figure 4: Confusion matrices of 10-shot TCProF(UniXcoder) performance.
• Figure 5: CodeT5+ self-learning performance visualization for CodeComplex (Java) 10-shot.