Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning-Based AI for Startup Evaluation (R.A.I.S.E.): A Memory-Augmented, Multi-Step Decision Framework

Jack Preuveneers, Joseph Ternasky, Fuat Alican, Yigit Ihlamur

TL;DR

The paper tackles the need for explainable startup evaluation by integrating LLM-based reasoning with explicit rule-based decision policies. It presents R.A.I.S.E., a memory-augmented, multi-step framework that converts chain-of-thought reasoning into verifiable rules and constructs an editable decision policy. Through data ingestion, reasoning log generation, rule extraction, policy formation, and persistent memory, the approach achieves substantial gains in precision and accuracy, culminating in a fully combined pipeline that reaches around 0.700 accuracy on a balanced startup dataset. The work demonstrates that combining reasoning traces with structured rules yields transparent, high-performance predictions suitable for expert oversight and continuous policy refinement in high-stakes investment environments.

Abstract

We present a novel framework that bridges the gap between the interpretability of decision trees and the advanced reasoning capabilities of large language models (LLMs) to predict startup success. Our approach leverages chain-of-thought prompting to generate detailed reasoning logs, which are subsequently distilled into structured, human-understandable logical rules. The pipeline integrates multiple enhancements - efficient data ingestion, a two-step refinement process, ensemble candidate sampling, simulated reinforcement learning scoring, and persistent memory - to ensure both stable decision-making and transparent output. Experimental evaluations on curated startup datasets demonstrate that our combined pipeline improves precision by 54% from 0.225 to 0.346 and accuracy by 50% from 0.46 to 0.70 compared to a standalone OpenAI o3 model. Notably, our model achieves over 2x the precision of a random classifier (16%). By combining state-of-the-art AI reasoning with explicit rule-based explanations, our method not only augments traditional decision-making processes but also facilitates expert intervention and continuous policy refinement. This work lays the foundation for the implementation of interpretable LLM-powered decision frameworks in high-stakes investment environments and other domains that require transparent and data-driven insights.

Reasoning-Based AI for Startup Evaluation (R.A.I.S.E.): A Memory-Augmented, Multi-Step Decision Framework

TL;DR

The paper tackles the need for explainable startup evaluation by integrating LLM-based reasoning with explicit rule-based decision policies. It presents R.A.I.S.E., a memory-augmented, multi-step framework that converts chain-of-thought reasoning into verifiable rules and constructs an editable decision policy. Through data ingestion, reasoning log generation, rule extraction, policy formation, and persistent memory, the approach achieves substantial gains in precision and accuracy, culminating in a fully combined pipeline that reaches around 0.700 accuracy on a balanced startup dataset. The work demonstrates that combining reasoning traces with structured rules yields transparent, high-performance predictions suitable for expert oversight and continuous policy refinement in high-stakes investment environments.

Abstract

We present a novel framework that bridges the gap between the interpretability of decision trees and the advanced reasoning capabilities of large language models (LLMs) to predict startup success. Our approach leverages chain-of-thought prompting to generate detailed reasoning logs, which are subsequently distilled into structured, human-understandable logical rules. The pipeline integrates multiple enhancements - efficient data ingestion, a two-step refinement process, ensemble candidate sampling, simulated reinforcement learning scoring, and persistent memory - to ensure both stable decision-making and transparent output. Experimental evaluations on curated startup datasets demonstrate that our combined pipeline improves precision by 54% from 0.225 to 0.346 and accuracy by 50% from 0.46 to 0.70 compared to a standalone OpenAI o3 model. Notably, our model achieves over 2x the precision of a random classifier (16%). By combining state-of-the-art AI reasoning with explicit rule-based explanations, our method not only augments traditional decision-making processes but also facilitates expert intervention and continuous policy refinement. This work lays the foundation for the implementation of interpretable LLM-powered decision frameworks in high-stakes investment environments and other domains that require transparent and data-driven insights.

Paper Structure

This paper contains 63 sections, 7 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methodology
  4. Founder Training Data Ingestion and Preprocessing
  5. LLM-Based Reasoning Log Generation
  6. Rule Extraction from Reasoning Logs
  7. Decision Policy Generation
  8. Start Up Success Prediction
  9. Evaluation and Metrics
  10. Results
  11. 4o vs o3
  12. o3 vs two-step refinement
  13. o3 vs simulated reinforcement learning (RL)
  14. o3 vs 3 picks
  15. o3 vs memory
  16. ...and 48 more sections

Figures (7)

  • Figure 1: Overview of the LLM-Powered Investment Decision Framework Pipeline.
  • Figure 2: Overview of the Two-Step Reasoning Process.
  • Figure 3: Overview of the Simulated Reinforcement Learning Module used to refine the quality of the model’s reasoning
  • Figure 4: Overview of the Ensemble Candidate Sampling Process
  • Figure 5: Overview of the Persistent Memory Integration, showing how dynamic summarisation maintains context across multiple interactions.
  • ...and 2 more figures