A Picture Is Worth a Thousand Words: Exploring Diagram and Video-Based OOP Exercises to Counter LLM Over-Reliance

TL;DR

This work addresses LLM over-reliance in introductory OOP by introducing a novel diagram- and video-based notation to specify OO tasks, enabling design-centric problem solving that discourages copy-paste prompts. The authors evaluate the approach in a semester-long OO design course, presenting five diagram types (Algorithmic function, State-change function, Class declaration, Inheritance, State transition rules) plus video demonstrations to express required behaviors. Survey results (n=56) indicate positive student reception, with videos improving interpretability and motivation, and the notation reducing reliance on LLMs for code generation, while ad-hoc vision-model tests show current GPT-4 and Bard struggle with diagram-based tasks. The work demonstrates that visual specifications can counteract LLM misuse and foster deeper computational thinking, suggesting directions for broader adoption and further study of design-oriented, LLM-resistant pedagogy in OOP education.

Abstract

Much research has highlighted the impressive capabilities of large language models (LLMs), like GPT and Bard, for solving introductory programming exercises. Recent work has shown that LLMs can effectively solve a range of more complex object-oriented programming (OOP) exercises with text-based specifications. This raises concerns about academic integrity, as students might use these models to complete assignments unethically, neglecting the development of important skills such as program design, problem-solving, and computational thinking. To address this, we propose an innovative approach to formulating OOP tasks using diagrams and videos, as a way to foster problem-solving and deter students from a copy-and-prompt approach in OOP courses. We introduce a novel notation system for specifying OOP assignments, encompassing structural and behavioral requirements, and assess its use in a classroom setting over a semester. Student perceptions of this approach are explored through a survey (n=56). Generally, students responded positively to diagrams and videos, with video-based projects being better received than diagram-based exercises. This notation appears to have several benefits, with students investing more effort in understanding the diagrams and feeling more motivated to engage with the video-based projects. Furthermore, students reported being less inclined to rely on LLM-based code generation tools for these diagram and video-based exercises. Experiments with GPT-4 and Bard's vision abilities revealed that they currently fall short in interpreting these diagrams to generate accurate code solutions.

Figures (9)

• Figure 1: Example algorithmic exercise. Students should understand that the function must create an array with elements from the two input arrays, in alternating fashion.
• Figure 2: State-change function diagram for the withdraw function. It changes the 'Account' object's state and returns true if the operation succeeded, or false otherwise.
• Figure 3: Class declaration exercises. Students should infer that the 'Person' class has a composition relationship with the 'Apartment' class. Some data-types were omitted, since students should be able to derive them (e.g. name should be a 'String').
• Figure 4: Example inheritance exercise. Students were expected to understand that some common elements exist between the 'Manager' and the 'IT Technician' and thus create an inheritance relationship with a super-class above those.
• Figure 5: Example state-change function exercise. Students were expected to understand that the function would receive 3 objects (a 'Company', an 'Employee', and a 'Task') and associate them if some rules were respected. The rules were described in another diagram which can be seen in Figure \ref{['fig:diagram-task-state-transitions']}.