How Could AI Support Design Education? A Study Across Fields Fuels Situating Analytics

TL;DR

This work investigates how AI can support design education by grounding analytics in the situated practices of design instructors. It employs a qualitative, co-design methodology with 11 instructors across five fields to derive a suite of AI-ready, learning-objectives-based design creativity analytics and a paradigm of situating analytics, delivered through dashboards integrated with design environments. Key findings reveal that rubrics function via the family resemblance principle, that assessing individual contribution in teams is challenging, and that students often fail to act on feedback without better tracking. The authors propose a framework of situating analytics to provide transparent, controllable, on-demand feedback that complements human instruction, offering both theoretical and practical guidance for building AI that supports design education without supplanting educators. This work advances human-centered AI by connecting analytics to lived teaching practices and outlining a path toward algorithm-in-the-loop design education tools.

Abstract

We use the process and findings from a case study of design educators' practices of assessment and feedback to fuel theorizing about how to make AI useful in service of human experience. We build on Suchman's theory of situated actions. We perform a qualitative study of 11 educators in 5 fields, who teach design processes situated in project-based learning contexts. Through qualitative data gathering and analysis, we derive codes: design process; assessment and feedback challenges; and computational support. We twice invoke creative cognition's family resemblance principle. First, to explain how design instructors already use assessment rubrics and second, to explain the analogous role for design creativity analytics: no particular trait is necessary or sufficient; each only tends to indicate good design work. Human teachers remain essential. We develop a set of situated design creativity analytics--Fluency, Flexibility, Visual Consistency, Multiscale Organization, and Legible Contrast--to support instructors' efforts, by providing on-demand, learning objectives-based assessment and feedback to students. We theorize a methodology, which we call situating analytics, firstly because making AI support living human activity depends on aligning what analytics measure with situated practices. Further, we realize that analytics can become most significant to users by situating them through interfaces that integrate them into the material contexts of their use. Here, this means situating design creativity analytics into actual design environments. Through the case study, we identify situating analytics as a methodology for explaining analytics to users, because the iterative process of alignment with practice has the potential to enable data scientists to derive analytics that make sense as part of and support situated human experiences.

Figures (3)

• Figure 1: D6's feedback on a student's design through redlining. D6 marks up the problem areas and draws arrows connecting feedback written in respective boxes. The student receives feedback of using too much white space on the left and right sides, title taking more than needed space, text not styled per the instructor's guidelines, dominating background and small building size in the lower section, and an expected human figure.
• Figure 2: Diagram of a situating analytics approach to transparent design assessment and feedback. Transparent, learning objectives-based design creativity analytics— based on building with contextual properties derived through understanding practices— are presented via interactive dashboards to support teaching and learning, and simultaneously, to gather feedback for validating and refining analytics. (Use of icons under Creative Commons license. See attribution noun_project_icons.)
• Figure 3: A mockup of dashboard integration with design learning environments. Design analytics dashboards are indexical: they present analytics that refer to and can be understood more effectively within the context of actual design work. For a quick overview, activating an analytic can present its underlying basis, right there on the dashboard. To aid comprehensibility, on activating the presented information, dashboards can support highlighting/redlining the elements— that form the basis for analytic computation— within the actual design work. To aid controllability, the information presented within dashboards and learning environments can include affordances for users to challenge the assessment and rationale behind it. This mock-up shows clusters (in orange) used as the basis for Visual Consistency analytics and inconsistencies identified (in red). Users can challenge assessment (activating cross symbols) and input rationale (in the presented text box). See attribution for 'NASA Technology' design example adobeSparkTemplate.