Engaging with Children's Artwork in Mixed Visual-Ability Families

Abstract

We present two studies exploring how blind or low-vision (BLV) family members engage with their sighted children's artwork, strategies to support understanding and interpretation, and the potential role of technology, such as AI, therein. Our first study involved 14 BLV individuals, and the second included five groups of BLV individuals with their children. Through semi-structured interviews with AI descriptions of children's artwork and multi-sensory design probes, we found that BLV family members value artwork engagement as a bonding opportunity, preferring the child's storytelling and interpretation over other nonvisual representations. Additionally, despite some inaccuracies, BLV family members felt that AI-generated descriptions could facilitate dialogue with their children and aid self-guided art discovery. We close with specific design considerations for supporting artwork engagement in mixed visual-ability families, including enabling artwork access through various methods, supporting children's corrections of AI output, and distinctions in context vs. content and interpretation vs. description of children's artwork.

Figures (9)

• Figure 1: P8's great-granddaughter's drawing of Love Monster with AI-generated descriptions. Both Be My AI and ChatGPT4 begin with a declaration "a child's drawing of a creature" but also intermix varying levels of interpretation ("whimsical", "cheerful", "resembles a collar or perhaps a bow"). Though some descriptions are accurate---its color, smile, and the background--others, such as the wide open eyes, or the hearts being a necklace, are incorrect.
• Figure 2: P5's son's sketch of a Centaur-like figure with two creatures at the bottom with the AI-generated descriptions. Again, both AI engines intermix descriptive and interpretative elements. They begin with the type of drawing: "hand-drawn sketch" before describing the creature, its posture, and even its expression. For the surrounding animals, the AI-generated descriptions use suggestive language: "resemble a mix of cats" and "interpreted as fauns, imps, ...". ChatGPT4 outputs more presumptive interpretation and context: "This character has a relaxed or confident posture" and "casual or impromptu creation, possibly doodled in a classroom".
• Figure 3: P10's daughter's bulldog art with AI-generated descriptions. Both descriptions begin with a declaration "a bulldog's face", then diverge in descriptive vs. interpretive language. Be My AI describes the bulldog's eyes as "wide and round... looking directly at the viewer", whereas ChatGPT4 makes inferences about the bulldog's state of being from its eyes: "round and expressive, conveying a sense of attentiveness and curiosity." ChatGPT4 also infers the technique used as: "stippling or pointillism."
• Figure 4: Drawings by P6's son (left) and P7's daughter (right). The children's descriptions emphasized both context (inspired by a YouTube channel; made for a school art event) and content (flag-themed balls; flower with seeds and big purple petals).
• Figure 5: P6's son's coloring of a cartoon turkey with two AI-generated descriptions. Both begin with very different inferences (or lack thereof) about the turkey: Be My AI calls the turkey a "cat", which bothered P6's son, while ChatGPT4 describes it as "an anstract figure with prominent circular eyes...". Both tools again intermix descriptive and interpretive traits---"surprised or curious", "stained glass effect", and "playful and whimsical".