Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explain in Plain Language Questions with Indic Languages: Drawbacks, Affordances, and Opportunities

David H. Smith, Viraj Kumar, Paul Denny

TL;DR

This work evaluates the efficacy of a recently introduced approach called Code Generation Based Grading (CGBG) in enabling language agnostic ``Explain in Plain Language'' (EiPL) activities and initially evaluates the correctness of code generated from correct EiPL responses provided in 10 of India's most commonly spoken languages.

Abstract

Background: Introductory computer science courses use ``Explain in Plain English'' (EiPE) activities to develop and assess students' code comprehension skills, but creating effective autograders for these questions is challenging and limited to English. This is a particular challenge in linguistically diverse countries like India where students may have limited proficiency in English. Methods: We evaluate the efficacy of a recently introduced approach called Code Generation Based Grading (CGBG) in enabling language agnostic ``Explain in Plain Language'' (EiPL) activities. Here students' EiPL responses generate code that is tested for functional equivalence to the original which was being described. Objectives: We initially evaluate the correctness of code generated from correct EiPL responses provided in 10 of India's most commonly spoken languages. To evaluate the effectiveness of the approach in practice, we assess student success and perceptions of EiPL questions in a NPTEL (National Programme on Technology Enhanced Learning) course. Results: We find promising results for the correctness of code generated from translations of correct EiPL responses, with most languages achieving a correctness rate of 75% or higher. However, in practice, many students preferred to respond in English due to greater familiarity with English as a technical language, difficulties writing in their native language, and perceptions of the grader being less capable of generating code from prompts in their mother tongue.

Explain in Plain Language Questions with Indic Languages: Drawbacks, Affordances, and Opportunities

TL;DR

This work evaluates the efficacy of a recently introduced approach called Code Generation Based Grading (CGBG) in enabling language agnostic ``Explain in Plain Language'' (EiPL) activities and initially evaluates the correctness of code generated from correct EiPL responses provided in 10 of India's most commonly spoken languages.

Abstract

Background: Introductory computer science courses use ``Explain in Plain English'' (EiPE) activities to develop and assess students' code comprehension skills, but creating effective autograders for these questions is challenging and limited to English. This is a particular challenge in linguistically diverse countries like India where students may have limited proficiency in English. Methods: We evaluate the efficacy of a recently introduced approach called Code Generation Based Grading (CGBG) in enabling language agnostic ``Explain in Plain Language'' (EiPL) activities. Here students' EiPL responses generate code that is tested for functional equivalence to the original which was being described. Objectives: We initially evaluate the correctness of code generated from correct EiPL responses provided in 10 of India's most commonly spoken languages. To evaluate the effectiveness of the approach in practice, we assess student success and perceptions of EiPL questions in a NPTEL (National Programme on Technology Enhanced Learning) course. Results: We find promising results for the correctness of code generated from translations of correct EiPL responses, with most languages achieving a correctness rate of 75% or higher. However, in practice, many students preferred to respond in English due to greater familiarity with English as a technical language, difficulties writing in their native language, and perceptions of the grader being less capable of generating code from prompts in their mother tongue.
Paper Structure (18 sections, 4 figures, 1 table)

This paper contains 18 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Code Reading and Comprehension Activities
  4. Explain in Plain Language Questions
  5. Language Diversity in India and Computer Science Education
  6. Methods
  7. Evaluation of Expert Translations
  8. Evaluating EiPL in an Online Course
  9. RQ1: Success of Expert Translations
  10. RQ2: Students' Interactions with and Perceptions of EiPL Questions
  11. Student Prompts for EiPL Activites
  12. Perceptions of Prompting in English vs Mother Tongue
  13. Students' greater familiarity with English as a technical language
  14. Difficulties writing in mother tongue
  15. Perceptions of the model as less capable in mother tongue
  16. ...and 3 more sections

Figures (4)

  • Figure 1: The percentage of each state's population which speaks the given language as their mother tongue as reported by the 2011 census for the top 10 most spoken languages of India's 22 scheduled languages. It is important to note that this census predates the separation of the states of Ladakh and Telangana from the states of Jammu-Kashmir and Andhra Pradesh, respectively. As such, it may not be fully representative of the languages spoken in those states today.
  • Figure 2: An example of an EiPL question hosted on PrairieLearn.
  • Figure 3: Languages spoken by students who participated in the activities (\ref{['fig:languages']}) and their proficiency in reading and writing in both their mother-tongue as well as English (\ref{['fig:lang_fluency']}).
  • Figure 4: Students' perceptions of the accuracy of the EiPL questions when responding in their mother tongue versus English (\ref{['fig:lang_comp']}) and their preference for responding in English or their mother tongue (\ref{['fig:lang_pref']}).