Automatic Generation of Examinations in the Automatic Control Courses

TL;DR

A decision support Matlab-LATEX toolkit based on random number generators for selection of examination problems is described in this report to facilitate the alignment.

Abstract

Final written examination is the most important part of summative assessment in automatic control courses. Preparation of the examinations with a given number of points according to the concept of Constructive Alignment (which could be the main concept in future automatic control education) takes significant amount of time of the educator and motivates development of a toolkit for automatic compilation of examination problems. A decision support Matlab-LATEX toolkit based on random number generators for selection of examination problems is described in this report to facilitate the alignment. The toolkit allows application of Stepwise Constructive Alignment (a new method described in this report), where the alignment is achieved by a number of software runs associated with random trials. In each step the educator manually selects suitable problems before each run based on evaluation of the random choice from the previous run. Automatic generation of the examination together with solutions for the course 'Process control and measurement techniques' is presented as an example.

Figures (2)

• Figure 1: The flow chart for automatic generation of examination. This system consists of two large parts. The first part is written in Matlab and is associated with the database, which contains examination problems and solutions, learning outcomes, SOLO model, indices of difficulty and dates of previous examinations. The educator specifies the number of points and the number of problems in each area to be included in the examination. The educator may select several problems manually for CA. The Matlab program creates the LaTeX file by merging items of randomly selected problems in the subareas. The merged LaTeX file can be reviewed in WinEdt editor for example, where the educator gets opportunity to edit the examination and solutions. The LaTeX source file is compiled by Miktex which produces the pdf output with examination and solutions.
• Figure 2: Flow chart for stepwise constructive alignment. The alignment starts with selection of an initial decision vector, where the educator enters any manually chosen problems (M) and indicates subareas where the problems should be chosen randomly (R). For example, the problem which has the number $M$ is chosen manually for the second subarea of the decision vector, i.e. $[R~M~R~R~R~R~R~R]$, and all other problems in the seven other subareas are chosen randomly. Educator runs the program, reviews information provided by the system in terms of ILOs, SOLO levels, indices of difficulty etc., wace24, evaluates compiled pdf output file and makes the decision about the inclusion of the problems in examination by updating the decision vector. The educator runs the program again with the same decision vector, if suitable problems could not be found in the trial. The desired number of points is guaranteed by the system for each run and the system automatically rejects the problems which have been used recently. After a number of reruns the educator should be able to find the best possible solution for CA with appropriate indices of difficulty. The alignment is performed in a stepwise way where each step is associated with an update of the decision vector.