Faculty: Voicu Popescu (CS) & Nicoletta Adamo (CGT)
Students: Jian Cui (CS); Suren Rajasekaran (CGT); Meng-Lin Wu (CS); Saikiran Anasingaraju (CGT)
Sponsor: NSF- Cyberlearning (2012-2014)
This two-year collaborative exploratory project researches, develops and evaluates a software system for creating interactive learning activities for K5 mathematics that are delivered by a teaching avatar that not only speaks, but also gestures, and writes and draws on a nearby virtual whiteboard. Gestures augment the delivery of the learning activity in three ways. Gestures help the student parse the speech of the teaching avatar, they help clarify the educational content, and they help convey the engaging personality of the teaching avatar. The project team comprises computer science, computer graphics technology, educational psychology, and psychology researchers.
A learning activity is defined through a script. The script specifies what the teaching avatar says, writes, and draws as the avatar interacts with students. Some avatar gestures are specified explicitly in the script, and some are inferred automatically using linguistics, educational content, and personality based gesturing rules. Once the necessary gestures are identified, the animation data is either retrieved from a database of existing gestures or it is computed on-the-fly using animation algorithms.
The system is intended to allow educators with only basic computer literacy and with no computer science or animation expertise to develop interactive digital learning activities for students. The scripting language has a small number of commands selected through drop-down lists in a specialized editor that does not require detailed knowledge of the scripting language. The system allows educators to extend the database of gestures through a graphical interface designed for non-expert users based on principles such as non-ambiguous selection in 2-D, manipulation of a single degree-of-freedom of the avatar at a time, and automatic view change to track the editing focus. The script editor, the gesture creation interface, and the system as a whole will be formatively evaluated with pre-service and in-service teachers over three iterations.
Using the system, the project team is developing a set of learning activities for studying mathematical equivalence. The learning activities will be used to formatively evaluate hypotheses about the use of gesture in computer-based instruction: whether the gesturing avatar is more appealing, whether the gesturing avatar promotes learning, whether the gesturing avatar promotes learner gesturing, whether introducing ideas in gesture first facilitates learning, and whether individualized gestures promote learning.