If learning is combined simultaneously with movements, learning success demonstrably increases. Salzburg Research developed a method to test the correct performance of gesture-assisted learning automatically and without special hardware.
Challenge
If a person simultaneously performs one or more successive body gestures when learning a new piece of information, this has a positive effect on learning success. When training the content, the same gesture must be performed repeatedly in order to establish the link in the brain. Especially in distance learning, however, the correct execution of the gesture cannot be checked with conventional means, or only in a very rudimentary way.
Solution
Salzburg Research was commissioned to develop a virtual learning platform for gesture-aided learning. In a web interface, course material can be combined with previously recorded gestures. These gestures are recorded with acceleration sensors and transferred to an avatar. Learners then see the content together with this avatar, which demonstrates the gestures that match the content.
The learners also perform these gestures themselves. From the camera image of a commercially available webcam, the system automatically calculates 2D skeleton models from the movements with distance measures compared to the recorded gestures. The user immediately gets feedback for the performed gestures.
Benefits
The gesture-aided learning platform developed by Salzburg Research is characterised by its user-friendliness and diverse areas of application: It can be used for learning various languages, gestures for remote control of machines, and many more.
No specialised hardware is required for the platform developed by Salzburg Research. The users only need a standard webcam. This means that they are not tied to devices, people or places, which is especially important in times of virtual learning.
More information:
