Evaluation of DIVA Performers

We have a unique opportunity to perform various types of evaluation of highly skilled performers on a new type of human computer interface. Over the next three years, people who have never used the device will spend enough time to create expressive performances with the DIVA. We can study the learning process as well as the performance aspects themselves.

We are actively looking for graduate students and undergraduates who would like to participate in the activities below. Please contact us if you are interested in getting involved.

We plan the following areas of study:

Skill Acquisition

We record and analyze the performer in order to characterize the skill-acquisition/learning process, to evaluate the gesture control aspects of the system and their impact on concurrent production of vocal and gestural output, and to validate the perceptual coherence of the various audible (natural and synthetic voices) and visible (natural hands and face, synthetic face) components of the production. These results along with user evaluations are used to fine-tune the system and set critical thresholds for the second phase of DIVA refinement.

As this project provides the motivation for a performer to become an expert with a new human-computer interface. Thus, one of the unique aspects of this project is that performers start as novices and invest the many hours needed to become expert. Documenting this process over three years and the impact of the design of the interface on the process and vice-versa, provide valuable contribution to understanding expert performance and HCI design.

Spatio-temporal coherence

The audiovisual behavior of our skilled performer are recorded during acquisition and performance. The spatial and temporal correspondences between hand gestures and combinations of the natural and synthesized visible motions (face and head) and the speech acoustics are assessed and compared (e.g., kinematic validation through comparison of the synthetic and natural head and face). Recorded data provides stimuli for perceptual evaluations to test the intelligibility, expressiveness, and naturalness of the gesturally synthesized output. The results of these studies establish practical methods for our final DIVA design.

Behaviour Studies of Performance

Once we have three speaker/singers, we can conduct large behavioral studies of the performers and their performances. Production studies examine whether ensemble performance facilitates the coordination of the vocal and hand-gestured behavior of a given performer into one bi-modal entity, or whether each modality attains independence. Perceptual testing can determine what effects the auditory and visual sensory modalities, alone and in combination, have on perceived coordination of the two modes of performance. These studies set the stage for more complex questions about human ability to generate multiple streams of highly-skilled artistic behavior, whatever the degree of coordination, in multiple neuromotor domains. Also, in the context of interactive performance we investigate when both vocal and hand gestures are active, will one mode be more capable of adaptation to sudden or even longer term (i.e., learned) demands of the performance? Performer evaluations of selected performances are compared to the perceptions of non-performers (audience) to determine, among other things, the role of motor participation in attentional acuity.