Adaptive Visualisations Using Spatiotemporal and Heuristic Models to Support Piano Learning
- Learning the piano is hard and many approaches including piano-roll visualisations have been explored in order to support novices in this process. However, existing piano roll prototypes have not considered the spatiotemporal component (user’s ability to press on a moving target) when generating these visualisations and user modelling. In this PhD, we are going to look into two different approaches: (i) exploring whether existing techniques in single-target spatiotemporal modelling can be adapted to a multi-target scenario such as when learners use several fingers to press multiple moving targets when playing the piano, and (ii) exploring heuristics by experts marking various difficult parts of songs, and deciding on specific interventions needed for marked parts. Using models and input from the experts we will design and build an adaptive piano roll training system to better support piano learners. We will evaluate and compare these models in various user studies involving novices trying to play piano pieces and develop their improvisation skills. We intend to uncover whether these adaptive visualisations will be helpful in the overall training of piano learners. Additionally, these models and adaptive visualisations will allow us to discover affordances that can potentially improve piano learning in general.
- Specifically, our research is guided by the following research questions:
- RQ0: What other technological interventions have been introduced to support piano learning?
- RQ1: Can we build multi-target pointing models to predict user errors while using the piano roll?
- RQ2: Can we better support learners using adaptive visualisations designed from spatiotemporal and/or heuristic models?
- RQ3: How do novices learn in AR piano under different learning conditions (no visualisation vs non adaptive visualisation vs adaptive visualisation)?
- RQ4: Can we support learners in developing their improvisation skills using adaptive visualisations?
You may view my full research statement here.
UPDATE! 31/03/2021 - My research statement has been accepted as part of the Doctoral Consortium track of ACM 29th Conference on User Modeling, Adaptation and Personalization (UMAP 2021).