PhD focus - Music Learning, Computational Interactions, Improvisation and Viz

Encouraging Improvisation in Piano Learning Using Adaptive Visualisations and Spatiotemporal Models

  • The process of learning the piano for novices is usually difficult and time-consuming. Several approaches in augmented reality such as piano-roll visualizations have been explored but have not garnered enough success and adoption. These piano roll prototypes have introduced several features and modules that assist novices on aspects such as sight-reading, timing and many others. However, improvisation, the act of allowing the piano user to incorporate their personal touch into their performance, and personalised learning have not been much explored in this domain. In this PhD, we are going to explore how we can encourage piano learners to improvise with the use of adaptive piano roll visualisations. Specifically, we are going to investigate how heuristics defined by experts and spatiotemporal models can be used to design visualisations that motivate and encourage learners based on their personalised learning patterns. Using these models and inputs, we will design and build a piano roll training system integrated with adaptive visualisations that serve as an intervention helping learners. We will evaluate and compare these visualisations in various user studies where they get to play piano pieces and develop their improvisation skills. We intend to uncover whether these adaptive visualisations will be helpful in the overall training of piano learners. Additionally, we wish to explore whether these adaptive visualisations will allow us to discover affordances that can potentially improve piano learning in general. Our research is guided by the main research question: How can we encourage piano learners to improvise with the help of adaptive visualisations based on spatiotemporal models and expert heuristics? UMAP 2021, MobileHCI 2021

  • Specifically, we are guided by the following research questions:

    • RQ0: What other technological interventions have been introduced to support piano learning?
    • RQ1: How to build multi-target pointing models to predict user errors while using the piano roll?
    • RQ2: How to improve the learner performance, user experience and sound quality of novices when learning the piano using adaptive visualisations designed from spatiotemporal and/or heuristic models?
    • RQ3: How do novices learn improvisation using adaptive visualisations designed from spatiotemporal and/or heuristic models?

You may view my research statement here UMAP DC 2021 and here MobileHCI DC 2021.

Here is the poster that I will be using for the DC track of MobileHCI 2021.

As of the moment, we are trying to build an open source version of PIANO 2.0 (inspired from PIANO by Rogers et al. It’s current schema can be viewed here.

UPDATE! 05/07/2021 - My improved research statement has been accepted as part of the Doctoral Consortium track of ACM International Conference on Mobile Human-Computer Interaction (MobileHCI 2021).

UPDATE! 31/03/2021 - My research statement has been accepted as part of the Doctoral Consortium track of ACM 29th Conference on User Modeling, Adaptation and Personalization (UMAP 2021).