Assembling the heterogeneous elements for digital learning

Harnessing learning analytics to inform/improve learning and teaching

The following is an early attempt to formulate a method by which learning analytics could be used to make a “Golf GTI”. The context for this is an attempt to develop a grant application for an OLT grant.

Standing on the shoulders of giants

This work has to build on earlier work, especially work funded by the same/related organisations. The two most obvious examples of earlier work involved @shaned07 and many talented collaborators:

  1. Investigating the application of IT generated data as an indicator of learning and teaching performance (Dawson and McWilliam, 2008); and
  2. “Seeing” networks: visualising and evaluating student learning networks .

This work has made a range of findings and recommendations for further work. This proposal aims to build directly on that foundation.

The aim

The overall aim of this project is to further address the “lack of research regarding the application of academic analytics to inform the design, delivery and future evaluations of individual teaching practices” (Dawson, Heathcote, & Poole, 2010).

In achieving this, the project will:

  • Explore and validate established indicators of student learning performance (Dawson and McWilliam, 2008).
    This project will be based on work that is doing exactly this (which is in turn based on the work of others). The nature of this project, however, will almost certainly help identify new and interesting indicators and further test established indicators.
  • Investigate and supply what data, indicators, tools, and knowledge are required to better assist teaching staff to inform the design, delivery, and evaluation of individual teaching practice (Dawson, Bakharia, Lockyer, & Heathcote, 2011).
    “The transformation of user-data from analysis to informed pedagogical action is for the vast majority of academic teaching staff, a complex and potentially labour intensive process” (Dawson, Heathcote, & Poole, 2010). The project will draw heavily on existing work, but also seek to make new contributions.
  • Explore how and with what impact learning analytics tools and insights are used to inform pedagogical interventions.

One approach

Action research – a methodology generally adopted to generate and evaluate innovations within a particular context – seems an appropriate fit for a project aiming to explore the application of learning analytics to inform the design, delivery and evaluation of individual teaching practices. Hence one idea for this project would be to based it on a a range of cycles of action research.

The cycles – intended to be run in parallel across the institutions involved – would generally aim to

  • Be focused on a small group of academics (perhaps 5-6) responsible for teaching a range of courses.
    Given the difficulty involved in understanding the data and insights revealed by analytics and translating that into action, initial cycle(s) could be targeted at academics with appropriate content knowledge. For example, a group of academics from statistics, psychology and education who have a mix of knowledge that would help interpreting the data and recommending useful theories and approaches that might inform changes to learning.

    Arguably this might also increase the variety of ideas for expanding learning analytics, opportunities for further publication and research, and the chance that the academics from initial rounds could become part of the support team for subsequent cycles.

  • Include a small team from the project with expertise around learning analytics (perhaps including academic participants in prior AR cycles).
    The aim of this team is both to help the academics involved understand and explore what learning analytics insights and tools are currently available, but also to make aid in exploring and developing new insights and tools.
  • Draw upon existing tools, patterns and experience around learning analytics to identify areas of interest in each of the courses.
    Where possible these cycles would draw heavily an established foundation of learning analytics tools and indicators. One of the aims of this project is to learn more about how and with what impacts these existing tools and indicators have on learning and teaching.
  • Collaboratively identify areas and processes for intervention and research within those courses with analytics.
  • Implement those interventions.
  • Evaluate and reflect on the results.
  • Start again.

The idea is that the act of using analytics to make interventions in a range of real teaching contexts with a diversity of teaching staff and courses will reveal new and interesting insights.

Outcomes

After a year or two of these cycles, it is assumed that the outcomes of the project could include:

  • Insights into if, how and with what impact learning analytics insights and tools were used to inform the design, delivery and evaluations of individual teaching practices.
  • Validation of existing and identification of new predictors of student learning performance.
  • A range of tools, support resources, and recommendations to aid in the use of learning analytics to inform the design, delivery and evaluation of individual teaching practice.
  • Increase the use of these tools and insights to inform the design, delivery and evaluation of individual teaching practice.

Foundations of the approach

The following is a mixed-bag of theories and references that give a vague idea of the perspectives informing this project. There is no narrative in the following.

From this post

Key enabling factors for knowledge creation is knowledge sharing and integration [36,54]. Research in organizational learning has emphasized the value of practice; people acquire and share knowledge in socially situated work. Learning in the organization occurs in the interplay between tacit and explicit knowledge while it crosses boundaries of groups, departments, and organizations as people participate in work [17,54]. The process should be situated in shared practice with a joint, collective purpose [12,14,15].

and also this from Seely-Brown and Duiguid

We, by contrast, suggest that practice is central to understanding work. Abstractions detached from practice distort or obscure intricacies of that practice. Without a clear understanding of those intricacies and the role they play, the practice itself cannot be well understood, engendered (through training), or enhanced (through innovation).

Given this focus, it does not appear surprising when Green et al (2009) report that “many academic staff continue to employ inappropriate, teacher-centered, content focused strategies”.

there is a significant body of literature that establishes the conceptions of learning and teaching held by academics and links those conceptions to the quality of student learning outcomes (Kember and Kwan 2000; Biggs 2001; Trigwell 2001; Norton, Richardson et al. 2005; Eley 2006; Gonzalez 2009).

Trigwell (2001) suggests that focusing more holistically on the combination of elements – especially on the teachers’ conceptions of teaching and a focus on students – makes the differences between teaching qualities more discernible and judgements easier. A focus on the strategies and technologies used by a teacher ignores the influence that their conceptions can have on how such strategies and technologies are used. Approaches to staff development that focus on the provision of prescribed skills and teaching recipes result, in many cases, in participants querying the feasibility of presented methods, defending methods they are already using, using new methods mechanically, or modifying methods intended to facilitate student learning into didactic transmission modes (Gibbs 1995; Trigwell 1995). A focus on strategies also ignores the likelihood that contextual factors also influence the appropriateness and implementation of strategies and techniques. Even a teacher with a student-centred conception of learning will adopt alternate strategies if the context is not appropriate.

The relationship between conceptions of learning and teaching has implications for educational change (Tutty, Sheard et al. 2008). Change towards more sophisticated forms of teaching is only possible if the pedagogue’s conception of teaching are addressed first (Ho, Watkins et al. 2001). There is little evidence to show that pedagogue’s conceptions of teaching will develop with increasing teaching experience or from formal training (Richardson 2005). Pedagogue’s approaches to teaching change slowly, with some change coming after a sustained training process (Postareff, Lindblom-Ylanne et al. 1997). Given that it appears most university pedagogues hold content-centred conceptions of learning and teaching and that the majority of e-learning appears focused on distributing content, there appears to be a need to change the conceptions held by pedagogues.

Changing pedagogues’ conceptions of teaching, however, are a necessary but not sufficient condition for improved student learning. While pedagogue’s are likely to adopt teaching approaches that are consistent with their conceptions of teaching there may be differences between espoused theories and theories in use (Leveson 2004). While pedagogues may hold higher-level view of teaching other contextual factors may prevent use of those conceptions (Leveson 2004). Environmental, institutional, or other issues may impel pedagogues to teach in a way that is against their preferred approach (Samuelowicz and Bain 2001). While conceptions of teaching influence approaches to teaching, other factors such as institutional influence and the nature of students, curriculum and discipline may also influence teaching approaches (Kember and Kwan 2000). Prosser and Trigwell (1997) found that pedagogue’s with a student-focused approach were more likely to report that their departments valued teaching, that their class sizes were not too large, and that they had control over what was taught and how it was taught. Other contextual factors that frustrate pedagogues’ intended approaches to teaching may include senior staff with traditional teacher-focused conceptions raising issues about standards and curriculum coverage and students who induce teachers to adopt a more didactic approach (Richardson 2005)

Efforts to improve teaching have often failed because the complexity of teaching has been underestimated and such attempts should consider the integrated system of relationships that constitute the teaching experience as a whole (Leveson 2004). One such important complicating influence are differences that have found differences between discipline areas (Lindblom-Ylanne, Trigwell et al. 2006), which suggest a need to understand teaching from both a general and discipline-specific perspective (Leveson 2004). Beliefs about teaching vary markedly across different disciplines and these variations are related to the pedagogue’s beliefs about the naure of the discipline they are teaching (Richardson 2005).

Rhetorical claims espousing e-learning seek to appeal to a pedagogues’ vision with an emphasis on innovation at the expense of reflection on pedagogues’ thinking and practices (Convery 2009). The unrealistic expectations of e-learning inhibit pragmatic attempts by pedagogues to integrate technology into classroom contexts and contribute to pedagogues being blamed for the failure of technology to fulfill its promise (Convery 2009).

Any approach to one technology and pedagogy for all is pretty much doomed to flap and then crash (Salmon 2005). Apart from the above reasons there are also those associated with the technology. Technological artifacts often generate new, unforeseen behaviours that may deviate from initial intentions, it is likely that secondary changes in patterns and behaviours will occur that will not be predictable (Westera 2004). E-learning practice cannot remain static because e-learning pedagogies are evolving through the continual emergence of new modes of practice and enhanced technological tools (Nichols and Anderson 2005).

More thinking to consider

The above is influence by my perspectives on what works in terms of changing/improving learning and teaching within universities. I need to re-read some of my earlier writings, including the following

References

Dawson, S., & McWilliam, E. (2008). Investigating the application of IT generated data as an indicator of learning and teaching performance. Canberra: Australian Learning and Teaching Council. Retrieved from http://www.olt.gov.au/system/files/resources/grants_cg_report_itgenerated_qut_feb09.pdf

Dawson, S., Heathcote, L., & Poole, G. (2010). Harnessing ICT potential: The adoption and analysis of ICT systems for enhancing the student learning experience. International Journal of Educational Management, 24(2), 116-128. doi:10.1108/09513541011020936

Dawson, S., Bakharia, A., Lockyer, L., & Heathcote, E. (2011). “Seeing”Âť networks : visualising and evaluating student learning networks Final Report 2011. Main. Canberra. Retrieved from http://research.uow.edu.au/content/groups/public/@web/@learnnet/documents/doc/uow115678.pdf

css.php