What follows will eventually be a summary of my contribution to a the ASCILITE'2017 panel titled Are learning analytics leading us towards a utopian or dystopian future, and what can we as practitioners do to influence this?. Below you'll find a summary of my prediction and the argument that underpins it, suggestions for more reading, the slides, and references....more...
The following provides a collection of information and resources associated with a paper and presentation given at ALASI 2017 - the Australian Learning Analytics Summer Institute in Brisbane on 30 November, 2017. Below you'll find an abstract, a recording of a version of the presentation, the presentation slides and the references....more...
The following post contains the content from a poster designed for the 2017 USQ Toowoomba L&T celebration event. It provides some rationale for a technology demonstrator at USQ based on the Moodle Activity Viewer....more...
A few of us recently submitted a paper to ALASI'2017 that examined a "case study" of a teacher (me) engaging in a bit of DIY learning analytics. The case was used to drawing a few tentative conclusions and questions around the institutional implementation of learning analytics. The main conclusion is that teacher DIY learning analytics is largely ignored at the institutional level and that there appears to be a need and value to support it. The question is how (and then if supported, what happens)?...more...
In which I play with some analytics and use some literature in an attempt to understand why the institutional implementation of learning analytics as a starvation problem (like most institutional attempts to leverage digital technologies). In this context, I'm using the definition of starvation from computer science....more...
"failure" (CC BY 2.0) by tinou bao...more...
Last year I started using with Perl to play with analytics around Moodle Book usage. This year, @beerc and I have been starting to play with Jupyter Notebooks and Python to play with analytics for meso-level practitioners (Hannon, 2013). Plotly provides a fairly useful platform for generating graphs of various types and sharing the data. Works well with a range of languages and Jupyter Notebooks....more...
My current institution is - like most other universities - attempting to make some use of learning analytics. The following uses a model of system conditions for sustainable uptake of learning analytics from Colvin et al (2016) to think about how/if those attempts might be enhanced. This is done by...more...
So the indicators notebooks/platform is on github. The one and only bit of analysis is almost completely useless and still requires a fair bit of set up code. The aims in this post are...more...
Following on from the last post the following documents how to share the "indicators platform" for analytics via github. It's largely intended to help @beerc. I doubt there's nothing (at the moment) that makes this inherently interesting for anyone else....more...
The last post documented early explorations of Jupyter notebooks ending with a simple query of a Moodle database. This post takes the first baby steps toward some sort of indicators platform using Jupyter notebooks, Python and github. The focus here is to find some form of ORM or other form of database independent layer....more...
This is the third in a series of posts documenting "thinking" and progress around the next step of some bricolage with learning analytics and attempts to make some progress with the Indicators project....more...
This is the 2nd in 3 posts thinking about learning analytics and how we might engage with it better. The first rambled on about reproducible research and hunted for what we might do....more...
There are moves afoot to revisit some of the earlier research arising from Indicators project and hopefully move beyond. To do that well we need to give some thought to updating the methods we use to analyse, share and report what we're doing. In particular, because we're now a cross-institutional project. The following captures some early exploration of this problem....more...
A small group of teacher educators that I work with are starting to explore some research ideas around engagement, initial teacher education, and in particular the questions that arise out of the Quality Indicators for Learning and Teaching (QILT)...more...
"More student details" (see image below) is the most visible part of the additional systems I've put in place to make teaching bearable (this presentation talks more about it). The problem I'm dealing with here is that a new semester has started, which means I need to re-configure these feral systems to work with the new semester....more...
I started playing around with what became learning analytics in 2007 or so. Since then every/any time "learning analytics" is mentioned in a university there's almost an automatic mention of dashboards. So much so I was lead to tweet....more...
The Moodlemoot'AU 2015 conference is running working groups one of which is looking at assessment analytics. In essence, trying to think about what can be done in the Moodle LMS code to enhance assessment....more...
The following documents some initial exploration into why, if, and how sentiment analysis might be added to the BIM module for Moodle. BIM is a tool that helps manage and mirror blog posts from individual student blogs. Sentiment analysis is an application of algorithms to identify the sentiment/emotions/polarity of a person/author through their writing and other artefacts. The theory is that sentiment analysis can alert a teacher if a student has written something that is deemed sad, worried, or confused; but also happy, confident etc....more...
The following is a summary and ad hoc thoughts on Macfadyen et al (2014)....more...
The following is a place holder for two presentations that are related. They are:...more...
A couple of weeks ago @kateMfD wrote a post asking "What next for the LMS?". (one of a raft of LMS posts doing the rounds recently). Kate's point was in part that...more...
Paper accepted to ASCILITE'2014 and nominated for best paper....more...
The institutional LMS seems to be having some problems, so I'll post this instead....more...
The following is a summary and probably some thoughts on...more...
In short, there is definitely a pattern. In fact, there are two patterns evident:...more...
I'm currently analysing the structure of a course I teach and have been using @damoclarky's Moodle Activity Viewer to help with that. In the process, I've discovered that I'm an idiot in having missed the much more interesting and useful application of MAV than what I've mentioned previously. The following explains (at least one example of) how I'm an idiot and how MAV can help provide a type of process analytics as defined by Lockyer et al (2013)....more...
The following is a summary of McNaught et al (2009). This is one of three papers that from the same institution around the LMS that I've looked at recently....more...
In 2014 I am hoping to make some changes to BIM that will enhance the course I'll be teaching. The hope is to leverage various learning analytics to enhance student learning. The following is an attempt to use the IRAC framework to think about what might be done. Essentially a bit of brainstorming about possible future development....more...
The following is a draft version of the presentation I'll be giving at ASCILITE tomorrow. Hopefully this will go close to fitting within the 12 minute time frame....more...
The following is a summary and some thoughts on Lam et al (2010). It's a paper from the same authors/research from which I summarised an earlier paper....more...
One of the on-going research projects we have underway (really just starting up) is an analysis of the learning analytics literature. The following is an ad hoc record of a search into the literature around different approaches to literature analysis. The aim is to further inform the work. Essentially a summary of some readings....more...
The following is a summary and some thoughts on Lam et al (2012). The abstract from the chapter is...more...
The following is a summary of an exploration of the recent literature analysing University LMS usage and some thinking about further research....more...
The following is a summary and some thinking around...more...
The following is a summary and some thoughts on...more...
With the vague idea of the IRAC framework done, it's time to take the next step, i.e. "to use the framework to analyse the extant learning analytics literature". The following is some initial thinking behind why, what and how we're thinking of doing....more...
The following is the final version of a short paper that's been accepted for ASCILITE'2013. It's our first attempt to formulate and present the IRAC framework for analysing and designing learning analytics applications. This presentation from last week expands on the IRAC framework a little and touches on some of the future work....more...
The following is a "webified" version of an invited presentation at the Blended Learning 2013 Summit. This version is largely an experiment with this medium/approach to sharing the talk. The slides from the talk are shown below with a written version of what I might have hoped to have said during the talk....more...
And now for the second MOOC of the year - Data Mining with Weka from the University of Waikato. Weka is an open source data mining application. Time to find out a bit more about one of the technical aspects of learning analytics. Hoping this might be useful in terms of research with The Indicators Project (we should really update that site)....more...
Learning analytics is one of the new management fashions sweeping higher education. As a consequence, every Australian University I know has some sort of project around learning analytics underway. Some of these projects are actually considering how to help the folk at the coal face - students and teaching staff - use and benefit from learning analytics....more...
The following is an early description of work arising out of The Indicators Project an ongoing attempt to think about learning analytics. With IRAC (Information, Representation, Affordances and Change) Colin Beer, Damien Clark and I are trying to develop a set of questions that can guide the use of learning analytics to improve learning and teaching. The following briefly describes:...more...
Sadly, but not surprisingly, I missed out on an invite to LASI 2013. But I am able to follow the conversation via other means such as the LASI blog aggregator. Via the aggregator I've come across this post from Mike Tissenbaum reflecting on the opening sessions....more...
Came across Perrotta et al (2013) in my morning random ramblings through my PLN and was particular struck by this...more...
The following is a summary and some thoughts on the following paper...more...
It seems to be the day for a backlash against learning analytics or its parent big data. This morning my PLN has filtered to the top Taleb's "Beware the big errors of 'big data'" and Why big data is not truth. Not that surprising to me given that I've argued that learning analytics in Universities has all hallmarks of yet another fad....more...
Was skimming through Mark Drechsler's slide deck from THETA 2013 when I came across the following slide....more...
The following describes the process and results of using Gephi to generate some visualisations of the inter-connections between the blogs of students in the course I'm teaching. The process is heavily informed by the work of Tony Hirst....more...
The following is a summary of and commentary on Greller and Drachsler (2012). I come to this via the JISC/CETIS report I summarised yesterday...more...
The following is a summary and some reflection upon a CETIS report (Harmelen and Workman, 2012) titled Analytics for Learning and Teaching. One of the reasons for reading this is the current considerations around reworking and resubmitting a unsuccessful OLT application around learning analytics....more...
I'm currently thinking about a potential contribution to the SoLAR Southern Flare Conference on Learning Analytics in about a month or so. The early shape of that contribution is online and my last few posts have been summarising some explorations through various areas of the literature. Not the learning analytics literature, but broader literature on which much of the learning analytics literature should be based, but of which much isn't....more...
The following is a summary of / reflection upon...more...
The following has some reflection/questions generated while reading...more...
Had some time this morning to read Analytics in Higher Education: Benefits, Barriers, Progress and Recommendations from the EDUCAUSE Centre for Applied Research. It's a report on the results of a survey and seven focus groups (involving IT and Institutional Research - IR - people from universities) attempt to "address the topics of defining analytics, identifying challenges, and proposing solutions" (Bischel, 2012, p. 5). The following uses the quotes from this report that really struck me to identify and suggest what might be the core problem I have with learning analytics. The assumption of rationality....more...
Thanks to @cj13 for the heads up about the EDUCAUSE analytics sprint in the midst of moving, conferences, end/start of term and grant writing I missed it. Found it interesting that the first thing the struck my eye was a link to this discussion titled "Faculty need how-to information for the data they do have". It's interesting because the grant application I'm writing is directly aimed in this general area. Though we perhaps have a slightly different take on the problem....more...
Another day and another #ascilite12 paper to think about. This is the one where my periodic engagement is potentially driving my co-author slightly crazy. I'm sure this contribution will add further to that....more...
The following is an attempt to build a bit more on an earlier idea around the use of learning analytics. It's an attempt to frame a different approach to the use of learning analytics and to share these ideas in preparation for a potential project....more...
The following is an early attempt to formulate a method by which learning analytics could be used to make a "Golf GTI". The context for this is an attempt to develop a grant application for an OLT grant....more...
Or, an attempt to share some thinking about the idea behind an - almost obligatory - application for external funding....more...
The following is a slightly edited version of a internal grant application that has just been funded. It is a small step towards actually moving the Indicators project a little further toward the targets we talked about so many years ago. Â My co-investigators on this grant are: Colin Beer and Professor Patrick Danaher....more...
This article from Inside Higher Ed reports on some initial findings from the Gates Foundation data mining project in the US. The key finding reported in the article is...more...
Following on from recent posts around learning analytics, this post is going to try and drill down a bit further on one of the useful questions around analytics identified by Shelia MacNeil's summary of a Gardner Campbell talk. The questions...more...
Continuing a re-engagement with analytics I spent some time listening to Gardner Campbell's talk to the LAK'12 MOOC - Here I Stand and from there followed various links....more...
It looks like we're going to start playing around with the Indicators project again. So it's time to start reading up a bit on the learning analytics literature and see where the interesting work is. First up is...more...
Am trying to slowly get back into the learning analytics literature as part of writing a paper. The following is an ad hoc collection of comments/reflections on a few learning analytics papers....more...
Just over two years ago some colleagues and I wrote up some initial exploration of the logs from the different Learning Management Systems (LMS) used by a university. The initial explorations included:...more...
In the following I ruminate on a possible correlation between academic participation in a course and response rates on end of term student evaluation. There are two points:...more...
The following is a summary and some initial reflections on the paper...more...
Have been listening to a Dave Snowden podcast of a "101 organic KM course". Amongst many familiar themes is the mention of the pitfalls of too much transparency hurting innovation....more...
I'm currently participating in the Learning and Knowledge Analytics MOOC being run by George Siemens and others. This post outlines the process I used to create a podcast of the presentations (click on that link if you want to subscribe to the podcast) being given as part of the course....more...
One of the activities for the first week of the lak11 MOOC is to get started with using Hunch and reflect on it as a model for learning....more...
The following floats/records some initial ideas for connection two of my current projects, BIM and lak11. The ideas arise out of some discussion with my better half who is currently using BIM in one of the courses she is teaching....more...
The following is the summary of my first reading for the LAK11 MOOC and follows on from my initial thoughts....more...
So, the year of study begins. First up is an attempt to engage in a MOOC (Massive Open Online Course) on Learning and Knowledge Analytics. This first post aims to contain some reflection on the course syllabus and what I hope to get out of the course....more...
A couple of days ago I floated the idea of scaffolding, context-sensitive conglomerations as one idea/model/suggestion for how e-learning systems (currently mostly LMS, but hopefully other models will arise)....more...
The following is a slightly edited copy of a message I've just sent off to the Learning Analytics GoogleGroup set up by George Siemens. I'm into reuse. Essentially it tries to highlight a small subset of the uses of learning analytics that I see as most interesting....more...
In some reading for the thesis today I came across the concept of McNamara's fallacy. I hadn't heard this before. This is somewhat surprising as it points out another common problem with some of the more simplistic approaches to improving learning and teaching that are going around at the moment. It's also likely to be a problem with any simplistic implementation of academic analytics....more...
Col's recent blog post has just started the necessary process of the Indicators project paying closer attention to the question of ethics when applied to learning analytics. The following are some of my initial responses to Col's post and an attempt to invite some additional suggestions from other folk around the question...more...
The following is a report of my attendance at Moodlemoot'AU 2010 during the first half of this week. The aim is to engage in a touch of reflection, outline tasks to do, and inform colleagues back at CQUniversity about the conference....more...
The following presents details of feature adoption by courses using the Webfuse system from 1997 through 2009. It presents the data generated by the work described in the last post....more...
I'm in the throws of finalising the second last bit of data analysis for the thesis. For this I'm trying to examine the level of feature adoption within courses supported by the Webfuse system (the main product for the the thesis). The following describes an attempt to formalise the process for this evaluation....more...
In the last post I outlined some early rough usage figures for Webfuse from 2000-2009. The following goes into a bit more detail with the usage of the wf applications....more...
In terms of the thesis, my current focus is on chapter 5. This chapter seeks to describe what we did from 2000 through 2004 and beyond. I'm currently working on the evaluation section, a major component of the evaluation is looking at the usage statistics of Webfuse during this period. It's slow going and there is still a fair bit of analysis to go, but some initial figures are available....more...
Last week I was thinking that academics shouldn't manually create course sites. That arose out of the process of writing up the why/what behind what we did with Webfuse from 1999 through 2004. Today, I've been continuing that and looking at the usage statistics from that period....more...
A blog post on the website of Gilfes Education Group (apparently a "network of independent education experts") picks up on the Indicators project and its take on academic analytics. The post seems to largely in agreement with what we're doing and the reasons behind it....more...
This post is a summary of work being done to update the Moodle indicators block so that it can support multiple visualisation tools and approaches....more...
The following reports on some work on the indicators block to move it towards using some object-orientation and the factory design pattern....more...
Yesterday's post introduced Col's initial work on the indicators block. This post reports on some minor tweaks I've been doing this afternoon, trying to find escape in something concrete....more...
Col has been playing around with some ideas for a Moodle indicators block. This is a record of my first attempt to install and play with the block. Might also do a bit of reflection and setting up of processes etc so we can go further with this....more...
The Indicators project team are off to Moodlemoot AU 2010. All three are giving presentations, but only two (i.e. not me) are on the indicators project. In thinking about how we might make the most of this opportunity, we've floated the idea of announcing an "indicators" Moodle block at the conference. This post is an attempt to make concrete our thinking and take some steps towards making it happen, hopefully even generate some interest and external comments....more...
The following is an attempt to describe early attempts at developing a framework for evaluating existing and developing new interventions aimed at improving learning and teaching within universities. It builds on some ideas from an earlier post....more...
The following draws on principles/theory from psychology to guide thinking about how to incorporate "data" from "academic analytics" into an LMS in a way that encourages and enables academic staff to improve their learning and teaching. It's based on some of the ideas that underpin similar approaches that have been used for students such as this Moodle dashboard and the signals work at University of Purdue....more...
A growing interest of mine is an investigation of how the design of the environment and information systems to support university learning and teaching can be improved with a greater consideration given to factors which can help encourage improvement and change. i.e. not just building systems that do a task (e.g. manage a discussion forum) but design a discussion forum that encourages and enables an academic to adopt strategies and tactics that are known to be good. If they choose to....more...
Just floating another idea for a research project around the Indicators project....more...
The question of the value of lurking is a common one for learning and teaching. Is it okay for students to simply sit quietly and listen, especially in an online forum? Should we "force" them to participate?...more...
I've been back working at the PhD thesis, hopefully down to months before submission. At this stage, I need to work on the 2 chapters that are reflecting on the usage of Webfuse during two periods: 1996 through 1999 and 1999 through 2004 (and a bit later). In doing some, the main tasks I need to achieve are:...more...
The following started as an attempt to develop an argument that business intelligence/data warehouse tools are not a perfect fit for what is broadly called academic analytics. In fact, as I was writing this post I realised that it's actually an argument that business intelligence tools aren't a perfect fit for what I'm interested in doing and that is not academic analytics....more...
The following is an attempt to make concrete some idea and tasks associated with writing a journal paper for the Australian Journal of Educational Technology (AJET) based on the ASCILITE'09 paper based on the work of the Indicators project. The paper will be co-authored by a group and the aim of this post is to start discussion about content of the paper and the tasks we need to do to get it written....more...
One of the good and also problematic things about the Indicators project (attempting to identify patterns in LMS usage data) is that there are so many interesting idea for further research that arise from the patterns being identified. What's best/worst is that the project has really only scratched the surface. This post is an attempt to write down an idea I had today in order to save it for another day, perhaps see if anyone has any insights that can improve it, and if anyone is keen to get involved....more...
The Indicators project is an attempt to enable the examination, analysis and comparison of LMS usage across time, systems and institutions. The project is nothing new. Projects around academic analytics have been around for a while and I and others at my current institution have been talking about using the collective data in systems logs to inform the practice of L&T at universities for a long time....more...
The Excellence in Research for Australia (ERA) Initiative combined with various local institutional and personal factors has increased the interest of some colleagues and I in biting the bullet and entering the next stage of academic development. i.e. the preparation, submission and hopefully successful receipt of applications for external funding. This post is the start of some reflections and thinking about how and what to do....more...
I'm currently sitting in the QANTAS club at Canberra airport waiting to return home after a week at ANU working on the PhD (being done through ANU). I decided to read a copy of CIO magazine while having brunch. In doing so I came across this article (Rodgers, 2009) title "Mind your own Business Intelligence....more...
A couple of colleagues and I are trying to kickstart a little thing we call the Indicators project. We've developed a "tag line" for the project which sums up the core of the project....more...
This post is an attempt to capture some adhoc, over night thoughts about how the indicators project might move forward....more...
After at least a decade of "wouldn't it be a good idea if" and at least one aborted attempt (hint: an organisational restructure in which you are a loser, is not a great context for a new project with intra-organisational implications), the Indicators Project is getting started. This post is my attempt to define what the project means for me. What I hope to get out of the project, and what I hope others might get out of the project....more...
I recently posted about the need to develop an approach that allows for the simple and consistent comparison of usage and feature adoption between different Learning Management Systems (aka LMS, Virtual Learning Environments - VLEs - see What is an LMS?). That last post on the need didn't really establish the need. The aim of this post is to explain the need and make some first steps in identifying how you might go about enabling this sort of comparison....more...
Some colleagues and I are putting the finishing touches on a paper that has arisen out of the indicators project. The paper is an exploratory paper, seeking to find interesting patterns that might indicate good or bad things about the use of LMS (learning management systems, aka course management systems, virtual learning environments etc) that might help improve decision-making by all participants (students through management). I hope to post the paper in coming days....more...
As part of the thesis I've been engaging with some of the literature around LMS feature usage to evaluate usage of Webfuse. A good first stab of this was reported in an earlier post. There were a number of limitations of that work, it's time to expand a bit on it. To some extent for the PhD and to some extent because of a paper....more...
Over the last week I've posted a bit around papers and ideas about how you evaluate the use of a learning management system (LMS). This post is intended to summarise findings from an initial evaluation of the use of features in the Webfuse system from 1997 through 2009. Webfuse is the system I designed as part of my PhD and has been used at Central Queensland University since 1997. At least the "LMS" part of Webfuse will cease being used in 2010....more...
This post follows on from a previous post and continues an examination of some papers written by Malikowski and colleagues examining the adoption of features of an LMS/VLE/CMS. This one focuses on the 2006 paper....more...
I'm in the midst of looking at the work of Malikowski et al in evaluating the usage of VLE features. The aim of this work is an attempt to provide information that can help those who help academics use VLEs. The following is an idea to address those problems and arrive at something that might be useful for cross-institutional comparisons....more...
As a step towards thinking about how you judge the success of an LMS/VLE, this post looks at some work done by Steven Malikowski. Why his work? Well he is co-author on three journal papers that provide one perspective on the usage of features of an LMS, including one that proposes a model for research into course management systems. A list of the papers in the references section....more...
My PhD is essentially arguing that most institutional approaches to e-learning within higher education (i.e. the adoption and long term use of an LMS) has some significant flaws. The thesis will/does describe one attempt to formulate an approach that is better. (Aside: I will not claim that the approach is the best, in fact I'll argue that the notion of there being "one best way" to support e-learning within a university is false.) The idea of "better" raises an interesting/important question, "How do you measure success with institutional use of an LMS?" How do you know if one approach is better than another?...more...
In a recent post I messily wrote about the start of the process of evaluating the use of Webfuse for my thesis. This post takes the ideas/process from that post and applies it to the course websites produced by Webfuse from 2006 through 2007. The data in here is in a similar time frame to the work being done by Col and Ken on their indicators project....more...
For the last couple of weeks I've been working on chapter 4 of my thesis. I've worked my way through explaining the context (general context and (use of e-learning), the design guidelines and the implementation (parts 1, 2 and 3). I've now reached the evaluation section, where I'm meant to describe what happened with the use of Webfuse and make some judgement calls about how it went....more...
Phillips (2005) includes the following quote...more...
Over the last week or so I've been criticising essentially all current practice used to improve learning and teaching. There are probably two main prongs to my current cynicism:...more...
This post adds another perspective borrowed from Gonzalez (2009) as a framework to report or evaluate findings from Col and Ken's indicators project. Col added an update on his work recently. Like previous post this one borrows a table of dimensions around conceptions of online learning because it may be helpful....more...
Col Beer has been doing some work around the "indicators" project - an attempt to mine system logs and databases of a course management system (CMS) to generate data of some use....more...
The following is a quick mind dump of ideas that have arisen today about how you might make use of the usage data and content of course websites from course management systems (CMS) to find out interesting things about learning and teaching. i.e. Col is aiming to develop inidcators that might be of use to the participants associated with e-learning - management, support staff, academics, students etc....more...