So, the year of study begins. First up is an attempt to engage in a MOOC (Massive Open Online Course) on Learning and Knowledge Analytics. This first post aims to contain some reflection on the course syllabus and what I hope to get out of the course.
The problem and the promise
As the course description suggests
The growth of data surpasses the ability of organizations or individuals to make sense of it
This is a general observation, but it also applies to learning and teaching related activities.
The promise is that analytics through techniques such as modelling, data mining etc will aid the analysis of this data and help people and organisations to make sense of all the data. To improve their decision making, learning and other tasks.
The aim of the course is as
a conceptual and exploratory introduction to the role of analytics in learning and knowledge development
. It is an introductory course, no heavy math.
I’ve dabbled in work that is close to analytics, but have always had some reservations about its promise. One of the aims of engaging in the course is to encourage me to read and reflect more on these reservations. A quick summary/mind dump of those reservations includes:
- The data is not complete;
At the moment, the data that is available for analytics is limited. e.g. data from an LMS gives only a very small picture of what learning and learning related activities are going on. Consequently, data driven decision making is overly influenced by the data that is available, rather than the data that is important.
- Models and abstractions are by nature lossy;
A lot of analytics is based on mathematical/AI models or abstractions. By definition these “abstract away” details that are deemed to be not important. i.e. information is lost.
- Not every system is causal, except in retrospect;
There often feels to be an assumption of (near) causality in some of this work. There are some events/systems/processes which simply aren’t causal. There is no regular, repeating pattern of “a leading to b”. Just because a lead to b this time, doesn’t mean it will next time. Some of this is related to the previous two points, but some of it is also related to the nature of the systems, especially when they are complex adaptive systems. It will be interesting to hear Dave Snowden’s (one of the invited speakers) take on this later in the course as this reservation is directly influenced by his presentations.
- People aren’t rational;
Personally, I don’t think most people are rational. This shouldn’t suggest that people aren’t somewhat sensible in making their decisions. One’s decisions always make sense to oneself, but they are almost certainly not the decisions that someone else would have made in the same situation. As part of that, I think our experiences constrain/influence our decision making and actions.
This generates two concerns about analytics. First, I wonder just how much change in decision outcomes will arise from the folk seeing all the nice, lovely new visualisations produced by analytics. Are people going to make new decisions or simply use the visualisations to justify the same sub-set of decisions that their experiences would have led them to make. Second, how common amongst learners will be the patterns, models and correlations that arise from analytics? Just because the model says I did “A-B-C” does that really imply I was doing it for the same reasons as the other 88% of the population?
- Is there enough information;
I believe, at this currently ill-informed stage, that some (much?) of the usefulness of analytics arises from a reliance on big number statistics. i.e. there’s so much data that you get useful correlations, patterns….How many existing institutions are going to have sufficiently big data to usefully use these techniques?
- The technologists alliance;
Geohegan suggests there is a technologists’ alliance that has alienated the mainstream through the inability to produce an application of technology that is of absolutely compelling value in pragmatic, mainstream terms that provides the compelling reason to adopt. I think it’s important that there be researchers and innovators pushing the boundaries, but there is too little thought given to the majority and applications of innovations/new technologies/fads that they see as useful. SNAPP is a good start, but there’s some more work to be done.
- Yet another fad;
Analytics is showing all the hallmarks of a fad. There will almost certainly be some interesting ideas here, but the combination of the previous reservations will end up it in being misapplied, misunderstood and ultimately have limited widespread impact on practice.
As evidence of the fad, I offer the photo below that comes from this blog post (which I reference again below).
- Ethical related questions;
A post from Johnathan MacDonald on “The Fallacy of Data Bubble Ignorance” includes the following quote
People don’t want to be spied on. It’s an abuse of civil liberty. The fact that people don’t realise they are being spied on, is not justification to do so. Betting on a business model that goes against how society really works, will ultimately end in disaster.
If this holds, does it hold for analytics. Will the exploitation of learning analytics lead to blow back from the learners?
For some of the above reasons, I am not confident in the ability for most organisations to engage in the use of analytics in ways not destined to annoy and frustrate learners. Many are struggling to implement existing IT systems, let alone manage something like this. I can see the possibilities of disasters.
- Teleological implementation.
This remains my major reservation about all these types of innovations. In the end, they will be applied to institutional contexts through teleological processes. i.e. the change will be done to the institution and its members to achieve some set plan. Implementation will have little contextual sensitivity and thus will have limited quality adoption and will be blind to some of the really interesting context innovations that could have arisen.
A bit of duplication and perhaps some limited logic, but a start.
Onto the week 1 readings.