Assembling the heterogeneous elements for digital learning

Evaluating an LMS by understanding the underpinning "model"

Currently, CQUni is undertaking an evaluation of Sakai and Moodle as a replacement for Blackboard as the organisation’s Learning Management System. The evaluation process includes many of the standard activities including

  • Developing a long list of criteria/requirements and comparing each LMS against that criteria.
  • Getting groups of staff and students together to examine/port courses to each system and compare/contrast.

Personally, I feel that while both approaches are worthwhile they fail to be sufficient to provide the organisation with enough detail to inform the decision. The main limitation is that neither approach tends to develop a deep understanding of the affordances and limitations of the systems. They always lead to the situation that after a few months/years of use people can sit back and say, “We didn’t know it would do X”.

A few months, at least, of using these systems in real life courses would provide better insight but is also prohibitively expensive. This post is the start of an attempt to try another approach, which might improve a bit on the existing approaches.

What is the approach?

The approach is based somewhat on some previous ramblings and is based on the assumption that an LMS is a piece of information technology. Consequently, it has a set of data structures, algorithms and interfaces that either make it hard or easy to perform tasks. The idea is that if you engage with and understand the model, you can get some idea about what tasks are hard or difficult.

Now there is an awful lot of distance between saying that and actually doing it. I’m claiming that the following posts are going to achieve anywhere near what is possible to make this work effectively. My existing current context doesn’t allow it.

At best this approach is going to start developing some ideas of what needs to be done and which I didn’t do. Hopefully it might “light” the way, a bit.

Using the concept elsewhere

We’ve actually been working on this approach as a basis for staff development in using an LMS. Based on the assumption that understanding the basics of the model will make things work somewhat easier for folk to use. The first attempt at this is the slidecast prepared by Col Beer and shown below.

Blackboard@CQ Uni

View SlideShare presentation or Upload your own.

What will be done?

Given time constraints I can only work with a single course, from a single designer. More courses, especially those that are different would be better. But I have to live with one. I’ve tried to choose one that is likely to test a broader range of features of the LMSes to minise this. But the approach is still inherently limited by this small sample set.

The chosen course is the T2, 2008 offering of ACCT19064, Auditing and Professional Practice. For 2008 this course underwent a complete re-design driven by two talented and committed members of staff – Nona Muldoon (an instructional designer) and Jenny Kofoed (an accounting academic). As part of this re-design they made use of machinima produced in Second Life. The re-design was found to be particularly successful and has been written about.

The basic steps will be

  1. Explain the model underpinning Blackboard and how it is used within the course.
  2. Seek to understand and explain the model underpinning Moodle and then Sakai.
  3. Identify and differences between the models and how that might impact course design.

Hopefully, all things being equal, you should see a list of posts describing these steps linked below.

Leave a comment

Your email address will not be published. Required fields are marked *

3 thoughts on “Evaluating an LMS by understanding the underpinning "model"”

css.php