Assembling the heterogeneous elements for (digital) learning

Month: October 2006 Page 1 of 2

The research "foundation" for the missing Ps

The work on the missing Ps is intended to contribute some research publications within the information systems field and also form the structure for chapter 2 of my PhD thesis. Consequently, it is important that the work have a good research foundation. This post is intended to provide that foundation, or at least document my development of the foundation.

The attempt is to develop a descriptive type of theory, a framework or taxonomy. Literature that talks about this type of research, appropriate processes and content include

The missing Ps – Process

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

In this section I’m expanding out my thoughts associated with process. It will include

  • The plan-driven assumption
    The almost automatic assumption that a plan-driven process will be adopted. The almost complete ignorance of the adaptive alternative.
  • IT governance model
  • Process and tool alignment
  • The importance of process change

The Plan-driven assumption

Over emphasis on plan-driven development at the cost of adaptive approaches
This is the one I’ve banged on about in previous papers (e.g. an early one, and the most recent one). The on-going acceptance of agile development methodologies, the Enterprise 2.0 meme incorporating emergence and the idea of rapid incrementalism from the two Johns make pushing this view a bit easier.

There are a number of reasons why this plan-driven approach is not appropriate to elearning

  • elearning, what it is and how best to do it are two very open questions.
    It is impossible to properly plan because what we know now is not going to be the best practice into the future. Examples include the arrival of blogs and social software onto the scene. Also LAMS and the learning design folks.
  • A large portion of any system cost is enhancement.
    This is a finding from software engineering research.

There are problems with the plan driven approach

  • The blind men and an elephant story
    The type of top-down design characteristic of plan driven development leads to the loss of the whole.

Some Peter Drucker quotes

from “The Effective Executive”

“Most discussions of the knowledge worker’s task start with the advice to plan one’s work. This sounds eminently plausible. The only thing wrong with it is that it rarely works. The plans always remain on paper, always remain good intentions. They seldom turn into achievement.”

“Innovation and Entrepreneurship”

“‘Planning’ as the term is commonly understood is actually incompatible with an entrepreneurial society and economy….innovation , almost by definition, has to be decentralized, ad hoc, autonomous, specific and microeconomic.”

IT Governance

The Wikipedia defintion

Corporate governance aims to:

  • align the actions of the individual parts of an organisation toward aggregate mutual benefit
  • provide the means by which each individual part of the organisation can trust that the other parts each make their contribution to the mutual benefit of the organisation and that none gain unfairly at the expense of others
  • provide a means by which information can quickly flow between the various stakeholders to ensure that the changing nature of both the stakeholder needs and desires and the environment in which the organisation operates get effectively factored into decision processes

Need to look more into the idea of IT governance and get some nice definitions.

How this is typically implemented in organisations is through a hierarchical committee structure. i.e. there is some central committee that meets to decide what should happen with IT. That committee has representatives from each part of the organisation. If someone at the coal-face identifies a way to make a potential large improvement their suggestion has to climb up the hierarchy, either within their part of the organisation or up the IT division, until it makes it to the committee. Along the way some small suggestions may get implemented, but anything that is large, crosses organisational boundaries or is a significant change from current practice will generally have to wend its way to the committee.

This is a rather negative description of the problem. On the surface this approach does appear to be somewhat logical if you buy into the rational model of people, organisations and decision making.

The problems with this model include

  • Even if it works it is incredibly slow.
    Often this committees meet no more than 4 times a year. Not exactly agile.
  • At any stage the idea can get knocked on the head.
    Any one of the people on the ladder up to the committee can kill off the idea, especially if it is perceived as significantly different, difficult or threatening.
  • It assumes the committee itself always acts rationally in the best interests of the organisation.
    e.g. the killing off of the AIS idea.
  • It assumes that the decisions of this committee will actually be followed and acted upon.
  • It assumes that people at the coal-face will even bother to start the ball rolling
    This working paper from the Harvard Business School reports on research that indicates that people don’t speak up.

    Qualitative data collected in 190 interviews with employees from all levels and functions suggest that fear of speaking up, even with pro-organizational suggestions, is pervasive and, for many, a source of intense negative affect.

Process and tool alignment

When you’ve been told to use a particular tool, you can’t use a process that doesn’t fit the tool. Or if you do there are going to problems of inefficiency or poor quality.

The implication for higher education when adopting an LMS (or any information system) is that how things are currently done and the system being adopted have to achieve some sort of alignment.

The problem with the implementation of many information systems is that it is assumed that it is the tool that cannot be changed and instead how things are done in the organisation must be changed to meet the tool. With enterprise resource planning systems this is seen as a good thing because these systems are meant to encapsulate “best practice”. But this assumption is highly questionable.

It also ignores the difficulty of forcing process change on an organisation. Especially when the organisation is full of knowledge workers, like a University. This difficulty often means that process change is often ignored or not completed and consequently leads to inefficiencies and poor quality.

Need to mention the idea that the organisation now becomes captive to the system. When the system changes, often due to the vendors needs, not the organisations, the organisation must go through yet another round of difficult process change (or simply ignore it).

There is an alternative. Modern information technology can be implemented in a way when it becomes significantly more malleable than previously thought. It is possible to mould the technology to fit the organisation. This makes it possible to enable a conversation between the organisation, it’s members and the technology where both are modified to provide significantly improved processes. .. this needs to be expanded.

The importance of process change

One way to build a comprehensive model is to place IT in a historical context. Economists and business historians agree that IT is the latest in a series of general-purpose technologies (GPTs), innovations so important that they cause jumps in an economy’s normal march of progress. Electric power, the transistor, and the laser are examples of GPTs that came about in the nineteenth and twentieth centuries. Companies can incorporate some general purpose technologies, like transistors, into products, and others, like electricity, into processes, but all of them share specific characteristics. The performance of such technologies improves dramatically over time. As people become more familiar with GPTs and let go of their old ways of thinking, they find a great many uses for these innovations. Crucially, general purpose technologies deliver greater benefits as people invent or develop complements that multiply the power, impact, and uses of GPTs. For instance, in 1970, fiber-optic cables enabled companies to employ lasers, which had already been in use for a decade, for data transmission. (McAfee 2006)

What should the institution provide in a Web 2.0 world?

Clarence Fisher has an interesting experiential post about the difficulties that start to arise when using a lot of cool, new “Web 2.0” apps in a teaching/learning context. Free(ish) services he lists are blogs, wikis, podcasts/vlogs, superglu, flickr etc.

But it’s not all free.

But we do pay, having to access different accounts across the web. The kids need to remember URLs, passwords, and how to navigate through different interfaces. They need to remember how to run WordPress, make a photostream in Flickr, add blogs to their aggregator, and format a wiki.

His reason for not using an integrated system?

I have consciously not worked with a single piece of software such as Moodle because I wanted kids to see the power of collecting resources across the web into a single environment

He sees the need for an interface that would combine the pieces.

When I thought/dribbled about Web 2.0 course sites one initial negative thought was, “why keep the whole course structure idea?”. Isn’t this an example of the horseless carriage problem. That’s how we did it the old way, so we must do it that way in the new Web 2.0 world?

Sort of, yes. Most people, students and staff, aren’t going to make that leap straight away. They need the comfort of the familiar to ease the pain.

As the above post points out. Integrating these disparate applications together is going to require some glue to reduce the difficulty of using it. Otherwise it won’t work.

Starting my "blogs for learning" article about BAM

blogs for learning is an “online resource about instructional blogging being set up by Michigan State University. I’ve volunteered/been asked to write an article about the use of blogs in COIS20025 and BAM.

This post where I’m going to start my writing.

Thinking about the article

Would be nice to link in with some of the previous articles on the site. Here’s a summary of articles I might link to and why

The site also contains various other posts, some of the ones to refer to include

  • Nicole Elison’s post on Non-Pseudonymous Student Blogging: Ethical and Legal Concerns
    The advantage of BAM is that students public blogs can be anonymous. The identification of which student owns which blog is only visible to CQU staff. Of course, in the absence of any specific advice students have tended to name their blogs using their student identifier.

What will I write about?

  • Why and what we did
  • How it worked out
  • What’s next

Additional improvements to BAM

It’s almost the end of term. We’ve started marking the blogs using BAM and not surprisingly some rough edges are showing. Things to look at

  • Merging feeds
    Currently BAM simply copies over the local mirror of the feed with the new one. This causes problems because WordPress, and likely other providers, only provide a feed with the last 10 posts. The solution would be to merge the new feeds with the local mirrored copy. XML::Feed, the Perl module used in BAM, offers the splice method to do this. So should be easy.
  • Making copy detection, pro-active
    I find it hard to believe but we have had students who have copied the blog entries of other students. At least half a dozen. Now that it is written, need to have this run as a more regular event and notify students as soon as it happens.
  • Increase visibility to students
    The students don’t really see anything from CQU’s end. This causes problems in terms of whether they’ve completed the questions etc. Implement a view and email notification system that gives the student a visible view of CQU’s summary of where they are up to. Notify students when the miss certain events.
  • Remove the special characters
    Students tend to cut and paste their posts from Word. This introduces special characters which, after the blog has finished with them, can cause problems.

Who is to blame for plagiarism: technology, lecturers or context?

At the Plagiarism Conference 2006 Baroness Ruth Deech, the first Independent Adjudicator for Higher Education gave a presentation which has been widely reported on the Web and in the press. It was interesting to note that the conference was co-sponsored by Turnitin’s UK arm. Rather than sound entirely negative the conference proceedings are online and, from the titles, there looks to be some good stuff.

Some of the comments online, especially from Digg and ArsTechnia which both include a fair number of student comments.

It is being widely reported, because of the title of the Times article, that plagiarism is the fault of indulgent lecturers. The Chronicle of Higher Education also carries the story with some interesting comments.

It suprises me that such views are getting such airplay as they miss the point entirely. My problems with these views include

  • The idea that there is a single cause of academic misconduct
  • That the primary cause is rooted in either the students or the teachers
  • The idea of placing blame is not exactly a helpful way to frame the conversation
  • That a change or introduction of technology will reduce academic misconduct

What was said

First, in the press Baroness Deech’s views are often reported along with a comment from Gill Clarke of the UK “education watchdog the Quality Assurance Agency”. From the Times article

Gill Clarke, of the education watchdog the Quality Assurance Agency, said that students should be made aware that most universities were using detection software, such as Turnitin, to spot plagiarism.

Baroness Deech’s views include the following, taken from an article in the Australian newspaper

“There is a culture of expectation among today’s students,” she says. “They just take whatever is put in their hands, be it a handout or a PowerPoint presentation. That way you end up boiling down complex things to three bullet points.

Taking down notes in longhand from a book is better than cutting and pasting from the internet, she says, because it requires students to digest material.

Taking notes is the solution

This point basically ignores most of what the educational literature has to say about learning. Learning isn’t enabled by copying content by hand. Information distribution is not learning.

The limited influence of copy detection software

In the course I’m currently teaching we have used copy detection on all assessment items. Part of the assessment is designed using strategies suggested as best practice for helping minimise plagiarism.

Teaching staff have repeatedly mentioned that copy detection software will be used on all assignments. It’s mentioned in the course profile and in the course discussion forum.

Currently (term not quite finished) the copy detection reports on the 3 assignments are showing about 40 students (about 14% of the class) have something that might be academic misconduct. The course staff already have 17 offences entered into the institutions system for managing academic misconduct. That 17 only includes assignment 1 and a small part of assignment 2.

For assignment 2 the students were provided with 3 references (including direct links to online versions) to help them with a report. The copy detection report shows students with reports where 20%, 23%, 49%, 58% and 70% of the report is a direct copy from one of the references we provided.

What should change?

Changing the technology doesn’t appear to make any difference.

Baroness Deech comes close to a possible solution when she says

Students need to be told that their own thoughts about a subject are very important. They need to be challenged to respond in their own way instead of downloading, cutting and pasting. The weighing up of views, the encouraging of nonconformity, the imbuing of intellectual tradition of inquiry are getting lost. If lecturers can imbue students with the view that they are searching rather than copying, then we might go some way towards tackling plagiarism.

The only way I see that we can “imdbue” our students with this view is if our primary form of teaching moves away from the information distribution model that emphasizes traditional lectures and textbooks and move towards more modern, effective pedagogies.

The Program in Course Redesign was a project that used technology as the basis for 30 different US Universities to redesign their instructional approaches to achieve cost savings and quality improvements.

The results from the last round included

  • Average savings of 39%
  • 8 of the 10 projects reporting improved learning outcomes, the other two reported no significant difference
  • 5 of the 10 projects showed improved course completion, 1 reported no change, 1 showed reduced completion (the other 3 did not measure course completion)

From the lessons page

All ten projects have effected significant shifts in the teaching-learning enterprise, making it more active and learner-centered. The primary goal is to move students from a passive, note-taking role to an active, learning orientation. Lectures are replaced with a wide variety of learning resources, all of which involve more active forms of student learning or more individualized assistance. In moving from an entirely lecture-based to a student-engagement approach, learning is less dependent on words uttered by instructors and more dependent on reading, exploring, and problem-solving undertaken actively by students.

Why isn’t it being done – who is to “blame”?

The type of change described by the Program for Course Redesign is large scale innovation within existing organisational settings. This type of thing is extremely difficult and time consuming and involves a great deal of battling against organisational norms. How is a poor academic expected to do something like this without organisational support at the highest levels?

For example, think of yourself as an academic who is thinking of implementing this sort of change. But your organisational context has the following characteristics

  • Increasing emphasis on research.
    The rhetoric from senior management is that research, in the form of top journal publications, is important. You hear from selection committees for new jobs at your institution using the publication/research record of new applicants as the main criteria.
  • On-going restructure and review
    For the last 4 or 5 years the academic unit and the institution as a whole has been either on-hold waiting for, or involved in organisational restructures and the introduction of new senior management. You’ve just heard that there are also going to a number of other reviews in the coming months.
  • Limitations of current models and processes
    The complexity of your institutions teaching means that there are a range of support units and other commercial organisations which manage aspects of teaching delivery. Those units have adopted specific models for the delivery of teaching that enshrine the traditional lecture/tutorial approach for face-to-face delivery. For a range of reasons they are very reluctant to change those models.

In this context, is it really the fault of the academic if he/she decides that it’s better to concentrate on the research than trying to actively engage with the large scale innovation around teaching and learning that is necessary to seriously address this issue?

Even if the issue is made to engage, how is it suggested that necessary organisational changes be made?

The missing Ps – People

LMS adoption decisions are made by people. LMSes are used by people. The nature of those people who they are, how they think and what they believe, amongst other characteristics, have a significant impact.

They myth that people are rational

Extensive research shows that our brains have certain hardwired propensities that might be exploited. For example, our brains tend to register frequently heard facts as true, even if they are patently false. As a result, our memories and beliefs are highly malleable and unreliable. We also tend, if unchecked by the conscious reasoning mind, to focus overly on risk, inconvenience, hassles—anything negative. And researchers have found that we all carry around an innate hostility toward “otherness,” which means anyone not like us. (Herbert, 2006)

An alternative is bounded rationality which suggests that people employ heuristics in decision making rather than strict optimization. This is in response to the complexity of the situation is to great and the individual is unable to process and understand all of the alternatives.

Various fields including cognitive science and social psychology have identified a range of cognitive biases that distort how people perceive reality. Wikipedia provides a list of cognitive biases. The following list includes some related to the LMS adoption decision – they are from the decision-making and behavioural biases section of the list

  • Bandwagon effect – tendency to do things because many others are doing it.
  • Bias blind spot – tendnecy not to compensate for one’s own cognitive biases
  • Choice-supportive bias – the tendency to remember one’s choices as better than they actually were
  • Confirmation bias – the tendency to search/interpret information to confirm your pre-conceptions
  • Congruence testing – the tendency to use tests that only confirm your hypotheses rather than disprove it, or prove another
  • Deformation professionnelle – the tendency to look at things through the conventions of your own profession with no thought to the broader view
  • Disconfirmation bias – tendency to critically examine information that contradicts your prior beliefs but accept uncritically information congruent with your beliefs
  • Endowment effect – the tendency for people to value something more as soon as they own it
  • Focusing effect – prediction bias when you place too much importance on one aspect
  • Hyperbolic discounting – tendency to have a stronger preference for immediate payoffs relative to later payoffs
  • Illusion of control – tendency to believe you can influence outcomes over which you have no control
  • Impact bias – tendency to overestimate the intensity of impact on future states
  • Information bias – tendency to seek information even when it cannot affect attention
  • Loss aversion – tendency to strongly prefer avoiding losses over acquiring gains
  • Outcome bias – tendency to judge a decision but its eventual outcome instead of on the quality of the decision at the time it was made
  • Planning fallacy – tendency to under estimate task completion times
  • Post-purchase rationalisation – tendency to persuade oneself through rational argument that a purchase was a good one.
  • Pseudocertainty effect – tendency to make risk averse choices if the expected outcome is positive, but make risk-seeking choices to avoid negative outcomes
  • Status quo bias – the tendency for people to like things to stay relatively the same
  • Zero-risk bias – preference for reducing a small risk to zero over a greater reduction in a larger risk

But let’s not leave out the social biases

  • False consensus effect – tendency to overestimate the degree to which others agree with you
  • Fundamental attribution error – tendency to over-emphasize personality-based explanations for the behaviour of others while under-emphasizing the role of power and situation influences
  • Halo effect – the tendency for a person’s positive or negative traits in one area to influence your perception of them in other areas
  • Illusion of asymmetric insight – the perception that your knowledge of your peers is greater than their knowledge of you
  • Illusion of transparency – you overestimate your ability to know others, and they overestimate their ability to know you
  • Ingroup bias – preferential treatement to people who you percieve to be part of your group
  • Project bias – tendency to unconsciously assume that others share your thoughts, beliefs, values or propositions
  • Self-server bias – tendency to claim more responsibility for successes than failures.

The missing Ps – Past Experience

While elearning is still relatively new there has been significant levels of implementation and research around elearning. It seems fairly obvious that any LMS adoption process should make use of the knowledge generated by this past experience.

The categories within this missing P include:

  • Quality benchmarks
  • eLearning Maturity Model

Quality benchmarks

A research project, funded by the National Education Assciation and Blackboard, from the Institute for Higher Education Policy in the USA drew on the experiences of pioneers in elearning from 6 institutions of higher education and developed 24 quality benchmarks spread across 7 categories (Phipps & Merisotis, 2000):

  1. Institutional support
  2. Course development
  3. Teaching and learning
  4. Course structure
  5. Student support
  6. Faculty support
  7. Evaluation and assessment

The specific benchmarks provide a very interesting set of questions to ask in the presentation.

eLearning Maturity Model

Work coming out of NZ and finding application in wider environments is application of the CMM into elearning. Stephen Marshall has a good summary of his work in this post.

In particular, it talks about the absence of process and the waste of money associated with the traditional, individual coordinator approach to elearning. Good stuff to include.


Phipps, R. and J. Merisotis. (2000). “Quality on the Line.” Retrieved 25 October, 2006, from

Moving beyond functional evaluation – scenario-based evaluation

Tony Byrne has posted an article entitled A Scenario-based approach to evaluating CMS vendors. It’s part of a website that examines/reviews content management systems. The post tells the story of how they evolved their evaluation of such systems from strictly functional evaluations to the addition of “vendor intangibles” and beyond.

Their reason for moving beyond this?

But still, the methodology remains incomplete, because it doesn’t explicitly answer key situational questions. When does a larger or smaller vendor make better sense? When does workflow matter more or less? More pointedly, when does a robust (read: complex) workflow subsystem become a drawback rather than an advantage?

Their solution is to add 12 common scenarios for implementing a CMS. Essentially 12 different type of contexts/organisations requiring a CMS and a list of the characteristics of those organisations that differ and may effect the selection of a CMS.

Obviously there’s a connection between this and the selection of a learning management system (LMS). Not all Universities are the same, not all Universities have the same aims or problems.

They need to think about their purpose and place

The missing Ps – Purpose and place

A continuation of the elements of the “missing Ps”.

This one looks at the Purpose and Place. The idea is that before a University starts thinking about selecting an LMS, or even starts to question whether or not an LMS is what it needs, it should think long and hard about the purpose to which it will put the LMS (need to try and avoid that label). In doing so it needs to think hard about what it considers to be the purpose of information technology and all of this needs to be driven by consideration of place, the context of the individual University.

Sections include

  • What’s the purpose of the tool
  • What’s the purpose of IT
  • Place – moving beyond purpose

What’s the purpose of the tool

Suppose I give you the choice of three vehicles

  1. Ferrari Enzo
  2. Humvee
  3. Toyota Prius

and I ask, “Which is the most appropriate car?”

The response of most people will be something like, “Most appropriate for what?”. An Enzo is a great car for completely different purposes than a Humvee and a Prius. The same applies to information systems.

You can’t choose the most appropriate tool unless you know what the purpose is.

For an elearning information system it’s often argued, quite logically, that the purpose/focus should be on learning. It also happens to be one of the most forgotten aspects.

One of the most crucial prerequisites for successful implementation of e-Learning is the need for careful consideration of the underlying pedagogy, or how learning takes place online. In practice, however, this is often the most neglected aspect in any effort to implement e-Learning. (Govindasamy 2002)

George Siemens (2006) suggests a review of LMSes should start with an organisational definition of learning, created through input from all stakeholders, as the foundation of decision making and the boundaries of platform selection.

A concern for learning is certainly an important consideration for an organisation making choices around technology to support education. But there are problems created by trying to start with an organisational definition of learning. Many of these problems are associated with this being an example of a plan-driven process. It assumes that it is possible for an institution to define a single view of learning and that such a single view is actually a good thing. Both are somewhat questionable and are (or will be) addressed in more detail in the process section of the missing Ps framework.

While an emphasis on education is essential it is not the only potential area of consideration for an organisation attempting to choose a technology platform for elearning. An organisation like CQU that has undergone considerable growth and expansion in the complexity of its offerings has a number of other problems and future aims to consider. There may be other business needs to consider.

Once the company’s business needs are clear, the technologies it requires will come into focus. (McAfee, 2006)

McAfee goes onto say in his Harvard Business Review article

I recommend that business leaders accept three IT-related responsibilities: selection (picking appropriate new tech investments), adoption (getting the technology ready for first use) and exploitation (maximizing value delivered once the system is up and running). Rules of thumb are:

  • During selection, work ‘inside-out’ by first determining what IT-based capabilities are required, then looking out at the technology landscape. This is in contrast to an ‘outside-in’ approach that attempts to determine IT needs by examining available technologies and deciding which ones to bring in.
  • During adoption and exploitation, put in place the organizational complements that will maximize the postive impat of the selected technology

The missing ingredients from most e-learning programs are clear and measurable objectives and cohesive strategies. Before an organization can evaluate any offerings from an e-learning provider or implement any internal initiative, it must first create a cohesive strategy that clearly defines and documents the value each program must deliver—before any program moves beyond the concept stage (Ismail 2002)

What’s the purpose of IT

Executives usually operate without a comprhensive model of what IT does for companies, how it affects organisations and what they must do to ensure that IT initiatives succeed (McAfee 2006).

Place, moving beyond purpose

Much of the enterprise systems IT world, including the LMS idea, is based on the assumption that a single piece of software and its encapsulated processes are equally applicable to just about every different organisation. This assumption ignores the central importance of context. In fact, it assumes that the type of organisation and the characteristics of that organisation and its environments are either the same or will exert no influence over the potential fit or success between the organisation and the enterprise system.

ERP systems grew out of the manufacturing industry. …need to say more about that type of organisation and then talk about the move of ERPs into other organisations eventually leading up to Universities and the characteristics of those institutional types. The various references I have will fit in nicely here.

Even within one organisation type there is significant difference due to a range of factors associated with the place and purpose of the University. For example, CQU is a very different organisation than MIT. What makes sense for MIT, does not make sense for CQU.

Scenario-based evaluation covers some aspects of this.


Govndasamy, T. (2002). “Successful implementation of e-Learning pedagogical considerations.” Internet and Higher Education 4(3-4): 287-299

McAfee, A. (2006). “Mastering the Three Worlds of Information Technology.” Harvard Business Review, November 2006.

Teleological approach to organisational reviews and change projects

I know of a few organisations that are currently undergoing that favourite past-time of all new management and the greatest fear of people in their new organisations, the organisational restructure. As part of another thread of reading I came across a post that has a nice take on these events.

Too often, the top management hires a high-priced expert who claims to have the best knowledge while ignoring the numbskull advice of the front-line workers because they are deemed less educated or not as “sophisticated”. All expert knowledge is limited to the fact that it is knowledge gained from experience. It is the rearview-mirror perspective of the world which, although can be very useful, is limited. The knowledge that counts in a new project and/or change effort is the tacit knowledge gained by the participants as they go through the change and learn from it. During a crisis or change, people will generate many numbskull ideas but they will also gain valuable insights that can later become the expert knowledge that creates a great CBOK.

The missing Ps – Perception

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

This post looks at perceptions.

The sections will include

  • Perceptions drive adoption
  • IT perceived as a cost
  • Sub-cultures drive perceptions

Perceptions drive adoption

(The following is a bit of plagiarism from a previous post.)

If you draw on the information systems literature you come to TAM (I talked about this before) which posits two main factors influencing adoption of technology

  1. Perceived ease of use
  2. Perceived usefulness
    1. Perceived usefulness is defined as

      The degree to which a person believes that using a particular system would enhance his or her job performance.

      A recent EDUCAUSE Review article focused on finding out what faculty and students want from future LMSes contains reference to this.

      For the most part, faculty members expressed a desire for easier-to-use functions that would reduce time and effort on their part. As one noted: “Maybe it can’t be made easier, but it shouldn’t be made harder.â€? Grading papers, entering grades, and returning files were seen as barriers that do not in exist in the traditional classroom but that are enormously problematic in the L/CMS environment.


      Tools provided within an L/CMS are not, in general, utilized to their fullest capacity. The interviewees gave many reasons for this phenomenon. The amount of time needed to use many tools demands too much of both learner and instructor. The need to change teaching styles in order to incorporate collaborative tools is not rewarded by the institution.

      IT perceived as a cost

      IT continues to be seen by higher education as a cost to be minimised.

      Sub-cultures drive perceptions

The missing P – Product

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

This post is the first step in expanding out one of the Ps into more detail. This post covers “Product”. Like the other posts in this discussion, this is a work in progress. I’m following the open source model – release early and often – though I’ll probably succeed more at the early, than the often.

The sections to (eventually) be covered here include:

  • The product fascination – this is just about the only thing they concentrate on.
  • The LMS model – what it encapsulates
  • The limitations of this model
  • The alternatives – what other options are being considered/are available
  • The broader product picture

The Product Fascination

Almost all LMS evaluations become comparisons of product features.

The LMS model

Almost all current LMSes encapsulate the same basic model which Blackboard recently obtained a patent for.

The limitations of the LMS model

First generation e-learning systems tend to focus almost entirely on the management and measurement of training processes. They add little or no value to the learning process. Furthermore, they do not provide any means to support internal content production processes, relying instead on commercial courseware. These ‘‘learning management systems’’ (LMS) were seen to be nothing more than launch pads for third party content that the organization would purchase or outsource. (Ismail 2002)

The role and importance of technology in the development of e-learning systems is often overstated by technology providers. It is often stated that the deployment of an LMS alone is all it takes to implement e-learning. The problem is, in many cases the development of e-learning projects devolved into a purely technical process, resulting in expensive software implementations, essentially unused by uninformed, fearful, or resentful employees. (Ismail 2002)

Most LMS vendors deliberately distance themselves from pedagogical issues, often adopting an indifferent attitude or sometimes even trying to disguise it as a praiseworthy act of impartiality. This finding is coherent with Firdiyiyek’s (1999) argument that there is a serious mismatch between the abundance of features in LMS and the lack or total absence of explanation on the pedagogy underlying the inclusion of these tools. (Govindasamy, 2002)

Often many features and tools of LMS are left unused. This is a terrible waste of resources since these tools account for the cost of implementing e-Learning. In a worse case scenario, the tools may end up being used in a manner entirely opposed to pedagogical principles, and in turn, will hamper learning. In either case, the impact inevitably will be reflected in the return on the e-Learning investment. (Govindasamy, 2002)

The alternatives

A recent EDUCAUSE article suggests

Furthermore, we need to investigate the limitations and shortcomings of current L/CMS systems so as to determine whether the existing system design has the capacity to fulfill the needs and requirements of today’s learners and other stakeholders.

The current LMS model encapsulates three assumptions

  1. One application provides all the services
  2. That application is hosted by a single entity – usually the University
  3. Only members of the host institution can get access

The same EDUCAUSE Review article mentioned above includes the following

Interestingly, today all major L/CMS products, both commercial and open source, are using a system design and framework very similar to that of the original L/CMS system invented in the mid-to-late 1990s. This system uses a dedicated or isolated L/CMS computer server that services a campus community or several campuses within a university system. The L/CMS software may run within a computer server located on the campus, or it may use a hosting service, which in most cases is supplied via an ISP (Internet Service Provider) by the vendor marketing the L/CMS software. Under this model, L/CMS services and collaboration are characteristically limited to members of that institution.

The rise of Web 2.0 means that neither of these assumptions need, nor possibly should, hold. I’ve talked about the concept of Web 2.0 course sites elsewhere as a stepping stone from the current comfort zone to something more radical (dropping the course sites entirely).

The recent announcement of Google Apps for Education makes this even more likely. Other institutions are also aware of this idea. A report from Monash/Melbourne Unis (Wise and Quealy, 2006) suggests

Broader innovative opportunities exist in terms of the possibility of partnering with popular Web 2.0 service providers (eg Yahoo, Google, MySpace. Flickr) for interactive web services in the same way as universities currently provide academic versions of popular software. Instead of having to log in to a range of online services depending whether it is for personal use, work use, or for study, a student could login to their service of choice for each functionality, and link the service back to their university course or other activity. Online access to the university through partner service providers could act as a ‘virtual gateway’ to the university. This would be the online equivalent of coming on to the physical campus.

Ismail (2002) talks about three types of systems

  1. Learning design system (LDS)
    Alloow the quick anlysis and esign of instructionaly sound learning programs.

    Many e-learning projects do not realize their full potential because they fail to adequately meet basic instructional goals and objectives. In the worst cases, these goals and objectives are never even defined beyond a broad statement of direction. The key to developing effective material lies in combining clear learning goals with pedagogical models. Often this is not done because content developers are not trained in instructional design principles.

  2. Learning Content Management System (LCMS)
    Provide a collaborative authoring environment for the content. Enable workflow between SMEs, content developers and media developers.
  3. Learning Support System (LSS)
    Web environment for supporting the teaching and learning activities.

    While most LMSs tend to provide these capabilities, the features implement in the LMS has tended to support self-directed learning. While this is an important mode of online learning, e-learning should be first and foremost about creating a social space that must be managed for the teaching and learning needs of the particular group of people inhabiting that space. This requires a platform that can be easily modified to take into consideration the needs of the particular learners in the course.

Derek Wenmoth has a post/image covering the difference between PLEs (owned/managed by student) and MLEs (owned/managed by University).

The broader product picture

Instead, designers should seek to understand the basic components of what constitutes an e-learning ‘‘ecosystem.’’ This systems framework is crucial in guiding the decisions relating to the choice and development of each component in relation to the objectives outlined in the organizational e-learning strategy. The framework will specify a learning systems architecture for pedagogical development and systems integration. Learning and the needs associated with supporting learning evolve and change over time, and so should learning systems. The reference architecture provided by a systems framework will allow an organization to progressively select and construct systems depending on requirements and budget. (Ismail 2002)

Ismail (2002) then goes on to talk about the Learning Technologies Systems Architecture as a reference model. This will need to be mentioned but it will need to be translated into something CQU specific, something a little more concrete, to enable people to more easily grasp it.


Ismail, J. (2002). “The design of an e-learning system: Beyond the hype.” Internet and Higher Education 4(3-4): 329-336

The Missing Ps are a waste of time

The missing Ps framework is very much under construction attempt to identify the holes and flaws in the methods used by Universities to evaluate and select a learning management system (LMS).

In reading around about this topic, in particular via a document from George Siemens, I came across a report funded by the Melbourne-Monash Collaboration in Educational Technologies, LMS Governance Project Report (Wise and Quealy, 2006).

Skimming through this report I was struck by the following quote from the foreword

We were also very mindful of the number of internal and external reviews of LMS and educational technology (not all of which were made available to us) and the lack of impact these reports appear to have had in decision-making.

A great little quote to bring me back to ground. The best that can come from this idea of the missing Ps is that I get a few publications and hopefully a couple of trips to nice places. Assuming you ignore the personal value that will be generated by simply reading and thinking about this stuff.

Another review of LMS Reviews

George Siemens has posted a document entitled “Learning or Management System? A Review of Learning Management System Reviews” which, as the title suggests, reviews a recent collection of reviews of LMSes in an attempt to offer a more nuanced view.

The abstract states

The position offered in this report encourages an organizational definition of learning as the starting point for selecting a technology platform for creating and delivery learning content. A clear definition of learning vision and desired future states, created through input from stakeholders (administrators, faculty, students, and information services) should provide the foundation for decision making, and the boundaries of platform selection.

This is extremely timely given I’ve only just recently embarked publically on the development of my “missing Ps framework”. My attempt at organising all of the flaws and problems I have with how institutions of higher education are currently going about the process of selecting an LMS.

I find myself agreeing with much of what is in the document. However, I also think there is more here, a broader perspective to be considered.

For example, the quote from the abstract above strongly suggests that an “organisational definition of learning” should be the starting point for this process. In a perfect world I would agree but I have two arguments against this.

  1. Establishing an organisational definition is likely impossible and may in fact be damaging in the short and long term.
    Have you ever tried to get two academics to agree on anything? How about two academics from different disciplines? Most of the organisational definitions of anything created by Universities are often so general as to be meaningless. If it isn’t so high level the definition means that some part of the organisational community has be sidelined. Their understanding, their definition deemed not to be acceptable to the organisation. This loss of diversity is often problematic for the institution. Finally, I’m sure if you could find an organisational definition of learning from 5, 10 or 50 years ago, it would be considerably different from the definition in vogue/use today. Such definitions can be a drag.
  2. Is learning always the primary focus behind the selection of an LMS?
    Sure, most places pay lip service to the rhetoric that the decision is all about learning. But scratch the surface and there is often a very different reason. If you take the perspective that LMSes do little or nothing to support learning then perhaps it’s acceptable for an organisation to choose a different purpose.

The starting place I would suggest should be that the organisation state up-front what is the purpose the tool will be put to. Why does the organisation need it and what is it hoped you can achieve.

The purpose might, hopefully would, include a significant/primary focus on learning. But I think honesty is more valuable in this sort of decision than lip service.

This seems to connect with a quote in the paper from Wise and Quealy (2006)

“the educational significance of LMS is largely overemphasized and misunderstood …[suggesting it is critical for a university to] … understand itself—what it values, what it does well and how it does it, what it would like to do, and how it might do thisâ€?

Question allocation difficulties – BAM

A long time ago I identified “reducing meaningless freedom” as a key design principle for elearning type apps – basically anything web-based and possible broader. It ties in with ideas others have identified, it’s even discussed in the sad textbook we’re using in COIS20025 this term.

The idea is, that if a user shouldn’t be able to do something, don’t let them.

With BAM as used in COIS20025 this term students are meant to use their blog to respond to a fixed set of questions.

In doing this I had assumed that

  • Each post would use the question title as the subject of the post
  • The answer to each question would be included in one post
  • That students could identify the URL for their blog and register it with the CQU system

We’ve had a number of students break each of those. Especially identifying the URL of their blog. We’ve had two students who have registered the home URL of their blog host (e.g. as the URL for their blog!

This will be one of the hurdles in using real applications, ones we don’t design and control. It means we’re going to have build more smarts into the integration apps we do build.

Page 1 of 2

Powered by WordPress & Theme by Anders Norén