Month: October 2006 Page 1 of 2

In bam, Uncategorized

It’s almost the end of term. We’ve started marking the blogs using BAM and not surprisingly some rough edges are showing. Things to look at

Merging feeds
Currently BAM simply copies over the local mirror of the feed with the new one. This causes problems because WordPress, and likely other providers, only provide a feed with the last 10 posts. The solution would be to merge the new feeds with the local mirrored copy. XML::Feed, the Perl module used in BAM, offers the splice method to do this. So should be easy.
Making copy detection, pro-active
I find it hard to believe but we have had students who have copied the blog entries of other students. At least half a dozen. Now that it is written, need to have this run as a more regular event and notify students as soon as it happens.
Increase visibility to students
The students don’t really see anything from CQU’s end. This causes problems in terms of whether they’ve completed the questions etc. Implement a view and email notification system that gives the student a visible view of CQU’s summary of where they are up to. Notify students when the miss certain events.
Remove the special characters
Students tend to cut and paste their posts from Word. This introduces special characters which, after the blog has finished with them, can cause problems.

Who is to blame for plagiarism: technology, lecturers or context?

By admin

In plagiarism, Uncategorized

At the Plagiarism Conference 2006 Baroness Ruth Deech, the first Independent Adjudicator for Higher Education gave a presentation which has been widely reported on the Web and in the press. It was interesting to note that the conference was co-sponsored by Turnitin’s UK arm. Rather than sound entirely negative the conference proceedings are online and, from the titles, there looks to be some good stuff.

Some of the comments online, especially from Digg and ArsTechnia which both include a fair number of student comments.

It is being widely reported, because of the title of the Times article, that plagiarism is the fault of indulgent lecturers. The Chronicle of Higher Education also carries the story with some interesting comments.

It suprises me that such views are getting such airplay as they miss the point entirely. My problems with these views include

The idea that there is a single cause of academic misconduct
That the primary cause is rooted in either the students or the teachers
The idea of placing blame is not exactly a helpful way to frame the conversation
That a change or introduction of technology will reduce academic misconduct

What was said

First, in the press Baroness Deech’s views are often reported along with a comment from Gill Clarke of the UK “education watchdog the Quality Assurance Agency”. From the Times article

Gill Clarke, of the education watchdog the Quality Assurance Agency, said that students should be made aware that most universities were using detection software, such as Turnitin, to spot plagiarism.

Baroness Deech’s views include the following, taken from an article in the Australian newspaper

“There is a culture of expectation among today’s students,” she says. “They just take whatever is put in their hands, be it a handout or a PowerPoint presentation. That way you end up boiling down complex things to three bullet points.

Taking down notes in longhand from a book is better than cutting and pasting from the internet, she says, because it requires students to digest material.

Taking notes is the solution

This point basically ignores most of what the educational literature has to say about learning. Learning isn’t enabled by copying content by hand. Information distribution is not learning.

The limited influence of copy detection software

In the course I’m currently teaching we have used copy detection on all assessment items. Part of the assessment is designed using strategies suggested as best practice for helping minimise plagiarism.

Teaching staff have repeatedly mentioned that copy detection software will be used on all assignments. It’s mentioned in the course profile and in the course discussion forum.

Currently (term not quite finished) the copy detection reports on the 3 assignments are showing about 40 students (about 14% of the class) have something that might be academic misconduct. The course staff already have 17 offences entered into the institutions system for managing academic misconduct. That 17 only includes assignment 1 and a small part of assignment 2.

For assignment 2 the students were provided with 3 references (including direct links to online versions) to help them with a report. The copy detection report shows students with reports where 20%, 23%, 49%, 58% and 70% of the report is a direct copy from one of the references we provided.

What should change?

Changing the technology doesn’t appear to make any difference.

Baroness Deech comes close to a possible solution when she says

Students need to be told that their own thoughts about a subject are very important. They need to be challenged to respond in their own way instead of downloading, cutting and pasting. The weighing up of views, the encouraging of nonconformity, the imbuing of intellectual tradition of inquiry are getting lost. If lecturers can imbue students with the view that they are searching rather than copying, then we might go some way towards tackling plagiarism.

The only way I see that we can “imdbue” our students with this view is if our primary form of teaching moves away from the information distribution model that emphasizes traditional lectures and textbooks and move towards more modern, effective pedagogies.

The Program in Course Redesign was a project that used technology as the basis for 30 different US Universities to redesign their instructional approaches to achieve cost savings and quality improvements.

The results from the last round included

Average savings of 39%
8 of the 10 projects reporting improved learning outcomes, the other two reported no significant difference
5 of the 10 projects showed improved course completion, 1 reported no change, 1 showed reduced completion (the other 3 did not measure course completion)

From the lessons page

All ten projects have effected significant shifts in the teaching-learning enterprise, making it more active and learner-centered. The primary goal is to move students from a passive, note-taking role to an active, learning orientation. Lectures are replaced with a wide variety of learning resources, all of which involve more active forms of student learning or more individualized assistance. In moving from an entirely lecture-based to a student-engagement approach, learning is less dependent on words uttered by instructors and more dependent on reading, exploring, and problem-solving undertaken actively by students.

Why isn’t it being done – who is to “blame”?

The type of change described by the Program for Course Redesign is large scale innovation within existing organisational settings. This type of thing is extremely difficult and time consuming and involves a great deal of battling against organisational norms. How is a poor academic expected to do something like this without organisational support at the highest levels?

For example, think of yourself as an academic who is thinking of implementing this sort of change. But your organisational context has the following characteristics

Increasing emphasis on research.
The rhetoric from senior management is that research, in the form of top journal publications, is important. You hear from selection committees for new jobs at your institution using the publication/research record of new applicants as the main criteria.
On-going restructure and review
For the last 4 or 5 years the academic unit and the institution as a whole has been either on-hold waiting for, or involved in organisational restructures and the introduction of new senior management. You’ve just heard that there are also going to a number of other reviews in the coming months.
Limitations of current models and processes
The complexity of your institutions teaching means that there are a range of support units and other commercial organisations which manage aspects of teaching delivery. Those units have adopted specific models for the delivery of teaching that enshrine the traditional lecture/tutorial approach for face-to-face delivery. For a range of reasons they are very reluctant to change those models.

In this context, is it really the fault of the academic if he/she decides that it’s better to concentrate on the research than trying to actively engage with the large scale innovation around teaching and learning that is necessary to seriously address this issue?

Even if the issue is made to engage, how is it suggested that necessary organisational changes be made?

The missing Ps – People

By admin

LMS adoption decisions are made by people. LMSes are used by people. The nature of those people who they are, how they think and what they believe, amongst other characteristics, have a significant impact.

They myth that people are rational

Extensive research shows that our brains have certain hardwired propensities that might be exploited. For example, our brains tend to register frequently heard facts as true, even if they are patently false. As a result, our memories and beliefs are highly malleable and unreliable. We also tend, if unchecked by the conscious reasoning mind, to focus overly on risk, inconvenience, hassles—anything negative. And researchers have found that we all carry around an innate hostility toward “otherness,” which means anyone not like us. (Herbert, 2006)

An alternative is bounded rationality which suggests that people employ heuristics in decision making rather than strict optimization. This is in response to the complexity of the situation is to great and the individual is unable to process and understand all of the alternatives.

Various fields including cognitive science and social psychology have identified a range of cognitive biases that distort how people perceive reality. Wikipedia provides a list of cognitive biases. The following list includes some related to the LMS adoption decision – they are from the decision-making and behavioural biases section of the list

Bandwagon effect – tendency to do things because many others are doing it.
Bias blind spot – tendnecy not to compensate for one’s own cognitive biases
Choice-supportive bias – the tendency to remember one’s choices as better than they actually were
Confirmation bias – the tendency to search/interpret information to confirm your pre-conceptions
Congruence testing – the tendency to use tests that only confirm your hypotheses rather than disprove it, or prove another
Deformation professionnelle – the tendency to look at things through the conventions of your own profession with no thought to the broader view
Disconfirmation bias – tendency to critically examine information that contradicts your prior beliefs but accept uncritically information congruent with your beliefs
Endowment effect – the tendency for people to value something more as soon as they own it
Focusing effect – prediction bias when you place too much importance on one aspect
Hyperbolic discounting – tendency to have a stronger preference for immediate payoffs relative to later payoffs
Illusion of control – tendency to believe you can influence outcomes over which you have no control
Impact bias – tendency to overestimate the intensity of impact on future states
Information bias – tendency to seek information even when it cannot affect attention
Loss aversion – tendency to strongly prefer avoiding losses over acquiring gains
Outcome bias – tendency to judge a decision but its eventual outcome instead of on the quality of the decision at the time it was made
Planning fallacy – tendency to under estimate task completion times
Post-purchase rationalisation – tendency to persuade oneself through rational argument that a purchase was a good one.
Pseudocertainty effect – tendency to make risk averse choices if the expected outcome is positive, but make risk-seeking choices to avoid negative outcomes
Status quo bias – the tendency for people to like things to stay relatively the same
Zero-risk bias – preference for reducing a small risk to zero over a greater reduction in a larger risk

But let’s not leave out the social biases

False consensus effect – tendency to overestimate the degree to which others agree with you
Fundamental attribution error – tendency to over-emphasize personality-based explanations for the behaviour of others while under-emphasizing the role of power and situation influences
Halo effect – the tendency for a person’s positive or negative traits in one area to influence your perception of them in other areas
Illusion of asymmetric insight – the perception that your knowledge of your peers is greater than their knowledge of you
Illusion of transparency – you overestimate your ability to know others, and they overestimate their ability to know you
Ingroup bias – preferential treatement to people who you percieve to be part of your group
Project bias – tendency to unconsciously assume that others share your thoughts, beliefs, values or propositions
Self-server bias – tendency to claim more responsibility for successes than failures.

The missing Ps – Past Experience

By admin

While elearning is still relatively new there has been significant levels of implementation and research around elearning. It seems fairly obvious that any LMS adoption process should make use of the knowledge generated by this past experience.

The categories within this missing P include:

Quality benchmarks
eLearning Maturity Model

Quality benchmarks

A research project, funded by the National Education Assciation and Blackboard, from the Institute for Higher Education Policy in the USA drew on the experiences of pioneers in elearning from 6 institutions of higher education and developed 24 quality benchmarks spread across 7 categories (Phipps & Merisotis, 2000):

Institutional support
Course development
Teaching and learning
Course structure
Student support
Faculty support
Evaluation and assessment

The specific benchmarks provide a very interesting set of questions to ask in the presentation.

eLearning Maturity Model

Work coming out of NZ and finding application in wider environments is application of the CMM into elearning. Stephen Marshall has a good summary of his work in this post.

In particular, it talks about the absence of process and the waste of money associated with the traditional, individual coordinator approach to elearning. Good stuff to include.

References

Phipps, R. and J. Merisotis. (2000). “Quality on the Line.” Retrieved 25 October, 2006, from http://www.ihep.org/Pubs/PDF/Quality.pdf

Moving beyond functional evaluation – scenario-based evaluation

By admin

On October 22, 2006

In elearning, Uncategorized

Tony Byrne has posted an article entitled A Scenario-based approach to evaluating CMS vendors. It’s part of a website that examines/reviews content management systems. The post tells the story of how they evolved their evaluation of such systems from strictly functional evaluations to the addition of “vendor intangibles” and beyond.

Their reason for moving beyond this?

But still, the methodology remains incomplete, because it doesn’t explicitly answer key situational questions. When does a larger or smaller vendor make better sense? When does workflow matter more or less? More pointedly, when does a robust (read: complex) workflow subsystem become a drawback rather than an advantage?

Their solution is to add 12 common scenarios for implementing a CMS. Essentially 12 different type of contexts/organisations requiring a CMS and a list of the characteristics of those organisations that differ and may effect the selection of a CMS.

Obviously there’s a connection between this and the selection of a learning management system (LMS). Not all Universities are the same, not all Universities have the same aims or problems.

They need to think about their purpose and place

The missing Ps – Purpose and place

By admin

On October 22, 2006

In elearning, information systems, missingPs, Uncategorized

A continuation of the elements of the “missing Ps”.

This one looks at the Purpose and Place. The idea is that before a University starts thinking about selecting an LMS, or even starts to question whether or not an LMS is what it needs, it should think long and hard about the purpose to which it will put the LMS (need to try and avoid that label). In doing so it needs to think hard about what it considers to be the purpose of information technology and all of this needs to be driven by consideration of place, the context of the individual University.

Sections include

What’s the purpose of the tool
What’s the purpose of IT
Place – moving beyond purpose

What’s the purpose of the tool

Suppose I give you the choice of three vehicles

and I ask, “Which is the most appropriate car?”

The response of most people will be something like, “Most appropriate for what?”. An Enzo is a great car for completely different purposes than a Humvee and a Prius. The same applies to information systems.

You can’t choose the most appropriate tool unless you know what the purpose is.

For an elearning information system it’s often argued, quite logically, that the purpose/focus should be on learning. It also happens to be one of the most forgotten aspects.

One of the most crucial prerequisites for successful implementation of e-Learning is the need for careful consideration of the underlying pedagogy, or how learning takes place online. In practice, however, this is often the most neglected aspect in any effort to implement e-Learning. (Govindasamy 2002)

George Siemens (2006) suggests a review of LMSes should start with an organisational definition of learning, created through input from all stakeholders, as the foundation of decision making and the boundaries of platform selection.

A concern for learning is certainly an important consideration for an organisation making choices around technology to support education. But there are problems created by trying to start with an organisational definition of learning. Many of these problems are associated with this being an example of a plan-driven process. It assumes that it is possible for an institution to define a single view of learning and that such a single view is actually a good thing. Both are somewhat questionable and are (or will be) addressed in more detail in the process section of the missing Ps framework.

While an emphasis on education is essential it is not the only potential area of consideration for an organisation attempting to choose a technology platform for elearning. An organisation like CQU that has undergone considerable growth and expansion in the complexity of its offerings has a number of other problems and future aims to consider. There may be other business needs to consider.

Once the company’s business needs are clear, the technologies it requires will come into focus. (McAfee, 2006)

McAfee goes onto say in his Harvard Business Review article

I recommend that business leaders accept three IT-related responsibilities: selection (picking appropriate new tech investments), adoption (getting the technology ready for first use) and exploitation (maximizing value delivered once the system is up and running). Rules of thumb are:

During selection, work ‘inside-out’ by first determining what IT-based capabilities are required, then looking out at the technology landscape. This is in contrast to an ‘outside-in’ approach that attempts to determine IT needs by examining available technologies and deciding which ones to bring in.

During adoption and exploitation, put in place the organizational complements that will maximize the postive impat of the selected technology

The missing ingredients from most e-learning programs are clear and measurable objectives and cohesive strategies. Before an organization can evaluate any offerings from an e-learning provider or implement any internal initiative, it must first create a cohesive strategy that clearly defines and documents the value each program must deliver—before any program moves beyond the concept stage (Ismail 2002)

What’s the purpose of IT

Executives usually operate without a comprhensive model of what IT does for companies, how it affects organisations and what they must do to ensure that IT initiatives succeed (McAfee 2006).

Place, moving beyond purpose

Much of the enterprise systems IT world, including the LMS idea, is based on the assumption that a single piece of software and its encapsulated processes are equally applicable to just about every different organisation. This assumption ignores the central importance of context. In fact, it assumes that the type of organisation and the characteristics of that organisation and its environments are either the same or will exert no influence over the potential fit or success between the organisation and the enterprise system.

ERP systems grew out of the manufacturing industry. …need to say more about that type of organisation and then talk about the move of ERPs into other organisations eventually leading up to Universities and the characteristics of those institutional types. The various references I have will fit in nicely here.

Even within one organisation type there is significant difference due to a range of factors associated with the place and purpose of the University. For example, CQU is a very different organisation than MIT. What makes sense for MIT, does not make sense for CQU.

Scenario-based evaluation covers some aspects of this.

References

Govndasamy, T. (2002). “Successful implementation of e-Learning pedagogical considerations.” Internet and Higher Education 4(3-4): 287-299

McAfee, A. (2006). “Mastering the Three Worlds of Information Technology.” Harvard Business Review, November 2006.

Teleological approach to organisational reviews and change projects

By admin

On October 22, 2006

In Uncategorized

I know of a few organisations that are currently undergoing that favourite past-time of all new management and the greatest fear of people in their new organisations, the organisational restructure. As part of another thread of reading I came across a post that has a nice take on these events.

Too often, the top management hires a high-priced expert who claims to have the best knowledge while ignoring the numbskull advice of the front-line workers because they are deemed less educated or not as “sophisticated”. All expert knowledge is limited to the fact that it is knowledge gained from experience. It is the rearview-mirror perspective of the world which, although can be very useful, is limited. The knowledge that counts in a new project and/or change effort is the tacit knowledge gained by the participants as they go through the change and learn from it. During a crisis or change, people will generate many numbskull ideas but they will also gain valuable insights that can later become the expert knowledge that creates a great CBOK.

The missing Ps – Perception

By admin

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

This post looks at perceptions.

The sections will include

Perceptions drive adoption
IT perceived as a cost
Sub-cultures drive perceptions

Perceptions drive adoption

(The following is a bit of plagiarism from a previous post.)

If you draw on the information systems literature you come to TAM (I talked about this before) which posits two main factors influencing adoption of technology

Perceived ease of use
Perceived usefulness

Perceived usefulness is defined as

The degree to which a person believes that using a particular system would enhance his or her job performance.

A recent EDUCAUSE Review article focused on finding out what faculty and students want from future LMSes contains reference to this.

For the most part, faculty members expressed a desire for easier-to-use functions that would reduce time and effort on their part. As one noted: âMaybe it canât be made easier, but it shouldnât be made harder.â? Grading papers, entering grades, and returning files were seen as barriers that do not in exist in the traditional classroom but that are enormously problematic in the L/CMS environment.

Consequently

Tools provided within an L/CMS are not, in general, utilized to their fullest capacity. The interviewees gave many reasons for this phenomenon. The amount of time needed to use many tools demands too much of both learner and instructor. The need to change teaching styles in order to incorporate collaborative tools is not rewarded by the institution.

IT perceived as a cost

IT continues to be seen by higher education as a cost to be minimised.

Sub-cultures drive perceptions

The missing P – Product

By admin

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

This post is the first step in expanding out one of the Ps into more detail. This post covers “Product”. Like the other posts in this discussion, this is a work in progress. I’m following the open source model – release early and often – though I’ll probably succeed more at the early, than the often.

The sections to (eventually) be covered here include:

The product fascination – this is just about the only thing they concentrate on.
The LMS model – what it encapsulates
The limitations of this model
The alternatives – what other options are being considered/are available
The broader product picture

The Product Fascination

Almost all LMS evaluations become comparisons of product features.

The LMS model

Almost all current LMSes encapsulate the same basic model which Blackboard recently obtained a patent for.

The limitations of the LMS model

First generation e-learning systems tend to focus almost entirely on the management and measurement of training processes. They add little or no value to the learning process. Furthermore, they do not provide any means to support internal content production processes, relying instead on commercial courseware. These ‘‘learning management systems’’ (LMS) were seen to be nothing more than launch pads for third party content that the organization would purchase or outsource. (Ismail 2002)

The role and importance of technology in the development of e-learning systems is often overstated by technology providers. It is often stated that the deployment of an LMS alone is all it takes to implement e-learning. The problem is, in many cases the development of e-learning projects devolved into a purely technical process, resulting in expensive software implementations, essentially unused by uninformed, fearful, or resentful employees. (Ismail 2002)

Most LMS vendors deliberately distance themselves from pedagogical issues, often adopting an indifferent attitude or sometimes even trying to disguise it as a praiseworthy act of impartiality. This finding is coherent with Firdiyiyek’s (1999) argument that there is a serious mismatch between the abundance of features in LMS and the lack or total absence of explanation on the pedagogy underlying the inclusion of these tools. (Govindasamy, 2002)

Often many features and tools of LMS are left unused. This is a terrible waste of resources since these tools account for the cost of implementing e-Learning. In a worse case scenario, the tools may end up being used in a manner entirely opposed to pedagogical principles, and in turn, will hamper learning. In either case, the impact inevitably will be reflected in the return on the e-Learning investment. (Govindasamy, 2002)

The alternatives

A recent EDUCAUSE article suggests

Furthermore, we need to investigate the limitations and shortcomings of current L/CMS systems so as to determine whether the existing system design has the capacity to fulfill the needs and requirements of today’s learners and other stakeholders.

The current LMS model encapsulates three assumptions

One application provides all the services
That application is hosted by a single entity – usually the University
Only members of the host institution can get access

The same EDUCAUSE Review article mentioned above includes the following

Interestingly, today all major L/CMS products, both commercial and open source, are using a system design and framework very similar to that of the original L/CMS system invented in the mid-to-late 1990s. This system uses a dedicated or isolated L/CMS computer server that services a campus community or several campuses within a university system. The L/CMS software may run within a computer server located on the campus, or it may use a hosting service, which in most cases is supplied via an ISP (Internet Service Provider) by the vendor marketing the L/CMS software. Under this model, L/CMS services and collaboration are characteristically limited to members of that institution.

The rise of Web 2.0 means that neither of these assumptions need, nor possibly should, hold. I’ve talked about the concept of Web 2.0 course sites elsewhere as a stepping stone from the current comfort zone to something more radical (dropping the course sites entirely).

The recent announcement of Google Apps for Education makes this even more likely. Other institutions are also aware of this idea. A report from Monash/Melbourne Unis (Wise and Quealy, 2006) suggests

Broader innovative opportunities exist in terms of the possibility of partnering with popular Web 2.0 service providers (eg Yahoo, Google, MySpace. Flickr) for interactive web services in the same way as universities currently provide academic versions of popular software. Instead of having to log in to a range of online services depending whether it is for personal use, work use, or for study, a student could login to their service of choice for each functionality, and link the service back to their university course or other activity. Online access to the university through partner service providers could act as a ‘virtual gateway’ to the university. This would be the online equivalent of coming on to the physical campus.

Ismail (2002) talks about three types of systems

Learning design system (LDS)
Alloow the quick anlysis and esign of instructionaly sound learning programs.

Many e-learning projects do not realize their full potential because they fail to adequately meet basic instructional goals and objectives. In the worst cases, these goals and objectives are never even defined beyond a broad statement of direction. The key to developing effective material lies in combining clear learning goals with pedagogical models. Often this is not done because content developers are not trained in instructional design principles.
Learning Content Management System (LCMS)
Provide a collaborative authoring environment for the content. Enable workflow between SMEs, content developers and media developers.
Learning Support System (LSS)
Web environment for supporting the teaching and learning activities.

While most LMSs tend to provide these capabilities, the features implement in the LMS has tended to support self-directed learning. While this is an important mode of online learning, e-learning should be first and foremost about creating a social space that must be managed for the teaching and learning needs of the particular group of people inhabiting that space. This requires a platform that can be easily modified to take into consideration the needs of the particular learners in the course.

Derek Wenmoth has a post/image covering the difference between PLEs (owned/managed by student) and MLEs (owned/managed by University).

The broader product picture

Instead, designers should seek to understand the basic components of what constitutes an e-learning ‘‘ecosystem.’’ This systems framework is crucial in guiding the decisions relating to the choice and development of each component in relation to the objectives outlined in the organizational e-learning strategy. The framework will specify a learning systems architecture for pedagogical development and systems integration. Learning and the needs associated with supporting learning evolve and change over time, and so should learning systems. The reference architecture provided by a systems framework will allow an organization to progressively select and construct systems depending on requirements and budget. (Ismail 2002)

Ismail (2002) then goes on to talk about the Learning Technologies Systems Architecture as a reference model. This will need to be mentioned but it will need to be translated into something CQU specific, something a little more concrete, to enable people to more easily grasp it.

References

Ismail, J. (2002). “The design of an e-learning system: Beyond the hype.” Internet and Higher Education 4(3-4): 329-336

The Missing Ps are a waste of time

By admin

The missing Ps framework is very much under construction attempt to identify the holes and flaws in the methods used by Universities to evaluate and select a learning management system (LMS).

In reading around about this topic, in particular via a document from George Siemens, I came across a report funded by the Melbourne-Monash Collaboration in Educational Technologies, LMS Governance Project Report (Wise and Quealy, 2006).

Skimming through this report I was struck by the following quote from the foreword

We were also very mindful of the number of internal and external reviews of LMS and educational technology (not all of which were made available to us) and the lack of impact these reports appear to have had in decision-making.

A great little quote to bring me back to ground. The best that can come from this idea of the missing Ps is that I get a few publications and hopefully a couple of trips to nice places. Assuming you ignore the personal value that will be generated by simply reading and thinking about this stuff.

Another review of LMS Reviews

By admin