Decision making, problem solving traps and course management systems

As part of the missing Ps framework, in the “People” category, I’m planning to talk about how the decision making around adoption of course management systems (CMSs) is always posited as a rational act. When, in fact, most people involved in and the decision making processes around CMSs is far from rational.

In a discussion on IT-Forum, Clark Quinn has mentioned in one email a few problem solving traps

  • Set effects – solving subsequent problems like previous problems, even if a different approach might be more effective
  • Functional fixedness – only seeing one way to use a tool
  • Premature evaluation – rejection or acceptance of an alternative prior to full consideration

I believe these three, as well as others, are very obvious in most of the literature and practice around the evaluation of course management systems.

In fact, the phrase “evaluation of course management systems” to me indicates one example of premature evaluation. The decision has already been made that it must be a course management system rather than some alternative approach that may be more appropriate and effective in the longer term.

Enterprise systems and shadow systems: What can the miner's canary tell us?

The following is the first cut at developing a submission for the 2007 EDUCAUSE Annual Conference in Seattle, Oct 23-36. The theme for the conference is “Information Futures: Aligning our Missions”.

Proposals are due February 6, 2007.

In this presentation I’d like to do something along the following lines

  • Enterprise systems – including ERP systems, CMSs etc – are an essential part of university operations.
  • The mismatch between these types of systems and universities (and many other types of organisations) means that there will always be shadow systems.
  • Most organisations and indeed much of the literature positions shadow systems as abberrations that must be killed. That they are problems for the organisation. Which in turn drives shadow systems further underground.
  • As we showed earlier organisations can actually learn lessons from shadow systems.
  • What lessons are shadow systems trying to tell us about “our information futures”?

I’d like to develop two main categories of lessons grouped around types of enterprise systems within universities

  1. ERPs
    I’d like to draw on the MyCQU/MyInfocom work to identify the lessons universities should be taking with respect to ERPs. Essentially and updated version of the previous work with additional insights.
  2. Course Management Systems
    The idea is that factors outlined in two previous posts (one and two) are starting to imply that many CMS features will no longer be provided by Universities but instead draw on free external services such as YouTube and Google Video. What I’ve called the web 2.0 course site idea. At CQU this change can be seen in the BAM project that has students using freely available external blog services (rather than a CQU provided blog) and CQU having systems that help integrate those services.
  3. Potentially wider.
    The web 2.0 course site idea is essentially drawing on software as a service ideas. This has potential application to many other services. For example, what implications does Amazon S3 (Simple Storage Service) have for services such as off site backups, personal backups, network shares?

I’d like to suggest that the lessons might include things like

  • Increasing use of software as a service
    There will have been IS research on this issue. We need to look at and learn from those issues.
  • Where “service” is not some tightly defined standard, but instead a lightweight standard like RSS
    A personal prejudice. I don’t think heavy weight web services are a good basis, perhaps better phrased as “the best basis”, for this sort of thing.
  • That universities will have to change from concentrating on implementation related issues (maintaining servers, lots of code development) to integration related issues (making sure all the services play nicely and providing integration with uni needs).

The Web 2.0 Course site trial

I’d like to further investigate this by developing some trial web 2.0 course sites.

A web 2.0 course site would consist of

  • A site on a CQU server that groups everything together
    A bit like this page. This site would serve to increase the compatibility of the new style with the old and encourage adoption. It may also serve as the interface to modify/change the course site used by staff. It’s the CQU wrapper around the other services.
  • Most of the content and services would be provided by external services.
  • There might also be an OPML file (or similar) that allows students/staff to access the information/services via a news reader. i.e. bypassing the course website entirely.

I would think the process for doing this might be something like

  • Take some existing course websites and figure out how the features used on those can be translated into web 2.0 course sites
  • Identify new features that might be included
  • Develop some prototypes that people can see and experiment with
  • Figure out the workload/resources required to implement these prototypes
  • Do some evaluations to figure out if this is a worthwhile thing to actually implement.

What needs to be done for the presentation

  • Identify which co-authors at CQU want to participate?
  • Select a topic area that fits
  • Write the proposal
  • Get the web 2.0 course site proposal off the ground

Proposal Content

Proposals must include

  • Abstract
    No more than 50 words. This is what appears on the conference website. Advice includes “Please be concise, accurate, and specific with your writing. Avoid flowery and overly descriptive prose. Use the abstract to state clearly what you will present during your session”. Examples of effective abstracts are available
  • Statement of the problem or issue
  • Description of activity, project or solution
  • Outcome/achievements
  • Importance or relevance to other institutions
  • Suggested Audience

Drawbacks

This particular project is focused at the organisational/infrastructure around how universities implement elearning and related features. It has little or nothing to do with increasing the educational value of elearning.

At least not directly. I am hoping that one reason that this approach might be useful is that it could save CQU money in terms of licensing and other costs. Money that could be reinvested in ways that does directly aim to increase the educational value of how elearning is applied.

LMS Governance Report – a summary and reflection

It’s that time in my thesis work (trawling through references for the lit review) when I have finally come back to the “LMS Governance report” written by Lisa Wise and James Quealy. This went through the blogosphere a while back. You can access the PDF of the report and a MediaWiki based annotated bibliography from this blog post.

The following is my attempt at a summary and a bit of reflection (the reflection may be very light)

The first paragraph in the foreword seems to give a good summary

This report represents our best effort to provide a review of current LMS governance with some usable recommendations (which was the primary focus of the project brief) while at the same time presenting an alternative viewpoint in which the potential of “Elearning 2.0â€? would obviate the need for a central LMS as the main focus of an elearning strategy. The tension between the need to address LMS Governance and our own view that the educational significance of LMS is largely overemphasised and misunderstood is evident throughout the report.

And I particular like these sentiments

Existing problems must be made visible, so that they can be discussed and worked through, so that the skills and professionalism of the people that make up the university can be applied to the ongoing series of solutions, experiments and evaluations that will keep educational technology supporting new applications of the university’s work. As it has always done, the university must adapt, using technologies and models of understanding, in this case to reconcile teaching, research, IT, a changing environment, financial accountability and managerial models.

The report covers the following

  • A finding that LMS implementation did not include many of the standard expected components for organisational information system implementation.
  • Suggests 7 recommendations related to good governance practice
  • Talks about elearning 2.0 as an alternate model for elearning that does not need the LMS
  • Propose recommendations to ensure that universities are ready to evaluate and integrate new technologies than occured with LMSs.

The objectives of the LMS Governance Project are listed as

  1. To better align the policy, procedures and governance structures involving the LMS with institutional strategic directions on teaching and learning.
  2. To develop a community of practice to enable the sharing of experiences, ideas and resources relating to migration, implementation and governance.
  3. To provide a theoretical basis and a coherent research framework for the ongoing use of an LMS and to generate transferable governance models and processes in order to demonstrate organisational value.

All of which seems to resonate with my nascent and developing ideas around “REACT”.

Of course, I wonder how possible it is to have “institutional strategic directions on teaching and learning” given the diversity of L&T, plus a range of other factors.

The use of “adaptive cybernetic networks for regenerative organisational change” as a theoretical framework for understanding online learning governance is also of interest.

Governance

Relevant categories of governance to LMS design

  • corporate governance
    Including alignment of corporate activities with strategic direction, provision of reliable and salient information for executive action and enable management and board decision-making and accountability.
  • IT project and service governance
  • university governance

Best practice

The report includes the best paragraph that I’ve seen so far that explains my reservations around best practice

‘Best practice’ as a concept originates from the scientific management perspective of management articulated in the work of Taylor and Munsterberg (see Haslam, 2004). This approach assumes that efficiency is the most desirable aspect of work practice, and that there is ultimately a single “best possible manâ€?, “best possible workâ€? and “best possible effectâ€? towards which we should all strive. Best practice, even in domains where it could meaningfully be identified and applied, has a different aetiology from innovative practice and is in fact largely inimical to it as discussed elsewhere in this report. Best practice for LMS governance assumes that a single underlying model of LMS governance could capture the entire domain of LMS and we did not find this to be a readily supportable contention.

Metaphors and models considered harmful for e-learning

The adoption, implementation and support of e-learning is a difficult and extremely complex act. A variety of metaphors and models are drawn upon by participants to reduce the difficulty involved. These metaphors and models represent fundamental understandings of the participants and directly influences the decisions they make.

The position I’d like to take for a paper is that many of these metaphors and models are not at all helpful. Many are, in fact, incredibly harmful or limiting. In the following I try to develop a list of these metaphors and models.

The list includes (more to be added)

  • Herding cats
  • Not invented here syndrome
  • Techno-rationalist view of organisations and information systems
  • A central, enterprise level LMS is essential for a quality university
  • Universities must begin to act in a “business-like fashion” (Coaldrake et al, 2003)
  • The idea of benchmarking best practice in elearning

References

Coaldrake, P., Stedman, L and Little. P. (August 2003) Issues in Australian university governance. Brisbane: QUT.

Breakthrough learning – what lessons for staff development of academics?

Starting with a review by Will thalheimer I came across the book The Six Disciplines of Breakthrough Learning: How to Turn Training and Development into Business Results by Calhoun Wick et al.

I won’t bother summarising the book. The review by Will thalheimer does a good job of that and the Amazon link lets you take a look inside the book.

Instead I want to pontificate/consider/reflect upon what the six disciplines are and what they might mean in connection with my thoughts around what needs to change about how university academics are supported in their course design and staff development.

The six disciplines are

  1. Define outcomes in business terms.
  2. Design the Complete Experience.
  3. Deliver for Application
  4. Drive Follow-Through
  5. Deploy Active Support
  6. Document Results

How well does traditional university-based staff development and course design fit within this model? Not real well. I’ll limit my comments to those that I’ve experienced. How general these observations are could be a topic for more research.

What I like about this model, or at least how I interpret the model currently and that this interpretation is not based on actually reading the book, is that it moves beyond the disconnected staff development process that focuses on some specific goal and is complete once the training session is finished.

Reworking some of these disciplines in line with where I’m thinking of taking the REACT process

  1. Define outcomes
    Combine organisational goals with those of heads of school, the individual academics and students (the addition of students is a new bit that I need to think about a bit more). Use those to define the outcomes of any given year’s round of “REACT” sessions.
  2. Design the Complete Experience.
    I’m not sure about this one. Not a fan of upfront design. Potentially replace this with a more emergent design. This concern may also be due to a difference of scale. The original might be limited to a single learning intervention. I’m thinking about a process that might last for a term or a year.

    In my situation, one of the first sessions would require participants to develop their goals, based on some of the above outcomes, and from that collaboratively identify the learning/staff development (and other actions) that are required.

  3. Deliver for Application
    The design of the subsequent development is aimed at helping staff implement specific plans for their courses. It’s designed to provide what they need to apply their plans.
  4. Drive Follow-Through
    I feel this is linked to the above and below points.
  5. Deploy Active Support
    The participants in this whole process are just not the staff developers and the academics. A broad array of other folk including senior managers/policy makers and “support” staff (i.e. the graphic designers, instructional designers, technology people, editors etc. that will help implement the plans). They participate to give input into designs and collaborate with the planning of implementation.
  6. Document Results
    Write these up in a fixed format and provide them via electronic means to enable sharing and reuse. And, go the next step, ensure the subsequent rounds are informed by previous results.

The other obvious part of “document results” is an evaluation of the results against the defined outcomes. Assessment/evaluation is a topic on which Will Thalheimer has somethings to say about including mistakes people make and a quick assessment audit guide

Freeze and unfreeze – one problem with herding cats

I’m currently working on a paper/idea which is essentially seeking to argue that the herding cats metaphor. This metaphor’s most famous instantiation is the EDS herding cats commercial. It is a metaphor/phrase that is often used within higher education, especially when someone has tried to get academics to do something. It’s often equated to herding cats.

My basic premise is that herding cats is exactly the wrong way to encourage academics to change. To me it is indicative of the traditional top-down, I know best approach that is completely inappropriate, for a number of reasons, for academia.

Herding cats includes the herders and the cats. The assumption is that the herders know where they are supposed to go and how to effectively get the cats to this destination. In the current context within which most universities operate I don’t believe that any one actually knows what that destination should be and I think it questionable that anyone knows how to effectively get a significantly large group of academics to any fixed destination.

Related to this is the observation that I think the traditional change management mantra of “unfreeze, move, refreeze” is also inappropriate. I’ve written elsewhere about what I think is a more appropriate model. This is the model I’m struggling with how we might implement across a complex, contemporary university.

As part of my writing I wanted to identify the original source of this “freeze” idea. I may have achieve that this morning while reading Parchoma (2006) which includes the following

Lewin (1947) argued that in order to successfully facilitate change, organizational leaders need to undertake a three-step process: unfreezing, moving, and refreezing. Unfreezing involves destabilizing the status quo. Moving includes identifying and evaluating the relative strengths of forces within a social field, considering available options and initiating incremental change. A social field is defined as an “ecological setting”? in which “coexisting social entities, such as groups, subgroups, members, barriers, [and] channels of communication”? (p. 200) undergo periods of relative constancy and change. The “relative positions of the entities”? within the social field illustrate their roles as either driving or restraining forces (p. 200). Driving forces are defined as those forces that initiate and sustain change; restraining forces are defined as those forces that restrain or decrease the driving forces. Refreezing is the process of supporting a return to a sense of stability in the changed environment.

My preference for emergent development means that I don’t believe that there should be any freezing and consequently no need for unfreezing. Instead there should be continual, on-going emergence and change as the institution responds to changes in the environment.

Parchoma (2006) is even more useful as it covers the criticisms of Lewin’s field theory and also covers the responses to it. The criticisms include

  • Its linearity, simplicity and mechanistic approach
    This is one of the arguments for the emergent approach. How this has been responded to will be intersting.
  • Field theory can only enable incremental change and not more transformational change
    This is an argument that can also be attached to emergent development. There are some arguments against this. Parchoma (2006) identifies Burnes (2004) as arguing against the criticism of field theory and also of stating that “over time, incremental change can lead to radical transformations”. This is something I need to follow up.
  • The naive exclusion of issues of power and politics within organisations
  • Field theory has been percieved as a top-down approach to change management which lacks relevance to the culture of contemporary organisations (Dawson, 1994)
    Does that sound familiar at all? The response to this is the Lewin had a focus on identifying forces that included those within/between group and in particular those that head variant levels of power within and among organisations.

Elord and Tippett (2002) are also referenced as having responded to criticisms of Lewin.

The quality of Parchoma (2006) continues to improve, the further I read. Of course, the fact that it is reinforcing my current beliefs is only a very small contributing factor in that opinion.

Current structures and functions of the traditional academy may not reflect the “network enterprise”? norm of the corporate world (Norton, 2000). Networked enterprises are described in terms of a triangulation of initiatives, each of which work toward the goal of achieving maximum flexibility as a strategy for dealing with complexity, ambiguity, and continual change. Implementing a networked system effectively involves an inter-related and complex set of changes to conventional business practices, which can only be accomplished “if managers and workers understand”? that the changes do not constitute “a fixed way of doing things but, rather, a method, or philosophy of experimentation, of constantly testing existing procedures against proposed changes, of always searching for small ways to improve”? (Alcaly, 2003, p. 148).

References

Burnes, B. (2004). Kurt Lewin and the planned approach to change: A re-appraisal. Journal of Management Studies, 41(6), 977-1002

Elrod, P. D. II, & Tippett, D. D. (2002). The “Death Valley”? of change. Journal of Organizational Change Management, 15(3), 273-91.

Gale Parchoma, A proposed e-learning policy field for the academy, International Journal of Teaching and Learning in Higher Education, 18(3): 230-240

A problem with credentialism around teaching and learning

For at least the last 5 years there has been an increasing drive for university academics to have formal credentials in learning and teaching. The essential argument is that you wouldn’t go to a medical doctor who hadn’t been trained in medicine, so why should you be taught at university by a person who has been trained in teaching.

One problem with this is the assumption that a credential in teaching is the same as being a good, or even capable, teacher. But that’s not the issue I’m thinking about here.

In thinking about the REACT approach to helping staff design courses and elearning I’ve been moving towards a participatory, emergent process through which academics pick up the necessary training while they are developing their courses. Put another way, the learning design process provides the necessary training as part of doing learning design.

This idea has many potential problems and shortcomings. Apart from the fact that it is only in its early days of development. One of those problem could well be the clash with the widespread acceptance of credentialism for university teaching.

Parchoma (2006) writes

While the new economy’s reliance upon a well educated workforce for survival and success suggests a strong role for the academy in the future, cultural and value differences may impede corporate-academic collaboration. Corporate demands for knowledge workers who continually renew their knowledge for the purpose of sustaining innovation—but do not necessarily seek formal credentials for that knowledge—and may not be attuned to traditional university culture and values. The norms of the traditional academy may not well serve the corporate agenda, and may not wish to do so.

Will this be a problem? How might it be overcome?

There’s a bit more. Parchoma (2006) also says

Coping with the ambiguities of work as an experimental arena where there are no fixed processes or procedures will require an adaptable, informed, and innovative workforce, capable of high levels of effective interpersonal communication and collaboration. Members of this workforce will need to continuously renew their knowledge; and therefore, adopt learning as a life-long process. The resultant pressures on existing post-secondary educational institutions to provide continuing personalized education for adult learners via flexible, affordable, distributed learning options may become an increasingly strong driving force for change within the institutions themselves.

The implication is that universities need to change their teaching to keep up with the demands of their students which are created by the knowledge economy. For me, the implication is that universities also need to change how they teach their staff about learning and teaching to keep up with the demands created by the knowledge economy.

References

Gale Parchoma, A proposed e-learning policy field for the academy, International Journal of Teaching and Learning in Higher Education, 18(3): 230-240

Issues around staff development for e-learning

Just working my way through Pettit (2005) which tells the story of how the OUK used online conferencing for staff development around e-learning. What I’m finding useful to my situation is the literature review which covers some of the issues around this issue and which connects nicely with some earlier ideas.

For example

  • academics must be familiar with technologies before thinking about how to apply them in their course….many references on this
  • staff suffer from time-poverty
  • there are some issues around using online technology to help
  • rather than being “developed” through centralised programmes there should be local support within the existing discipline-based structures
  • this can make it more palatable for those that are reluctant and avoids the difficulties of top-down change in universitites
  • But, local support is costly in a large institution and can also result in fragmentation and entrenching of local differences
  • mentions the problems with reinventing the wheel
  • Not all exemplars are suitable across different disciplines
  • The problem of “staff developers” being seen as the university’s agent
  • the importance of selecting the participants

The ideas I have around the REACT process take a slightly different approach which might be useful.

This particular quote is interest, and is something I need to follow up.

it is important to acknowledge that academics possess varied knowledge bases and that one valuable source for informing the design of creative and effective programmes
of professional development may be found in the insights, experiences and knowledge
of the academics themselves’ (Ferman, 2002, p. 146).

References

John Pettit, Conferencing and Workshops: a blend for staff development, Education, Communication and Information, 5(3), 2005, 251-263

FERMAN , T. (2002) Academic professional development practice: what lecturers find valuable, The International Journal for Academic Development, 7(2), pp. 146–58.

Process and product are inextricably linked

Simon (1996, p 130) writing about the connection between what is designed (product) and the process used to designed it, says

What we ordinarily call “style”? may stem just as much from these decisions about the design process as from alternative emphases on the goals to be realized through the final design … both the shape of the design and organization of the design process are essential components of a theory of design.

The process used to design something, influences the end result. The nature of the process used, to some extent, limits what can be produced.

Earlier I wrote about my concerns around traditional instructional design as practiced at CQU in the late 1990s and how I thought participatory design might offer a different approach.

This quote from Simon has a nice resonance with this idea. It also provides a potential frame for a publication comparing the original approach with the new approach (whatever form that takes) and then doing a comparison of the outcomes.

Participatory design as the basis for staff development and course design

Origins

Teemu Leinonen has a good post about Participatory Design and Scenarios in Learning. It outlines some norms behind participatory design, explains the use of scenarios and makes connections between participatory designs and other “paradigms” (e.g. social constructivist theory of learning and open source development).

Relevance

The value I take from the post is that it resonates with the the many ideas that are floating around in my head in connection with the new position I’ll be taking up soon (shh, don’t tell any one). I’ll be the clueless middle manager responsible for the group that supports CQU’s academics design their learning and teaching.

The problem

As one of those academics, I was always a bit critical of the traditional instructional design approach taken at CQU. Well, at least traditional for institutions that were into print-based distance education, like CQU.

This model can be characterised as having

  • SME(s) – subject matter experts
    A single (or small group) of academics in charge of the course. The people who need the advice on how to “package” their content to achieve a goal.
  • ID – instructional designer
    The expert in education that was going to help the SME(s) achieve their goals.

Problems I had with this model (which may say more about me than the model itself) included

  • The IDs having little or no knowledge of the discipline.
  • The IDs having a limited understanding of education and what was possible with
    technology.
    We all suffer from bounded rationality. So this is no surprise. I, as the SME, also suffered from the same limitation, though my limited circle was different. This was especially problematic during the 90s (when most of my teaching was done) because of the advent of the Internet and the Web and the impact this was having on distance education. The poor print-based IDs didn’t have the knowledge (or support) to fully grasp the implications.

The above is not meant to read as a criticism of the individuals involved. This is not a discussion about the limitations of the individuals. It’s an attempt to get to the limitations of the model and start thinking about a better model.

Some of the points from Teemu’s post help pick out some of what was wrong with the model

  • “Respect of the people – they are the best experts of their own life and activities. They should be supported to have a voice in the design process.”
    Under the old model, the design of course material was goverened by
    a single centralised model. SMEs did not have a voice in the design process.
  • “Knowing that the people are the primary course of innovation. The ideas emerge in collaboration with participants who are representing different stakeholders and backgrounds. ”
    The old model usually only involved two people: the SME and the ID. A very limited set of stakeholders and backgrounds. For example, no students were directly involved.
  • “Focus on systems. Systems are networks of people, their practices, technologies and artifacts embedded to the actions in a particular context.”
    The old model emphasised the individual course as the level of design. Little or no thought was given to the wider systems at CQU and the potential need for it to change. This was especially problematic during the 90s and the “Internet revolution”.

Solutions

In the past, I’ve been involved in a number of half-hearted attempts to address this and related problems. There was some work on the use of design patterns in online learning (publication 1 and 2)
which then morphed into the REACT idea.

Both these failed, for various reasons but mostly due to my lack of follow through. I’m hoping that we can solve some of those going forward.

In particular, I am interested in taking lessons from participatory design, the REACT process and the patterns work to develop a different model for curriculum design at CQU.

Some initial ideas

  • All curriculum design is done in 1 or 2 day sessions, typically off-campus
  • The sessions include a number of SMEs, IDs and representatives from other related groups. For example, librarians, various technologists and potentially some “important” guests.
  • The sessions would essentially be participatory design sessions where each academic would present their problem and the group would help develop scenarios. The aim being to draw on the diversity of the participants.
  • The scenarios and other outcomes would be developed in some set, shareable, digital format that others can use and benefit from.
  • There might be different stages in the sessions equating to steps in the design process. For example, start with an analysis sessions (emphasis on analysing the current state of the course and figuring out what should be done), design (figure out how to achieve the outcome/address the problem) and evaluation (report back on what was done).
  • The presence of other staff would also aim to enable some level of system design. For example, if there is a trend amongst many of the academics the other representatives could use this as a spur to redesign the entire system (e.g. policies, technology etc.)