Thesis abstract v1.0b

The following is yet another sign that the thesis is finally getting close to submission. The following is the second version of an abstract of the thesis. There have been some changes, feedback always welcome.

Abstract

This thesis seeks to offer an answer to the problem of how to design, implement and support information systems that effectively and efficiently support e-learning within universities. This is a problem that is increasingly prevalent and important to the operation of universities. It is also a problem where existing solutions are limited in terms of variety, quality and explicit theoretical guidance. This thesis formulates a specific Information Systems Design Theory (ISDT) – An Information Systems Design Theory for Emergent University E-learning Systems – as one answer to this problem.

The ISDT is formulated using an iterative action research cycle that encompasses the design, support and evolution of the Webfuse information system at Central Queensland University (CQU) from 1996 through 2009. The Webfuse system was used by tens of thousands of staff and students. It is the knowledge gained through this experience that, at two separate stages, is used to formulate ISDTs culminating in An Information Systems Design Theory for Emergent University E-learning Systems.

The final ISDT recognises that diversity and rapid, on-going change are, for a number of reasons, the key characteristics of e-learning within universities. Consequently, the ISDT specifies both process and product models that aim to enable the e-learning information systems to be emergent. In particular the ISDT proposes that emergent elearning information systems will: encourage and enable greater levels of e-learning adoption in terms of quantity, quality and diversity; as well as provide a level of differentiation and competitive advantage for the institution.

This thesis makes two additional contributions. First, the Ps Framework is developed and used to analyse the current, dominant practice of providing e-learning information systems within universities. The resulting analysis reveals a significant mismatch between the requirements of e-learning within universities and the characteristics of the product and process models used by the dominant approach to supporting e-learning within universities. It is this mismatch that the ISDT seeks to address. Second, problems with the existing approach for specifying ISDTs led to the development and widespread acceptance of an improved method of ISDT specification.

No burden for TEQSA, yea right

The quote attributed to Lisa Paul illustrates the vast distance between her experience and that of staff within universities. The phrase “we’ll get to that after the AUQA audit” has been oft repeated at one institution I’m familiar with for at the best part of a year.

It signifies important tasks being put to the side while the entire focus of the institution is on presenting the same, appropriate story to the AUQA auditors when they arrive. It signifies how important it is to senior management on limited term contracts to get a good tick from external agencies, regardless or the reality. It signifies how the quality of teaching materials and experience suffers while the institution expends resources on printing posters and handouts, and purchasing coffee mugs to spread the “AUQA story”.

A new regulator, especially one that is expected to have some additional “bite”, is only going to make this problem worse.

People talk about “teaching to the test”, (at least some) universities are now “managing to the audit”.

Amplify’d from www.theaustralian.com.au

Asked about the burden under a new regulator, department secretary Lisa Paul said universities already had to contend with audits by the Australian Universities Quality Agency.

“There is no guarantee yet, I would have thought, that there is going to be any more burden on universities than there already has been,” she said.

Read more at www.theaustralian.com.au

 

Ideas for extending the flexibility of BIM

The following explores some ideas for extensions to BIM based on a blog post from Dave Cormier.

Dave’s post

My summary of the post is that it describes the evolution of Dave’s approach to teaching/learning through a variety of attempts to build systems to support learning through to the eventual realisation that the system he wants to use is just the Internet.

The end realisation of this is in a recent course Dave led. The students did their work “in the wild”. In online tools like twitter and WordPress.com. So, how did Dave evaluate the work, in his words

I told them that they would need to keep track of each and everything that they wanted me to evaluate and put it in a googledoc. I said “I”m not going to look for it, I want you to interact with people, keep track of it, and tell me why it was important.”

Connection with BIM

BIM is a Moodle activity Module I wrote. It’s aimed to allow students to have their own blog and do their work on that blog, but still provide a way that teaching staff can easily track, comment, mark, and manage that work. i.e. students do their work where they want, staff get the support/integration they need. BIM arguably stands somewhere between Dave’s “phase 4” and “the internet is my PLE”.

However, there’s some significant limitations in BIM. Dave’s post has given me some ideas for extending BIM.

Ideas for extending BIM

In Dave’s model, the students identify what is important and gather it together. The model in BIM at the moment, is that the teacher who creates the BIM activity specifies a set of questions or posts for the student. For example, the teacher wants 5 reflective posts from the student, so they create 5 questions in BIM with the titles “Reflective post #1”, “Reflective post #2” etc. After registering their blog with BIM, the students eventually create posts with the necessary fixed titles.

What if BIM allowed the students to specify what was interesting? This could be as simple as the existing idea of allowing them to allocate blog posts to specific BIM questions. But perhaps it might be necessary to allow the students to submit as many, or as few, items as they want. Not necessarily submit some pre-defined number.

Mmm, I wonder if this could be achieved by the student using a specific tag in del.icio.us to identify and comment on what they thought was important and then registering the del.ico.us RSS feed for that tag? This would work, however, BIM probably wouldn’t be able able keep a complete copy of the original artifact. Thought it would allow you to aggregate resources from a range of different places, just not a blog.

Which raises the idea of a student being able to register multiple feeds, not just the one. But this brings up the requirement for the student to be able to curate what BIM takes from these feeds. This could be done through the use of certain tags or keywords in the source, or through an interface in BIM.

Mm, is this becoming a e-portfolio? If I argued no, the distinction would be that BIM serves a single purpose, supporting evaluation/assessment of student work. BIM has nothing to do with sharing that work with others. That’s up to the student to do, where they chose to do it (e.g. on the social media they used to create what’s been put into BIM).

Of course, this will have to wait until post thesis and also post production of BIM v2.0

Time for a career change?

As work on the thesis is slowly finishing, I’m increasingly starting to think about the next chapter. What am I going to do post-PhD? The answer, potentially surprising for some, is probably going to high school teaching.

Some context

After 20 years working as an academic for a local university, around August this year my position was made redundant. With not a little relief I left the institution eager to complete the thesis and see what the next chapter would bring.

As part of this process I did apply for a few jobs around university L&T support, mostly around e-learning, but a few months down the track nothing has come of them. I don’t necessarily recommend this type of process as it can do some interesting things to your self-confidence. That said, there were some folk who were very nice. But, in the end, there was always a problem with those potential jobs, they required a move away from where we live.

First shot with the new camera

While there’s something to be said for living in some of those places. The family is quite settled where we are and it’s a brilliant place to bring up the kids in a rural setting. Not to mention that the wife has a hobby that isn’t easy to translate to other settings, breeding race horses.

Mother and foal

More importantly, there are my two young sons (3 and 5). Being away from them at this time, would not be fun by any measure.

What can I do?

So the question becomes what type of career can I build while staying in the local area. I’m legally not able to work at the one university in the local area until at least late next year. Even then I’m not sure going back as a member of staff would be a good thing. Not to mention the fact that the local university is being effected as much, if not more so, by the downturn in overseas student enrolments that is currently impacting many Australian universities. So, there aren’t likely to be jobs at that institution, and even less likely folk who would want to appoint me. It is interesting and somewhat related that, as I found when applying for some jobs, I’m not really “technical enough” for some jobs and not really “educational enough” for some others.

That said, I like to teach. So high school teaching is an option. It requires me to return to study for a one year graduate diploma. Thankfully that’s easily possible. Though returning to the same local institution as a student, after 20 years as a staff member will be very, very interesting.

High school teaching doesn’t pay well, but that’s not a problem for us. It has great holidays that provide the opportunity to spend more time with the boys. It hopefully allows us to stay where we are, and it enables me to become a bit more “educational”. I also think that through this career change, I can make some sort of contribution to teaching and the local community, at least in some small way.

What will I do?

I’ve started talking to a few local high school teachers who I know in various ways to get a better feel for the life style. So far so good, but I have a few more to talk with and who knows what might happen. It’s another four or five months until any study would commence, a lot can happen in that time.

So, if you have any insights about high school teaching as a second career, or have a brilliant high-paying job to offer me, feel free to share.

Change in education, failure to learn and the commodification of university

20 years ago, straight from graduating, I started work as a part-time tutor within a Department of Mathematics and Computing. Within a few years it was obvious to a naive computing person that the mathematics part of the department was in trouble. Fewer and fewer people were enrolling in the mathematics programs, but they still had a fairly large group of academics (mostly doing service teaching – i.e. stats for business undergrads). At the same time, computing was exploding in enrolments. We had too few staff and too many students. The perception was that it wasn’t fair on us, to carry them.

How the worm turns. The descendant of that department is now a combination of computing (information technology), mathematics (mostly service), and information systems (a business flavour of computing). Computing had flourished around the turn of the century with the Y2K and dot-com influences, combined with international students in Australia. But the bubble has burst and the computing disciplines now find themselves struggling with large numbers of staff and bugger all students. They are suffering the same problem as the mathematicians.

One of the reasons I got out of computing and into L&T support within the university sector a few years ago was that I could see writing on the wall. What’s worse, I was not confident that the strategies being adopted (or likely to be adopted) would do anything to turn this around. I’m still not confident.

Infancy is perpetual

As a sign of getting older, this (as with many other events) continually reminds me of the Santayana quote

Progress, far from consisting in change, depends on retentiveness. When change is absolute there remains no being to improve and no direction is set for possible improvement: and when experience is not retained, as among savages, infancy is perpetual. Those who cannot remember the past are condemned to repeat it.

Here’s how this problem was planned to be solved in the past:

  • Back in the mid to late 1990s, the mathematicians (led by a new professor) redesigned/recreated a mathematics degree that would be sexy to potential students and thus create a huge influx of new enrolments.
  • In the mid to late 2000s, the computing folk (led by a Dean and a professor) redesigned/recreated the computing degree so it would be sexy to potential students and thus create a huge influx of new enrolments.
    A key part of that initial plan was “games programming”, the apparent saviour of the information technology discipline in Australia.

Both failed.

What do you think the solution is to the new problem? Especially for computing?

Yep, you’re right. They are going to discuss changes needed to programs that will help increase enrolments. I hear that games programming is being thrown around again as possible solution.

The commodification of knowledge

Apart from the problem of failing to learn from the past, the other fundamental problem that I see in this response is a limited and incorrect view of higher education. It’s a view that sees the product of a university to be the degree program. It’s the view that leads to the above solution. Our product is our program, people don’t like our programs, so we have to reorganise the program or create new ones. The trend to creating new programs leads to the situation where “Forensic science graduates outnumber criminals”.

It’s the focus on the product that has led university leaders to place less emphasis on the process and the people. More importantly there hasn’t been an emphasis on the alignment between process, people, and product. It’s assumed that a university can create and teach any program. Oh, there is demand for a program in Paramedic Science. Oh, then we’d better employ a few folk in Paramedic Science, crank the product creation process and offer a program in Paramedic Science and people will come.

It’s a market driven, techo-rational approach that assumes a traditional analyse, design, implement, evaluate cycle that fails to understand the full complexity of what is required and the changing nature of surrounding environment.

Alignment of process, people and product

This type of process is externally driven and led by “leaders”. It assumes that there are people who are smart enough to predict what “consumers” will want from the University. It assumes that those folk can then create and control an environment that leads to the creation of what the consumers want. After 20 years within the university sector, I question all of those assumptions.

It also ignores the centrality of the people, but then is a long running theme of Australian higher education. I’m talking here about the academics. This process assumes that the academics are interchangeable. That there isn’t a significant difference between academics. That as long as you have an academic assigned to a course the outcome will be a good one. Especially if the institution is quality assured and has a significant number of checklists to control what is done.

As but one example, I’ve been trying to think of the programs I’m familiar with and asking the question “How many of the staff teaching courses in that program, actively engage in research/practice within the area they are teaching?”. In all of the examples I can think of, the answer is very, very few.

There are many reasons for this, but a major one is that this type of process starts with the end-product, and jury rigs a connection with the people. Often very badly.

The alternatives

There’s no easy fix for this problem. The idea that a new program will change things assumes that there are easy fixes. Any truly effective solution is going to be really, really hard and require some guts and intelligence on the part of the people involved. From that foundation, I can only see two alternatives:

  1. “Right-size” the program; or
    i.e. if you accept the status quo, then there are only every going to be X students enrolled in the program (where X may be 0) and the number of staff you have should match that. Bite the bullet and effectively and appropriately “right-size” the number of staff required.
  2. Re-think the problem.
    This takes guts and is briefly commented on below.

Re-think the problem

It doesn’t take a genius to see that society (in the broadest possible sense), how it works, and what it expects from education is changing. This post talks about Sir Ken Robinson’s take and mentions some of the societal changes. Tomaz captures a list of the almost taken for granted changes and starts talking about the fears that prevent people from engaging with them. Both of these are mostly school-focused, but they exist within higher education. Stephen Downes talks about these and other factors more broadly.

With all of those factors (and more) influencing how and what is expected from education, then old thinking isn’t going to cut it. Especially old thinking that assumes it knows the answers. Which influences what I’d suggest as the solution.

Focus on what you do well and learn

Let’s get specific. For the computing program, my suggestion would be figure out what you are good at. Or what the academics in your program are interested in. Focus on that, build on it, tend to ignore the rest, create networks around that, and learn.

Remember, I said above that there is no easy answer. This is not a simple process. It will be ruined by simple answers such as let everyone do there own thing or let them do what they’ve always done.

Some brief expansion on only a couple of those points.

Good at

Importantly, I would define “good at” not be the number of formal qualifications held, self-reporting, or number of research grants/journal publications. I would measure “good at” by the size, diversity and quality of the network surrounding the teaching and research the academics do in the area. The number of folk that are using or following the work the person does in that area.

For example, back in the mid-1990s I taught Systems Administration. We produced some resources. Those resources were widely used across the world. People were translating them into other languages. That’s being good at something.

But the network doesn’t have to be this focused on production. One of the significant limitations of the work in Systems Administration is that we never really grew a reciprocal network with these people where we used their materials and insights as much as they used ours.

This doesn’t mean that “good at” means you are the expert in the network, but that you are actively participating in the network. That you are learning.

Learning

Learning doesn’t apply to just to what you are teaching. It also applies to how you teach and how the institution supports you in your teaching and your practice. That learning has to be used to build on what you’re doing.

Build on

Too much of what goes on institutions is historical. It doesn’t change. It doesn’t learn. The “learning” isn’t enough, practice and the organisation has to build on what is learned. It has to be continually emerging.

For example, back when we were doing the work on Systems Administration, we had requests from hundreds of people from throughout the world to do the course. They wanted to pay and they wanted to do it entirely online.

The institutional rules at the time required overseas students to pay $1200 to take the course. There was no freedom for us to charge less because we planned to offer and suppor the course in a different way. There was no way for us to build on what was being done.

Time to get back to the thesis.

The problem with blended learning

Once I get out from under the PhD, I’m hoping to expand an initial idea attacking the rhetoric of blended learning currently in vogue in some parts of higher education. The purpose of this quote is to save a quote that I’ve just come across for that expansion, and also to summarise the idea.

At the very least, this has helped me realise that it isn’t the original idea of blended learning that is at fault, it’s how the idea has been translated into action within some institutions. i.e. as a fad, with little understanding of the complex complexities and even less engagement with them.

Blended learning described

In describing blended learning, Garrison and Kanuka (2004) offer this

A blended learning design represents a significant departure from either of these approaches. It represents a fundamental reconceptualization and reorganization of the teaching and learning dynamic, starting with various specific contextual needs and contingencies (e.g., discipline, developmental level, and resources). In this respect, no two blended learning designs are identical. This introduces the great complexity of blended learning.

This is a significantly more nuanced understanding of blended learning than is typically used. In such typical rhetoric, blended learning is typically equated with the adoption of an LMS. Any “fundamental reconceptualization and reogranization” of learning can only occur within the confines of the LMS.

More broadly, it can only occur within the confines of the existing practices within the institution. Practices that typically include workload calculation formulas that have the provision of lectures and tutorials embedded within them. Institutional management don’t engage with the complexity of Garrison and Kanuka’s conception of blended learning.

What’s worse, the idea that “no two blended learning designs are identical” clashes horribly with the commodification of learning and requirements for accountability.

References

Garrison, R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7(2), 95-104.

Making the LMS more like the Globe Theatre: distributed cognition, the extended mind and Moodle

The following draws together some work on distributed cognition in the Globe Theatre and some early thoughts on how that might be useful in improving the design of learning management systems such as Moodle (there’s also an argument to be made about university teaching and learning, but I’ll mostly leave that alone). While primarily focusing on helping teachers teach and thus improve the student learning experience, the ideas could equally be applied to learners (and probably have been).

At the core of the following seems to be the question of whether o not the design of LMS, and e-learning tools in general, have been limited due to the following observation made by Tribble (2005)

consistently distorted by a tendency to view cognition as individual rather than social, which has caused us to imagine the workings of complex group structures in mechanistic terms. In other words, we have mistakenly assumed that properties of the system as a whole must be possessed by each individual within it.

Shakespeare's - Globe Theatre

The post continues to build some recent thoughts about this issue and was sparked by listening to this episode from ABC Radio National show – The Philosophers Zone. This led me to the article Distributing Cognition in the Globe by Tribble (2005).

After finishing the post, the idea of adding “plots” to LMS tools might be the most interesting outcome. Will need to do more reading and thinking. Must be other folk looking at this, pointers?

Cognitive demands of the Elizabethan repertory system

Imagine you are a member of an Elizabethan repertory theatre company in the late 16th century. Just how cognitively demanding is your work? Tribble (2005) draws on a number of sources to give the following examples:

  • As a leading player you would have to remember 71 different roles, around 52 of which were new.
  • Your company would be putting on 6 plays a week, there would be infrequent repetition, and would have a new play every fortnight.

This appears to be quite different, and cognitively a lot more demanding, than more modern theatre companies.

So how did they do it? Tribble (2005) summarises some literature that suggests it was achieved through “mechanistic” approaches such as: actors performing the same type of role, learning roles by imitation, and having little or not interest in creative interpretation.

How did the Globe and distributed cognition help?

Tribble’s (2005) argument is that the design of theatre system – embedded in the theatres, the plots, actors’ roles, the plays’ verbal structures, the apprentice system, and the organisational practices of the companies – provided elements of cognitive structure that helped individuals and the company to fulfill their cognitive demands. In particular, she argues that the design of the theatres played a part

The more thinking that can be off-loaded onto the environment, the more mental energy remains available for those tasks that are primarily internal (memory for the spoken lines, for instance).

For example, it is argued that “plots” – summaries/directions of when, who enters the scene – hanging back stage from pegs by stage entrances provided players with reminders of the order of scenes. Especially helpful for actors playing multiple small roles. Another is that rather than all players having the full script, they instead have their lines in detail and various cues to indicate when they are to be performed. Similarly, it’s argued that some of the physical spaces within the theatre (e.g. a balcony) provided cognitive hints as to what type of scene you were meant to perform.

The podcast that brought me to Tribble and her work includes a better description starting at about 13m48s. Tribble’s summary

The productive constraint of the stripped down part reduces the need to filter signal (one’s own part) from noise (everyone else’s); the plot provides a schematic diagram of the shape of the play as a whole to supplement the part; the physical space of the theater and the conventions of movement it supports enable the transition from the two-dimensional maps of plot and part to its three-dimensional embodiment onstage; and the structures and protocols of the theatrical company pass on its practices to new members. Such a theater can best be understood, in other words, through a framework that takes group practices seriously, that assumes that systems can work well, and that sees individual agency as constrained but not contained by these practices.

Tribble (2005) draws inspiration from Hutchins (1995) and describes on of his contentions as

that the lines between “inside” and “outside” are frequently misdrawn or misidentified, “creat[ing] the impression that individual minds operate in isolation and encourag[ing] us to mistake the properties of complex sociocultural systems for the properties of individual minds.”

and that when the design of these systems are done well

“Novice[s]” can thus be “embed[ded]” in the system and perform at a level far above that which they could attain by means of their individual cognitive powers alone.

The design of the LMS and the tyranny of the individual mind

It is my suggestion that the design of the LMS (and probably the assumptions underpinning much of the actions around improving university teaching) is limited by the tyranny of the individual mind. Sturgess and Nouwens (2004) describe one universities rationale for an LMS as being

to enable teaching staff to develop and
manage online courses with little professional support

. This fits with the broader practice of university teaching being essentially a solo act. It’s my course, I’ll teach it my way. Give me control of the course on the LMS and let me do my thing.

It is assumed that the academic (or their proxy who has been employed to create the course) has the skills and knowledge to effectively use the features of the LMS to prepare the course site. When adopting a new LMS, a focus for most institutions is on training sessions to provide these skills and knowledge to staff.

But what additional scaffolding or “distributed cognition” does the design of most LMS provide? In my experience it is generally limited to either some canned “help” document or the contact details of a support group. The “help” resources are usually fairly limited and it’s not unusual for them to be out of date. The support group are generally keen and willing to help (often above and beyond the call of duty) but are often limited to knowing where the academic might find a query, or about how to use the LMS features as they exist.

I’m not sure that most LMS are designed in a way such that

“Novice[s]” can thus be “embed[ded]” in the system and perform at a level far above that which they could attain by means of their individual cognitive powers alone.

. The evidence from past experience with LMS seems to indicate that most e-learning is not all that good, perhaps indicative of how well the LMS support distributed cognition.

Some ideas for BIM

BIM is a Moodle module that I’ve written. It’s probably going to be the first place I experiment with some of these ideas. Some initial thoughts:

  • Improve the help documentation provided with BIM to include the videos that have been produced and also provide links and summaries of some of the literature around the use of blogs in higher education.
    This is a simple first step that arguably should have been done already.
  • Link the help documentation with some the BIM community.
    Rather than just static help documentation, provide mechanisms for asking questions of and seeing answers/resources from a broader BIM community.
  • Design a scaffolded “BIM configuration” process informed by common problems and linked to the help documentation, especially the literature.
    As it stands, the BIM configuration process is your typical IT artifact. It is a better representation of the data requirements of BIM than a process that helps academics create and design a BIM activity that is effective.
  • Figure out a way to develop standard BIM “plots”.
    There are common events in the use of a BIM activity (e.g. not all students have registered a blog, not all the students within the course have been allocated to groups and markers, etc.). These can be picked up from the BIM interface, but the academic has to apply a fair bit of cognition in understanding how to do this.

    A better approach might be to implement some “plots” (in the sense described above) that suggest when and how to handle a particular event. For example, when BIM sees that not all students in a course are allocated to groups, generate a “plot” entry that includes a list of the unallocated students and links to the Moodle group allocation process and the help documentation associated with groups explaining this problem.

Ideas for Moodle

Some of the ideas for BIM could potentially be applied to any complex process within the use of an LMS/Moodle such as the design of a course site or the management of a discussion forum. This links to some extent to the earlier suggestion of combining analytics and discussion forums.

Misc reflections

Beginning to wonder how much of this is related to the design of computer systems being done by programmers more interested in representing the models underpinning the system, rather than providing cognitive support to the folk using the system?

Wondering how much the type of support given to academics using an LMS within a University context in terms of the skills of the support staff and the organisational structures within which they work limits the provision of appropriate cognitive support?

Wondering if an increasingly hierarchical organisational structure can effectively provide the type of distributed cognition that might actually help improve the LMS and university teaching?

Wondering if the hierarchical organisational structure can effectively grok the need for this type of approach? Can it provide the environment in which this type of work can grow?

References

Hutchins, E. (1995). Cognition in the Wild. Cambridge, MA, MIT Press.

Sturgess, P. and F. Nouwens (2004). “Evaluation of online learning management systems.” Turkish Online Journal of Distance Education 5(3).

Tribble, E. B. (2005). “Distributing cognition in the Globe.” Shakespeare Quarterly 56(2): 135-155.