Month: February 2009 Page 2 of 3

Wesley Fryer has a post summarising a talk given by Sir Ken Robinson. I’m pulling out a few relevant quotes/recollections for later use.

Update: One of the comments on Wesley Fryer’s post points to video of Sir Ken giving a similar talk in another venue.

Common sense and innovation

Common sense (the things we take for granted) is the big obstacle for innovation

Resonates for me because on the main aims of my research in e-learning is that the “common sense” that surrounds current practice in e-learning is a big obstacle for innovation (and adoption, acceptance…). A perspective expanded in this post.

Excellence comes through customizing

The enemy of raising standards is conformity

Quality through consistency has been one of my bug bears for over 10 years. Talked about briefly here. I’m a fan of Oscar Wilde’s take on consistency

Consistency is the last refuge of the unimaginative.

Metaphors for higher education

a better metaphor for education is not manufacturing, but is agriculture

When you consider much of higher education is importing practices from manufacturing…

Modified TPACK diagram to illustrate the reality of the entanglement/mess

Frameworks and representation – tidy versus messy

By admin

On February 16, 2009

In Chapter 2, missingPs, PsFramework, Uncategorized

I’m a fan of frameworks and taxonomies. Also known as theories for understanding (Gregor, 2006). It’s the understanding part that I like. They provide, or at least good ones do, a leg up in understanding difficult concepts. As Mischra and Koehler (2006, p 1019) say

Having a framework goes beyond merely identifying problems with current approaches; it offers new ways of looking at and perceiving phenomena and offers information on which to base sound, pragmatic decision making.

As it happens, I’m currently doing a lot of work around one framework and its application and the following arises out of that work.

Two of my current most used frameworks include Dave Snowden’s cynefin framework (Snowden and Boone, 2007) and Mischra and Koehler’s TPACK (2006). Representation is important to frameworks. The cynefin framework, in particular, has a very specific representation that has very specific meaning and purpose.

The TPACK crew have just released an updated representation of their framework (see the image to the left). I particularly like the addition of ‘contexts’ around the outside. The use of ven diagrams is important, one of the contributions of TPACK is the overlaps.

Tidy versus messy

One of the things I don’t like about frameworks is that they have (for very good reasons) to be tidy. This certainly helps understanding, a key purpose of frameworks, but it also can give the false impression of tidiness, of simplicity of a tame problem. My interest is currently in e-learning within universities, which I consider to be extremely messy. To me it is an example of a wicked problem.

Last week I ran a session on course analysis and design for some CQUniversity academic staff. I used TPACK as one of the major themes. However, at one point I really wanted to emphasise to the participants that none of our discussions should be taken to assume that this is a neat and simple problem. The image to the left is the one I used to reinforce this (they’d already seen the tidy version of TPACK).

In doing this, I sacrificed much of the representational value of TPACK to highlight the messiness involved.

The Ps Framework – Tidy versus messy

For about 3 years (this presentation is the first public evidence) I’ve been working on what is now known as the Ps Framework as part of my PhD.

The first representation of the Ps Framework, taken from the first presentation is included below. A photo of some frozen peas used as a “pun”. The arrows are included, but don’t really mean anything. Still very early days.

The next public iteration of a graphical representation of the Ps Framework was the following one for a more recent presentation (you can even watch the video of this one). In this “Place” becomes the underlying context for all the other Ps. Much like the addition of context in TPACK. The frozen peas disappear for nice tidy circles (to some extent each one is meant to be a pea) and the arrows are still there. The arrows are meant to indicate that each of the Ps impacts upon the other in some unspecified way.

I had to prepare the above images to deadlines for presentations. I never liked them. Too tidy and they appeared to indicate linear or simple connections between the individual Ps. I don’t believe that. The relationships between these Ps when talking about e-learning implementation within universities is messy, complex and unpredictable – at least beforehand.

So I had to come up with something else. For the last few months of last year Jocene, Nathaniel and I spent a lot of time discussing and arguing about how to represent the Ps Framework. The following is my current best effort – it’s the effort I’ll be using this week at ANU.

I have a range of problems with this representation including:

It’s still a little too structured.
i.e. People only overlaps with Past Experience, Purpose and Process. Those overlaps aren’t intentional. They aren’t meant to represent some specific connection. I’m not sure what the connections are, I have an inkling that each and everyone is connected/overlaps with the other but I am stuck with this current conceptualisation.
It’s too static.
The relationships between these components is forever changing. Universities and the place they inhabit are continually changing, each of the other components are changing and each change has some, unpredictable impact on the other components. In my mind I see this dynamic representation of this image where each component is seething and roiling and impacting upon each other.
It doesn’t capture perspective.
Still not certain if this should be another P added to the framework or whether different instantiations of the Ps Framework represent different perspectives. I tend to prefer the latter, but then that leaves unsaid the important point about the perspectives of different groups being very diverse and that this is one of the fundamental problems with e-learning within universities.

Any suggestions?

References

Gregor, S. (2006). “The nature of theory in information systems.” MIS Quarterly 30(3): 611-642.

Mishra, P. and M. Koehler (2006). “Technological pedagogical content knowledge: A framework for teacher knowledge.” Teachers College Record 108(6): 1017-1054.

Snowden, D.J. Boone, M. “A Leader’s Framework for Decision Making”. Harvard Business Review, November 2007, pp. 69-76.

New ways of thinking – quote

By admin

On February 15, 2009

In quotes, Uncategorized

Came across the following quote in Mischra and Koehler (2006), storing it here for future use.

The important thing in science is not so much to obtain new facts as to discover new ways of thinking about them.
— Sir William Henry Bragg

References

Mishra, P. and M. Koehler (2006). “Technological pedagogical content knowledge: A framework for teacher knowledge.” Teachers College Record 108(6): 1017-1054.

RSS feeds into course management systems – why?

By admin

On February 15, 2009

In elearning, ples@CQUni, Uncategorized, web 2.0 course sites

Last night I was looking for some information about recording audio for powerpoint presentations in order to create a slidecast

Aside: I like Slideshare and I like creating slidecasts. However, synchronising the audio with each slide is a pain, even using the interface provided by Slideshare. I’d much prefer being able to record the audio while giving the presentation and having it automatically synchronised. A while ago I thought we had a process using Powerpoint, but no. Bloody powerpoint keeps cutting off the last few seconds of the audio for each slide. To get it to work you have to pause for 5 seconds at the end of each slide. If you have any insight into how to fix this, please let me know. I can’t even find any mention of this problem via Google.

While searching for some information I came across the TLT Group’s wordpress blog because of the low threshold applications included some stuff on narrations. It also had an LTA on integrating RSS feeds into a course management system.

I sent this around to some folk at the PLEs@CQU project and some others. One of them responded with

I am not sure of the advantages of having RSS feeds go through the CMS. It is an easy thing for individuals to set up in their own, online personal learning environments.

It’s easy to do, not

Some of the other low technology applications included on the TLT site include

Personally, I’d class these tasks as much simpler and more familiar to people than integrating RSS into a CMS.

The definition for an LTA used on the TLT blog is

A Low Threshold Application (LTA) is a teaching/learning application of information technology that is reliable, accessible, easy to learn, non-intimidating and (incrementally) inexpensive.Each LTA has observable positive consequences, and contributes to important long term changes in teaching and/or learning. “… the potential user (teacher or learner) perceives an LTA as NOT challenging, not intimidating, not requiring a lot of additional work or new thinking.LTAs… are also ‘low-threshold’ in the sense of having low INCREMENTAL costs for purchase, training, support, and maintenance.”

Even though they are low threshold, you would be surprised at the number of academics who do not know how to carry out these tasks. Computer literacy amongst academics remains fairly low. I also think the same applies for students. Most of these folk know how to do what they do regularly – email, IM etc. But there are few people who are comfortable with and able to explore applications and think of how they can harness the features of technology to improve education.

Especially if it requires a rethinking of how they teach.

Advantages

The uncertainty held about the advantages of this approach is, potentially, one example of this difficulty people have of applying new features of technology to learning and teaching. Some possible examples follow, but they mostly come down to the following description

Incorporating a newsfeed into your WebCT course is a great way to get dynamic, changing content into the password protected environment of WebCT.Potential uses include creating an up to date ‘breaking information’ news source for your class.

which comes from this page which is pointed to from the LTA RSS page.

The example used on that page is for the academic to maintain a course blog that they use to keep students aware of events. This is similar to what was done on the EDED11448 website for “latest discussion”.

The EDED11448 website also shows a more interesting example of this practice in the portfolio, weblog and resources sections. Each of these pages show an example of aggregating individual RSS feeds from students into a single RSS feed and then including it in the course site.

As was pointed out above it is easy enough for students and staff to make use of these RSS feeds in their own personal RSS readers. They don’t need to go to the course site. However, I can think of two reasons why this is a good thing:

It helps maintain an identity for the course.
Like it or not, course websites remain an important contributor to the identity of a course offering and/or to the staff member coordinating a course. Many folk like, in part because it has become the accepted practice, to have a course website that can be seen as a product of a course. Having it distributed into everyone’s personal learning environment removes that sense of identity. There has been some work around learning networks that suggests that this is one of the requirements of a learning network. For example, look at this paper and search for the section titled “requirements of a learning network”.
It’s still not easy for everyone to use an RSS reader.
As I pointed out in the previous section. RSS readers are still not common place. A lot of people don’t know what they are. A lot of students have become indoctrinated into the practices associated with a course website. Having the RSS feed in the course website helps the transition. The advantage of this idea is you can support both the course website and those with RSS readers.
For example, the EDED11448 website looks like a fairly typical course website, this serves the traditional students. There is also an OPML feed that allows the entire site and all its contents and updates to be tracked via an RSS reader.

Isn’t a key feature of personal learning environments allowing the students to make their own choice. They choose, course website or RSS reader, or both.

Getting half-baked ideas out there: improving research and the academy

By admin

On February 15, 2009

In Chapter 2, cognitiveEdge, design theory, elearning, evaluation, innovation, lmsEvaluation, react, thesis, Uncategorized

In a previous post examining one reason folk don’t take to e-learning I included the following quote from a book by Carolyn Marvin

the introduction of new media is a special historical occasion when patterns anchored in older media that have provided the stable currency for social exchange are reexamined, challenged, and defended.

In that previous post I applied this idea to e-learning. In this post I’d like to apply this idea to academic research.

Half-baked ideas

In this post Jon Udell talks about the dissonance between the nature of blogs, the narrative form he recommends for blogs and the practices of academics. In it he quotes an academic’s response to his ideas for writing blogs as

I wouldn’t want to publish a half-baked idea.

Jon closes the blog post with the following paragraph

That outcome left me wondering again about the tradeoffs between academia’s longer cycles and the blogosphere’s shorter ones. Granting that these are complementary modes, does blogging exemplify agile methods — advance in small increments, test continuously, release early and often — that academia could use more of? That’s my half-baked thought for today.

I think this perspective sums it up nicely. The patterns of use around the old/current media for academic research (conference and journal papers) are similar to heavyweight software development methodologies. They rely on a lot of up-front analysis and design to ensure that the solution is 100% okay. While the patterns of use of the blogosphere is very much more like that of agile development methods. Small changes, get it working, get it out and learn from that experience to inform the next small change.

Update: This post talks a bit more about Udell’s views in light of a talk he gave at an EDUCAUSE conference. There is a podcast of the presentation.

There are many other examples of this, just two include:

Dave Snowden’s distinction between fail-safe design (old style) and safe-fail design probes (new style).
Lucas Introna’s (1996) distinction between teleological design (old style) and ateleological design (new style).

Essentially the standard practices associated with research projects in academia prevent many folk from engaging in getting the “half-baked ideas” out into the blogosphere. There are a number of reasons, but most come back to not looking like a fool. I’ve seen this many times with my colleagues wanting to spend vast amounts of time completing a blog post.

As a strong proponent and promoter of ateleological design processes, I’m interested in how this could be incorporated into research. Yesterday, in discussions with a colleague, I think we decided to give it a go.

What we’re doing and what is the problem?

For varying reasons, Col and I are involved, in different ways, with a project going under the title of the indicators project.. However, at the core of our interest is the question

How do you data mine/evaluate usage statistics from the logs and databases of a learning management system to draw useful conclusions about student learning, or the success or otherwise of these systems.

This is not a new set of questions. The data mining of such logs is quite a common practice and has a collection of approaches and publications. So, the questions for use become:

How can we contribute or do something different than what already exists?
How can we ensure that what we do is interesting and correct?
How do we effectively identify the limitations and holes underpinning existing work and our own work?

The traditional approach would be for us (or at least Col) to go away, read all the literature, do a lot of thinking and come up with some ideas that are tested. The drawback of this approach is that there is limited input from other people with different perspectives. A few friends and colleagues of Col’s might get involved during the process, however, most of the feedback comes at the end when he’s published (or trying to publish) the work.

This might be too late. Is there a way to get more feedback earlier? To implement Udell’s idea of release early and release often?

Safe-fail probes as a basis for research

The nature of the indicators project is that there will be a lot of exploration to see if there are interesting metrics/analyses that can be done on the logs to establish useful KPIs, measurements etc. Some will work, some won’t and some will be fundamentally flawed from a statistical, learning or some other perspective.

So rather than do all this “internally” I suggested to Col that we blog any and all of the indicators we try and then encourage a broad array of folk to examine and discuss what was found. Hopefully generate some input that will take the project in new and interesting directions.

Col’s already started this process with the latest post on his blog.

In thinking about this I can come up with at least two major problems to overcome:

How to encourage a sufficient number and diversity of people to read the blog posts and contribute?
People are busy. Especially where we are. My initial suggestion is that it would be best if the people commenting on these posts included expertise in: statistics; instructional design (or associated areas); a couple of “coal-face” academics of varying backgrounds, approaches and disciplines; a senior manager or two; and some other researchers within this area. Not an easy group to get together!
How to enable that diversity of folk to understand what we’re doing and for us to understand what they’re getting at?
By its nature this type of work draws on a range of different expertise. Each expert will bring a different set of perspectives and will typically assume everyone is aware of them. We won’t be. How do you keep all this at a level that everyone can effectively share their perspectives?
For example, I’m not sure I fully understand all of the details of the couple of metrics Col has talked about in his recent post. This makes it very difficult to comment on the metrics and re-create them.

Overcoming these problems, in itself, is probably a worthwhile activity. It could establish a broader network of contacts that may prove useful in the longer term. It would also require that the people sharing perspectives on the indicators would gain experience in crafting their writing in a way that maximises understandability by others.

If we’re able to overcome these two problems it should produce a lot of discussion and ideas that contributes to new approaches to this type of work and also to publications.

Questions

Outstanding questions include:

What are the potential drawbacks of this idea?
The main fear I guess of folk is that someone, not directly involved in the discussion, steals the ideas and publishes them unattributed and before we can publish. There’s probably a chance that we’ll also look like fools.
How do you attribute ideas and handle authorship of publications?
If a bunch of folk contribute good ideas which we incorporate and then publish, should they be co-authors, simply referenced appropriately, or something else? Should it be a case by case basis with a lot of up-front discussion?
How should it be done?
Should we simply post to our blogs and invite people to participate and comment on the blogs? Should we make use of some of the ideas Col has identified around learning networks? For example, agree on common tags for blog posts and del.icio.us etc. Provide a central point to bring all this together?

References

Lucas Introna. (1996) Notes on ateleological information systems development, Information Technology & People. 9(4): 20-39

Alternatives for the institutional implementation of e-learning: Lessons from 12 years of Webfuse

By admin

On February 15, 2009

In design theory, elearning, thesis, Uncategorized, webfuse

This is a submission to a conference later in the year. The conference has its own specific requirements for submissions, hence the structure of this post. In some cases I’ve left in quotes or italics the directions given by the conference as to the content of the submission. It’s still a work in progress. Submission is not due for a few days. More than keen to hear suggestions and criticisms.

The submission was accepted and will be presented at EDUCAUSE’09 on the 6th November, 2009. The slides should be here before the presentation.

Background

The topic of the submission is essentially a summary of the results of my PhD work. I’ve been working on this submission, in part, to prepare for a presentation on the PhD at ANU late next week that will cover some of the same content. The slides are included below, and an early version of the presentation is available as video on ustream

Abstract

50 word maximum

The practice of e-learning in universities suffers from a number of unquestioned perspectives that limit outcomes. This presentation describes a framework for understanding the full diversity of alternate perspectives and examines one successful set of perspectives arising out of 12+ years of designing, supporting and competing with the Webfuse system.

Slides

Presentation outline

Statement of the problem or issue: State specifically what problem or issue you will address during your session.

Even though the implementation of e-learning within institutions of higher education is no longer new, it remains difficult and somewhat less than successful. While participation and use of e-learning continues to increase (Allen and Seaman, 2008), most universities are struggling to engage a significant percentage of students and staff in e-learning (Salmon, 2005). There remain concerns about the quality of the learning and teaching and about the return on the investment in e-learning (Bates, 2009). Some have found linkages between the practice of e-learning within universities and the idea of management fads and fashions (Pratt, 2005). Technology has not yet transformed education.

That there remains a problem with e-learning is illustrated by it being ninth in the top-ten issues facing IT in the 2008 EDUCAUSE current issues survey (Allison, DeBlois, et al, 2008). There continues to be a need for a vision on the structures, processes and technology that can be drawn upon effectively to implement e-learning within universities (Alavi and Leidner, 2001). There is a need for work that develops theories and models of change related to human intervention and sustainability of e-learning within universities (Salmon, 2005).

The fundamental problem at the core of this work is the question

How do you design, implement and support information systems that effectively and efficiently supports e-learning within an institution of higher education?

In attempting to develop answers to this question over 12+ years of designing, supporting and competing with the Webfuse e-learning system it has become obvious that many of the assumptions and perspectives underpinning current and accepted practice are of questionable and, in some cases, highly negative value. It has also become obvious that there are many different and contradictory perspectives of the highly complex, confusing and almost over-whelming issues associated with the e-learning within universities and that there is value in moving beyond the obvious and accepted views.

Description of activity, project or solution: This is the core of your proposed presentation and should include such things as historical background; who was involved with the project; examples of what it took to analyze and define problems and solutions; methods used to work toward solutions, and so on.

In 1996 the Department of Mathematics and Computing at Central Queensland University, like many other university departments of the time, embarked on the design (Jones and Buchanan, 1996) and development of a system called Webfuse to enable it to more effectively make use of e-learning. Since that time the system and associated processes have evolved through a number of phases in response to changes in the local institutional context and a growing insight into what works and what doesn’t. The phases include:

Initial design – 1996/1997.
Performed by an individual academic given a term off teaching by the Department. The university’s central IT division was not directly involved.
Early, limited support – 1997-2000.
First, as part of the Department of Maths and Computing and then as part of the Faculty of Informatics and Communication, Webfuse is supported by a small group of 1 full-time and 1 part-time positions. The central IT division is still not involved with the system. There is little development of Webfuse, mostly supporting staff using existing functionality.
Growth and development – 2000-2004.
During this time support was provided by a group varying between 3 and 5 people. This included an academic position as lead designer. The group supported and developed Webfuse within the Faculty of Informatics and Communication. Still no direct involvement from central IT, though significant disagreements commence. These are largely due to increased usage of Webfuse services by academic staff from across the institution. Increased usage directly attributable to the development of a number of useful services.
Traditional systems development – 2005-2007.
The academic lead developer of Webfuse returns to teaching and a group of 3 people continue to support Webfuse using more traditional governance and development models. The group is based within the Faculty of Informatics and Communication and then, after a restructure, within the Faculty of Business and Informatics. Still no direct involvement of the central IT division. Significant use of Webfuse services outside of the host faculty continues. Development of new services somewhat limited, but still going on.
Centralisation – 2007-now.
The Webfuse staff move into the central information technology division and the group is eventually reduced to 1 and a bit positions. Usage across the institution of some Webfuse services decrease, while some increase. All development of new services is halted and it is likely that Webfuse will be phased out over the coming year or two.

Throughout the history of Webfuse the author has taken on a number of different roles including:

the academic teaching staff member charged with the initial design of Webfuse (1996-1997);
lead designer in a team of between 2 and 5 staff (2000-2004);
an academic member of staff using the system in his own teaching (1996-2000 and 2004-2006);
manager of a unit responsible for providing staff and student support in the use of Blackboard, the institutions official and competing e-learning system (2007-2008);
a researcher publishing the principles underlying the design and the use of Webfuse (1996-now); and
an information systems PhD student developing an information systems design theory based on the design and support of Webfuse (2000-2008).

Throughout its lifespan Webfuse has been developed with a range of different methods ranging from naive design and “cowboy” coding, agile and participative development through to more traditional heavyweight software development and governance methods. It has predominantly made use of open source software and the Perl scripting language with an object-oriented, patterns-based architecture with an emphasis on small pieces, loosely joined.

The development of the perspectives expressed in this presentation and other publications has been based on an interative, action-research process underpinning the design, re-design and use of Webfuse for e-learning. A process that has led to a number of peer-reviewed publications (Jones, 1996; Jones, 1999; Jones, Jameson and Clark, 2001; Jones and Behrens, 2001), the formulation of the Ps Framework (Jones, Vallack and Fitzgerald-Hood, 2008) and an information systems design theory (ISDT) for e-learning implementation within Universities (Jones and Gregor, 2004).

Outcome: Here you should talk about your outcomes and achievements. Did you accomplish what you set out to do? In addition to anecdotal evidence, include quantitative examples of how your project resolved a problem or issue. It’s important to talk about the successes and failures, especially in respect to what you learned from either.

There are three main outcomes arising out of the design and support of Webfuse:

The Ps framework (Jones, Vallack and Fitzgerald-Hood, 2008);
The Ps Framework is a descriptive theory that helps reduce the complexity associated with making decisions about the implementation of e-learning within universities. It helps identify the diverse perspectives that exist of the components of e-learning. The framework can be used to guide discussion and evaluation of the perspectives most appropriate for a given context.
In this presentation the Ps Framework will be used to compare and contrast a diverse set of perspectives of how to go about the impelementation of e-learning. This will include examination of the many different perspectives of success and failure associated with Webfuse over its 12+ years. It will also be used to illustrate the existence and potential limitations of existing standard practice.
An information systems design theory for e-learning (Jones and Gregor, 2004).
Information system design theories (ISDTs) are prescriptive theories formulated to provide guidance on how to solve specialised classes of information systems design problems. ISDTs increase the likelihood of successful development by providing principles that limit the range of system features and development processes to a more manageable set.
In this presentation the Webfuse ISDT will be used to explain the foundations and describe one set of alternate perspectives around the implementation of e-learning that may be an improvement of existing standard practice.
A successful instantiation (Webfuse) and related experience.
The 12 years of designing and supporting Webfuse has seen both failures and success with both contributing to the development of the ISDT and the Ps Framework. The failures have provided the opportunity to learn lessons that have led in turn to improvements that have enabled the successes.

In this presentation stories and statistics will be used to illustrate the different possible perspectives on aspects of e-learning implementation and provide supporting evidence for the ISDT.

Some of the positive outcomes arising from this work include:

Recognition by leadership of the importance of Webfuse.
The annual teaching and learning report for the Faculty of Informatics and Communication for 2003 (Central Queensland University, 2004, p. 21 of 50) included the following comment.

the best thing about teaching and learning in this faculty in 2003 would be the development of technologically progressive academic information systems that provide better service to our students and staff and make our teaching more effective. Webfuse and MyInfocom development has greatly assisted staff to cope with the complexities of delivering courses across a large multi-site operation.
Higher levels of adoption by students and staff.
In 2005 91% of students and 87% of staff (including a large number of casual staff) made use of Webfuse. This is in comparison to reports (Sausner, 2005) from institutions held up as examples of best practice reporting no greater than 55% of staff adopting institutional e-learning systems.
Positive reactions from academic staff
The following quotes from Webfuse users are indicative of academic staff reactions to the system and its processes and the subsequent level of trust that was created.

..the precedent of other IT systems made available in Infocom suggests that it would be extremely user friendly for people with very limited computer competence/confidence

my positive experience with other Infocom systems gives me confidence that OASIS would be no different. The systems team have a very good track record that inspires confidence
Greater flexibility and innovation.
The architecture of Webfuse and the development processes as outlined in the ISDT are specifically designed to be flexible and to enable innovation. Some of the many examples of flexibility and innovation within Webfuse include:
- Earlier and better adoption of online assignment submission and management.
  The online submission and management (OASM) of student assignments has been a feature of Webfuse since 1996. By 2000 the system had evolved into a system (Jones and Behrens, 2003) that is perceived to be significantly easier and more useful than both traditional methods and those in other learning management systems. A concrete example of this is that in 2008 the university recommended that staff make use of the Webfuse OASM system rather than that provided by Blackboard, the institution’s official learning management systems. This recommendation was due to the significant inefficiency, lack of integration into institutional processes and lack of features of the Blackboard OASM system.
- Early and innovative use of new technology.
  For example, Webfuse has used RSS in various ways since 2000 and blogs were first used in 2002 through the inclusion into Webfuse of the Movable Type blog engine. In 2006, the Blog Aggregation Management (BAM) system was added to Webfuse. BAM is discussed in the ELI Guide to Blogging (Coghlan, Crawford et al, 2007), which observed
  
  One of the most compelling aspects of the project was the simple way it married Web 2.0 applications with institutional systems. This approach has the potential to give institutional teaching and learning systems greater efficacy and agility by making use of the many free or inexpensive – but useful – tools like blogs proliferating on the Internet and to liberate institutional computing staff and resources for other efforts.
Greater integration with the organisational context.
The architecture and development process underpinning Webfuse places an emphasis on responding to the local context. Consequently, many of the services provided by Webfuse are not typically found in traditional e-learning systems and have made a significant difference to adoption. Examples of the unusual services offered by Webfuse include: a system to request informal review of grades, uploading of end of term results, a gallery of student photos, an email merge facility, a system to track incidents of academic misconduct, an online timetable generator, a web-based interface to student records, the automatic creation and population of default course sites and many others. It is these services specific to the local context, that have been adopted across the entire institution.

Importance or relevance to other institutions: Please extract and describe the overarching principals or lessons learned that attendees can focus on during your presentation so that they can better understand the importance and relevance of your findings to their institutions.

The Ps Framework identifies the following components associated with the institutional implementation of e-learning (Jones, Vallack and Fitzgerald-Hood, 2008):

Purpose.
What is the purpose or reason for the organization in adopting e-learning or changing how it currently implements e-learning? What does the organization hope to achieve? How does the organization conceptualise its future and how e-learning fits within it? What is the purpose of the organisation?
Place.
What is the nature of the organization in which e-learning will be implemented? What is the social and political context within which it operates? How is the nature of the system in which e-learning will be implemented understood?
People.
What type of people and roles exist within the organization? What are their beliefs, biases and cultures? How do people make decisions?
Pedagogy.
What are the conceptualisations about learning and teaching that people within the place bring to e-learning? What practices are being used? What practices might people like to adopt? What practices are most appropriate?
Past experience.
What has gone on before with e-learning, both within and outside of this particular place? What worked and what didn’t? What other aspects of previous experience at this particular institution will impact upon e-learning?
Product.
What type of “systems” or products are being considered? What is the nature of these products? What are their features? What are their affordances and limitations?
Process.
What are the characteristics of the process used to choose how or what will be implemented? What process will be used to implement the chosen approach?

Using the Ps Framework it is possible to identify a growing orthodoxy around the organisational implementation of e-learning within Universities. This orthodoxy is characterised by a particular set of perspectives on the components of the Ps Framework. For example, the orthodoxy generally selects as the product one of a number of integrated, enterprise learning management systems (e.g. Blackboard, Sakai, Moodle). For process the orthodoxy will generally use selection and governance processes based on teleological design (Jones and Muldoon, 2007). This presentation will use the Ps Framework, the ISDT and the Webfuse instantiation to make the following points:

There is an identifiable orthodox view on the organisational implementation of e-learning within universities.
Parts of this orthodox view are not necessarily a good fit for the requirements of e-learning within universities and, in some cases, can directly contribute to the problems and limitations of existing practice.
There are a wide array of alternate perpsectives that can be more appropriate and consequently improve the quality of outcomes.
Webfuse and the associated ISDT illustrate one combination of these alternate perspectives that has worked well and may be useful in other contexts.
There is value in investigating, discussing, comparing and expanding the combinations of these perspectives drawn upon to inform the organisational implementation of e-learning.

These points are important and relevant to other institutions because there are concerns that current decision making around the organisational implementation of e-learning is showing signs of succumbing to fads and fashions (Birnbaum, 2000; Pratt, 2005). It is suggested that a contributing factor to this tendency is a less than complete understanding of the diversity of alternate perspectives and their relative merit. This presentation aims specifically to broaden the awareness of these different perspectives.

Frameworks offer new ways of looking at phenomena and provide information on which to base sound, pragmatic decisions (Mishra & Koehler, 2006). This presentation will see to provide a basis for increasing the awareness of the full diversity of perspectives associated with e-learning. Greater awareness of different perspectives will help improve decision making, the implementation of e-learning and consequently the quality of outcomes.

References

Alavi, M. and D. E. Leidner (2001). “Research commentary: technology-mediated learning – a call for greater depth and breadth of research.” Information Systems Research 12(1): 1-10.

Allen, I. E. and J. Seaman (2008). Staying the course: Online education in the United States, 2008, Sloan Consortium.

Allison, D. and P. DeBlois (2008). “Top-Ten IT Issues, 2008.” EDUCAUSE Review 43(3): 36-61.

Bates, T. (2009). Technology should be used as integral part of teaching and learning activities. EQUIBELT Project Newsletter, No. 6. Z. Bekic.

Birnbaum, R. (2000). Management Fads in Higher Education: Where They Come From, What They Do, Why They Fail. San Francisco, Jossey-Bass.

Central Queensland University. (2004, September). Faculty teaching & learning report, Retrieved February 16, 2009, from: http://www.cqu.edu.au/academic_board/academicboard/2004/September/8.2.1-DVCAR-FacultyAnnualTLreport.doc, p. 21)

Coghlan, E., J. Crawford, et al. (2007). ELI Discovery Tool: Guide to Blogging, EDUCAUSE.

Jones, D. (1999). Webfuse: An integrated, eclectic web authoring tool. Proceedings of EdMedia’99, World Conference on Educational Multimedia, Hypermedia & Telecommunications, Seattle, Association for the Advancement of Computing in Education.

Jones, D. and S. Behrens (2003). Online Assignment Management: An Evolutionary Tale. 36th Annual Hawaii International Conference on System Sciences, Hawaii, IEEE.

Jones, D. and R. Buchanan (1996). The design of an integrated online learning environment. Proceedings of ASCILITE’96, Adelaide.

Jones, D. and S. Gregor (2006). The formulation of an Information Systems Design Theory for E-Learning. First International Conference on Design Science Research in Information Systems and Technology, Claremont, CA.

Jones, D., K. Jamieson, et al. (2003). A model for evaluating potential Web-based education innovations. 36th Annual Hawaii International Conference on System Sciences, Hawaii, IEEE.

Jones, D. and N. Muldoon (2007). The teleological reason why ICTs limit choice for university learners and learning. ICT: Providing choices for learners and learning. Proceedings ASCILITE Singapore 2007, Singapore.

Jones, D., J. Vallack, et al. (2008). The Ps Framework: Mapping the landscape for the PLEs@CQUni project. Hello! Where are you in the landscape of educational technology? ASCILITE’2008, Melbourne.

Mishra, P. and M. Koehler (2006). “Technological pedagogical content knowledge: A framework for teacher knowledge.” Teachers College Record 108(6): 1017-1054.

Pratt, J. (2005). “The Fashionable Adoption of Online Learning Technologies in Australian Universities.” Journal of the Australian and New Zealand Academy of Management 11(1): 57-73.

Salmon, G. (2005). “Flying not flapping: a strategic framework for e-learning and pedagogical innovation in higher education institutions.” ALT-J, Research in Learning Technology 13(3): 201-218.

Sausner, R. (2005). Course management: Ready for prime time? University Business.

Innovation – rendering the hitherto non-existent blindingly obvious

By admin

On February 13, 2009

In innovation, quotes, Uncategorized

A lot of what I do could be seen as innovation. In the past I was a “faculty teaching and learning innovation officer” and I may well take on such a role in the future. Today, I came across the following quote from Douglas Adams which seems connected to the idea of innovation.

It is a rare mind indeed that can render the hitherto non-existent blindingly obvious. The cry ‘I could have thought of that’ is a very popular and misleading one, for the fact is that they didn’t, and a very significant and revealing fact it is too.

The emperor has no clothes – why is the learning and teaching peformance fund naked

By admin

On February 13, 2009

In Chapter 2, elearning, evaluation, Uncategorized

The Australian Federal Government has a Learning and Teaching Performance Fund (LTPF) that is meant to allocate money to Australian universities on the quality and/or improvement in their learning and teaching.

Based on what I know of this approach I think it is fundamentally broken. It’s probably that the “emperor has no clothes”. i.e. Australian universities know it is broken, but can’t point it out because they want to get the money.

The fund and how it works

According to this story in the Australian newspaper the fund allocated $AUD73 million this year. The administrative information for providers document outlines the process for 2009. The process has changed over the 3 years it has been run.

There are two data sources used by the fund:

Australian Graduate Survey; and
All Australian University graduates get a survey in the months after they graduate which asks them two broad sets of questions: are they working and in what, and how satisfied were they with their study/university. The LTPF uses two sets of indicators from this survey
- Student satisfaction indicators
  - Satisfaction with generic skills
  - Satisfaction with good teaching
  - Overall satisfaction
- Outcome indicators
  - Full-time employment
  - Further full-time and part-time study
- Higher education student collection.
  The statistics are used in the LTPF to examine progress rates amongst Bachelor students and the retention rate for the same students.
The process used goes something like
- An adjustment process is applied to the raw indicators data.
- Each university gets a package describing details of the findings for their students.
- The university provide a submission offering information that may explain some of the results.
- An expert panel looks at the information and provides advice to the government, back to the institutions and generally ensures the process is effective.
The trouble is that I think the majority of the data that is at the foundation of this process is less than reliable.

Why is it broken

A couple of weeks ago I published a post titled “Somethings that are broken with the evaluation of university teaching”. Essentially it is a collection of links pointing out that “level 1 smile sheets” (surveys that ask learners “were you satisfied”) have significant and well-known limitations in validity and value. In particular the following quote is from this article

In some instances, there is not only a low correlation between Level I and subsequent levels of evaluation, but a negative one.

From my perspective the course experience questionnaire (the bit of the Australian graduate survey that asks student satisfaction) essentially takes the “level 1 smile sheet” approach and applies it to a graduates entire university experience. I don’t see this move to a broader area of coverage (whole university experience, up from individual course/unit/subject) helping address the concerns about “level 1 smile sheets”. In fact, I see it getting much worse.

The course experience is likely to cover at least 3 years experience. For some part-time students this might as much as 6, 9 or more years. Do we really believe that their experience over the last 6 to 12 months isn’t going to over shadow and be more in their mind than their previous experience?

There are also problems with the graduate destination survey – the part of the survey that asks about what they are doing now. This article points out some of the limitations.

To some extent having institutions comment on the data before the expert panel examines it might address some of this. But I don’t think that goes anyway towards addressing the significant limitations of this form of evaluation.

Solutions

So if it’s broken, what are the alternatives?

I don’t know. It’s a difficult question and I don’t have the knowledge, experience or time to recommend a solution. Given the nature of this problem, I’m not even sure that there is a single correct solution.

Based on what I know, some suggestions I think might be worth more consideration include:
- Measure fit for context/purpose, not comparison.
  To some extent the LTPF process acknowledges that comparing the quality of learning and teaching across all of the diversity of the Australian higher education sector is extremely difficult. So why continue to try and do it. Why not focus on how well the learning and teaching is for the given context. Measure fit for purpose, not comparison against others. That said, any form of measurement has some potential downsides and negative outcomes.
- Concentrate on improvement.
  This is somewhat similar to the last point. It’s also linked to one of my common sayings, “It’s not how bad you start, but how quickly you get better”. Rather than measure fit for context, measure and reward how much better the learning and teaching at a particular institution has become. Also some potential negative concequences.
- Use other forms of evaluation/data.
  This and I’m sure many other places talk about additional forms of evaluation beyond level 1 smile sheets. Dave Snowden also has some interesting approaches to evaluation which might apply.
In general, I would suggest that rather than wringing hands over how difficult, spending inordinate amounts of time reflecting on how hard it is and arguing over which of many options is the best, spending lots of money on consultants that will push their own barrow at the expense of any knowledge of the local context, leaping at the latest fad, or say it’s all too hard we can’t change the governments mind. I think it would suggest an organisation attempt a lot of safe-fail probes to investigate different potential solutions. Apply the lessons learned to improve evaluation within the institution and then promote it amongst the sector.

It's always easier to say sorry than to ask for permission

By admin

On February 12, 2009

In Chapter 2, design theory, elearning, thesis, Uncategorized, webfuse

The video from NASA referenced in a post from yesterday illustrates many of the difficulties in attempting to be innovative within any largish type of organisation.

I was talking with a colleague today about this and other associated problems and his question is “But what are the solutions, the alternatives?”. As a cynical bastard who has a long history of forever pointing out the flaws in various organisational practices (there really is no challenge in doing this), a common response from those nasty folk in power has been – “But what are the solutions?”.

These folk really started getting annoyed, just like those in the video, when I started actually outlining solutions. So annoyed that you knew you would never actually be able to do anything.

So, what’s the solution?

The solution

Talk about coincidence, the 37Signals blog has just had a post published that outlines the solution I’ve used in the past. They call it “going rouge”. My explanation was “it’s always easier to say sorry than to ask for permission”. i.e. do it and apologise later, don’t ask.

There are some problems with this approach, including:

You can become a “pariah”.
The negative connotations that arise form this approach can be seen in the words used to describe it. For example, “rogue” in the 37Signals post or “shadow systems” when it comes to IT systems.
You don’t get the resources or access you may need.
In some cases you actually need resources or access to materials/information in order to do something innovative/effective. If you’re a “rogue” you won’t get that access. iStandford would never have happened without some folk within Stanford helping out the “rogues”.
Folk may never ever get it.
If you’re approach is so different, folk may never understand it. Even if you show some early results, in the long run the established perspectives will rule. Especially because of the next point.
The quality might not be up to it.
A lot of what I did with “rogue” e-learning systems wasn’t seen as innovative because it didn’t look pretty (because I’m not an interface/graphic designer). People couldn’t see beyond the surface to see the idea. The lack of quality limited the ability to demonstrate results. The quality was limited because I had to go rogue and couldn’t get access to resources – like a good interface/graphic designer.

Barriers to innovation in organisations: teleological processes, organisational structures and stepwise refinement

By admin

On February 11, 2009

In Chapter 2, design theory, elearning, information systems, missingPs, thesis, Uncategorized

This video speaks to me on so many levels. It summarises many of the problems I have faced and encountered trying to implement innovative approaches to e-learning at universities over the last 15 plus year. I’m sure I am not alone.

Today, I’ve spent a lot of time not directly related to what I wanted to achieve. Consequently, I had planned not to do or look at anything else until I’d finished. But this video resonates so strongly that I couldn’t resist watching, downloading it and blogging it.

I came across the video from a post by Punya Mishra. Some more on this after the video. I should also link to the blog post on the OpenNASA site. Would your University/organisation produce something similar?

If Nona ever gets around to watching this video, I am sure she will see me in a slightly different role in the video. Until recently I had the misfortune to be in the naysayer role. That’s no longer the case. Who said no good could come of organisational restructures?

Barriers to innovation and inclusion

The benefits of being open

Coming across this video, provides further evidence to support an earlier post I made today on the values of being open. I became aware of Punya’s post because of the following process:

Almost a year ago Punya published this post on his blog that openly shares the video of a keynote he and Mat Koehler gave.
I came across it not long afterwards through my interest in TPACK (formerly known as TPCK).
About two weeks ago I decided to use part of the video in some sessions I was running on course analysis and design.
A couple of days ago I blogged on an important part of the presentation (not used in the sessions I ran) that resonated with my PhD work.
My blog software told Punya’s blog software about my post and added it as a comment to his blog.
This afternoon Google Alerts sent me an email that this page on Punya’s blog was linking to my blog (because of the comment – see the comments section in the right hand menu).
Out of interest (some might say in the interest of procrastination) I followed the link and saw the video.

I plan to use parts of this video in future presentations around my PhD research. I believe it will resonate with people so much better than me simply describing the abstract principles.

So while not directly contributing to what I wanted to do today. It’s provided with a great advantage in the future.

of a Google Alert I have set on my site. Google emailed me to say that Punya had made this post because his blog software includes a list of the

I’ve spent a lot of time today doing stuff not necessarily directly related to what I wanted to achieve today. To such an extent I’d decided not to blog anymore.

One reason people don't take to new e-learning technology

By admin

On February 11, 2009

In Chapter 2, design theory, elearning, missingPs, ple, ples@CQUni, quotes, thesis, Uncategorized

In a recent post I started my collection of quotes on this blog. I also talked about the “mere exposure effect” and suggested it’s one reason behind the horseless carriage approach to using new technology. It’s also one reason why people resist new technology – especially e-learning/computer technology.

In working on another post, one directly related to the PhD, I came across this article from EDUCAUSE Quarterly titled “The Three-E Strategy for Overcoming Resistance to Technological Change “.

One of the quotes it uses to as evidence of why adoption of new technology is hard is from a book by Carolyn Marvin

For if it is the case, as it is fashionable to assert, that media give shape to the imaginative boundaries of modern communities, then the introduction of new media is a special historical occasion when patterns anchored in older media that have provided the stable currency for social exchange are reexamined, challenged, and defended.

The EDUCAUSE Quarterly article also says the following

As technology professionals, we often fail to see how intimidating technology can be to the user community.

I’d expand this out to include instructional designers and management. Instructional designers often don’t see how intimidating many of their pedagogical innovations (forget the use of technology) are to many academic staff. Many management folk I’ve seen make similar mistakes, though generally worse. Management generally don’t see how new pedagogy and technology, if used effectively, needs a radically different approach to teaching and learning practice. More importantly they don’t see or engage in the fact that this type of radical change often brings into question many of the accepts administrative processes, policies and organisational structures within institutions.

The article also quotes an EDCAUSE review article titled “My Computer Romance”. An expanded quote from this review article

What kept me from seeing and acting on those benefits? The question interests me, and not only out of self-regard. The question is at the heart of “faculty development,” a crude, even misleading phrase that cannot suggest the trick of imagination needed to bring substantial, important knowledge into plain sight and to develop in faculty the resolve and courage to risk failure. For an academic, “failure” is often synonymous with “looking stupid in front of someone.” For many faculty, and maybe for me back in the 1980s, computers mean the possibility of “pulling a Charlie Gordon,” as the narrator poignantly terms it in Daniel Keyes’s Flowers for Algernon.

This has significant implications for personal learning environments that surely represents a significant shift in practice created by the capabilities of a new medium and offers an even greater opportunity for academics to “pull a Charlie Gordon”. The Quarterly article

finishes it’s introduction with the following paragraph

Consider for a moment the impact of Web 2.0 on a professor working in academia for 20 or 30 years. The flattening of knowledge production and the ease of access to information represented by Web 2.0 technologies in many ways negates the concept of the “sage on the stage” or even traditional notions of scholarship. This world is not what most professors are used to, and many are threatened by and therefore resist this kind of change.

The solution

The Quarterly article

suggests that the solution is a strategy for gaining acceptance of technology that embodies “Three Es”

Evident – as potentially useful in making life easier.
Easy to use – to avoid feelings of adequacy.
Essential – as part of going about their business.

Sort of sounds a bit like the insights from TAM and Diffusion of Innovations.

The wrong view

The Quarterly article finishes with this sentence

Only then will faculty effectively use the complex technical infrastructure that we technologists labor so hard to put into place.

God I hate the mindset that underpins that sentence. Or at least the common mindset amongst “support” folk in higher education. This isn’t limited to just information technology people. Instructional designers, quality folk and management all suffer from this view from time to time.

How do we get these poor ill-informed and/or obstinate academics to use the great technology/idea. If only we could do this we would solve all the problems of learning/teaching/research in one fell swoop.

This has been a problem with most people peddling innovation. Indeed, diffusion theory (Rogers, 1995) one of the best known innovation theories, has been criticised for having a pro-innovation bias that, amongst other effects, can separate members of a social system into the superior innovators group and the inferior recalcitrants group (McMaster and Wastell, 2005).

In this paper (Jones and Lynch, 1999) we talk about

developer based; and
A developer-based focus assumes that the new product will automatically replace the old and that adopters will see the benefits of the new product automatically and in the same way as the developers.
adopter-based approaches to software development.
These approaches focus on the adopters and their setting in order to understand the social context and the social function the innovation will serve.

The Es approach strikes me as someone who comes from a developer-based culture taking the first steps towards a more adopted-based approach. But someone who still has the same underlying belief that we build it and they use it.

References

Jones, D. and T. Lynch (1999). A Model for the Design of Web-based Systems that supports Adoption, Appropriation and Evolution. First ICSE Workshop on Web Engineering, Los Angeles.

Rogers, E. (1995). Diffusion of Innovations. New York, The Free Press.

Carolyn Marvin, When Old Technologies Were New: Thinking About Electric Communication in the Late Nineteenth Century (New York: Oxford University Press, 1988), p. 4.

The value of being open

By admin

On February 11, 2009

In Chapter 2, design theory, elearning, ples@CQUni, thesis, Uncategorized

Moving into the “web 2.0″/social media/online world can be confronting for people. Especially the “open” part. With blogs, photo sharing, social bookmarking etc a lot of what used to be private (or closed) becomes public (or open). This can challenge people. Damien talked about this and other problems with social media in a blog post from late last year.

In the last week three separate things have happened that have reinforced the value of being open. They are:

Finding out about an Instructional design group on Slideshare.
Last week I ran some sessions on Course analysis and design. I used powerpoint slides to structure the sessions and like all of my presentations, uploaded them to slideshare. Within a few hours someone else on Slideshare had seen these presentations and suggested (via slideshare) that I add them to the Instructional design group. Consequently, these presentations have been viewed many more times than might have happened otherwise. For example, the slides on teacher thinking, which includes audio, has been viewed 81 times in under a week.
Making a connection with a “name” and enhancing the PhD
A couple of days ago I posted something related to the PhD. This post arose out of some work I did for the course analysis and design sessions and was only possible because a keynote presentation was publicly available and because Google revealed a blog post that expanded more on this presentation and highlight a particular point.

The ideas in this work connected with my PhD work and has led to an enhancement of the ideas within it. Also, within 40 minutes of publishing the blog post, one of the presenters of the keynote had made a comment on my post.
WordPress’ auto-generated related posts highlighting other interesting stories.
A few days ago a friend asked whether I had any ideas for iPhone applications. Apparently one of the folk we know of has made a bit of money out of a fairly simple iPhone application. I said I didn’t know of any. I have some ideas now.

Those ideas have been sparked by this story from Standford. It’s at least a month and a half old but I hadn’t heard of it. At least not until I published this post this morning. I viewed the post after publishing to check formatting and WordPress had automatically included a link to the “Can iStanford take on Facebook mobile?” story from Time.

Making technology more protean?

Taking up the last point, the “Can iStanford take on Facebook mobile?” story highlights a few things related to making technology more protean.

First, there’s a university information technology group that actually gets this stuff. At least to some extent. They realise that they don’t have to provide all the tools for students to access institutional services.

Second, iStanford, based on the little I learnt from the article in Time, is an example of a “small-step” towards making systems more protean. I’m going to assume that iStanford is a traditional application. Only the original developers can modify it. It can’t be easily mashed up and modified by others. Chances are it’s going to be more difficult for other people to go to Stanford and say can we get access to the same information.

A much larger step might have been Stanford making student information available via open APIs or RSS feeds so anyone could produce their own application.

BAM – making e-learning technology more protean

By admin

On February 11, 2009

In bam, Chapter 5, design theory, elearning, missingPs, ple, ples@CQUni, thesis, Uncategorized, webfuse

In a post yesterday I talked about how most applications of e-learning within universities seems to actively prevent students and staff leveraging the protean nature of information technology. That is the nature of computer software to be flexible, malleable and customisable.

The rise of “web 2.0” and related concepts has made it easier to put in place elearning technology that is designed to be more protean. In this post I talk about the Blog Aggregation Management (BAM) project and reflect on some of its ideas and implications for making e-learning technology more protean.

What is BAM?

It’s a research project aimed at extending ideas around how to implement e-learning technology at universities, particularly with an emphasis on what the rise of “web 2.0”, software as a service and other related concepts might mean for this practice.

BAM is a set of Perl scripts that aggregates RSS feeds, it matters little from where those RSS feeds originate, registered to individual students and then provides a range of additional services required by university educators. For example, (click on the screenshots to see larger images)

Link with the institutional teaching responsibilities database so that staff can see which of their students have (or haven’t) registered their RSS feed.
Show how many posts each staff member’s students have made.
If required, award a mark and make comments on a student’s posts.

Because of the original context in which BAM was designed (explained in some of the publications and presentations listed in the next section) there are also some scripts to detect plagiarism between student posts.

Origins of BAM

BAM started life as a more flexible way of implementing student journals in a particular course with the intent of encouraging reflection, increasing interactions between students and staff and hopefully increasing student performance. The initial use of BAM for this purpose is talked about in a number of places including:

Two presentations given at CQUniversity in 2006 that are available on Google Video. The first talks about the initial design ideas while the second reflects on the experience about half way through the course.
A paper describing the use in the initial course.
This initial use was also covered in the ELI Guide To Blogging as one of the three case studies.

System usage – examples of the protean nature

The initial application of BAM was intended to encourage student reflection and interaction between staff and students. It succeeded to varying levels of success depending on the staff involved. BAM has been used in all 8 offerings of that course from 2006 to 2008.

It has also been used in 8 other course offerings for a variety of different purposes. The most different was in the course EDED11448, Creative Futuring. EDED11448 was CQUni’s first “Web 2.0 course site” where all of the services used by students and staff in the courses were hosted on external services including del.icio.us, WordPress.com, Wetpaint, and RedBubble.

For EDED11448, BAM was used, in conjunction with Yahoo Pipes to create the Portfolio and Weblog pages. This was done by

Students create RedBubble accounts and using this to create their portfolio and their blog.
Students register their RedBubble account with BAM.
BAM aggregates both the portfolio and blog and produces aggregated RSS feeds.
Pipes is used to turn those RSS feeds into a bit of JSON data that can be used by the Javascript on the course website to present the data.

The same idea has been used to create RSS feeds from BAM that aggregate all a staff members students’ posts into one feed. A number of the courses that use BAM can have hundreds of students and tens of staff.

These application of BAM have moved beyond the original design. The protean nature of BAM includes the following:

There is choice in what application they use to generate the RSS feeds.
In some courses that choice is left to the student. In EDED11448 the course designer made a specific choice – RedBubble – for her purpose.
There is choice in what BAM is used for.
The original use was aimed specifically at individual student reflective journals reviewed and marked by staff. EDED11448 aggregated and made public to all students the work of individual students. In some cases the blog posts haven’t been marked.

Implications

I think BAM and the way it operates has the following implications for the practice of e-learning within universities.

Increase efficacy and agility while liberating institutional resources.
This isn’t my view. It’s the one expressed by the authors of the ELI Guide to Blogging. When talking about BAM they say

One of the most compelling aspects of the project was the simple way it married Web 2.0 applications with institutional systems. This approach has the potential to give institutional teaching and learning systems greater efficacy and agility by making use of the many free or inexpensive—but useful—tools like blogs proliferating on the Internet and to liberate institutional computing staff and resources for other efforts.
There is no need to pre-determine and specify all of the technology that staff and students must use.
Most of the students who have used BAM haven’t really known what a blog is and very few have already had a blog. This lack of knowledge is not a reason to say we must use the blog provided by our LMS in order to minimise confusion. With BAM we recommend that students, who aren’t sure what to do, should make use of WordPress.com to create a blog. But we enable those with more knowledge to be able to use their own.
Small pieces loosely joined works.
To me this is a fundamental characteristic of Web 2.0, the ability to create something larger out of a bunch of small pieces that are all loosely joined. Where each small piece can be replaced or re-tasked depending on the contextual needs. This is simply not possible with traditional enterprise software such as a course management system.
There are potential problems but they can be solved, and generally cheaper and easier with this approach.
The most common question that is asked about BAM is “What happens if a student’s blog provider goes belly up and we can’t access the student’s work?”. This is the “can we depend on external providers” question. The assumption is that organisationally provided systems are more reliable. While that is somewhat questionable, the concern can be mitigated quite easily.
In BAM’s case, this is done by mirroring. Every hour BAM
- Visits each student’s RSS feed.
- If there have been any changes it creates/updates a local copy of the RSS feed.</li
If the external blog provider ever disappears, we have a copy. These types of problems can be solved.
Making existing systems more protean is a good thing.
A number of benefits arise from systems being more protean. For example, the tools are able to be used for a number of unintended applications and the users are able to use tools that they are familiar with and have a sense of ownership over. For me, this means that making existing systems more protean is a good and worthy thing to do.

Future work

Future work might include:

Making BAM more self-serve.
Currently setting up BAM requires some additional input from technical folk. Wouldn’t be too hard to make this self-serve.
Extending the RSS generation capabilities in BAM.
These are still fairly limited in terms of capabilities. The need some extension in capabilities, especially in increasing the protean nature of such capabilities.
Improvements to the BAM interface.
It was designed by me. Enough said.
Enabling more complex group-based manipulation, tagging and commenting within BAM.
Beyond simple aggregation there is little that can be done. Even marking is not performed with RSS but with databases. One extension might be to create RSS feeds that include comments/marks from markers. Enabling peer marking, commenting and tagging and a range of more complex approaches might also be useful.
Looking at supporting privacy capabilities in BAM.
At the simplest form adding the ability for the student’s RSS feed to be password protected might be useful. At the moment the RSS feed fed into BAM must be freely available. Supporting broader privacy settings makes the tool more flexible.
Making existing systems more protean.
Add RSS feeds to the discussion forums and other features of a learning management system to enable staff and students to start mashing up.
Integrating BAM into an existing LMS.
BAM’s current use is limited to CQUniversity. BAM, at the moment, is essentially a set of scripts that integrate RSS feeds with several CQUniversity systems (online assignment marking, results processing, staff teaching responsibilities, student enrollment etc). This means it doesn’t make sense to sell or release BAM’s code (beyond having people look at it). Another institution would have to rewrite all of BAM to fit with its systems and practices.
One solution to this might be to integrate BAM with a system like Moodle. These systems already should have data about which staff are responsible for which students, which students are in which course etc.
Working closely with a range of different staff to explore and enable different applications of BAM, to extend its protean capabilities and leverage them to improve the learning and teaching experience.
This is where the real benefit is. Working with staff with different purposes and problems to collaboratively identify approaches and necessary changes to BAM.

The protean nature of modern technology – another limitation of most views of e-learning

By admin

On February 9, 2009

In Chapter 2, design theory, elearning, missingPs, thesis, Uncategorized, webfuse

A part of my thinking around the Ps Framework I suggest that there are a number of dominant assumptions that underpin the current implementation of e-learning within institutions of higher education. I believe these dominant assumptions limit the quality, efficiency, effectiveness and innovativness of e-learning at Universities. In this post I am trying to identify one of the dominant assumptions associated with the “Product” component of the Ps Framework and its implications.

The dominant assumption I’d like to explore here is that people forget that modern technology is protean. Worse than that, how most universities implement e-learning significantly limits the ability to take full advantage of this protean nature and subsequently limits the quality and innovation possible within e-learning.

Origins

This though has arisen through a combination of work I’ve observed at CQUni over the last year or so and some resources I looked at for the course analysis and design sessions I ran last week.

The resource that got it all started was the keynote presentation given by Mat Koehler and Punya Mishra at the SITE’2008 conference. I originally used the presentation for the first part where they talked about teaching as a wicked design problem. The fact that most people treat teaching as a tame design problem is another one of the major assumptions that negatively impact on e-learning, but that’s a story for another post.

Late last night, as I was putting together the resources used in the course analysis and design sessions I did a Google search for the SITE’2008 keynote. Doing so I came across this blog post from Wesley Fryer. The post provides a summary of the keynote and includes the following snippet

Difference between traditional technologies (pencil, microscope, blackboard) are specific – new technologies are PROTEAN

What does protean mean?

Wikipedia offers this description

with the general meaning of “versatile”, “mutable”, “capable of assuming many forms”: “Protean” has positive connotations of flexibility, versatility and adaptability.

The source of this adjective is Proteus, a sea-god from Greek mythology who can tell the future but will change his shape so he can escape doing so.

Mishra and Koehler draw on the work of a number of folk in describing the digital computer as protean in nature – inherently flexible. For example Kay (1984) describing computers as a meta-medium that can dynamically simulate the details of any other medium (including non-physical media) and his suggestion that we have barely begun to investigate this freedom for representation and expression. A computer is a tool to manipulate symbol systems be they visual, acoustic, textual or numeric.

They also point out that digital computer systems can also mean different things to different people. That different people given the same computer will achieve radically different things with it because of its protean nature.

Indications of the current limitation

I can see indications of ignorance of the protean nature in current organisational practice of e-learning in two main forms.

Almost universal use of processes for the selection and support of information systems that treat information systems as “non-protean”.
Unspoken and unquestioned acceptance of information systems serving only a narrow purpose of its original design.

“Non-protean” processes

Look at how most organisations implement computer systems, including those associated with e-learning. Almost without exception (please let me know if you can point out some exceptions) they follow the traditional teleological systems development life cycle

Identify all requirements.
Examine/build a software system that meets those requirements.
Have a long period of system use where it does not change in order to recoup the costs of the first two stages (or simply minimise costs).
Eventually, when the disconnect between the features of the system and the requirements of the organisation become so great, return to step #1.

Truex, Baskerville and Klein (1998) go into more detail about this traditional approach and why it’s inappropriate.

This approach ignores the protean nature of software in that during the long period of use the system is not to be changed and/or changes are kept to a minimum. The only people who are allowed to change them are the central IT folk and anyone who even thinks about changing or working around these systems is sought out and “dealt with”. Hence the negative conatations of shadow systems.

Traditional organisational approaches to the implementation of e-learning within universities do not leverage one of the greatest strengths of modern computers – its protean nature.

In this paper (Danaher, Luck et al, 2004) my colleagues and I make the following points

Software based systems, such as course management systems, represent a set of cultural patterns frozen for now into a reproducible and constraining form (Clear, 2002). Application areas that have low volatility in requirements make it possible for stable, precisely designed systems – the outcome of traditional development methodologies – to operate satisfactorily with minimal changes for long periods (Truex, Baskerville and Klein, 1999). The criterion for this proposition is the capability of the course management system and the supporting organisation to adapt itself and themselves in response to an ongoing process of shared requirements negotiated within the local institution.

That is, we argued that it was important that organisational implementation of e-learning actively enable and encourage the harnessing of the protean nature of information systems in order to

to support and enhance ongoing and shared constructions of innovation, rather than enact a pragmatic set of existing practices or follow a single conceptualization of what is innovative

In fact, when it comes to enterprise systems standard “best practice” from the research and practitioner literature is to implement a vanilla system. i.e. don’t make any adaptations to the enterprise system as it is too expensive to maintain those changes as the enterprise system evolves.

This is important to e-learning because since early this century a university’s course management system forms the academic system equivalent of enterprise resource planning (ERP) systems in terms of pedagogical impact and institutional resource consumption (Morgan, 2003).

I know of one university that is currently in the process of adopting an open source course management system that is actively explaining how they will be implementing a vanilla system. At least that seems to be the narrative emerging from the information technology division.

Limited views on capabilities of technology

As with the monk in the Introducing the book sketch struggling to come to grips with the new fangled technology of the book. Less than experienced designers of e-learning often don’t grasp or see the full potentials of a new technology. Too often falling into the horseless carriage problem.

For example, it continues to be a trend that when universities or academics purchase some space in SecondLife the first building they construct is some form of auditorium in which they can give lectures.

Even if they are particularly innovative in terms of learning and teaching most folk using SecondLife still automatically think that any potential application of Second Life will require students to be able to get in world. Something that can be expensive, if not impossible for them because of bandwidth and hardware requirements.

Few folk make the connection that Second Life can also be a multimedia production platform. Similar to what folk I’ve worked with have done with machinimas (see example below) to support case studies in an Auditing course.

The use of Second Life to produce machinimas provided a cheap and effective way to produce “video” of authentic scenarios without requiring students to access Second Life and without the huge cost of a “real life” production. It’s also an example of the “protean” nature of Second Life that enables its use in new and interesting ways.

Making systems and processes more protean

One of the core aims behind the design of the Webfuse e-learning system (Jones and Buchanan, 1996) was “flexibility and the ability to adapt to change”. In that paper I wrote

The one unchanging characteristic of the Internet, and the computing field in general, is that it never stops changing. This characteristic makes flexibility and adaptability essential features for any online or computing system. Without these characteristics an organisation runs the risk of either retaining an out of date system because it is too expensive to replace, or having to throw away the investment in a system because it has not kept up with change. This risk is demonstrated by the problems faced by Universities that have only recently stopped (and some who haven’t stopped) using mid-80s style, text based computer mediated communications systems.

As the system and the knowledge used to design and support it improved over time there were two main approaches used to make Webfuse “more protean”

The technical design of the system enables and supports modification and enhancement.
The design and support processes place emphasis on close contact between the system developers and the system users.

The trend is towards more protean technology

Originally the design of Webfuse, in terms of “technical design” was enabled by the amount of open source software associated with the Web and Internet that was available in the late 1990s. At any other time it would not have been possible.

Webfuse’s flexibility was based on provide an institutional specific wrapper around this open source software. The wrapper provides a standard interface for users and does some translation from institutional specific abstractions into the abstractions used by the open source software.

With this design, if the discussion was no good. We could find another discussion forum and incorporate it. However, this model was still based on the “run everything on our server” model.

The rise of “web 2.0” and in particular mashable services helps remove the need for this model and significantly increases the protean possibilities of the Webfuse model. The first experiment with these possibilities was the Blog Aggregation (BAM) Project. The second was the “web 2.0 course site” idea.

Both extended the Webfuse model beyond open source software to making use of RSS to mash up services.

The development of immersive 3d applications/worlds like Second Life continue this trend towards more protean software. Not only in that it continues the ability to mash up (to some extent) it also provides an environment more like the real world.

Conclusions

This has been a quick mind dump about the nature of digital computer systems as being protean and the implications for e-learning within universities. The basic outcome is I believe most institutional e-learning actively negates the protean nature of information systems and that e-learning is the worse for this.

This is not to say that enabling and supporting the protean features of e-learning is a silver bullet without any problems. There are issues, mostly associated with how you operate under a completely different set of assumptions that question accepted practice.

References

Patrick Danaher, Jo Luck, David Jones, Jeanne McConachie, Course management systems: Innovation versus managerialism, Proceedings of 11th International Conference, ALT-C 2004, 14-16 September 2004, University of Exeter, Devon, England, 2004. pp 23-35.

Jones, D. and R. Buchanan (1996). The design of an integrated online learning environment. Proceedings of ASCILITE’96, Adelaide.

Morgan, G. (2003). Faculty use of course management systems, Educause Centre for Applied Research: 97.

Truex, D., R. Baskerville, et al. (1999). “Growing systems in emergent organizations.” Communications of the ACM 42(8): 117-123.

On the silliness of "best practice" – or why you shouldn't (just) copy successful organisations

By admin

On February 9, 2009

In Chapter 2, cognitiveEdge, design theory, elearning, missingPs, thesis, Uncategorized

The very idea of “best practice” is silly. In any meaningful complex activity the idea of simply copying what someone else did is destined to fail because it doesn’t seek to understand the reasons why that best practice worked for them and what are the differences between “them” and “us”.

This post over at 37 Signals expounds a bit more on this and references an article titled Why your startup shouldn’t copy 37signals or Fog Creek. The article gives one of the explanations of why best practices are silly.

Dave Snowden has an article called Managing for Serendipity: why we should lay off “best practice” in Knowledge Management that takes the discussion even further. Some of the reasons he gives include:

Human beings naturally learn more effectively from failure than success.
There is only a very limited set of circumstances in which you are able to identify some “best way” of doing something (see wicked problems).
It’s very unlikely that we can codify this “best way” in a way that makes it possible for others to fully understand and adopt the practice.
People are unlikely to actually follow the best practice.

My favourite one, from a number of sources, is that “best” practice, “good” practice and even “bad” bad practice from somewhere else tends to be adopted because it is easier than attempting to really understand the local context and draw on expertise and knowledge to develop solutions appropriate to that context.

Doing that is hard. Much easier to see what “important organisation X” has done and copy them. This is where fads come from.

This is a small part of the argument made in the book Management Fads in Higher Education: Where They Come From, What They Do, Why They Fail by Robert Birnbaum that I’m currently reading. More on this soon.

Plato on the problems that writing will create for student learning

By admin

On February 9, 2009

In Chapter 2, design theory, elearning, quotes, thesis, Uncategorized

This is the first post in my new “quotes” section. Acting as a storage place for all the interesting insights I come across.

I came across this one as part of the keynote presentation given by Mat Koehler and Punya Mishra at the SITE’2008 conference.

They quote it as being from Plato’s Phaedrus

[Writing] will implant forgetfulness in their souls… calling things to remembrance no longer from within themselves, but by means of external marks…[students will] seem to know much, while for the most part they know nothing, and as men filled, not with wisdom, but with the conceit of wisdom

This illustrates the fear and the actual loss of something as the world moved from an oral to a written culture. For me it highlights that there will always be fear around the potential implications of a new technology. It also really highlights that people can be limited by their in-built and unquestioned assumptions that have been formed by their environment and can find it difficult to evaluate the positives of the new in some sort of objective way.

An example of the mere exposure effect

Month: February 2009 Page 2 of 3

Common sense and innovation

Excellence comes through customizing

Metaphors for higher education

Tidy versus messy

The Ps Framework – Tidy versus messy

References

References

It’s easy to do, not

Advantages

Half-baked ideas

What we’re doing and what is the problem?

Safe-fail probes as a basis for research

Questions

References

Background

Abstract

Slides

Presentation outline

References

The fund and how it works

Why is it broken

Solutions

The solution

Barriers to innovation and inclusion

The benefits of being open

The solution

The wrong view

References

Making technology more protean?

What is BAM?

Origins of BAM

System usage – examples of the protean nature

Implications

Future work

Origins

What does protean mean?

Indications of the current limitation

“Non-protean” processes

Limited views on capabilities of technology

Making systems and processes more protean

The trend is towards more protean technology

Conclusions

References