Situated shared practice, curriculum design and academic development

Am currently reading Faegri et al (2010) as part of developing the justificatory knowledge for the final ISDT for e-learning that is meant to be the contribution of the thesis. The principle from the ISDT that this paper connects with is the idea of a “Multi-skilled, integrated development and support team” (the name is a work in progress). The following is simply a placeholder for a quote from the paper and a brief connection with the ISDT and what I think it means for curriculum design and academic development.

The quote

The paper itself is talking about an action research project where job rotation was introduced into a software development firm with the aim of increasing the quality of the knowledge held by software developers. The basic finding was that in this case, there were some benefits, however, the problems outweighed them. I haven’t read all the way through, I’m currently working through the literature review. The following quote is from the review.

Key enabling factors for knowledge creation is knowledge sharing
and integration [36,54]. Research in organizational learning has emphasized the value of practice; people acquire and share knowledge in socially situated work. Learning in the organization occurs in the interplay between tacit and explicit knowledge while it crosses boundaries of groups, departments, and organizations as people participate in work [17,54]. The process should be situated in shared practice with a joint, collective purpose [12,14,15].

Another related quote

The following is from a bit more related reading, in particular Seely Brown & Duguid (1991) – emphasis added

The source of the oppositions perceived between working, learning, and innovating lies primarily in the gulf between precepts and practice. Formal descriptions of work (e.g., “office procedures”) and of learning (e.g., “subject matter”) are abstracted from actual practice. They inevitably and intentionally omit the details. In a society that attaches particular value to “abstract knowledge,” the details of practice have come to be seen as nonessential, unimportant, and easily developed once the relevant abstractions have been grasped. Thus education, training, and technology design generally focus on abstract representations to the detriment, if not exclusion of actual practice. We, by contrast, suggest that practice is central to understanding work. Abstractions detached from practice distort or obscure intricacies of that practice. Without a clear understanding of those intricacies and the role they play, the practice itself cannot be well understood, engendered (through training), or enhanced (through innovation).

Relevance?

I see this as highly relevant to the question of how to improve learning and teaching in universities, especially in terms of the practice of e-learning, curriculum design and academic development. It’s my suggestion that the common approaches to these tasks in most universities ignore the key enabling factors mentioned in the above quote.

For example, the e-learning designers/developers, curriculum designers and academic developers are generally not directly involved with the everyday practice of learning and teaching within the institution. As a result the teaching academics and these other support staff don’t get the benefit of shared practice.

A further impediment to shared practice is the divisions between e-learning support staff, curriculum designers and academic developers that are introduced by organisational hierarchies. At one stage, I worked at a university where the e-learning support people reported to the IT division, the academic staff developers reported to the HR division, the curriculum designers reported to the library, and teaching academics were organised into faculties. There wasn’t a common shared practice amongst these folk.

Instead, any sharing that did occur was either at high level project or management boards and committees, or in design projects prior to implementation. The separation reduce the ability to combine, share and create new knowledge about what was possible.

The resulting problem

The following quote is from Seely Brown and Duiguid (1991)

Because this corporation’s training programs follow a similar downskilling approach, the reps regard them as generally unhelpful. As a result, a wedge is driven between the corporation and its reps: the corporation assumes the reps are untrainable, uncooperative, and unskilled; whereas the reps view the overly simplistic training programs as a reflection of the corporation’s low estimation of their worth and skills. In fact, their valuation is a testament to the depth of the rep’s insight. They recognize the superficiality of the training because they are conscious of the full complexity of the technology and what it takes to keep it running. The corporation, on the other hand, blinkered by its implicit faith in formal training and canonical practice and its misinterpretation of the rep’s behavior, is unable to appreciate either aspect of their insight.

It resonates strongly with some recent experience of mine at an institution rolling out a new LMS. The training programs around the new LMS, the view of management, and the subsequent response from the academics showed some very strong resemblances to the situation described above.

An alternative

One alternative, is what I’m proposing in the ISDT for e-learning. The following is an initial description of the roles/purpose of the “Multi-skilled, integrated development and support team”. Without too much effort you could probably translate this into broader learning and teaching, not just e-learning. Heaven forbid, you could even use it for “blended learning”.

An emergent university e-learning information system should have a team of people that:

  • is responsible for performing the necessary training, development, helpdesk, and other support tasks required by system use within the institution;
  • contains an appropriate combination of technical, training, media design and production, institutional, and learning and teaching skills and knowledge;
  • through the performance of its allocated tasks the team is integrated into the everyday practice of learning and teaching within the institution and cultivates relationships with system users, especially teaching staff;
  • is integrated into the one organisational unit, and as much as possible, co-located;
  • can perform small scale changes to the system in response to problems, observations, and lessons learned during system support and training tasks rapidly without needing formal governance approval;
  • actively examines and reflects on system use and non-use – with a particular emphasis on identifying and examining what early innovators – to identify areas for system improvement and extension;
  • is able to identify and to raise the need for large scale changes to the system with an appropriate governance process; and
  • is trusted by organisational leadership to translate organisational goals into changes within the system, its support and use.

References

Faegri, T. E., Dyba, T., & Dingsoyr, T. (2010). Introducing knowledge redundancy practice in software development: Experiences with job rotation in support work. Information and Software Technology, 52(10), 1118-1132.

Seely Brown, J., & Duguid, P. (1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40-57.

The rider, elephant, and shaping the path

Listened to this interview of Chip Heath, a Stanford Professor in Organizational Behaviour about his co-authored book Switch: How to change things when change is hard. My particular interest in this arises from figuring out how to improve learning and teaching in universities. From the interview and the podcast this seems to be another one in a line of “popular science” books aimed at making clear what science/research knows about the topic.

The basic summary of the findings seems to be. If you wish to make change more likely, then your approach has to (metaphorically):

  • direct the rider;
    The rider represents the rational/analytical decision making capability of an individual. This capability needs to be appropriately directed.
  • engage the elephant; and
    The elephant represents the individual’s emotional/instinctive decision making approach. From the interview, the elephant/rider metaphor has the express purpose of showing that the elephant is far stronger than the rider. In typical situations, the elephant is going to win, unless there’s some engagement.
  • shape the path.
    This represents the physical and related environment in which the change is going to take place. My recollection is that the shaping has to support the first two components, but also be designed to make it easier to traverse the path and get to the goal.

There are two parts of the discussion that stuck with me as I think they connect with the task of improving learning and teaching within universities.

  1. The over-rationalisation of experts.
  2. Small scale wins.

Over-rationalisation of experts

The connection between organisational change and losing weight seems increasingly common, it’s one I used and it’s mentioned in the interview. One example used in the interview is to show how a major problem with change is that it is driven by experts. Experts who have significantly larger “riders” (i.e. rational/analytical knowledge) of the problem area/target of change than the people they are trying to change. This overly large rider leads to change mechanisms that over complicate things.

The example they use is the recently modified food pyramid from the United States that makes suggestions something like, “For a balanced diet you should consume X tablespoons of Y a day”. While this makes sense to the experts, a normal person has no idea of how many tablespoons of Y is in their daily diet. In order to achieve the desired change, the individual needs to develop all sorts of additional knowledge and expertise. Which is just not likely.

They compare this with some US-based populariser of weight loss who proposes much simpler suggestions e.g. “Don’t eat anything that comes through your car window”. It’s a simpler, more evocative suggestion that appears to be easier for the rider to understand and helps engage the elephant somewhat.

I can see the equivalent of this within learning and teaching in higher education. Change processes are typically conceived and managed by experts. Experts who over rationalise.

Small scale wins

Related to the above is the idea that change always consists of barriers or steps that have to be stepped over. Change is difficult. The suggestion is that when shaping the path you want to design it in such a way so that the elephant can almost just walk over the barrier. The interviewer gives the example of never being able to get her teenage sons to stop taking towels out of the bathroom and into their bedroom. Eventually what worked was “shaping the path” by storing the sons’ underwear in the bathroom, not their bedroom.

When it comes to improving learning and teaching in universities, I don’t think enough attention is paid to “shaping the path” like this. I think this is in part due to the process being driven by the experts, so they simply don’t see the need. But it is also, increasingly, due to the fact that the people involved can’t shape the path. Some of the reasons the path can’t be shaped include:

  • Changing the “research is what gets me promoted” culture in higher education is very, very difficult and not likely to happen effectively if just one institution does it.
  • When it comes to L&T path (e.g. the LMS product model or the physical infrastructure of a campus) it is not exactly set up to enable “shaping”.
  • The people involved at a university, especially in e-learning, don’t have the skills or the organisational structure to enable “shaping”.

Possible uses of academic analytics

The following is a slightly edited copy of a message I’ve just sent off to the Learning Analytics GoogleGroup set up by George Siemens. I’m into reuse. Essentially it tries to highlight a small subset of the uses of learning analytics that I see as most interesting.

Some colleagues and I have been taking some baby steps in this area. In terms of trying to understand where we might end up doing we’ve started talking about three aspects. All very tentative, but can highlight a small subset of potential uses.

I have perhaps used a broad definition for analytics.

1. What?

This is the visualisation of what has happened in learning.

Lots of work to be done here in terms of finding out what patterns to look for and how to represent them.

For example, we took factors identified by Fresen (2007) as promoting quality as a guide to look for particular patterns.

Use #1 – Analytics can be used to test theories/propositions around learning and teaching. Perhaps by supplementing existing methods.

Fresen, J. (2007). A taxonomy of factors to promote quality web-supported learning. International Journal on E-Learning, 6(3), 351-362.

2. Why?

Once we’ve seen certain patterns, the obvious question is why did that pattern arise?

e.g. A colleague found a pattern where, on average, the older a distance education student was, the more they used the LMS.

The obvious question is why? Various theories are possible, but which apply? The 2nd person to comment on the above post is a psychology research masters student who is just completing some research attempting to identify an answer to they why question.

Use #2 – Analytics can be used to identify areas for future research.

3. How?

How can you harness analytics to improve learning and teaching.

Rowan made the point about using analytics to encourage changes in policy. I’ve seen this happen. Some early analysis at one institution showed that very few course websites had a discussion forum and even fewer used it effectively. Policy changed.

Use #3 – Analytics can inform policy change.

As we work/worked in areas supporting university academics in their teaching we were most interested in this question. In particular, we were interested in how analytics could be used to improve academics teaching.

Use #4 – Analytics can be used to encourage academics to reflect on their teaching.

e.g. Another colleague used analytics to reflect on how his conceptions of teaching matched what analytics showed about what happened within his courses.

Use #5 – Analytics can be used to encourage students to think differently about their learning.

Presenting students with different visualisations of what they are doing (or not doing) around learning can also encourage them to change practice.

I’ve heard reports that use of the SNAPP tool has achieved this. I’ve heard similar reports about the
use of the Progress Bar block for Moodle.

It’s possible to see a common trend in the last few uses. To some extent analytics is being used to improve “distributed cognition” in terms of putting some more smarts into the environment, which in turn becomes more likely to be seen and acted upon by policy makers, students or teachers.

However, what these people do in response to the improved knowledge they have, is still fairly open. I have a particular interest in how to encourage and enable these folk to use their improved knowledge in useful and interesting ways.

Which in turns will generate changed behaviour and hopefully changed use of the system. This takes us back to the what question at the start.

Use #6: Analytics can be an important component to on-going learning about what works and what doesn’t in learning and teaching.

BIM, blog posts and special characters

The following is a summary/explanation of a common problem with BIM and its mirroring of blog posts and a common solution. The problem is generally caused by folk creating their blog posts in Word and then copying and pasting them into the blog post. For various reasons this process brings along some “special” characters which, while they work fine in Word, screw up royally within more constrained textual representations, like those of Web browsers and XML/RSS parsing libraries.

Reported problem

A student has made a post to their blog, the teacher can see it on the student’s blog, but it’s simply not present within BIM. BIM isn’t picking it up.

Diagnosis of the problem

Steps to diagnosing the source of the problem were:

  • Login to the Moodle course site and confirm the problem.
    Yes, student has posted it to his blog, but BIM not picking it up.
  • Register the student blog with a local copy of BIM.
    Ahh, the blog post shows up on my local copy, but only the first dozen or so characters.
  • Look at the feed for the student blog.
    Find the tell-tale signs of special characters exactly where my local copy of BIM cuts off the post.

Okay, BIM currently attempts to handle special characters, obviously it is missing something.

Common solution

This appears likely to be an on-going problem, so am going to leave a bit of commented code in place that I use to implement this “solution”. The “solution” is basically get BIM to print out each individual character in a blog post along with its ASCII value. Use this ASCII value to modify the bim_clean_content function to remove the offending special character.

The code that implements this character by character display looks like this

[sourcecode lang=”php”]
# KLUDGE: simple test to find out which special characters are
# causing problems
$contenta = str_split( $content);
print "<h1> $title </h1>";
foreach ( $contenta as $char ) {
echo "$char .. " . ord( $char ) . "<br />";
}
[/sourcecode]

For this particular problem the offending character is 189. So add the following to the function bim_clean_content. It appears that character 189 is some sort of dash.

[sourcecode lang=”php”]
$post = ereg_replace( chr(189), "-", $post );
[/sourcecode]

Re-register a student with the same blog and 189 has been replaced. Remove the kludge and it all appears to be registered correctly.

Nobody likes a do-gooder – another reason for e-learning not mainstreaming?

Came across the article, “Nobody likes a do-gooder: Study confirms selfless behaviour is alienating” from the Daily Mail via Morgaine’s amplify. I’m wondering if there’s a connection between this and the chasm in the adoption of instructional technology identified by Geoghegan (1994)

The chasm

Back in 1994, Geoghegan draw on Moore’s Crossing the Chasm to explain why instructional technology wasn’t being adopted by the majority of university academics. The suggestion is that there is a significant difference between the early adopters of instructional technology and the early majority. That what works for one group, doesn’t work for the others. There is a chasm. Geoghegan (1994) also suggested that the “technologists alliance” – vendors of instructional technology and the university folk charged with supporting instructional technology – adopt approaches that work for the early adopters, not the early majority.

Nobody likes do-gooders

The Daily Mail article reports on some psychological research that draws some conclusions about how “do-gooders” are seen by the majority

Researchers say do-gooders come to be resented because they ‘raise the bar’ for what is expected of everyone.

This resonates with my experience as an early adopter and more broadly with observations of higher education. The early adopters, those really keen on learning and teaching are seen a bit differently by those that aren’t keen. I wonder if the “raise the bar” issue applies? Would imagine this could be quite common in a higher education environment where research retains its primacy, but universities are under increasing pressure to improve their learning and teaching. And more importantly show to everyone that they have improved.

The complete study is outlined in a journal article.

References

Geoghegan, W. (1994). Whatever happened to instructional technology? Paper presented at the 22nd Annual Conferences of the International Business Schools Computing Association, Baltimore, MD.

The end of management – lessons for universities?

Yet another “death of X” article is the spark for this post. This one comes from the Wall Street Journal and is titled The end of management. There’s been a wave of these articles recently, but this one I like because it caters to my prejudice that most of the problems in organisations, especially in universities around learning and teaching, arise from an inappropriate management paradigm. The following has some connections to the oil sheiks thread.

Some choice quotes

Corporations are bureaucracies and managers are bureaucrats. Their fundamental tendency is toward self-perpetuation. They are, almost by definition, resistant to change. They were designed and tasked, not with reinforcing market forces, but with supplanting and even resisting the market.

and

The weakness of managed corporations in dealing with accelerating change is only half the double-flanked attack on traditional notions of corporate management. The other half comes from the erosion of the fundamental justification for corporations in the first place.

And a quote from Gary Hamel which summarises much of the problem with real innovation, including innovation around management

That thing that limits us is that we are extraordinarily familiar with the old model, but the new model, we haven’t even seen yet.

Moving onto the question of resources

In corporations, decisions about allocating resources are made by people with a vested interest in the status quo. “The single biggest reason companies fail,” says Mr. Hamel, “is that they overinvest in what is, as opposed to what might be.”

The challenge that strikes at the heart of improving learning and teaching within universities is capture in this quote

there’s the even bigger challenge of creating structures that motivate and inspire workers. There’s plenty of evidence that most workers in today’s complex organizations are simply not engaged in their work.

Does your university have large numbers of academic staff that are actively engaged in teaching? How does it do it?

I’d like to work for a university that gets this, or at least is trying to.

University e-learning systems: the need for new product and process models and some examples

I’m in the midst of the horrible task of trying to abstract what I think I know about implementing e-learning information systems within universities into the formal “language” required of an information systems design theory and a PhD thesis. This post is a welcome break from that, but is still connected in that it builds on what is perhaps fundamentally different between what most universities are currently doing, and what I think is a more effective approach. In particular, it highlights some more recent developments which are arguably a step towards what I’m thinking.

As it turns out, this post is also an attempt to crystalise some early thinking about what goes into the ISDT. So some of the following is a bit rough. Actually, writing this has identified one perspective that I hadn’t thought of, which is potentially important.

Edu 2.0

The post arises from having listened to this interview with Graham Glass the guy behind Edu 2.0, which is essentially a cloud-based LMS. It’s probably one of a growing number out there. What I found interesting was his description of the product and the process behind Edu 2.0.

In terms of product (i.e. the technology used to provide the e-learning services), the suggestion was that because Edu 2.0 is based in the cloud – in this case Amazon’s S3 service – it could be updated much more quickly than more traditional institutionally hosted LMSs. There some connection here with Google’s approach to on-going modifications to live software.

Coupled with this product flexibility was a process (i.e. the process through which users were supported and the system evolved) that very much focused on the Edu 2.0 developers interacting with the users of the product. For example, releasing proposals and screenshots of new features within discussion forums populated with users and getting feedback; and also responding quickly to requests for fixes or extensions from users. To such an extent that Glass reports users of Edu 2.0 feeling like it is “there EDU 2.0” because it responds so quickly to them and their needs.

The traditional Uni/LMS approach is broken

In the thesis I argue that when you look at how universities are currently implementing e-learning information systems (i.e. selecting and implementing an LMS) the product (the enterprise LMS, the one ring to rule them all) and the process they use are not a very good match at all for the requirements of effectively supporting learning and teaching. In a nut shell, the product and the process is aimed at reducing diversity and the ability to learn, while diversity is a key characteristic of learning and teaching at a university. Not to mention that when it comes to e-learning within universities, it’s still very early days and it is essential that any systemic approach to e-learning have the ability to learn from its implementation and make changes.

I attempted to expand on this argument in the presentation I gave at the EDUCAUSE’2009 conference in Denver last year.

What is needed

The alternative I’m trying to propose within the formal language of the ISDT is that e-learning within universities should seek to use a product (i.e. a specific collection of technologies) that is incredible flexible. The product must, as much as possible, enable rapid, on-going, and sometimes quite significant changes.

To harness this flexibility, the support and development process for e-learning should, rather than be focused on top-down, quality assurance type processes, be focused on closely observing what is being done with the system and using those lessons to modify the product to better suit the diversity of local needs. In particular, the process needs to be adopter focused, which is described by Surry and Farquhar (1997) as seeing the individual choosing to adopt the innovation as the primary force for change.

To some extent, this ability to respond to the local social context can be hard with a software product that has to be used in multiple different contexts. e.g. an LMS used in different institutions.

Slow evolution but not there yet

All university e-learning implementation is not the same. There has been a gentle evolution away from less flexible products to more flexible produces, e.g.

  1. Commercial LMS, hosted on institutional servers.
    Incredibly inflexible. You have to wait for the commercial vendor to see the cost/benefit argument to implement a change in the code base, and then you have to wait until your local IT department can schedule the upgrade to the product.
  2. Open source LMS, hosted on institutional servers.
    Less inflexible. You still have to wait for a developer to see your change as an interesting itch to scratch. This can be quite quick, but it can also be slow. It can be especially quick if your institution has good developers, but good developers cost big money. Even if the developer scratches your itch, the change has to be accepted into the open source code base, which can take some time if its a major change. Then, finally, after the code base is changed, you have to wait for your local IT shop to schedule the upgrade.
  3. Open source LMS, with hosting outsourced.
    This can be a bit quicker than the institutional hosted version. Mainly because the hosting company may well have some decent developers and significant knowledge of upgrading the LMS. However, it’s still going to cost a bit, and it’s not going to be real quick.

The cloud-based approach used by EDU 2.0 does offer a product that is potentially more flexible than existing LMS models. However, apart from the general slowness in the updating, if the change is very specific to an individual institution, it is going to cause some significant problems, regardless of the product model.

Some alternative product models

The EDU 2.0 model doesn’t help the customisation problem. In fact, it probably makes it a bit worse as the same code base is being used by hundreds of institutions from across the globe. The model being adopted by Moodle (and probably others), having plugins you can add, is a step in the right direction in that institutions can choose to have different plugins installed.
However, this model typically assumes that all the plugins have to use the same API, language, or framework. If they don’t, they can’t be installed on the local server and integrated into the LMS.

This requirements is necessary because there is an assumption for many (but not all) plugins that they provide the entire functionality and must be run on the local server. So there is a need for a tighter coupling between the plugin and the LMS and consequently less local flexibility.

A plugin like BIM is a little different. There is a wrapper that is tightly integrated into Moodle to provide some features. However, the majority of the functionality is provided by software (in this case blogging engines) that are chosen by the individual students. Here the flexibility is provided by the loose coupling between blog engine and Moodle.

Mm, still need some more work on this.

References

Surry, D., & Farquhar, J. (1997). Diffusion Theory and Instructional Technology. e-Journal of Instructional Science and Technology, 2(1), 269-278.

Oil sheiks, Lucifer and university learning and teaching

The following arises from a combination of factors including:

Old wine in new bottles

Perhaps the key quote from Mark’s post is

This post is simply to try and say what many people don’t want to say and that is, that most universities really don’t care about educational technology or elearning.

My related perspective is that the vast majority of university learning and teaching is, at best (trying to be very positive), just ok. There’s a small bit that is really, really bad; and a small bit that is really, really good. In addition, most interventions to improve learning and teaching are not doing anything to change this distribution. At best, they might change the media, but the overall distribution is the same.

There’s a quote from Dutton and Loader (2002) that goes something like

without new learning paradigms educators are likely to use technology to do things the way they have always done; but with new and more expensive technology.

I am currently of the opinion that without new management/leadership paradigms to inform how universities improve learning and teaching, the distribution is going to remain the same just with new and more expensive organisational structures. This article from the Goldwater Institute about administrative bloat at American universities might be an indicator of that.

Don’t blame the academics

The “When good people turn bad” radio program is an interview with Philip Zimbardo. He’s the guy responsible for the Stanford prisoner study, an example of where good people turned really bad because of the situation in which they were place. The interview includes the following from Prof Zimbardo

You no longer can focus only on individual freedom of will, individual rationality. People are always behaving in a context, in a situation, and those situations are always created and maintained by powerful systems, political systems, cultural, religious ones. And so we have to take a more complex view of human nature because human beings are complex.

This resonates somewhat with a point that Mark makes

the problem of adoption is primarily not a technical one but one of organisational culture

. I agree. It’s the culture, the systems, the processes and the policies within universities that are encouraging/enshrining this distribution where most university learning and teaching is, at best, just ok.

The culture/system doesn’t encourage nor enable this to change. When management do seek to do something about this, their existing “management paradigm” encourages an emphasis on requiring change without doing anything effective to change the culture/system.

The proposition and the interest

Which is where I am interested in and propose the following

If you really wish to improve the majority of learning and teaching within a university, then you have to focus on changing the culture/system so that academics staff are encouraged and enabled to engage in learning about how to teach.

In addition, I would suggest that requiring learning (e.g. through requiring all new academic staff to obtain a formal qualification in learning) without aligning the entire culture/system to enable academic staff to learn and experiment (some of these characteristics are summarised here) is doomed to failure.

I’d also suggest that there is no way you can “align” the culture/system of a university to enable and encourage academic staff learning about teaching. At best you can engage in a continual process of “aligning” the culture/system as that process of “aligning” is itself a learning process.

Easy to say

I can imagine some universities leaders saying “No shit Sherlock, what do you think we’re doing?”. My response is you aren’t really doing this. Your paradigm is fundamentally inappropriate, regardless of what you claim.

However, actually achieving this is not simple and I don’t claim to have all the answers. This is why this is phrased as a proposition, it’s an area requiring more work.

I am hoping that within a few days, I might have a small subset of an answer in the next, and hopefully final, iteration of the design theory for e-learning that is meant to be the contribution of my thesis.

References

Dutton, W. and B. Loader (2002). Introduction. Digital Academe: The New Media and Institutions of Higher Education and Learning. W. Dutton and B. Loader. London, Routledge: 1-32.

Extracting case study research and multiple contexts?

The following is an attempt to consider and respond to a suggestion on my thesis from my esteemed supervisor. It’s essentially some thinking and questions arising from further refinement of the research method being used in my thesis.

The suggestion was that perhaps my work is an example of van Aken’s (2004) extracting/ive case study research. Is this the case?

What’s the method?

I’m essentially using an action research method within a single case study to develop/abstract a design theory for e-learning. The action research cycle is somewhat disjointed in that the case – the development and evolution of Webfuse from 1997 through 2009 – involved an on-going process of action research as different aspects of the system were developed and changed (evidence of this is found in my publications where this process was played out).

In addition to this more traditional example of action research there was another round of action research which occurred after most of the initial Webfuse development was completed. This was the cycle that led to the development and re-development of the design theories that were abstracted from the Webfuse. This is almost an example of a meta action research cycle. The existence of this on-going cycle is also evident in the publications from about 2002 onwards as there are three different representations of the ISDT, with another being formulated now.

Extracting and developing case study research

This type of method does seem to be fairly evident in the broader business research methods with van Aken (2004) identifying two relevant types of case study:

  • extracting case study research; and
    Also labeled as “best-practice” research. The aim here is to extract technological rules (theory) that is already being used in practice.
  • developing case study research.
    Researchers collaborate with people within specific contexts to solve a local problem. However, there is a reflective cycle after each case in which the researcher seeks to develop knowledge that can be transferred to similar contexts. Also labeled as clinical research.

It appears that van Aken (2004) makes another distinction between these two in that extracting case study research generally involves multiple case studies in which the same best practices have been implemented in multiple contexts.

However, van Aken (2004) does make the suggestion that developing case study research isn’t necessarily multiple case study. Instead development can go through an “alpha test” in which the “rule” is evaluated within a single context. However, significant value arises when there is “beta testing” when the “rule” is translated into other contexts and “third parties use it, assess its effectiveness and make final improvements”. van Aken (2004) makes the connection with beta-testing and replication research.

However, this bit is interesting

An essential element of beta-testing is that testing is conducted by a third party to counteract the ‘unrecognized defences’ of the originator of the rule, which may blind him or her to possible flaws in its use (Argyris, 1996).

This somewhat later quote is also of interest to my thesis

Management Theory research as described here may have most in common with the approach to Action Research discussed by Eden and Huxham (1996), who stress among other things the need for generality of research results (p. 530) and write about ‘action research aimed at the study of organization and organizations . . . where it is likely that the researcher accepts the dominant managerial ideology’ (p. 529).

So which “case study research” is my thesis?

From this description, I think that my thesis research leans more towards the developing case study research. There has, however, been two different reflective cycles. The first was the immediate reflection on the intervention that contributed to changes in the intervention. Then there was some additional reflection afterwards to move into the design theory. It was this additional reflection that is aiming to generate the more general contribution to knowledge.

So when does a context change?

An interesting aside here is exactly how do you define a different context. To some extent it, I would argue that CQU in 1996/1997 – when Webfuse started development – is almost an entirely different context in 2001/2002 and certainly by 2005. It’s still the same institution, however, the internal structure, people and processes have changed quite significantly. Significantly enough to be a new context? Not sure. But answering that question might be interesting.

References

van Aken, J. (2004). “Management research based on the paradigm of the design sciences: The quest for field-tested and grounded technological rules.” Journal of Management Studies 41(2): 219-246.

PhD Update #28 – minimal work, feedback arrived

Am attempting to keep this weekly update thing going within the new context. Not much to report this week as I was out of town for three days, baby sitting and recovering for another, and thinking about jobs and family-based travel on the other.

Am going to try a more evolutionary, to do list oriented approach to this post as well. i.e. the finer grain tasks I need to complete will be added and hopefully checked off here during the week.

What I did last week

Two fairly small bits of work done this week:

  • Completed a draft of the “lessons learned” section of chapter 5.
    This is still fairly early and will be revised, but wanted to get something down on paper, so to speak.
  • Received and started processing some feedback from my esteemed supervisor.

This left at least 3 of the tasks I set myself, undone.

What I need to do next week

A growing list of tasks to do:

  • Read comments on chapter 6, modify what I’ve done, send it off to supervisor.
    • Main point is to have a good overview of findings, i.e. summarise the Ps Framework, New view on ISDT and the ISDT for e-learning.
    • Look at including the Gregor and Jones table for theory summary.
    • Implications for IS/Design researchers needs to mention contribution of Gregor and Jones
    • See what insights Eisenhardt (1989) can add.
    • Look into van Aken’s “extracting” and “developing” case study concepts and see how it can be worked into this chapter and also perhaps chapter 3
    • The conclusions section needs to be more of a “declaration of victory”.
  • Read comments on chapter 4, especially on the ISDT
    • Think about how chapter 4 ISDT may change in response;
      • Table 4.3 – the principles read more like requirements. This opens up the question of what form should a principle take? Perhaps Bunge/van Aken’s technological rules?
        van Aken (2004) “If you want to achieve Y in situation Z, then something like action X will help”
      • Is section 4.3 about high-level guiding requirements and Section 4.4 more about design principles? YES
      • Need to think about the names of the principles/guidelines, especially the important/innovative ones.
      • Add the Gregor and Jones “theory table” as a summary/overview
    • Start thinking about form of ISDT for chapter 5.
    • Make more general changes to chapter 4 and the rest of the thesis
      • Rework the Chapter 4 introduction, and perhaps other parts of the chapter, to better orient the reader to the purpose and the structure of the chapter.
      • Look at how much of the design work in the thesis was done prior to commencing the thesis and how this is outlined within the thesis, with some connection to the extractive case study conceptNO.
  • Complete evaluation section of chapter 5.
    • What can be evidenced about the trend in feature and broader user adoption about Webfuse systems post 2004/2005.
  • Revisit chapter 1 and get to final draft, start sending around.