Adding "deleted" to BIM

The following is a record of the process of adding a “deleted” state for a blog post in BIM. This is in response to an issue that has arisen out of BIM usage at CQU.

The problem

BIM always keeps a record of all posts ever found in a student’s blog feed. Even if the student deletes the post or changes the post, BIM keeps a record of the post in its internal database. This is a feature of BIM as one of its aims is to provide a “safe/secure” record of what is posted to the students’ blogs, just in case the blog service provider goes away.

In addition, BIM also attempts to allocate old posts in its internal database to questions. Sometimes, you do wish to “delete” a post to stop BIM from allocating it. To give the student a chance to post something new.

Requirements

Requirements for this feature include:

  • Only the coordinator can “delete” a post.
  • The post is not actually removed from the internal BIM database, it just is no longer able to be “allocated” by BIM, released by the coordinator, visible to the student, or put into the gradebook.
  • It must be possible to “undelete” a post.
  • Deletion should not change any of the other data about a post (e.g. marker comments).
  • Perhaps rather than be invisible to the student, it should just be obvious that it has been deleted.

Design

  • Add “Deleted” status for mdl_bim_marking.
    Modified setting via XMLDB, but the database is not being updated. This has been a problem before which I’ve kludged around. But it’s not something to rely upon for production. Will have to solve this problem.
  • Add interface for staff to “delete”
  • Modify student view to show that it is deleted

The alignment project as leadership

The following signals a slight change in direction around the curriculum mapping project. First, the project is now going under the label “alignment project” (curriculum mapping is just one aspect of the project). Second, the project is likely to be re-framed as an application for an ALTC leadership grant. This post is an attempt to begin this re-framing. It’s really just thinking out loud.

As a result, I am very interested in suggestions and criticisms. In terms of suggestions, I’m particularly keen for insights onto better/alternative theoretical frames. It has become a bit confused as I’ve tried out different lines of approach.

Note: In the following a “course” typically means the smallest unit of teaching offered by an institution. A “program” is a group of courses that form a degree or perhaps a major.

Summary of the alignment project

Alignment is an increasingly core component of teaching in Australian universities. At a basic level, alignment is where the learning resources, learning activities and assessment of learning all align with the stated outcomes or aims of teaching (in a course/unit or program/course). Such alignment is a core component of various “movements” within university learning and teaching including: graduate attributes, quality assurance, and improvements in learning and teaching. However, consideration of alignment is not a regular, everyday part of teaching or learning at universities.

The majority of academic teaching practice involves the teaching of an existing course, one the academic has usually taught before, and as such most “teaching practice” revolves around making minor modifications to material or content (Stark, 2000). Academics are not often required to engage in the development of new courses or major overhauls of existing courses (Stark and Lowther 1988). Alignment is most considered during the development of new courses, major overhauls of existing courses or in response to external quality assurance needs.

The following two sub-sections try to summarise the alignment project. The first is a more concrete description, the second more abstract or theoretical. The thinking behind this project has changed many times, the following are likely to change. Suggest away.

Process and intent

As currently thought, the alignment project can be described as four main tasks:

  1. Modify Moodle to allow mapping of alignment.
    Moodle has been chosen because it is the institutional e-learning system at both institutions that are part of the project. As such, support for Moodle is embedded into the institution and will continue to be supported. In addition, Moodle is an increasing part of the everyday practice of teaching academics. Lastly, Moodle, for a number of reasons, is very flexible and easy to modify and any modifications made could be usable by other institutions. The point of this project is not the Moodle modifications, it’s simply the best solution for embedding these changes into the institutions.

    The changes will focus on enabling the alignment of outcomes (be the course learning outcomes, program learning outcomes, those from accrediting bodies, graduate attributes etc) with the assessment, activities and resources within a course site. Having this functionality is a foundation for the rest of the project.

  2. Work with teaching staff to map their courses.
    Mapping the alignment of a course within Moodle is not going to be straight forward. Teaching staff are likely to be busy and may not be entirely familiar with the concepts around alignment and mapping. The content and design of a Moodle course site may not be currently appropriate in terms of developing a useful mapping of alignment. Using the alignment mapping functionality added to Moodle may not be straight forward.

    For these and other reasons staff from the L&T support services will have to work collaboratively with academic staff to overcome these problems. This will be the first cycle of action research and will identify specific problems and insights into potential solutions.

  3. Work with teaching staff to embed alignment into everyday practice.
    Once the initial mapping of alignment is complete, the focus moves onto helping academics maintain and reflect on the level of alignment within their course and programs. On helping them embed alignment considerations into everyday practice.

    This is the second cycle of action research and will likely include the development of models, tools and processes that address questions such as:

    • How do you encourage reflection and action based on the everyday consideration of alignment?
      Identify the support, processes and tools do teaching academics and program leaders need to encourage and enable reflection and action?
    • How do you measure and give feedback upon action based on the everyday consideration of alignment?
  4. Develop on-going and embedded institutional processes that take the lessons learned from the everyday consideration of alignment and use it to remove barriers within the institutional context.

Theoretical perspective

The alignment project is seeking to take the first steps towards what Biggs (2001) called the reflective institution. It seeks to do this by modifying the institutional systems around learning and teaching in ways suggested by Biggs (2001). These are:

  1. Make obvious the quality model.
    Most institutions espouse the theory of alignment, however the teaching systems and processes employed by institutions do not make this theory explicit. The first step is to modify these systems so that a focus on alignment is made explicit as a part of every day teaching practice.

    This is partly achieved through the modifications to Moodle to enable mapping of alignment. But more importantly it is achieved through the changes in L&T support, systems and processes that support academic in using those new Moodle capabilities. In part, these changes are the next step.

  2. Provide appropriate support for quality enhancement.
    Further modify these systems and process to enable and encourage teaching staff to reflect and improve their teaching through a focus on alignment. The modifications build on the changes in the previous stage to enable this support to be highly contextualised to everyday teaching practice. A focus on helping in what academics do most often, the fine-tuning of existing courses.
  3. Inform the quality feasibility process.
    Quality feasibility is the removal of factors in the teaching environment that are not conducive to good L&T. The everyday consideration of alignment will identify a range of barriers in the institutional setting, many of which will require the engagement of institutional leadership to remove.

ALTC leadership grants

The guidelines for the ALTC leadership grants scheme describe the grants as being for (my emphasis added)

projects that build leadership capacity in ways consistent with the promotion and enhancement of learning and teaching in contemporary higher education, and which reflect the ALTC’s values of excellence, inclusiveness, diversity and collaboration, and its commitment to long-term, systemic change.

The scheme has three priorities which can be summarised as being focused on: institutional leadership capacity building; disciplinary and cross-disciplinary leadership; and building on earlier projects. The alignment project seems to be best suited to Priority one

institutional leadership to enhance learning and teaching through leadership capacity-building at the institutional level.

  • Funding range: $150,000 to $220,000
  • Project duration: up to 2 years

In completing a lit review around leadership for the ALTC, Southwell and Morgan (2009) make the observation that

Leadership for Excellence in Learning and Teaching Program is expected to be a ‘demonstrable enhancement of learning and teaching through leadership capacity building’

What is leadership?

It seems required when discussing leadership to make the observation that “leadership eludes comprehensive definition” (Southwell and Morgan, 2009). Southwell and Morgan (2009) reference Marshall (2006) and Jameson (2006) as folk who have made that observation. Having made this claim, the idea is that you then define your familiarity with the broad array of perspectives, understandings and definitions of what leadership is. I’ll postpone that bit for now.

Southwell and Morgan (2009) draw on Leithwood and Levin (2005) who suggest that the core of most conceptions of leadership are two functions generally considered to be indispensable:

  1. Direction-setting: helping members of the organization establish a widely agreed on direction or set of purposes considered valuable for the organization; and
  2. Influence: encouraging organizational members to act in ways that seem helpful in moving toward the agreed on directions or purposes

Leithwood and Levin (2005) arrive at these two functions by adopting a definition where “the primary effect of organisational leadership would be significant change in a direction valued by the organisation”. In defending their definition or understanding of leadership, Leithwood and Levin (2005) agree that this may not be a precise definition, but that attempts to too narrowly define a complex topic like leadership is “more likely to trivialise than help bring greater clarity to its meaning”.

How the alignment project fits

Taking the emphasised phrases from the above purpose of the ALTC projects, I’d suggest/argue that the alignment project fits in the following ways:

  • leadership capacity;
    In terms of the above set of functions, the project is aiming to build into the systems and processes of each host institution the capacity to make consideration of alignment an every day part of practice. It helps establish a widely agreed direction for L&T and helps influence organisational members in moving towards that agreed direction.
  • promotion and enhancement of L&T;
    The educational literature is replete with evidence that consideration of alignment changes the conception of L&T held by academics and that it also results in improvements in student learning outcomes.
  • contemporary higher education;
    The project recognises and seeks to fulfill the increased requirements for accountability from a range of diverse source, however, it seeks to achieve it in a way that offers significant greater benefits that existing methods. As part of this the application seeks to engage with the on-going argument over centralised or devolved L&T support services by aiming for a focus on an approach to L&T support services that seeks to contextualise such support into the every day practice of teaching academics.
  • inclusiveness, diversity and collaboration; and
    The action research process suggested for use by the project is largely based on recognition within the project that engaging with the full diversity of higher education is essential. It recognises that this diversity will result in different approaches and benefits and that the process needs to enable this to happen. Similarly, collaboration is seen as essential to the project. Not just in the process used in this project, but in the aims of the project. An important aim of the project is to increase the collaboration around consideration of alignment in teaching.
  • long-term, systemic change.
    The project aims to embed consideration of alignment into the everyday practice of teaching staff. i.e. the aim of the project is long-term, systemic change. The process and approach being used is designed to achieve that aim.

The alignment project as leadership

Contemporary higher education is placing increasing importance on the concept of alignment in learning and teaching. In terms of quality assessment, program accreditation, graduate attributes and generally improving L&T alignment is broadly seen as a necessary component. However, many university courses are not all that well aligned and one explanation for this is that consideration and discussion of alignment is not a regular part of everyday teaching practice. Alignment is often only considered at the time of course and program reviews or accreditation.

The aim of the alignment project is to build leadership capacity into the system and processes of education within a university so as to encourage and enable effective and informed consideration of alignment as part of everyday teaching. This embedding of alignment into everyday practice then serves as the foundation for a range of other possibilities.

The alignment project is an example of leadership as it is attempting to encourage significant change – in the form of increased consideration of alignment at all levels – that is valued by the individual universities and the broader higher education sector. In addition, there is broad agreement in the education literature that alignment has significant positive effects on student learning outcomes.

The alignment project intends to fulfill the two indispensable functions of leadership identified by Leithwood and Levin (2005):

  • direction-setting; and
    By making considerations of alignment a visible and hopefully key aspect of everyday teaching, there should be an increased emphasis placed on learning outcomes, graduate attributes and other “outputs of learning”. This should encourage and assist academics teaching the same course or in the same program to increase discussion of these outcomes. To increase discussion of the purpose or direction of a course or program.
  • influence.
    A specific aim of the alignment project is to modify the teaching environment so that considerations, discussions and reflection upon alignment are directly encouraged and enabled. It encourages and enables them to think about how to move towards the stated directions or purpose.

Still not happy with that division. First attempt to make it concrete. Another query that arises from this is whether or not this project builds leadership capacity at two or more levels. For example, one approach is that it builds capacity at both the:

  • Institutional level; and
    At an institutional level it enables curriculum/program alignment, but also accountability etc.
  • Instructional level.
    This is the level between student and course/teacher. It makes alignment an more regular component of instruction.

Outcomes

If successful, the project should result in the consideration of alignment should becoming an every day component of teaching practice. This should/could generate the following outcomes:

  • Changes in the conceptions of L&T held by teaching staff;
    Being required to consider alignment encourages a different way of looking at teaching. This should encourage changes in the conceptions of L&T held by teaching staff towards those considered more appropriate.
  • Improvements in student learning outcomes;
    Effective considerations of alignment should increase the alignment within courses. The educational literature suggests that increases in alignment will result in improve student learning outcomes. In addition, it is likely that one of the likely additional outcomes will be making alignment more visible to students. Which should also encourage improved student learning outcomes.
  • Improvements in the quality and timeliness of quality assurance; and
    In terms of demonstrating alignment against outcomes or attributes, current quality assurance practices rely on special “mappings” that are held every few years. By embedding alignment considerations and mapping into every day teaching practice, the there is no longer any need for special “mappings”. “Mapping” information can be generated at any time as it is maintained as part of normal practice.
  • A variety of additional outcomes.
    Embedding alignment considerations as an everyday practice is the foundation of the project. The availability of “mapping” information and the on-going consideration of alignment will generate a range of additional outcomes. However, the diversity inherent in universities and higher education, combined with the fundamental change in everyday practice which this project aims to achieve means that it is unlikely we can predict successfully all of these outcomes.

Theoretical foundations

The ALTC place significant emphasis on applications having a sound and obvious theoretical base. The theoretical work that have informed my thinking about this project, and which may influence the project, include the following.

Project intent or outcome

Biggs (2001) presents an argument for a reflective institution that focuses on prospective, rather than retrospective quality. Having just re-read the paper, it seems to provide a good fit for a theory/model for the overall intent of the project.

The model is based on the idea of three aspects of AQ:

  • Quality Model.
    i.e. an espoused theory of teaching, for Biggs this is constructive alignment. For the alignment project this might be alignment a little more broadly.
  • Quality enhancement.
    A “teaching delivery system” that is designed in accordance to the quality model, i.e. one which encourages and enables alignments. In addition, the teaching delivery system should also have built-in mechanisms to continually review and improve current practice.
  • Quality feasibility.
    A process by which impediments to quality teaching are removed from the “teaching delivery system”.

The “quality model” is underpinned by a large, overlapping and diverse collection of literature from various areas including: outcomes-based quality/evaluation, instructional alignment (Cohen, 1987); curriculum alignment; graduate attributes; and, of course constructive alignment (Biggs and Tang, 2007; Biggs, 1999 ).

Understandings of universities and organisations

My personal conceptions of most organisations, but especially those like universities, are informed by Dave Snowden’s Cynefin framework and complex adaptive system.

Understandings of leadership

This is obviously an area which needs more consideration, beyond the summary given above.

Based on my limited reading, I like the description of “new leadership” (related to distributed leadership) attributed to Fullan (2008), which include:

  • respect of employees, rather than simplistic judgmentalism;
    A specific focus of this project is to help teaching staff consider the alignment of their courses. It is not to judge them. Biggs (2001) makes the point that under the type of reflective institution he outlines, the focus is on the teaching, not the teacher.
  • connecting peers with purpose and ownership;
    By embedding indications of alignment into the LMS the aim is to create connections between other teaching staff, academic leaders and also teaching support staff. The clear purpose is around considering alignment.
  • building employees’ and systems capacity; and
    This is the specific aim of the project. Building into the institution the capacity in both the employees and its systems to engage regularly in consideration of alignment.
  • transparency of practice and results.
    At the very least, the aim of this project is to enable teaching staff within the same program (i.e. group of courses/degree etc.) to see each others practice. To show what is aligned, where (or where not). It opens up the teaching practice to colleagues, hopefully in a way that is not judgmental.

Process

Based on some of the above and below, I’m leaning very much towards an action research process. The practice of L&T within a university is a complex-adaptive system. As we introduce change into the system, the system will change around us and unexpected event will happen. The type of change suggested involves a fairly widespread change in the practice of teaching academics, but also the institution. In addition, as far as I’m aware, no-one else has tried and reported on this type of change, hence it is novel. While driving towards a particular goal, we have to aim on learning as much as we can during the process.

In addition, Biggs (2001) offers the following

action research, a methodology designed precisely to generate and evaluate in-context innovations (Elliott 1991). As a result of engaging in action research, teachers change their conceptions of teaching, and teach more effectively (Kember 2000).

This type of approach also fits very closely with what is known about staff development.
i.e. the current recommendations are that staff development should be as contextualised as possible. My conception of how teaching staff would be helped to consider the alignment of their courses contains a very heavy assumption on contextualised staff development. In fact, the presence of alignment information and the transparency between courses is aimed at helping this support and development to be increasingly contextualised to the every day teaching practice of the teaching staff.

Teaching practice

One of the fundamental models/theories underpinning this project for me is the observations embodied in the following.

How academics design their teaching is not described by a rational planning model (Lattuca and Stark 2009). In part, this is because the dominant setting for academics is teaching an existing course, generally one the academic has taught previously. In such a setting, academics spend most of their time fine tuning a course or making minor modifications to material or content (Stark 2000). Academics are usually not often required to engage in the development of new courses or major overhauls of existing courses (Stark and Lowther 1988).

The practice of most academics does not separate planning from implementation, and rather than starting with explicit course objectives, starts with content (Lattuca and Stark 2009). In the absence of formally documented teaching goals, the actual teaching and learning that occurs is more in line with the teacher’s implicit internalised knowledge, than that described in published course descriptions (Levander and Mikkola 2009). Formal descriptions of the curriculum do not necessarily provide much understanding about how teachers put their curriculum ideas into action (Argyris and Schon 1974)

References

Argyris, C., & Schon, D. (1974). Theory in practice: Increasing professional effectiveness. Oxford, England: Jossey-Bass.

Biggs, J. (1999). Teaching for quality learning at university. Buckingham: Open University Press.

Biggs, J. (2001). The Reflective Institution: Assuring and Enhancing the Quality of Teaching and Learning. Higher Education, 41(3), 221-238.

Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16(8), 16-20.

Fullan, M. (2008). The Six secrets of change. San Francisco: Jossey-Bass.

Jameson, J. (2006). Leadership in post-compulsory education: Inspiring leaders of the future. London: David Fulton Publishers.

Kember, D. (2000). Action Learning and Action Research: Improving the Quality of Teaching and Learning. London: Kogan Page.

Lattuca, L., & Stark, J. (2009). Shaping the college curriculum: Academic plans in context. San Francisco: John Wiley & Sons.

Levander, L., & Mikkola, M. (2009). Core curriculum analysis: A tool for educational design. The Journal of Agricultural Education and Extension, 15(3), 275-286.

Marshall, S. (2006). Issues in the development of leadership for learning and teaching in higher education (Occasional paper). Sydney: Carrick Institute of Learning and Teaching in Higher Education.

Stark, J. (2000). Planning introductory college courses: Content, context and form. Instructional Science, 28(5), 413-438.

Stark, J., & Lowther, M. (1988). Strengthening the Ties That Bind: Integrating Undergraduate Liberal and Professional Study. Ann Arbor, MI: Professional Preparation Project.

Southwell, D., & Morgan, W. (2009). Leadership and the impact of academic staff development and leadership development on student learning outcomes in higher education: A review of the literature. Sydney: Australian Learning and Teaching Council.

The realities of the ERA and L&T support services

In mid-October last year I blogged about my search for a research publication outlet. The conclusion was that in my context, the Australasian Journal of Education Technology (AJET) was probably the best fit. It is an open journal and the first round of the Australian government’s Excellence in Research for Australia (ERA) initiative had ranked AJET as an A journal, second only to A*.

More recently that blog post got a reference in an AJET editorial (that the editors are referencing a blog post in an editorial is a good sign for AJET, especially if it is one of mine). The editors hope was

that his recommendation, written in 2009 when AJET was a Tier A journal, will not change as a result of AJET’s demotion to Tier B in the 2010 list

I’m sad to say, that it probably will!

Why?

There are two reasons why:

  1. the on-going uncertainty around L&T support services within Australian higher education; and
  2. the increasing force of having to comply with government indicators.

On-going uncertainty

Early in 2007 I moved from being a faculty-based academic in the information systems discipline to a role within a central L&T support division. Within about 6 months of taking on the role, the then Director of that L&T division was told her services were no longer needed. Apparently, they were after someone with a greater research focus, rather than on getting things done for the institution. The division limped on for around another 18 months while they failed in appointing such a person. During this time, rumour was rife that we would be restructured, though officially this was not confirmed. By the end of 2008 the restructure was complete, I was in position limbo for 6 months (theoretically better than having been made redundant, but I’m still trying to decide if that was the case) and the institution tried again (again unsuccessfully) to appoint a Professor to lead the unit.

So, as you might deduce, there has been little certainty for the folk in L&T support services. Oh, by the way, the institution has finally admitted that the last restructure was somewhat less than successful in its outcomes (of which they were informed many times back in 2008) and they recognise they need to address this problem. By another restructure. So, again, there isn’t a lot of certainty.

This is not something unique to my current institution. A regular topic of conversation at the last ASCILITE conference was around the restructuring of L&T services. It seemed that every second person from an L&T support area was going through, just been through or predicting they would go through a restructure. Just recently I heard that the Director of another L&T support area is being replaced with someone who has a more research focus. Deja vu all over again.

In the past, I have argued that this is, at least partly – if not mostly, to do with the absence of any broadly agreed indicators for the quality or impact of L&T support. Combine this with the fact (or at least strong likelihood) that every senior manager who has taught a course thinks he/she has a unique insight into how to support L&T at a university and the sheer complexity of such a task, and you have a recipe of on-going change, little or no success and little certainty for the folk within such centers.

Requirement to comply

The ERA itself is just one example of the Australian government specifying indicators of success. Now, don’t get me wrong, there is an argument to be made the universities should be accountable. However, as I’ve argued before such indicators are generally very poor and at best result in compliance and at worst task corruption.

For example, I’ve seen in the last few weeks the following:

  • A faculty dean’s report to academic board spend a lot of time explaining how many A* journal publications the faculty had produced.
  • New guidelines for course (my institution uses american terms, so a course is the smallest unit of education offered by the institution, a program is a collection of courses) coordinators that specify having a graduate certificate of higher education – or the intent to get one – a desirable criteria.
  • A job in a central L&T support area that listed having a graduate certificate in higher education as the first necessary criteria.

To me these are indications that the compliance has begun. Senior management at universities have decided, regardless of other reasoning, to adopt the government indicators as prime decision criteria. This is bad because a heavy focus on these types of indicators gives a false sense of security that we’re doing the right thing. It reinforces simplistic beliefs. e.g. that if teachers with a formal teaching qualification will be better teachers. A belief that ignores a whole bunch of literature around the impact of context – i.e. if the context of a university does not value L&T, it will not be good.

It will encourage a generation of senior leaders at universities that focus on these indicators rather than a broader and deeper understanding of what it takes to encourage and enable effective teaching and important research. This is because most of the current senior leadership roles at Australian universities are short-term appointments – 3 to 5 years. You can’t effectively and broadly improve the teaching and research at a university in 3 to 5 years, but you can sure as hell pragmatically hit some simplistic indicators.

The pragmatic defense

Worse still, this combination of factors encourages academics in L&T support areas to be seen to comply with the indicators. If you’re uncertain about the value senior leaders place on the contribution of L&T support areas, the best way to ensure a place at the table is to have lots of A* journal publications, important grants etc. You need to have a place at the table because you can’t effectively battle against the short-term focus on broken indicators when you’re on the outer. Even if you don’t get to the table, given the uncertainty and a likely need to retain some sort of salary, you need to have fulfilled these indicators so you can get a job at another institution.

What’s even sadder is that you probably can’t effectively battle against the short-term focus on broken indicators even when you’re sitting at the table. Sadder again is the observation is that your focus on these indicators may end up changing you.

My potential publication outlets

Sadly, at least in the short-term, AJET has slipped down the rankings. It probably won’t be my first target for publication in the next couple of years. I have to hit the A*/A journals. Hopefully, I can find one of those that is somewhat open – I believe ALT-J has some sort of set up that is somewhat open.

Inside out, Outside in or both?

During the last week I have been in Canberra for various events, including giving a presentation on BIM at University of Canberra. Somewhat surprisingly (as last I knew, he was in New Zealand), Leigh Blackall was in “audience” at the presentation, and as is Leigh’s wont, he asked some serious questions. I was troubled by those questions and needed time to reflect on what an answer might be.

This is an attempt to develop an answer to why I was troubled. In part, this attempts to pick up a comment I’d made earlier on Leigh’s blog about thesis, anti-thesis and synthesis

The question

The initial question Leigh asked which troubled me is repeated in his blog post on the presentation. It is

I asked the obvious question of why, or if BIM might consider developing outside the framework of Moodle say, as a Firefox based or other Feed Reader plug in, and offering a file that can be imported to Moodle (as well as a spreadsheet, a MediaWiki table, a Wikispaces Table, MySQL database, a text document and a PDF to email), and thereby offering the functionality of BIM to a wider user base than just Moodle.

The immediate answer I gave is reported faithfully in Leigh’s post

David explained that the project was constrained in many ways to the needs of the sponsoring Institution,

(as is the fact that we had broader discussions.

What troubled me (in part) about my response is that the constraints of the sponsoring institution is both more and less than it sounds. In part it is what is simply easiest, the institution pays my wage and it uses Moodle. But it is also, to me, what is the best way in terms of improving L&T.

It appears to be the difference between an “inside out” approach (which I’m taking and will argue has a chance of success) and an “outside in” approach (which is somewhat close to what Leigh suggests). At the very least, it makes for a good title.

Inside out

The main reason I didn’t start with Firefox or some other external way of developing BIM, is that I’m taking an inside out approach to improving learning and teaching. i.e. I’m starting with what is being used within the organisation and trying to change it for the better. The organisation is currently using Moodle, so if I want more people to be thinking about using Web 2.0 and reflective student journals in their L&T, I have to start with Moodle. Doing so lowers the barriers to entry and actually makes it somewhat likely that people will use it. It even fits (with some difficulty) within the constraints of how Moodle is being managed within the institution.

In commenting on Leigh’s post, Peter gets close to the approach

The central question though for an educational developer is how to promote innovation and change and it seems to me that it has to be based on an invitational ethos: teaching staff need be convinced of the benefits of technology adoption, they sometimes come to it slowly, they come to it in surprisingly unexpected ways at times and positive things happen.

I actually think more than this is needed and will pick up on it below. But first..

Outside in

To a limited extent, Leigh’s approach could be characterised as outside in. Start with the outside stuff, support those people and then perhaps change might eventually occur within the institution. However, in responding to Peter’s comment Leigh suggests a perspective that doesn’t really both with the “inside” (current universities)

To borrow your highway metaphor, a bypass is needed, one that goes around that old town, and offers a more direct route for the people in need of credentials with minimal debt. Remember, the experiences in the old town have become irrelevant. An old road can remain for those who like nostalgic tourist routes, but an alternative route is needed.

There are many within current institutions that react negatively to this perspective, however, I can see the need for it. Mainly because, as Leigh points out, there are significant barriers within universities that suggest that an inside out approach may not work.

I don’t believe a passive approach can be effective when considering our dense hierarchies, performance reviews, infrastructure, broken feedback systems, conservatism and the wrong sorts of incentives and rewards.

Limits in developing innovative pedagogy with Moodle

In fact, I’m hoping to use the development of BIM and the limitations of this approach as the basis for a couple of publications, including a presentation at MoodleMoot AU 2010. (still waiting to hear if the submission has been successful). The “theme” for this conference is “without limits….”. Rather than accepting that Moodle is “without limits”, I argue that

e-learning with Moodle, as currently practised, has a number of limits and that progress can be made through the recognition, understanding and removal of those limits.

So, what may you ask, is my problem with Leigh’s comments?

Both-and, not either-or

The questions around the quality of L&T within universities, the requirement from society for different approaches and the future of universities are complex. So complex, that it’s never going to be about a single answer, there is no such thing. For this, and other personal reasons, I prefer a both-and approach. It’s not about outside-in or inside-out, it’s about both.

Actually, I should paraphrase, in terms of improving L&T within universities, it’s a question of both-and. In terms of responding to societies changing needs around learning (or simply recognising a long-standing need that has been ignored), I’m not so sure there is a need for an “inside-out” perspective. However, as I’m paid by a university to improve the quality of L&T, I see the need for a both-and approach for the long-term benefit of the university (and hopefully society…there’s a big question in there).

What this means is that there is a huge need for folk like Leigh and many others (Leigh gives a list of some in this post) who are identifying and creating insights into what the “outside” should be. The value is not just for the “outside” it’s also for the universities and other institutions as it helps identify some options around where we need to be.

However, there is also a need – at least at the moment – for folk who are taking the “inside-out” approach. Thinking about how to effectively make the “inside” a better fit or enabler for what the outside should be like. That need may not exist in the future, but for the moment it does and because this is a complex area, I think we need both.

Inside-out is currently failing

As stated above and numerous times on this blog, I think the current approaches being used within universities are failing. Most L&T at universities is poor quality by traditional standards, let alone if measured by adoption of social media. For me, this is not a sign that it is impossible, it’s a sign that the principles of current approaches are just plain wrong. This is what my current work is looking at.

Why is it failing?

The following diagram represents what I think is a fundamental mismatch (The image is taken from this post by Donald Clark).

im

As I said above, this problem is a complex one. Based on the above diagram, the best type of solution arises from immersion in the problem. The problem is that most universities are attempting to solve this problem by analysis.

For along time I’ve been saying that learning and teaching is a wicked problem (Rittel & Webber, 1973). Clark’s blog post (source of the image) draws on another of Rittel’s publications to suggest that for a complex problem

This is because as Rittel (1972) discovered — the best experts within these types of environments are those affected by the solution — since they are the only ones to have experienced the complexity of the problem, they are the best experts for helping to improve that environment.

At the moment, universities are only paying lip service to involving the “experts” – the students and teachers. Most of it is being driven by “management” who don’t have in-depth experience of a specific context. Their decisions are driven more by other considerations than in-depth understandings of the current context. Importantly, this isn’t about asking the students and teachers what they want, for me, it’s about understanding what they are experiencing and where they are now as an important first step in helping them go somewhere else.

At the same time, the approach taken by the “outside in” folk – like Leigh – also has the same failure. It doesn’t seek to understand the existing context or practice of the students and staff. But that’s okay, that’s not what they are about, they are about figuring out and creating a better future. (I fully recognise that this is a gross simplification and generalisation. However, I do think it’s a distinction that has some value.)

What is both-and?

As an inside-out person, I believe any success comes from having deep knowledge of the current context (the experience of staff and students) and marrying that with “solution” knowledge from the experts and insights into how the environment can be changed in a way that encourages and enables the staff and students to improve their experience. This describes BIM:

  • knowledge of current context;
    The need for BIM arose out of my need to teach a 200+ student course that had a “reflective” journal assignment which had significant problems (a problem faced by other staff).
  • expert knowledge; and
    I knew about Web 2.0, the benefits of student-owned journals/blogs and the institutional need for and staff/teacher familiarity with the LMS.
  • environment change.
    Add a module to Moodle that enables staff to manage individual student blogs hosted on external services.

But it’s not enough

This is what really troubled me, and now I’m becoming repetitive in the same post. The above by itself is not enough. As a measly e-learning and innovation specialist I have no power to make the further changes in the environment that are necessary to make BIM truly attractive.

I guess the real reason why Leigh’s questions troubled me, is that I’m frustrated at my inability to make the change and the on-going blindness of institutional leadership.

How’s that for a positive end to a rambling post?

A response to Leigh

Couldn’t leave it there, let me return to the original question from Leigh. I believe that an inside-out approach is probably more likely to help improve L&T on a broad-scale within a university than an outside-in approach. I value the insights offered by outside-in folk, but I think I need to value the experiences of the students/staff within a university and build on that experience to help improvements happen. Perhaps, it’s simply a question of purpose. My purpose is to help improve L&T within universities, yours is more about helping those people who are already learning outside of universities.

That’s why I think building BIM on top of Moodle was a better fit for my purpose. A purpose, which I agree, is fairly narrow.

References

Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155-169.

Clarification of the alignment project

Am at the University of Canberra for a second day today talking about the possibility of an ALTC grant application around alignment/curriculum mapping. The aim today is to try and clarify where the project would go and this post is an attempt to make concrete some of my thinking. The final ideas for the project may be radically different.

I’m going to use the “project clarification” section of the ALTC’s Steps in developing an evaluation plan for an ALTC grant project as the structure for this post. Mainly because of a belief that getting into the “ALTC mindset” will increase the likelihood of success.

This remains very much a work in progress. Comments and thoughts all welcome.

What is the focus of the project?

The fundamental issue the project is aiming to address is in increasing the visibility of curricula and instructional alignment in the practice of university teaching.

The observation is that consideration of alignment is not an important consideration in what most academics do around learning and teaching. Consideration of alignment is not something that students see regular reminders of in their learning. It is done and visible in places, but those times and places are not a regular, visible part of what they do.

Since it is not visible in regular practice, it is often not thought of. It doesn’t inform what they do. This is bad because there’s vast bodies of research that suggests alignment between outcomes/graduate attributes, assessment, learning activities and instructional resources is a good thing. There is evidence to suggest that many university courses do not have strong alignment. There is also evidence (McDonald, 2008) that when considerations of alignment become part of what academics regularly do as part of teaching, it encourages them to begin questioning existing practice.

The focus of the project is on how to make considerations and discussions about the level of alignment in university courses a regular and on-going part of what academics do as part of their normal teaching practices.

The aim is to make changes within the teaching context that enable and encourage these considerations and discussion of alignment to take part regularly.

What is the scope of the project?

While the project will (hopefully) be funded by the ALTC over two years, the intent is that the aim of the project is an attempt to embed these practices within the institutions. i.e. to be successful, it should be an on-going project.

Initially, the ALTC funded project will be limited to University of Canberra and CQUniversity. Participants are likely to fall into these categories:

  • Round 1 teaching staff;
    A smallish group of academic staff who will be involved in the first cycle of making alignment a regular part of their practice. The nature of this group will depend on each institution.
  • Round 2 teaching staff;
    Includes the round 1 teaching staff plus additional staff that will work with modified practices from round 1. The intent is that round 1 teaching staff will be using these practice for the second time, perhaps with the same course. The round 2 only staff will likely be chosen on the basis of having some differences (on a broad array of possible variables) with the round 1 staff.
  • learning and teaching support staff;
    These are staff tasked with helping academics improve their L&T. This might include instructional designers, staff developers etc. These staff will be working with the teaching staff during both stages to help develop appropriate approaches to integrate alignment considerations into everyday practice.
  • project leaders; and
    The staff named on the ALTC grant application will be involved in various tasks in ensuring the project is moving along and may also fulfill some of the previous roles.
  • reference group.
    A body of experts that will offer oversight and provide feedback on the projects aims, outcomes etc.

Over the 2 years of ALTC funding the project would probably aim to, at least at one of the institutions, integrated alignment into the practice of at least one program in order to test the impact at the program level. Perhaps a target of 30 or courses at each institution?

The particular approach being taken with this project is to use Moodle as the tool that enables and encourages an increased consideration of alignment.

What are the intended outcomes?

  • Indications of the impact of significant consideration of alignment on the thinking/practice of teaching staff and on the student experience.
  • Guidelines, identified challenges and suggested processes (with accompanying documentation and resources) for increasing the consideration of alignment into everyday practice.
  • Guidelines, tools and processes describing how the practice of quality assurance can change once alignment considerations are embedded into every day practice.
  • A collection of Moodle enhancements.
    These are important only in terms of the affordances they enable which can then be harnessed to improve considerations of alignment. The enhancements are likely to fall into two categories:
    1. Addition of “mapping” into Moodle.
      The ability for Moodle to enable mapping of all activities and resources against specified outcomes, attributes etc.
    2. Moodle features that use the mapping information.
      This is the really interesting part. What applications make embedding alignment into every day practice useful and interesting for the broadest array of stakeholders.

In the elevator pitch for this project I identified some of the following as propositions:

  • Curriculum maps that are generated, and continue to evolve, in the same time and space as everyday teaching and learning will have stronger validity in terms of capturing reality and consequently be of more value.
  • A learning environment that makes visible to all stakeholders the alignment (or not) of a course and then provides scaffolding necessary to improve that alignment will help improve teaching.
  • Well designed extensions to an LMS that encourage and enable improvement of course alignment will increase the quantity and quality of usage of the institutional LMS and subsequent student outcomes.
  • Building curriculum mapping around Moodle’s student “tracking” functionality will enable and encourage greater use of the student tracking functionality.
  • Properly implemented, this approach can make it easier for curriculum designers to embed assistance into the context within which teaching is taking place. If this works well, relationships will develop.

Each of these could be thought of as potential outcomes. Yes, they should be developed some more.

What are the operational processes developed to achieve the outcomes?

The process to be used in this project should lean heavily towards being adopter-focused (Surry & Farquhar, 1997). The following description is taken from Jones and Lynch (1999)

In contrast to the developer-based approach, the adopter-based approach focuses on the human, social, and interpersonal aspects of innovation diffusion (Surry & Farquhar, 1997). Developers are interested in the individual who will ultimately implement the innovation in a practical setting as the primary force for change. The adopter-based theories reject the assumption that superior products will automatically be attractive to potential adopters. They ’seek to understand the social context in which the innovation will be used and the social function the innovation will serve’ (Surry & Farquhar, 1997).

Stages in the process for this project might include:

  1. Adding mapping to Moodle.
    The focus here is on obtaining a version of Moodle that allows the mapping of every activity, resource and assignment within Moodle to a set of outcomes. i.e. the aim is to get the data stored. This is almost a purely technical process that requires little involvement from academics.
  2. Planning and first roundtable. ????
    The project groups at each of the two institutions formulate the plans/approaches they plan to use for the first round. This identifies: the participants, programs/courses, current state of mapping, approaches to be used to complete the mapping…..Have a roundtable where the reference group are shown these plans and offer advice and suggestions.
  3. Round 1
  4. Reflection/roundtable.
  5. Round 2
  6. What is the conceptual and theoretical framework underpinning the project?

    • Alignment.
    • Adopter-focused development … more buzz words.
    • Teacher conception and behaviour is a key component of quality of L&T.

    What is the context of the project?

    • 2 institutions that have adopted Moodle as enterprise LMS.
    • Higher ed sector where alignment is important in terms of graduate attributes etc. and also more broadly quality assurance and quality enhancement are increasingly more important.
    • Institutional change.

    What key values drive the project?

    • Adopter-focused.
    • Open source/open.

    References

    Surry, D., & Farquhar, J. (1997). Diffusion Theory and Instructional Technology. e-Journal of Instructional Science and Technology, 2(1), 269-278.

Framing a body of research and innovation

Markus has finally posted the set of questions for framing a research proposal idea that he showed me weeks ago. This post is an attempt to use those questions to frame what I’m doing in my current position.

A simple test for BIM purposes, ignore

Write a research question

How can you improve the quality of learning and teaching within a university?

What is the important theoretical or conceptual setting?

Too much work in this area has not focused on the academic staff and factors or approaches that would encourage and enable them to change their teaching behaviour. The behaviour of the teacher is a key component of the quality of learning and teaching. Teaching and learning is getting more difficult.

Write 2-3 key points about why it is important (Significance? Innovation?)

  • Governments and other external stakeholders are increasingly demanding proof that university learning and teaching is good and getting better.
  • Most university learning and teaching is of less than good quality.
  • Most, if not all, interventions to improve L&T are not succeeding in widespread, long-term behaviour change (i.e. improving quality).

Write 1-2 brief objective points (formulate full aims later)

  • Evaluation of existing interventions, informed by behaviour change research, will reveal significant short-comings.
  • Interventions, informed by behaviour change research, will result in significantly greater levels of improvement of L&T.

Write a couple of points about the approach you might take

This is not necessarily just about a single research project and it is unlikely to have, at least initially, significant institutional support or recognition. A collaborative approach involving many different projects and people will be done. However, it seems likely that an on-going process of:

  1. Learning more about how behaviour change research may inform this project.
  2. Combining these insights with other knowledge to design interventions.
  3. Use the insights to evaluate both existing and new interventions. Also to further understanding of the current context.
  4. Publish.

Will I have all the expertise/resources? Should I be talking to collaborators? Might the proposal be strengthened by having a team?

A team is absolutely necessary. Knowledge of behaviour change research is essential. As is good knowledge of the local context and educational and technical skills to implement the interventions.

Do I or will I need some ‘proof of concept’, preliminary data or demonstration of competency to undertake the project?

Yes, the on-going process above will have to start small and learn from there.

Will the team/partners be competitive in this field?

At the moment, yes. There doesn’t appear to be too many people taking this approach.

What would the outcome(s) be and who benefits?

Hopefully, there will be improvements in L&T in the local context and research publications and grants around that work.

What is the ‘WOW’, ‘HOOK’, or ‘EXCITEMENT’ factor?

It appears that this is a new way of approaching an intractable problem.

Further thinking – behaviour change and improving L&T

This post is an attempt to synthesize and reflect more upon two posts from yesterday. One from me trying to explain an early form of a framework for improving L&T based on findings from psychology behavior change research. The second from Markus that expands on the potential psychological foundations for this thinking by mentioning:

  • The recommendations section of a UK National Health Service (NHS) document outlining some generic principles for initiatives to support attitude and behaviour change.
  • The concept of salutogensis, at least as a cliff hanger for a subsequent post.

The following is my attempt to summarise and make some initial connections between this work and ideas around improving learning and teaching.

What behaviour change programs need to do

The NHS document is said to highlight the need for “interventions and programmes aimed at changing behaviour” to:

  • Be based on a sound knowledge of community needs and should build upon the existing skills and resources within a community.
    To me, for improving and learning and teaching this speaks of a need for empathy-driven innovation and for a focus on what the teacher does.
  • Equip practitioners with the necessary competencies and skills to support behaviour change, using evidence-based tools.
    My understanding is that “practitioners” refer to the people helping the change, not the actual teachers. Interesting.
  • Evaluate all interventions.
    Something not really being done around L&T within universities.

These are derived somewhat from the principles for interventions presented. It is again the importance placed on understanding the community as the foundation for change that resonates for me. You have to start with knowledge of the community and their strengths, perceptions etc.

Salutogensis

The following is based on a quick reading of this paper.

Focus on origins of health, rather than causes of disease

Interesting thought, salutogensis arose from an attempt to understand why women who survived concentration camps in the Second World War, had the capability for good health.

Is there a connection hear with L&T?

I would perhaps characterise the current focus in Australian universities on changes driven by the need to respond to government as attempts to prevent disease, rather than encourage health. …tentative and tenuous.

The human is at the heart

Discussions this morning lead to some reflection on metaphor – an area in which my wife has done work. The idea is that the metaphors we use to describe complex concepts reveal how we think about those concepts, it informs our decisions.

Increasingly, some colleagues and I are seeing evidence of senior management holding a machine metaphor when it comes to learning and teaching. The academic staff involved in learning and teaching are seen as machines, as inter-changeable machines. You can take that machine from there, and place it over there and it will do the same job. It doesn’t matter which machine, they all do the same thing.

It is this metaphor which enables the adoption of techo-rational approaches to improving learning and teaching such as the idea that every academic should have a graduate certificate. The metaphor here is that if you add a new widget to the machine it will perform the task with greater efficiency or with greater effectiveness.

The trouble is that people aren’t machines. They don’t passively accept the addition of a widget. There’s no direct causal link between addition of a widget and changes in behaviour.

Lundstrom and Eriksson (2006) see health promotion as

three phases, the first one that recognizes the background (the determinants), the second one that sets an objective (to lead an active productive life), and the third one is the activity (the enabling process) where the determinants are used to reach the objective in a dialectic relationship between people, the setting and the enablers.

As a “dialectic relationship” it is an on-going dialog, the person doesn’t take it unquestioningly. They go on to say

At the heart there is the human being seen as an active participating subject, respected in her full human rights.

Something that is increasingly missing in L&T within universities.

The salutogenesis framework

A framework might be possible by drawing on the salutogenic focus on three aspects

  1. Focus on problem solving/finding solutions.
  2. Identifying GRRs that help people move in the right direction.
    GRR = General Resistance Resources – biological, material and pyschosocial factors that help people see their lives as consistent, structured and understandable. A person with these resources has a better chance of dealing with challenges.
  3. Identify the SOC – the pervasive sense that enables this move.
    The SOC is the ability to use the GRRs. The GRRs create life experiences that promote a strong SOC. (This is somewhat confusing but the message I’m getting from the paper.) The SOC is the capabaility to perceived you can manage in any situation. It is flexible and not focused on a specific set of strategies.

Will have to wait for Markus’ explanation to make this understandable to me.

There are, however, some positive resonances with this ideas and my current thoughts, will have to see how that plays out.

Identifying and designing interventions to improve L&T – a behaviour change framework?

The following is an attempt to describe early attempts at developing a framework for evaluating existing and developing new interventions aimed at improving learning and teaching within universities. It builds on some ideas from an earlier post.

Sorry, but the first couple of sections seem necessary to convince myself that this is connected with some of my earlier thinking.

Rationale and assumptions

As mentioned many times on this blog, I think most attempts to improve learning and teaching at Universities really don’t work as they don’t change what the majority of academics do. At best they get a bit of compliance. The aim here is to come up with a framework/guidelines that help evaluate and design such interventions.

Empathy driven innovation

While the following uses a framework from psychology, the basic message is the same as a range of other work. That message is, focus on the academics, understand what they are currently doing and experiencing, and design interventions that connect with that experience. Most recently this was discussed in a a post on empathy-driven innovation.

Much of my thinking around improving learning and teaching is based on this sort of approach, the following tries and provide some additional guidance into how to do this. But underpinning it all, is a focus on the actual teaching experience of the academic staff (and the learning experience of the students) as the primary focus.

The aim is to understand what they are experiencing and develop ideas about how that experience can be improved, as perceived by the staff and students and not as perceived by management, the government or some research ideal. The focus is on what the teacher does, not what management does.

Improving L&T is an attempt to change behaviour

An assumption underpinning this thinking is that when you are trying to improve learning and teaching, what you are actually trying to do is to change the behaviour of the teaching academics. If you don’t change the behaviour of the academics, then there is no way you can expect the quality of learning and teaching to change.

This is perhaps the main problem that I have with existing methods being employed to improve learning and teaching at Universities. Existing methods don’t actually consistently result in behaviour change amongst a significant percentage of university teaching staff. I’ve argued before that they don’t work and that there is evidence to support this.

The question is, what’s the solution?

Conceptions and teaching

An example of this mismatch can be seen with the on-going focus on teacher qualifications. For example, the Australian government has introduced KPIs for university learning and teaching that include the use of the % of university teaching staff with teaching qualifications as a KPI. I’ve argued this this attempt will encourage compliance behaviour on the part of institutions and academics.

The institutions are already at it. My current institution is currently introducing a new “desirable criteria” for people in charge of courses – having a formal teaching qualification. The assumption is that this will increase the quality of teaching. It won’t.

At best, a formal teaching qualification will change the conceptions of teaching held by an academic. The “teacher’s thinking” layer in the following diagram. There is significant research (e.g. Trigwell, 2001; Richardson, 2005) that suggests that unless all of the layers in the diagram are in alignment, you won’t get improvement in teaching.

Trigwell's model of teaching

That is, even with a formal teaching qualification, if the context, disciplinary expectations and many other factors aren’t right, then there will be no behaviour change. The teacher will continue to employ the same strategies as always.

How to encourage and enable behaviour change

How to change behaviour is something that the psychologists study. This current line of work is a collaboration between myself and a psychologist colleague. The current aim is to try and distill what is known about behaviour change in psychology into a form that helps explain how someone – with a deep knowledge of the current experience of academics – can encourage and enable behaviour change that results in improved learning and teaching.

In earlier work I’ve drawn on insights from Roger’s (1995) diffusion theory as a guide. There are definite connections between diffusion theory and the following.

The initial spark for this thinking is a paper by Michie et al (2008) which suggests that:

  • There are determinants of behaviour change, (see the table below) someone is unlikely to change behaviour when these determinants have the wrong value.
  • There are known techniques for encouraging behaviour change.
  • These techniques are each seen as likely to influence a certain subset of the determinants.
Key Determinants of Behaviour Change from Fishbein et al., 2001; Michie et al., 2008
Fishbein et al Michie et al
Self-standards Social/professional role and identity
Knowledge
Skills Skills
Self-efficacy Beliefs about capabilities
Anticipated outcomes/attitude Beliefs about consequences
Intention Motivation and goals
Memory, attention and decision processes
Environmental constraints Environmental context and resources
Norms Social influences
Emotion
Action planning

From a L&T perspective, the application of this knowledge seems to be something like:

  • Identify something that is known to be good practice, that improves L&T, but which isn’t being done in the current context.
  • Use the determinants to identify areas of weakness, areas for improvement, around the current context. Especially in terms of the “good” practice.
  • Identify generic behaviour change techniques that address those areas of weakness.
  • Draw on empathy driven innovation to develop specific interventions, which are based on the information gathered in the previous two steps.

An example

In a previous post that seeks to summarise the Michie et al (2008) paper, I use the following example of the level of interaction or participation of staff in a course site. In the following, I compare/contrast the more traditional approach and the type of approach that I think might work better.

The traditional approach

The more traditional approach is what I suggests is “what management does”. Management will see that there is a problem, i.e. staff aren’t participating enough. They then take it upon themselves to perform some action, which might include:

  • Require staff to get a teaching qualification.
    The assumption being that with a teaching qualification they know more about the importance of staff interaction and hence will increase their interaction.
  • Create a policy that requires a certain level of participation.
    At the very least they might say that every course site must have a discussion forum. Over time they might add requirements like: every student query must be answered within two days, or every staff member must make 5 posts a week to the course discussion forum.
  • Run a staff development session.
    Organise an external expert on the benefits of staff participation to run a session at the institution and invite teaching staff to come along to listen.
  • Reward staff who interact at appropriate levels.
    Give them extra money or recognition as good teachers. etc.

Most of the above interventions would require a fair bit of work. Most, if not all, of the senior management at the institution would need to be involved, a working party might need to be formed, appropriate consultation, and if they’re really smart they’ll make sure that someone important within the institution is publicly seen encouraging this goal.

I’ve seen the above approach used again and again. I don’t think I’ve ever seen it work to any great extent.

The behavioural change/empathy-driven approach

Using the process above:

  • Identify the good practice.
    Done, we wish to increase the level of staff participation in a course site/forum.
  • Use the determinants to identify areas of weakness in the current context.
    As an example, let’s assume that after an appropriate process we have identified the following:
    • Skills: many staff don’t have the skills or insight necessary to increase their interaction with their course in effective ways.
    • Environmental constraints: there’s nothing in the existing context that helps them perform this task. In fact, the rewards system tends to suggest that research is more important than teaching.
    • Norms: there is little visible evidence of what the norms are in terms of appropriate levels of staff participation. Staff don’t know how much other staff are participating, they don’t know what is acceptable and what is not.
    • Anticipated outcomes: many staff don’t see the connection between increasing their participation in a course site and benefits to students. They do see how increasing their level of participation increases the time they spend on teaching and decreases the time they spend on research.
  • Identify generic behaviour change techniques that target these determinants.
    Techniques that target skills include: monitoring; self-monitoring; graded task, starting with easy tasks; modelling/demonstration of behaviour by others; goal/target specified: behaviour or outcome. Norms can be addressed by: modelling/demonstration of behaviour by others; social processes of encouragement, pressure, support; and prompts, triggers, cues. etc.
  • Use empathy-driven innovation to design specific interventions.
    Embed into the LMS a graph that shows the academic staff members level of participation in their course site (self-monitoring) and compare it against the levels of participation in a group of related courses (social processes of encouragement). Have that graph show up whenever a staff member logs into the LMS. Include in the graph some easily visible representation of the connection between student failure rate and staff participation. Include a link “how to increase participation” to a page that outlines various techniques for increasing participation (prompts, triggers, cues) and includes comments or suggestions from other staff about how they did it (modeling/demonstration of behaviour by others).

    Once this intervention is implemented, spend a lot of time observing what happens with the use of this intervention and make on-going change.s

Differences

Apart from the obvious, there is one major difference between these two approaches which I must make explicit. The traditional method spends a lot more time on the design of idealistic solutions that are disconnected from the everyday experience (e.g. the setting up and encouraging of people to enrol in a graduate certificate in higher education). Often, such approaches all but ignore the reality of the lived experience (e.g. policies that state a requirement but for which there’s been no work to support implementation). The traditional approach tends to be design heavy.

The empathy-driven approach, focuses on the lived experience, on what happens day-to-day. There is some design work, but much of the time is spent understanding what is going on everyday and trying to respond in an informed way.

The questions

It is still very early days for this work. There are lots of questions, here are just some.

How do you evaluate the level of determinants?

In the above example, I outlined some of my ideas of what values the behavioural change determinants around increasing staff participation on a course site might be. Basing the design on my thoughts is not a good start. It has to be based on a deep understanding of the experience/beliefs of the teaching staff.

How can you develop this understanding?

Are there survey instruments or other approaches from psychology that allow you to get some understanding of the current “level” of these determinants across an institution?

Which are the most important determinants?

Michie et al (2008) identify 7 sets of determinants, there are certain to be complexities within each one. In any large population you are likely to get widely differing results.

How do you figure out which of the determinants is most important or difficult?

Answers to this could help you guide the selection of which determinants to address and/or which of the techniques to adopt.

What’s the relationship between the determinants?

It’s likely that there would be relationships between the determinants. For example, if you increase skills in an appropriate way, I would imagine that this would change beliefs about self-efficacy, at least to some level.

What are the known relationships between the determinants?

What are the factors within determinants?

Fishbein et al (2001) talk about norms as being one of the determinants. Michie (et al) 2008 expand this out to include social influences, emotion and action planning. This is just one example, it is likely that each of the determinants embody a collection of factors. For example, self-efficacy might be a combination of characteristics of the person, the type of task and the previous experience of those involved.

What are the factors or sub-components of each of the determinants?

What types of behaviour are there?

Are all behaviours the same? Is a particular teaching behaviour the same as diet or how you control your children?

Are there different categorisations of behaviours with different characteristics?

What are the specifics of change techniques?

Michie et al (2008) just give a short title to these techniques. I imagine that each one is worthy of a literature that seeks to define, describe and evaluate interventions that have been designed on the basis of a specific technique.

What are the specifics of each change technique? What are the success factors? How do you choose which technique might be most appropriate?

References

Michie, S., Johnston, M., Francis, J., Hardeman, W., & Eccles, M. (2008). From theory to intervention: Mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology: An International Review, 57(4), 660-680.

Rogers, E. (1995). Diffusion of Innovations (4th ed.). New York: The Free Press.

Two types of process and what university e-learning continues to get wrong

I should be writing other things, but there’s a wave amongst some of the “innovation bloggers” at the moment that I wanted to ride for the purposes of – once again – trying to get those driving university e-learning (and learning and teaching more generally) to realise they have something fundamentally wrong. They are using the wrong type of process.

I level this criticism at most of management, most of the support staff (information technology, instructional designers, staff developers etc) and especially at most of the researchers around e-learning etc.

For those of you at CQU who still don’t get what Webfuse was about. It wasn’t primarily about the technology, it was about this type of process. The technology was only important in so far as it enabled the process to be more responsive.

Empathy-driven innovation and a pull strategy

Over the weekend, Tim Kastelle wrote a post yesterday in which he proposes that a pull strategy is a key for empathy-driven innovation.

What is empathy-driven innovation, well Tim provides the following bullet points about empathy-driven innovation:

  • It requires a deep understanding of what the people that will use your innovation need and want.
    Most organisational e-learning assumes that steering committees and reference groups are sufficient and appropriate for understanding what is needed. This is just plain silly. The people who reside on such things are generally very different in terms of experience and outlook than the majority of academics involved with learning and teaching. If they aren’t different at the start, the very act of being a member of such groups will, over time, make them very different. These groups are not representative.

    What’s worse, is the support structures, processes, and roles that are created to sit under these committees and implement e-learning are more likely to prevent “deep understanding”, than help it. For example, different aspects of e-learning are divided along the lines of institutional structures. Anything technology related is the responsibility of the information technology folk, anything pedagogical is the responsibility of the instructional design folk and never shall the twain meet. As these folk generally report to different managers within different organisational units, the rarely talk and share insights.

    E-learning is more than the sum of its parts. Currently, there is generally a large gulf between the academics and students “experiencing” e-learning, the technology people keeping the systems going, the instructional design folk helping academics design courses, and the management staff trying to keep the whole thing going. This gulf actively works against developing deep understanding and limits the quality of e-learning within universities.

  • Using empathy for the users of our innovations is the best way to create thick value.
    A deep contextualised understanding and appreciation for the context of the academic staff and students helps develop truly unique and high quality e-learning applications and environment. Without it you are left with copying what every one else does, which is typically pretty limited.
  • We are creating ideas that entice people.
    Almost all of university-based e-learning is based on push strategies. i.e. someone or group who is/are “smart” identify the right solution and then push it out onto the academics and students. They have to do this because their understanding of the context and need of the academics and students is small to non-existent. They decisions are based more on their own personal observations and preferences, or even worse, on the latest fad (e.g. e-portfolios, open source LMS etc.).

    They aren’t creating ideas that entice people, they are creating ideas that people have to use.

    Researchers are particular bad at this.

  • Innovations that pull are inherently network-based.
    The idea is that to engage in empathy-driven innovation, you have to have connections to the people using the innovations.

    As argued above, it’s my suggestion that the structures and processes around e-learning within universities are such that they actively work against the formation of these connections. To have empathy-driven innovation you have to connect the folk involved in teaching, learning, technology and instructional design in ways that are meaningful and that actively strengthen the connections.

    At the moment, at least in my institution, there is no easy way for an academic to talk to a technical person that actually knows anything about the system, i.e. someone who can actively modify the system. The technology person can’t easily talk with someone with educational knowledge to better inform technological change. Instead each group retreats to talking amongst themselves. The necessary connections are generally only there in spite of the system, not because of it.

The Webfuse Thesis

I’m somewhat engaged in this discussion because I have seen, for a small period of time, this type of empathy-driven innovation work in the context of e-learning within a University. This is the topic of my PhD Thesis, an attempt to describe an information systems design theory for e-learning that encapsulates this.

At it’s simplest, the Webfuse thesis embodies two aspects:

  1. Process.
    There are two broad types of process: teleological and ateleological. I describe the two types here. Empathy-driven innovation is an example of an ateleological process. The table in the blog post describing teleological and ateleological mentions Seely Brown’s and Hagels push and pull distinction.

    University e-learning is always too teleological, it needs to be more ateleological.

  2. Product.
    Everyone focuses too much on the actual product or system when we talk about e-learning. With Webfuse the product was only important in terms of how flexible it could be. How much diversity could it support and how easy was it to support that diversity. Because the one thing I know about learning and teaching within universities, is that it is diverse. In addition, if you are to engage in ateleological (empathy-driven) design, you need to be able to respond to local needs.

    Most importantly, the question of how flexible the product is, is not limited to just the characteristics of the product. Yes, Moodle as an open source LMS implemented with technology (PHP and MySQL) that has low entry barriers, can be very flexible. However, if the organisation implements with technology with high entry barriers and inflexibility (e.g. Oracle) or if it adopts a process that is more teleological than ateleological, it robs Moodle of its flexibility.

Phd Update #26 – Getting to first draft stage

Two weekly PhD updates in a row, will be interesting to see how long this keeps going.

What I did last week

The aim for this week was to get chapters 2 and 3 of the thesis into first draft stage and sent to the supervisor for comment. If possible, get chapter 4 into the same stage.

2 and 3 were sent on Tuesday. Chapter 4 is about 3/4 of the way into first draft stage. Hope to have that done over the weekend.

The week has been slower than normal due to school holidays and other work.

The aim for next week

The time factor is going to raise its ugly head. Found out today that a deadline for a grant application is now a month earlier than I thought – it’s now May 4. On the downside, that means that much of the time from now until May 4 will probably be spent working on that application. On the plus side, I’ll have some more free time after May 4 for the thesis.

For the next week, the plan is to:

  • get chapter 4 into first draft stage;
  • get a rough version of chapter 6 completed; and
  • if possible, get started on chapter 5.

My ultimate goal is for submission by the end of June, or as close to that as I can get.