Month: April 2010

Adding "deleted" to BIM

On April 30, 2010

The following is a record of the process of adding a “deleted” state for a blog post in BIM. This is in response to an issue that has arisen out of BIM usage at CQU.

The problem

BIM always keeps a record of all posts ever found in a student’s blog feed. Even if the student deletes the post or changes the post, BIM keeps a record of the post in its internal database. This is a feature of BIM as one of its aims is to provide a “safe/secure” record of what is posted to the students’ blogs, just in case the blog service provider goes away.

In addition, BIM also attempts to allocate old posts in its internal database to questions. Sometimes, you do wish to “delete” a post to stop BIM from allocating it. To give the student a chance to post something new.

Requirements

Requirements for this feature include:

Only the coordinator can “delete” a post.
The post is not actually removed from the internal BIM database, it just is no longer able to be “allocated” by BIM, released by the coordinator, visible to the student, or put into the gradebook.
It must be possible to “undelete” a post.
Deletion should not change any of the other data about a post (e.g. marker comments).
Perhaps rather than be invisible to the student, it should just be obvious that it has been deleted.

Design

Add “Deleted” status for mdl_bim_marking.
Modified setting via XMLDB, but the database is not being updated. This has been a problem before which I’ve kludged around. But it’s not something to rely upon for production. Will have to solve this problem.
Add interface for staff to “delete”
Modify student view to show that it is deleted

The alignment project as leadership

By admin

On April 29, 2010

In addie, alignment, curriculumMapping, Uncategorized

The following signals a slight change in direction around the curriculum mapping project. First, the project is now going under the label “alignment project” (curriculum mapping is just one aspect of the project). Second, the project is likely to be re-framed as an application for an ALTC leadership grant. This post is an attempt to begin this re-framing. It’s really just thinking out loud.

As a result, I am very interested in suggestions and criticisms. In terms of suggestions, I’m particularly keen for insights onto better/alternative theoretical frames. It has become a bit confused as I’ve tried out different lines of approach.

Note: In the following a “course” typically means the smallest unit of teaching offered by an institution. A “program” is a group of courses that form a degree or perhaps a major.

Summary of the alignment project

Alignment is an increasingly core component of teaching in Australian universities. At a basic level, alignment is where the learning resources, learning activities and assessment of learning all align with the stated outcomes or aims of teaching (in a course/unit or program/course). Such alignment is a core component of various “movements” within university learning and teaching including: graduate attributes, quality assurance, and improvements in learning and teaching. However, consideration of alignment is not a regular, everyday part of teaching or learning at universities.

The majority of academic teaching practice involves the teaching of an existing course, one the academic has usually taught before, and as such most “teaching practice” revolves around making minor modifications to material or content (Stark, 2000). Academics are not often required to engage in the development of new courses or major overhauls of existing courses (Stark and Lowther 1988). Alignment is most considered during the development of new courses, major overhauls of existing courses or in response to external quality assurance needs.

The following two sub-sections try to summarise the alignment project. The first is a more concrete description, the second more abstract or theoretical. The thinking behind this project has changed many times, the following are likely to change. Suggest away.

Process and intent

As currently thought, the alignment project can be described as four main tasks:

Modify Moodle to allow mapping of alignment.
Moodle has been chosen because it is the institutional e-learning system at both institutions that are part of the project. As such, support for Moodle is embedded into the institution and will continue to be supported. In addition, Moodle is an increasing part of the everyday practice of teaching academics. Lastly, Moodle, for a number of reasons, is very flexible and easy to modify and any modifications made could be usable by other institutions. The point of this project is not the Moodle modifications, it’s simply the best solution for embedding these changes into the institutions.
The changes will focus on enabling the alignment of outcomes (be the course learning outcomes, program learning outcomes, those from accrediting bodies, graduate attributes etc) with the assessment, activities and resources within a course site. Having this functionality is a foundation for the rest of the project.
Work with teaching staff to map their courses.
Mapping the alignment of a course within Moodle is not going to be straight forward. Teaching staff are likely to be busy and may not be entirely familiar with the concepts around alignment and mapping. The content and design of a Moodle course site may not be currently appropriate in terms of developing a useful mapping of alignment. Using the alignment mapping functionality added to Moodle may not be straight forward.
For these and other reasons staff from the L&T support services will have to work collaboratively with academic staff to overcome these problems. This will be the first cycle of action research and will identify specific problems and insights into potential solutions.
Work with teaching staff to embed alignment into everyday practice.
Once the initial mapping of alignment is complete, the focus moves onto helping academics maintain and reflect on the level of alignment within their course and programs. On helping them embed alignment considerations into everyday practice.
This is the second cycle of action research and will likely include the development of models, tools and processes that address questions such as:
- How do you encourage reflection and action based on the everyday consideration of alignment?
  Identify the support, processes and tools do teaching academics and program leaders need to encourage and enable reflection and action?
- How do you measure and give feedback upon action based on the everyday consideration of alignment?
Develop on-going and embedded institutional processes that take the lessons learned from the everyday consideration of alignment and use it to remove barriers within the institutional context.

Theoretical perspective

The alignment project is seeking to take the first steps towards what Biggs (2001) called the reflective institution. It seeks to do this by modifying the institutional systems around learning and teaching in ways suggested by Biggs (2001). These are:

Make obvious the quality model.
Most institutions espouse the theory of alignment, however the teaching systems and processes employed by institutions do not make this theory explicit. The first step is to modify these systems so that a focus on alignment is made explicit as a part of every day teaching practice.
This is partly achieved through the modifications to Moodle to enable mapping of alignment. But more importantly it is achieved through the changes in L&T support, systems and processes that support academic in using those new Moodle capabilities. In part, these changes are the next step.
Provide appropriate support for quality enhancement.
Further modify these systems and process to enable and encourage teaching staff to reflect and improve their teaching through a focus on alignment. The modifications build on the changes in the previous stage to enable this support to be highly contextualised to everyday teaching practice. A focus on helping in what academics do most often, the fine-tuning of existing courses.
Inform the quality feasibility process.
Quality feasibility is the removal of factors in the teaching environment that are not conducive to good L&T. The everyday consideration of alignment will identify a range of barriers in the institutional setting, many of which will require the engagement of institutional leadership to remove.

ALTC leadership grants

The guidelines for the ALTC leadership grants scheme describe the grants as being for (my emphasis added)

projects that build leadership capacity in ways consistent with the promotion and enhancement of learning and teaching in contemporary higher education, and which reflect the ALTC’s values of excellence, inclusiveness, diversity and collaboration, and its commitment to long-term, systemic change.

The scheme has three priorities which can be summarised as being focused on: institutional leadership capacity building; disciplinary and cross-disciplinary leadership; and building on earlier projects. The alignment project seems to be best suited to Priority one

institutional leadership to enhance learning and teaching through leadership capacity-building at the institutional level.

Funding range: $150,000 to $220,000

Project duration: up to 2 years

In completing a lit review around leadership for the ALTC, Southwell and Morgan (2009) make the observation that

Leadership for Excellence in Learning and Teaching Program is expected to be a ‘demonstrable enhancement of learning and teaching through leadership capacity building’

What is leadership?

It seems required when discussing leadership to make the observation that “leadership eludes comprehensive definition” (Southwell and Morgan, 2009). Southwell and Morgan (2009) reference Marshall (2006) and Jameson (2006) as folk who have made that observation. Having made this claim, the idea is that you then define your familiarity with the broad array of perspectives, understandings and definitions of what leadership is. I’ll postpone that bit for now.

Southwell and Morgan (2009) draw on Leithwood and Levin (2005) who suggest that the core of most conceptions of leadership are two functions generally considered to be indispensable:

Direction-setting: helping members of the organization establish a widely agreed on direction or set of purposes considered valuable for the organization; and

Influence: encouraging organizational members to act in ways that seem helpful in moving toward the agreed on directions or purposes

Leithwood and Levin (2005) arrive at these two functions by adopting a definition where “the primary effect of organisational leadership would be significant change in a direction valued by the organisation”. In defending their definition or understanding of leadership, Leithwood and Levin (2005) agree that this may not be a precise definition, but that attempts to too narrowly define a complex topic like leadership is “more likely to trivialise than help bring greater clarity to its meaning”.

How the alignment project fits

Taking the emphasised phrases from the above purpose of the ALTC projects, I’d suggest/argue that the alignment project fits in the following ways:

leadership capacity;
In terms of the above set of functions, the project is aiming to build into the systems and processes of each host institution the capacity to make consideration of alignment an every day part of practice. It helps establish a widely agreed direction for L&T and helps influence organisational members in moving towards that agreed direction.
promotion and enhancement of L&T;
The educational literature is replete with evidence that consideration of alignment changes the conception of L&T held by academics and that it also results in improvements in student learning outcomes.
contemporary higher education;
The project recognises and seeks to fulfill the increased requirements for accountability from a range of diverse source, however, it seeks to achieve it in a way that offers significant greater benefits that existing methods. As part of this the application seeks to engage with the on-going argument over centralised or devolved L&T support services by aiming for a focus on an approach to L&T support services that seeks to contextualise such support into the every day practice of teaching academics.
inclusiveness, diversity and collaboration; and
The action research process suggested for use by the project is largely based on recognition within the project that engaging with the full diversity of higher education is essential. It recognises that this diversity will result in different approaches and benefits and that the process needs to enable this to happen. Similarly, collaboration is seen as essential to the project. Not just in the process used in this project, but in the aims of the project. An important aim of the project is to increase the collaboration around consideration of alignment in teaching.
long-term, systemic change.
The project aims to embed consideration of alignment into the everyday practice of teaching staff. i.e. the aim of the project is long-term, systemic change. The process and approach being used is designed to achieve that aim.

The alignment project as leadership

Contemporary higher education is placing increasing importance on the concept of alignment in learning and teaching. In terms of quality assessment, program accreditation, graduate attributes and generally improving L&T alignment is broadly seen as a necessary component. However, many university courses are not all that well aligned and one explanation for this is that consideration and discussion of alignment is not a regular part of everyday teaching practice. Alignment is often only considered at the time of course and program reviews or accreditation.

The aim of the alignment project is to build leadership capacity into the system and processes of education within a university so as to encourage and enable effective and informed consideration of alignment as part of everyday teaching. This embedding of alignment into everyday practice then serves as the foundation for a range of other possibilities.

The alignment project is an example of leadership as it is attempting to encourage significant change – in the form of increased consideration of alignment at all levels – that is valued by the individual universities and the broader higher education sector. In addition, there is broad agreement in the education literature that alignment has significant positive effects on student learning outcomes.

The alignment project intends to fulfill the two indispensable functions of leadership identified by Leithwood and Levin (2005):

direction-setting; and
By making considerations of alignment a visible and hopefully key aspect of everyday teaching, there should be an increased emphasis placed on learning outcomes, graduate attributes and other “outputs of learning”. This should encourage and assist academics teaching the same course or in the same program to increase discussion of these outcomes. To increase discussion of the purpose or direction of a course or program.
influence.
A specific aim of the alignment project is to modify the teaching environment so that considerations, discussions and reflection upon alignment are directly encouraged and enabled. It encourages and enables them to think about how to move towards the stated directions or purpose.

Still not happy with that division. First attempt to make it concrete. Another query that arises from this is whether or not this project builds leadership capacity at two or more levels. For example, one approach is that it builds capacity at both the:

Institutional level; and
At an institutional level it enables curriculum/program alignment, but also accountability etc.
Instructional level.
This is the level between student and course/teacher. It makes alignment an more regular component of instruction.

Outcomes

If successful, the project should result in the consideration of alignment should becoming an every day component of teaching practice. This should/could generate the following outcomes:

Changes in the conceptions of L&T held by teaching staff;
Being required to consider alignment encourages a different way of looking at teaching. This should encourage changes in the conceptions of L&T held by teaching staff towards those considered more appropriate.
Improvements in student learning outcomes;
Effective considerations of alignment should increase the alignment within courses. The educational literature suggests that increases in alignment will result in improve student learning outcomes. In addition, it is likely that one of the likely additional outcomes will be making alignment more visible to students. Which should also encourage improved student learning outcomes.
Improvements in the quality and timeliness of quality assurance; and
In terms of demonstrating alignment against outcomes or attributes, current quality assurance practices rely on special “mappings” that are held every few years. By embedding alignment considerations and mapping into every day teaching practice, the there is no longer any need for special “mappings”. “Mapping” information can be generated at any time as it is maintained as part of normal practice.
A variety of additional outcomes.
Embedding alignment considerations as an everyday practice is the foundation of the project. The availability of “mapping” information and the on-going consideration of alignment will generate a range of additional outcomes. However, the diversity inherent in universities and higher education, combined with the fundamental change in everyday practice which this project aims to achieve means that it is unlikely we can predict successfully all of these outcomes.

Theoretical foundations

The ALTC place significant emphasis on applications having a sound and obvious theoretical base. The theoretical work that have informed my thinking about this project, and which may influence the project, include the following.

Project intent or outcome

Biggs (2001) presents an argument for a reflective institution that focuses on prospective, rather than retrospective quality. Having just re-read the paper, it seems to provide a good fit for a theory/model for the overall intent of the project.

The model is based on the idea of three aspects of AQ:

Quality Model.
i.e. an espoused theory of teaching, for Biggs this is constructive alignment. For the alignment project this might be alignment a little more broadly.
Quality enhancement.
A “teaching delivery system” that is designed in accordance to the quality model, i.e. one which encourages and enables alignments. In addition, the teaching delivery system should also have built-in mechanisms to continually review and improve current practice.
Quality feasibility.
A process by which impediments to quality teaching are removed from the “teaching delivery system”.

The “quality model” is underpinned by a large, overlapping and diverse collection of literature from various areas including: outcomes-based quality/evaluation, instructional alignment (Cohen, 1987); curriculum alignment; graduate attributes; and, of course constructive alignment (Biggs and Tang, 2007; Biggs, 1999 ).

Understandings of universities and organisations

My personal conceptions of most organisations, but especially those like universities, are informed by Dave Snowden’s Cynefin framework and complex adaptive system.

Understandings of leadership

This is obviously an area which needs more consideration, beyond the summary given above.

Based on my limited reading, I like the description of “new leadership” (related to distributed leadership) attributed to Fullan (2008), which include:

respect of employees, rather than simplistic judgmentalism;
A specific focus of this project is to help teaching staff consider the alignment of their courses. It is not to judge them. Biggs (2001) makes the point that under the type of reflective institution he outlines, the focus is on the teaching, not the teacher.
connecting peers with purpose and ownership;
By embedding indications of alignment into the LMS the aim is to create connections between other teaching staff, academic leaders and also teaching support staff. The clear purpose is around considering alignment.
building employees’ and systems capacity; and
This is the specific aim of the project. Building into the institution the capacity in both the employees and its systems to engage regularly in consideration of alignment.
transparency of practice and results.
At the very least, the aim of this project is to enable teaching staff within the same program (i.e. group of courses/degree etc.) to see each others practice. To show what is aligned, where (or where not). It opens up the teaching practice to colleagues, hopefully in a way that is not judgmental.

Process

Based on some of the above and below, I’m leaning very much towards an action research process. The practice of L&T within a university is a complex-adaptive system. As we introduce change into the system, the system will change around us and unexpected event will happen. The type of change suggested involves a fairly widespread change in the practice of teaching academics, but also the institution. In addition, as far as I’m aware, no-one else has tried and reported on this type of change, hence it is novel. While driving towards a particular goal, we have to aim on learning as much as we can during the process.

In addition, Biggs (2001) offers the following

action research, a methodology designed precisely to generate and evaluate in-context innovations (Elliott 1991). As a result of engaging in action research, teachers change their conceptions of teaching, and teach more effectively (Kember 2000).

This type of approach also fits very closely with what is known about staff development.
i.e. the current recommendations are that staff development should be as contextualised as possible. My conception of how teaching staff would be helped to consider the alignment of their courses contains a very heavy assumption on contextualised staff development. In fact, the presence of alignment information and the transparency between courses is aimed at helping this support and development to be increasingly contextualised to the every day teaching practice of the teaching staff.

Teaching practice

One of the fundamental models/theories underpinning this project for me is the observations embodied in the following.

How academics design their teaching is not described by a rational planning model (Lattuca and Stark 2009). In part, this is because the dominant setting for academics is teaching an existing course, generally one the academic has taught previously. In such a setting, academics spend most of their time fine tuning a course or making minor modifications to material or content (Stark 2000). Academics are usually not often required to engage in the development of new courses or major overhauls of existing courses (Stark and Lowther 1988).

The practice of most academics does not separate planning from implementation, and rather than starting with explicit course objectives, starts with content (Lattuca and Stark 2009). In the absence of formally documented teaching goals, the actual teaching and learning that occurs is more in line with the teacher’s implicit internalised knowledge, than that described in published course descriptions (Levander and Mikkola 2009). Formal descriptions of the curriculum do not necessarily provide much understanding about how teachers put their curriculum ideas into action (Argyris and Schon 1974)

References

Argyris, C., & Schon, D. (1974). Theory in practice: Increasing professional effectiveness. Oxford, England: Jossey-Bass.

Biggs, J. (1999). Teaching for quality learning at university. Buckingham: Open University Press.

Biggs, J. (2001). The Reflective Institution: Assuring and Enhancing the Quality of Teaching and Learning. Higher Education, 41(3), 221-238.

Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16(8), 16-20.

Fullan, M. (2008). The Six secrets of change. San Francisco: Jossey-Bass.

Jameson, J. (2006). Leadership in post-compulsory education: Inspiring leaders of the future. London: David Fulton Publishers.

Kember, D. (2000). Action Learning and Action Research: Improving the Quality of Teaching and Learning. London: Kogan Page.

Lattuca, L., & Stark, J. (2009). Shaping the college curriculum: Academic plans in context. San Francisco: John Wiley & Sons.

Levander, L., & Mikkola, M. (2009). Core curriculum analysis: A tool for educational design. The Journal of Agricultural Education and Extension, 15(3), 275-286.

Marshall, S. (2006). Issues in the development of leadership for learning and teaching in higher education (Occasional paper). Sydney: Carrick Institute of Learning and Teaching in Higher Education.

Stark, J. (2000). Planning introductory college courses: Content, context and form. Instructional Science, 28(5), 413-438.

Stark, J., & Lowther, M. (1988). Strengthening the Ties That Bind: Integrating Undergraduate Liberal and Professional Study. Ann Arbor, MI: Professional Preparation Project.

Southwell, D., & Morgan, W. (2009). Leadership and the impact of academic staff development and leadership development on student learning outcomes in higher education: A review of the literature. Sydney: Australian Learning and Teaching Council.

The realities of the ERA and L&T support services

By admin

On April 28, 2010

In publication, Uncategorized

In mid-October last year I blogged about my search for a research publication outlet. The conclusion was that in my context, the Australasian Journal of Education Technology (AJET) was probably the best fit. It is an open journal and the first round of the Australian government’s Excellence in Research for Australia (ERA) initiative had ranked AJET as an A journal, second only to A*.

More recently that blog post got a reference in an AJET editorial (that the editors are referencing a blog post in an editorial is a good sign for AJET, especially if it is one of mine). The editors hope was

that his recommendation, written in 2009 when AJET was a Tier A journal, will not change as a result of AJET’s demotion to Tier B in the 2010 list

I’m sad to say, that it probably will!

Why?

There are two reasons why:

the on-going uncertainty around L&T support services within Australian higher education; and
the increasing force of having to comply with government indicators.

On-going uncertainty

Early in 2007 I moved from being a faculty-based academic in the information systems discipline to a role within a central L&T support division. Within about 6 months of taking on the role, the then Director of that L&T division was told her services were no longer needed. Apparently, they were after someone with a greater research focus, rather than on getting things done for the institution. The division limped on for around another 18 months while they failed in appointing such a person. During this time, rumour was rife that we would be restructured, though officially this was not confirmed. By the end of 2008 the restructure was complete, I was in position limbo for 6 months (theoretically better than having been made redundant, but I’m still trying to decide if that was the case) and the institution tried again (again unsuccessfully) to appoint a Professor to lead the unit.

So, as you might deduce, there has been little certainty for the folk in L&T support services. Oh, by the way, the institution has finally admitted that the last restructure was somewhat less than successful in its outcomes (of which they were informed many times back in 2008) and they recognise they need to address this problem. By another restructure. So, again, there isn’t a lot of certainty.

This is not something unique to my current institution. A regular topic of conversation at the last ASCILITE conference was around the restructuring of L&T services. It seemed that every second person from an L&T support area was going through, just been through or predicting they would go through a restructure. Just recently I heard that the Director of another L&T support area is being replaced with someone who has a more research focus. Deja vu all over again.

In the past, I have argued that this is, at least partly – if not mostly, to do with the absence of any broadly agreed indicators for the quality or impact of L&T support. Combine this with the fact (or at least strong likelihood) that every senior manager who has taught a course thinks he/she has a unique insight into how to support L&T at a university and the sheer complexity of such a task, and you have a recipe of on-going change, little or no success and little certainty for the folk within such centers.

Requirement to comply

The ERA itself is just one example of the Australian government specifying indicators of success. Now, don’t get me wrong, there is an argument to be made the universities should be accountable. However, as I’ve argued before such indicators are generally very poor and at best result in compliance and at worst task corruption.

For example, I’ve seen in the last few weeks the following:

A faculty dean’s report to academic board spend a lot of time explaining how many A* journal publications the faculty had produced.
New guidelines for course (my institution uses american terms, so a course is the smallest unit of education offered by the institution, a program is a collection of courses) coordinators that specify having a graduate certificate of higher education – or the intent to get one – a desirable criteria.
A job in a central L&T support area that listed having a graduate certificate in higher education as the first necessary criteria.

To me these are indications that the compliance has begun. Senior management at universities have decided, regardless of other reasoning, to adopt the government indicators as prime decision criteria. This is bad because a heavy focus on these types of indicators gives a false sense of security that we’re doing the right thing. It reinforces simplistic beliefs. e.g. that if teachers with a formal teaching qualification will be better teachers. A belief that ignores a whole bunch of literature around the impact of context – i.e. if the context of a university does not value L&T, it will not be good.

It will encourage a generation of senior leaders at universities that focus on these indicators rather than a broader and deeper understanding of what it takes to encourage and enable effective teaching and important research. This is because most of the current senior leadership roles at Australian universities are short-term appointments – 3 to 5 years. You can’t effectively and broadly improve the teaching and research at a university in 3 to 5 years, but you can sure as hell pragmatically hit some simplistic indicators.

The pragmatic defense

Worse still, this combination of factors encourages academics in L&T support areas to be seen to comply with the indicators. If you’re uncertain about the value senior leaders place on the contribution of L&T support areas, the best way to ensure a place at the table is to have lots of A* journal publications, important grants etc. You need to have a place at the table because you can’t effectively battle against the short-term focus on broken indicators when you’re on the outer. Even if you don’t get to the table, given the uncertainty and a likely need to retain some sort of salary, you need to have fulfilled these indicators so you can get a job at another institution.

What’s even sadder is that you probably can’t effectively battle against the short-term focus on broken indicators even when you’re sitting at the table. Sadder again is the observation is that your focus on these indicators may end up changing you.

My potential publication outlets

Sadly, at least in the short-term, AJET has slipped down the rankings. It probably won’t be my first target for publication in the next couple of years. I have to hit the A*/A journals. Hopefully, I can find one of those that is somewhat open – I believe ALT-J has some sort of set up that is somewhat open.

Inside out, Outside in or both?

By admin

On April 25, 2010

In bim, elearning, futures, moodle, phd, PsFramework, Uncategorized, webfuse

During the last week I have been in Canberra for various events, including giving a presentation on BIM at University of Canberra. Somewhat surprisingly (as last I knew, he was in New Zealand), Leigh Blackall was in “audience” at the presentation, and as is Leigh’s wont, he asked some serious questions. I was troubled by those questions and needed time to reflect on what an answer might be.

This is an attempt to develop an answer to why I was troubled. In part, this attempts to pick up a comment I’d made earlier on Leigh’s blog about thesis, anti-thesis and synthesis

The question

The initial question Leigh asked which troubled me is repeated in his blog post on the presentation. It is

I asked the obvious question of why, or if BIM might consider developing outside the framework of Moodle say, as a Firefox based or other Feed Reader plug in, and offering a file that can be imported to Moodle (as well as a spreadsheet, a MediaWiki table, a Wikispaces Table, MySQL database, a text document and a PDF to email), and thereby offering the functionality of BIM to a wider user base than just Moodle.

The immediate answer I gave is reported faithfully in Leigh’s post

David explained that the project was constrained in many ways to the needs of the sponsoring Institution,

(as is the fact that we had broader discussions.

What troubled me (in part) about my response is that the constraints of the sponsoring institution is both more and less than it sounds. In part it is what is simply easiest, the institution pays my wage and it uses Moodle. But it is also, to me, what is the best way in terms of improving L&T.

It appears to be the difference between an “inside out” approach (which I’m taking and will argue has a chance of success) and an “outside in” approach (which is somewhat close to what Leigh suggests). At the very least, it makes for a good title.

Inside out

The main reason I didn’t start with Firefox or some other external way of developing BIM, is that I’m taking an inside out approach to improving learning and teaching. i.e. I’m starting with what is being used within the organisation and trying to change it for the better. The organisation is currently using Moodle, so if I want more people to be thinking about using Web 2.0 and reflective student journals in their L&T, I have to start with Moodle. Doing so lowers the barriers to entry and actually makes it somewhat likely that people will use it. It even fits (with some difficulty) within the constraints of how Moodle is being managed within the institution.

In commenting on Leigh’s post, Peter gets close to the approach

The central question though for an educational developer is how to promote innovation and change and it seems to me that it has to be based on an invitational ethos: teaching staff need be convinced of the benefits of technology adoption, they sometimes come to it slowly, they come to it in surprisingly unexpected ways at times and positive things happen.

I actually think more than this is needed and will pick up on it below. But first..

Outside in

To a limited extent, Leigh’s approach could be characterised as outside in. Start with the outside stuff, support those people and then perhaps change might eventually occur within the institution. However, in responding to Peter’s comment Leigh suggests a perspective that doesn’t really both with the “inside” (current universities)

To borrow your highway metaphor, a bypass is needed, one that goes around that old town, and offers a more direct route for the people in need of credentials with minimal debt. Remember, the experiences in the old town have become irrelevant. An old road can remain for those who like nostalgic tourist routes, but an alternative route is needed.

There are many within current institutions that react negatively to this perspective, however, I can see the need for it. Mainly because, as Leigh points out, there are significant barriers within universities that suggest that an inside out approach may not work.

I don’t believe a passive approach can be effective when considering our dense hierarchies, performance reviews, infrastructure, broken feedback systems, conservatism and the wrong sorts of incentives and rewards.

Limits in developing innovative pedagogy with Moodle

In fact, I’m hoping to use the development of BIM and the limitations of this approach as the basis for a couple of publications, including a presentation at MoodleMoot AU 2010. (still waiting to hear if the submission has been successful). The “theme” for this conference is “without limits….”. Rather than accepting that Moodle is “without limits”, I argue that

e-learning with Moodle, as currently practised, has a number of limits and that progress can be made through the recognition, understanding and removal of those limits.

So, what may you ask, is my problem with Leigh’s comments?

Both-and, not either-or

The questions around the quality of L&T within universities, the requirement from society for different approaches and the future of universities are complex. So complex, that it’s never going to be about a single answer, there is no such thing. For this, and other personal reasons, I prefer a both-and approach. It’s not about outside-in or inside-out, it’s about both.

Actually, I should paraphrase, in terms of improving L&T within universities, it’s a question of both-and. In terms of responding to societies changing needs around learning (or simply recognising a long-standing need that has been ignored), I’m not so sure there is a need for an “inside-out” perspective. However, as I’m paid by a university to improve the quality of L&T, I see the need for a both-and approach for the long-term benefit of the university (and hopefully society…there’s a big question in there).

What this means is that there is a huge need for folk like Leigh and many others (Leigh gives a list of some in this post) who are identifying and creating insights into what the “outside” should be. The value is not just for the “outside” it’s also for the universities and other institutions as it helps identify some options around where we need to be.

However, there is also a need – at least at the moment – for folk who are taking the “inside-out” approach. Thinking about how to effectively make the “inside” a better fit or enabler for what the outside should be like. That need may not exist in the future, but for the moment it does and because this is a complex area, I think we need both.

Inside-out is currently failing

As stated above and numerous times on this blog, I think the current approaches being used within universities are failing. Most L&T at universities is poor quality by traditional standards, let alone if measured by adoption of social media. For me, this is not a sign that it is impossible, it’s a sign that the principles of current approaches are just plain wrong. This is what my current work is looking at.

Why is it failing?

The following diagram represents what I think is a fundamental mismatch (The image is taken from this post by Donald Clark).

As I said above, this problem is a complex one. Based on the above diagram, the best type of solution arises from immersion in the problem. The problem is that most universities are attempting to solve this problem by analysis.

For along time I’ve been saying that learning and teaching is a wicked problem (Rittel & Webber, 1973). Clark’s blog post (source of the image) draws on another of Rittel’s publications to suggest that for a complex problem

This is because as Rittel (1972) discovered — the best experts within these types of environments are those affected by the solution — since they are the only ones to have experienced the complexity of the problem, they are the best experts for helping to improve that environment.

At the moment, universities are only paying lip service to involving the “experts” – the students and teachers. Most of it is being driven by “management” who don’t have in-depth experience of a specific context. Their decisions are driven more by other considerations than in-depth understandings of the current context. Importantly, this isn’t about asking the students and teachers what they want, for me, it’s about understanding what they are experiencing and where they are now as an important first step in helping them go somewhere else.

At the same time, the approach taken by the “outside in” folk – like Leigh – also has the same failure. It doesn’t seek to understand the existing context or practice of the students and staff. But that’s okay, that’s not what they are about, they are about figuring out and creating a better future. (I fully recognise that this is a gross simplification and generalisation. However, I do think it’s a distinction that has some value.)

What is both-and?

As an inside-out person, I believe any success comes from having deep knowledge of the current context (the experience of staff and students) and marrying that with “solution” knowledge from the experts and insights into how the environment can be changed in a way that encourages and enables the staff and students to improve their experience. This describes BIM:

knowledge of current context;
The need for BIM arose out of my need to teach a 200+ student course that had a “reflective” journal assignment which had significant problems (a problem faced by other staff).
expert knowledge; and
I knew about Web 2.0, the benefits of student-owned journals/blogs and the institutional need for and staff/teacher familiarity with the LMS.
environment change.
Add a module to Moodle that enables staff to manage individual student blogs hosted on external services.

But it’s not enough

This is what really troubled me, and now I’m becoming repetitive in the same post. The above by itself is not enough. As a measly e-learning and innovation specialist I have no power to make the further changes in the environment that are necessary to make BIM truly attractive.

I guess the real reason why Leigh’s questions troubled me, is that I’m frustrated at my inability to make the change and the on-going blindness of institutional leadership.

How’s that for a positive end to a rambling post?

A response to Leigh

Couldn’t leave it there, let me return to the original question from Leigh. I believe that an inside-out approach is probably more likely to help improve L&T on a broad-scale within a university than an outside-in approach. I value the insights offered by outside-in folk, but I think I need to value the experiences of the students/staff within a university and build on that experience to help improvements happen. Perhaps, it’s simply a question of purpose. My purpose is to help improve L&T within universities, yours is more about helping those people who are already learning outside of universities.

That’s why I think building BIM on top of Moodle was a better fit for my purpose. A purpose, which I agree, is fairly narrow.

References

Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155-169.

Clarification of the alignment project

By admin

On April 22, 2010

In curriculumMapping, Uncategorized

Am at the University of Canberra for a second day today talking about the possibility of an ALTC grant application around alignment/curriculum mapping. The aim today is to try and clarify where the project would go and this post is an attempt to make concrete some of my thinking. The final ideas for the project may be radically different.

I’m going to use the “project clarification” section of the ALTC’s Steps in developing an evaluation plan for an ALTC grant project as the structure for this post. Mainly because of a belief that getting into the “ALTC mindset” will increase the likelihood of success.

This remains very much a work in progress. Comments and thoughts all welcome.

What is the focus of the project?

The fundamental issue the project is aiming to address is in increasing the visibility of curricula and instructional alignment in the practice of university teaching.

The observation is that consideration of alignment is not an important consideration in what most academics do around learning and teaching. Consideration of alignment is not something that students see regular reminders of in their learning. It is done and visible in places, but those times and places are not a regular, visible part of what they do.

Since it is not visible in regular practice, it is often not thought of. It doesn’t inform what they do. This is bad because there’s vast bodies of research that suggests alignment between outcomes/graduate attributes, assessment, learning activities and instructional resources is a good thing. There is evidence to suggest that many university courses do not have strong alignment. There is also evidence (McDonald, 2008) that when considerations of alignment become part of what academics regularly do as part of teaching, it encourages them to begin questioning existing practice.

The focus of the project is on how to make considerations and discussions about the level of alignment in university courses a regular and on-going part of what academics do as part of their normal teaching practices.

The aim is to make changes within the teaching context that enable and encourage these considerations and discussion of alignment to take part regularly.

What is the scope of the project?

While the project will (hopefully) be funded by the ALTC over two years, the intent is that the aim of the project is an attempt to embed these practices within the institutions. i.e. to be successful, it should be an on-going project.

Initially, the ALTC funded project will be limited to University of Canberra and CQUniversity. Participants are likely to fall into these categories:

Round 1 teaching staff;
A smallish group of academic staff who will be involved in the first cycle of making alignment a regular part of their practice. The nature of this group will depend on each institution.
Round 2 teaching staff;
Includes the round 1 teaching staff plus additional staff that will work with modified practices from round 1. The intent is that round 1 teaching staff will be using these practice for the second time, perhaps with the same course. The round 2 only staff will likely be chosen on the basis of having some differences (on a broad array of possible variables) with the round 1 staff.
learning and teaching support staff;
These are staff tasked with helping academics improve their L&T. This might include instructional designers, staff developers etc. These staff will be working with the teaching staff during both stages to help develop appropriate approaches to integrate alignment considerations into everyday practice.
project leaders; and
The staff named on the ALTC grant application will be involved in various tasks in ensuring the project is moving along and may also fulfill some of the previous roles.
reference group.
A body of experts that will offer oversight and provide feedback on the projects aims, outcomes etc.

Over the 2 years of ALTC funding the project would probably aim to, at least at one of the institutions, integrated alignment into the practice of at least one program in order to test the impact at the program level. Perhaps a target of 30 or courses at each institution?

The particular approach being taken with this project is to use Moodle as the tool that enables and encourages an increased consideration of alignment.

What are the intended outcomes?

Indications of the impact of significant consideration of alignment on the thinking/practice of teaching staff and on the student experience.
Guidelines, identified challenges and suggested processes (with accompanying documentation and resources) for increasing the consideration of alignment into everyday practice.
Guidelines, tools and processes describing how the practice of quality assurance can change once alignment considerations are embedded into every day practice.
A collection of Moodle enhancements.
These are important only in terms of the affordances they enable which can then be harnessed to improve considerations of alignment. The enhancements are likely to fall into two categories:
1. Addition of “mapping” into Moodle.
  The ability for Moodle to enable mapping of all activities and resources against specified outcomes, attributes etc.
2. Moodle features that use the mapping information.
  This is the really interesting part. What applications make embedding alignment into every day practice useful and interesting for the broadest array of stakeholders.

In the elevator pitch for this project I identified some of the following as propositions:

Curriculum maps that are generated, and continue to evolve, in the same time and space as everyday teaching and learning will have stronger validity in terms of capturing reality and consequently be of more value.
A learning environment that makes visible to all stakeholders the alignment (or not) of a course and then provides scaffolding necessary to improve that alignment will help improve teaching.
Well designed extensions to an LMS that encourage and enable improvement of course alignment will increase the quantity and quality of usage of the institutional LMS and subsequent student outcomes.
Building curriculum mapping around Moodle’s student “tracking” functionality will enable and encourage greater use of the student tracking functionality.
Properly implemented, this approach can make it easier for curriculum designers to embed assistance into the context within which teaching is taking place. If this works well, relationships will develop.

Each of these could be thought of as potential outcomes. Yes, they should be developed some more.

What are the operational processes developed to achieve the outcomes?

The process to be used in this project should lean heavily towards being adopter-focused (Surry & Farquhar, 1997). The following description is taken from Jones and Lynch (1999)

In contrast to the developer-based approach, the adopter-based approach focuses on the human, social, and interpersonal aspects of innovation diffusion (Surry & Farquhar, 1997). Developers are interested in the individual who will ultimately implement the innovation in a practical setting as the primary force for change. The adopter-based theories reject the assumption that superior products will automatically be attractive to potential adopters. They ’seek to understand the social context in which the innovation will be used and the social function the innovation will serve’ (Surry & Farquhar, 1997).

Stages in the process for this project might include:

Adding mapping to Moodle.
The focus here is on obtaining a version of Moodle that allows the mapping of every activity, resource and assignment within Moodle to a set of outcomes. i.e. the aim is to get the data stored. This is almost a purely technical process that requires little involvement from academics.
Planning and first roundtable. ????
The project groups at each of the two institutions formulate the plans/approaches they plan to use for the first round. This identifies: the participants, programs/courses, current state of mapping, approaches to be used to complete the mapping…..Have a roundtable where the reference group are shown these plans and offer advice and suggestions.
Round 1
Reflection/roundtable.
Round 2

What is the conceptual and theoretical framework underpinning the project?

Alignment.
Adopter-focused development … more buzz words.
Teacher conception and behaviour is a key component of quality of L&T.

What is the context of the project?

2 institutions that have adopted Moodle as enterprise LMS.
Higher ed sector where alignment is important in terms of graduate attributes etc. and also more broadly quality assurance and quality enhancement are increasingly more important.
Institutional change.

What key values drive the project?

Adopter-focused.
Open source/open.

References

Surry, D., & Farquhar, J. (1997). Diffusion Theory and Instructional Technology. e-Journal of Instructional Science and Technology, 2(1), 269-278.

Framing a body of research and innovation

By admin

On April 15, 2010

In elearning, herding cats, reflectivealignment, Uncategorized

Markus has finally posted the set of questions for framing a research proposal idea that he showed me weeks ago. This post is an attempt to use those questions to frame what I’m doing in my current position.

A simple test for BIM purposes, ignore

Write a research question

How can you improve the quality of learning and teaching within a university?

What is the important theoretical or conceptual setting?

Too much work in this area has not focused on the academic staff and factors or approaches that would encourage and enable them to change their teaching behaviour. The behaviour of the teacher is a key component of the quality of learning and teaching. Teaching and learning is getting more difficult.

Write 2-3 key points about why it is important (Significance? Innovation?)

Governments and other external stakeholders are increasingly demanding proof that university learning and teaching is good and getting better.
Most university learning and teaching is of less than good quality.
Most, if not all, interventions to improve L&T are not succeeding in widespread, long-term behaviour change (i.e. improving quality).

Write 1-2 brief objective points (formulate full aims later)

Evaluation of existing interventions, informed by behaviour change research, will reveal significant short-comings.
Interventions, informed by behaviour change research, will result in significantly greater levels of improvement of L&T.

Write a couple of points about the approach you might take

This is not necessarily just about a single research project and it is unlikely to have, at least initially, significant institutional support or recognition. A collaborative approach involving many different projects and people will be done. However, it seems likely that an on-going process of:

Learning more about how behaviour change research may inform this project.
Combining these insights with other knowledge to design interventions.
Use the insights to evaluate both existing and new interventions. Also to further understanding of the current context.
Publish.

Will I have all the expertise/resources? Should I be talking to collaborators? Might the proposal be strengthened by having a team?

A team is absolutely necessary. Knowledge of behaviour change research is essential. As is good knowledge of the local context and educational and technical skills to implement the interventions.

Do I or will I need some ‘proof of concept’, preliminary data or demonstration of competency to undertake the project?

Yes, the on-going process above will have to start small and learn from there.

Will the team/partners be competitive in this field?

At the moment, yes. There doesn’t appear to be too many people taking this approach.

What would the outcome(s) be and who benefits?

Hopefully, there will be improvements in L&T in the local context and research publications and grants around that work.

What is the ‘WOW’, ‘HOOK’, or ‘EXCITEMENT’ factor?

It appears that this is a new way of approaching an intractable problem.

Further thinking – behaviour change and improving L&T

By admin

On April 15, 2010

In elearning, herding cats, reflectivealignment, Uncategorized

This post is an attempt to synthesize and reflect more upon two posts from yesterday. One from me trying to explain an early form of a framework for improving L&T based on findings from psychology behavior change research. The second from Markus that expands on the potential psychological foundations for this thinking by mentioning:

The recommendations section of a UK National Health Service (NHS) document outlining some generic principles for initiatives to support attitude and behaviour change.
The concept of salutogensis, at least as a cliff hanger for a subsequent post.

The following is my attempt to summarise and make some initial connections between this work and ideas around improving learning and teaching.

What behaviour change programs need to do

The NHS document is said to highlight the need for “interventions and programmes aimed at changing behaviour” to:

Be based on a sound knowledge of community needs and should build upon the existing skills and resources within a community.
To me, for improving and learning and teaching this speaks of a need for empathy-driven innovation and for a focus on what the teacher does.
Equip practitioners with the necessary competencies and skills to support behaviour change, using evidence-based tools.
My understanding is that “practitioners” refer to the people helping the change, not the actual teachers. Interesting.
Evaluate all interventions.
Something not really being done around L&T within universities.

These are derived somewhat from the principles for interventions presented. It is again the importance placed on understanding the community as the foundation for change that resonates for me. You have to start with knowledge of the community and their strengths, perceptions etc.

Salutogensis

The following is based on a quick reading of this paper.

Focus on origins of health, rather than causes of disease

Interesting thought, salutogensis arose from an attempt to understand why women who survived concentration camps in the Second World War, had the capability for good health.

Is there a connection hear with L&T?

I would perhaps characterise the current focus in Australian universities on changes driven by the need to respond to government as attempts to prevent disease, rather than encourage health. …tentative and tenuous.

The human is at the heart

Discussions this morning lead to some reflection on metaphor – an area in which my wife has done work. The idea is that the metaphors we use to describe complex concepts reveal how we think about those concepts, it informs our decisions.

Increasingly, some colleagues and I are seeing evidence of senior management holding a machine metaphor when it comes to learning and teaching. The academic staff involved in learning and teaching are seen as machines, as inter-changeable machines. You can take that machine from there, and place it over there and it will do the same job. It doesn’t matter which machine, they all do the same thing.

It is this metaphor which enables the adoption of techo-rational approaches to improving learning and teaching such as the idea that every academic should have a graduate certificate. The metaphor here is that if you add a new widget to the machine it will perform the task with greater efficiency or with greater effectiveness.

The trouble is that people aren’t machines. They don’t passively accept the addition of a widget. There’s no direct causal link between addition of a widget and changes in behaviour.

Lundstrom and Eriksson (2006) see health promotion as

three phases, the first one that recognizes the background (the determinants), the second one that sets an objective (to lead an active productive life), and the third one is the activity (the enabling process) where the determinants are used to reach the objective in a dialectic relationship between people, the setting and the enablers.

As a “dialectic relationship” it is an on-going dialog, the person doesn’t take it unquestioningly. They go on to say

At the heart there is the human being seen as an active participating subject, respected in her full human rights.

Something that is increasingly missing in L&T within universities.

The salutogenesis framework

A framework might be possible by drawing on the salutogenic focus on three aspects

Focus on problem solving/finding solutions.
Identifying GRRs that help people move in the right direction.
GRR = General Resistance Resources – biological, material and pyschosocial factors that help people see their lives as consistent, structured and understandable. A person with these resources has a better chance of dealing with challenges.
Identify the SOC – the pervasive sense that enables this move.
The SOC is the ability to use the GRRs. The GRRs create life experiences that promote a strong SOC. (This is somewhat confusing but the message I’m getting from the paper.) The SOC is the capabaility to perceived you can manage in any situation. It is flexible and not focused on a specific set of strategies.

Will have to wait for Markus’ explanation to make this understandable to me.

There are, however, some positive resonances with this ideas and my current thoughts, will have to see how that plays out.

Identifying and designing interventions to improve L&T – a behaviour change framework?

By admin

On April 14, 2010

In elearning, herding cats, indicators, reflectivealignment, Uncategorized

The following is an attempt to describe early attempts at developing a framework for evaluating existing and developing new interventions aimed at improving learning and teaching within universities. It builds on some ideas from an earlier post.

Sorry, but the first couple of sections seem necessary to convince myself that this is connected with some of my earlier thinking.

Rationale and assumptions

As mentioned many times on this blog, I think most attempts to improve learning and teaching at Universities really don’t work as they don’t change what the majority of academics do. At best they get a bit of compliance. The aim here is to come up with a framework/guidelines that help evaluate and design such interventions.

Empathy driven innovation

While the following uses a framework from psychology, the basic message is the same as a range of other work. That message is, focus on the academics, understand what they are currently doing and experiencing, and design interventions that connect with that experience. Most recently this was discussed in a a post on empathy-driven innovation.

Much of my thinking around improving learning and teaching is based on this sort of approach, the following tries and provide some additional guidance into how to do this. But underpinning it all, is a focus on the actual teaching experience of the academic staff (and the learning experience of the students) as the primary focus.

The aim is to understand what they are experiencing and develop ideas about how that experience can be improved, as perceived by the staff and students and not as perceived by management, the government or some research ideal. The focus is on what the teacher does, not what management does.

Improving L&T is an attempt to change behaviour

An assumption underpinning this thinking is that when you are trying to improve learning and teaching, what you are actually trying to do is to change the behaviour of the teaching academics. If you don’t change the behaviour of the academics, then there is no way you can expect the quality of learning and teaching to change.

This is perhaps the main problem that I have with existing methods being employed to improve learning and teaching at Universities. Existing methods don’t actually consistently result in behaviour change amongst a significant percentage of university teaching staff. I’ve argued before that they don’t work and that there is evidence to support this.

The question is, what’s the solution?

Conceptions and teaching

An example of this mismatch can be seen with the on-going focus on teacher qualifications. For example, the Australian government has introduced KPIs for university learning and teaching that include the use of the % of university teaching staff with teaching qualifications as a KPI. I’ve argued this this attempt will encourage compliance behaviour on the part of institutions and academics.

The institutions are already at it. My current institution is currently introducing a new “desirable criteria” for people in charge of courses – having a formal teaching qualification. The assumption is that this will increase the quality of teaching. It won’t.

At best, a formal teaching qualification will change the conceptions of teaching held by an academic. The “teacher’s thinking” layer in the following diagram. There is significant research (e.g. Trigwell, 2001; Richardson, 2005) that suggests that unless all of the layers in the diagram are in alignment, you won’t get improvement in teaching.

That is, even with a formal teaching qualification, if the context, disciplinary expectations and many other factors aren’t right, then there will be no behaviour change. The teacher will continue to employ the same strategies as always.

How to encourage and enable behaviour change

How to change behaviour is something that the psychologists study. This current line of work is a collaboration between myself and a psychologist colleague. The current aim is to try and distill what is known about behaviour change in psychology into a form that helps explain how someone – with a deep knowledge of the current experience of academics – can encourage and enable behaviour change that results in improved learning and teaching.

In earlier work I’ve drawn on insights from Roger’s (1995) diffusion theory as a guide. There are definite connections between diffusion theory and the following.

The initial spark for this thinking is a paper by Michie et al (2008) which suggests that:

There are determinants of behaviour change, (see the table below) someone is unlikely to change behaviour when these determinants have the wrong value.
There are known techniques for encouraging behaviour change.
These techniques are each seen as likely to influence a certain subset of the determinants.

Key Determinants of Behaviour Change from Fishbein et al., 2001; Michie et al., 2008
Fishbein et al	Michie et al
Self-standards	Social/professional role and identity Knowledge
Skills	Skills
Self-efficacy	Beliefs about capabilities
Anticipated outcomes/attitude	Beliefs about consequences
Intention	Motivation and goals Memory, attention and decision processes
Environmental constraints	Environmental context and resources
Norms	Social influences Emotion Action planning

From a L&T perspective, the application of this knowledge seems to be something like:

Identify something that is known to be good practice, that improves L&T, but which isn’t being done in the current context.
Use the determinants to identify areas of weakness, areas for improvement, around the current context. Especially in terms of the “good” practice.
Identify generic behaviour change techniques that address those areas of weakness.
Draw on empathy driven innovation to develop specific interventions, which are based on the information gathered in the previous two steps.

An example

In a previous post that seeks to summarise the Michie et al (2008) paper, I use the following example of the level of interaction or participation of staff in a course site. In the following, I compare/contrast the more traditional approach and the type of approach that I think might work better.

The traditional approach

The more traditional approach is what I suggests is “what management does”. Management will see that there is a problem, i.e. staff aren’t participating enough. They then take it upon themselves to perform some action, which might include:

Require staff to get a teaching qualification.
The assumption being that with a teaching qualification they know more about the importance of staff interaction and hence will increase their interaction.
Create a policy that requires a certain level of participation.
At the very least they might say that every course site must have a discussion forum. Over time they might add requirements like: every student query must be answered within two days, or every staff member must make 5 posts a week to the course discussion forum.
Run a staff development session.
Organise an external expert on the benefits of staff participation to run a session at the institution and invite teaching staff to come along to listen.
Reward staff who interact at appropriate levels.
Give them extra money or recognition as good teachers. etc.

Most of the above interventions would require a fair bit of work. Most, if not all, of the senior management at the institution would need to be involved, a working party might need to be formed, appropriate consultation, and if they’re really smart they’ll make sure that someone important within the institution is publicly seen encouraging this goal.

I’ve seen the above approach used again and again. I don’t think I’ve ever seen it work to any great extent.

The behavioural change/empathy-driven approach

Using the process above:

Identify the good practice.
Done, we wish to increase the level of staff participation in a course site/forum.
Use the determinants to identify areas of weakness in the current context.
As an example, let’s assume that after an appropriate process we have identified the following:
- Skills: many staff don’t have the skills or insight necessary to increase their interaction with their course in effective ways.
- Environmental constraints: there’s nothing in the existing context that helps them perform this task. In fact, the rewards system tends to suggest that research is more important than teaching.
- Norms: there is little visible evidence of what the norms are in terms of appropriate levels of staff participation. Staff don’t know how much other staff are participating, they don’t know what is acceptable and what is not.
- Anticipated outcomes: many staff don’t see the connection between increasing their participation in a course site and benefits to students. They do see how increasing their level of participation increases the time they spend on teaching and decreases the time they spend on research.
Identify generic behaviour change techniques that target these determinants.
Techniques that target skills include: monitoring; self-monitoring; graded task, starting with easy tasks; modelling/demonstration of behaviour by others; goal/target specified: behaviour or outcome. Norms can be addressed by: modelling/demonstration of behaviour by others; social processes of encouragement, pressure, support; and prompts, triggers, cues. etc.
Use empathy-driven innovation to design specific interventions.
Embed into the LMS a graph that shows the academic staff members level of participation in their course site (self-monitoring) and compare it against the levels of participation in a group of related courses (social processes of encouragement). Have that graph show up whenever a staff member logs into the LMS. Include in the graph some easily visible representation of the connection between student failure rate and staff participation. Include a link “how to increase participation” to a page that outlines various techniques for increasing participation (prompts, triggers, cues) and includes comments or suggestions from other staff about how they did it (modeling/demonstration of behaviour by others).

Once this intervention is implemented, spend a lot of time observing what happens with the use of this intervention and make on-going change.s

Differences

Apart from the obvious, there is one major difference between these two approaches which I must make explicit. The traditional method spends a lot more time on the design of idealistic solutions that are disconnected from the everyday experience (e.g. the setting up and encouraging of people to enrol in a graduate certificate in higher education). Often, such approaches all but ignore the reality of the lived experience (e.g. policies that state a requirement but for which there’s been no work to support implementation). The traditional approach tends to be design heavy.

The empathy-driven approach, focuses on the lived experience, on what happens day-to-day. There is some design work, but much of the time is spent understanding what is going on everyday and trying to respond in an informed way.

The questions

It is still very early days for this work. There are lots of questions, here are just some.

How do you evaluate the level of determinants?

In the above example, I outlined some of my ideas of what values the behavioural change determinants around increasing staff participation on a course site might be. Basing the design on my thoughts is not a good start. It has to be based on a deep understanding of the experience/beliefs of the teaching staff.

How can you develop this understanding?

Are there survey instruments or other approaches from psychology that allow you to get some understanding of the current “level” of these determinants across an institution?

Which are the most important determinants?

Michie et al (2008) identify 7 sets of determinants, there are certain to be complexities within each one. In any large population you are likely to get widely differing results.

How do you figure out which of the determinants is most important or difficult?

Answers to this could help you guide the selection of which determinants to address and/or which of the techniques to adopt.

What’s the relationship between the determinants?

It’s likely that there would be relationships between the determinants. For example, if you increase skills in an appropriate way, I would imagine that this would change beliefs about self-efficacy, at least to some level.

What are the known relationships between the determinants?

What are the factors within determinants?

Fishbein et al (2001) talk about norms as being one of the determinants. Michie (et al) 2008 expand this out to include social influences, emotion and action planning. This is just one example, it is likely that each of the determinants embody a collection of factors. For example, self-efficacy might be a combination of characteristics of the person, the type of task and the previous experience of those involved.

What are the factors or sub-components of each of the determinants?

What types of behaviour are there?

Are all behaviours the same? Is a particular teaching behaviour the same as diet or how you control your children?

Are there different categorisations of behaviours with different characteristics?

What are the specifics of change techniques?

Michie et al (2008) just give a short title to these techniques. I imagine that each one is worthy of a literature that seeks to define, describe and evaluate interventions that have been designed on the basis of a specific technique.

What are the specifics of each change technique? What are the success factors? How do you choose which technique might be most appropriate?

References

Michie, S., Johnston, M., Francis, J., Hardeman, W., & Eccles, M. (2008). From theory to intervention: Mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology: An International Review, 57(4), 660-680.

Rogers, E. (1995). Diffusion of Innovations (4th ed.). New York: The Free Press.

Two types of process and what university e-learning continues to get wrong

By admin

On April 12, 2010

In Chapter 2, elearning, herding cats, phd, PsFramework, thesis, Uncategorized, webfuse

I should be writing other things, but there’s a wave amongst some of the “innovation bloggers” at the moment that I wanted to ride for the purposes of – once again – trying to get those driving university e-learning (and learning and teaching more generally) to realise they have something fundamentally wrong. They are using the wrong type of process.

I level this criticism at most of management, most of the support staff (information technology, instructional designers, staff developers etc) and especially at most of the researchers around e-learning etc.

For those of you at CQU who still don’t get what Webfuse was about. It wasn’t primarily about the technology, it was about this type of process. The technology was only important in so far as it enabled the process to be more responsive.

Empathy-driven innovation and a pull strategy

Over the weekend, Tim Kastelle wrote a post yesterday in which he proposes that a pull strategy is a key for empathy-driven innovation.

What is empathy-driven innovation, well Tim provides the following bullet points about empathy-driven innovation:

It requires a deep understanding of what the people that will use your innovation need and want.
Most organisational e-learning assumes that steering committees and reference groups are sufficient and appropriate for understanding what is needed. This is just plain silly. The people who reside on such things are generally very different in terms of experience and outlook than the majority of academics involved with learning and teaching. If they aren’t different at the start, the very act of being a member of such groups will, over time, make them very different. These groups are not representative.
What’s worse, is the support structures, processes, and roles that are created to sit under these committees and implement e-learning are more likely to prevent “deep understanding”, than help it. For example, different aspects of e-learning are divided along the lines of institutional structures. Anything technology related is the responsibility of the information technology folk, anything pedagogical is the responsibility of the instructional design folk and never shall the twain meet. As these folk generally report to different managers within different organisational units, the rarely talk and share insights.

E-learning is more than the sum of its parts. Currently, there is generally a large gulf between the academics and students “experiencing” e-learning, the technology people keeping the systems going, the instructional design folk helping academics design courses, and the management staff trying to keep the whole thing going. This gulf actively works against developing deep understanding and limits the quality of e-learning within universities.
Using empathy for the users of our innovations is the best way to create thick value.
A deep contextualised understanding and appreciation for the context of the academic staff and students helps develop truly unique and high quality e-learning applications and environment. Without it you are left with copying what every one else does, which is typically pretty limited.
We are creating ideas that entice people.
Almost all of university-based e-learning is based on push strategies. i.e. someone or group who is/are “smart” identify the right solution and then push it out onto the academics and students. They have to do this because their understanding of the context and need of the academics and students is small to non-existent. They decisions are based more on their own personal observations and preferences, or even worse, on the latest fad (e.g. e-portfolios, open source LMS etc.).
They aren’t creating ideas that entice people, they are creating ideas that people have to use.

Researchers are particular bad at this.
Innovations that pull are inherently network-based.
The idea is that to engage in empathy-driven innovation, you have to have connections to the people using the innovations.
As argued above, it’s my suggestion that the structures and processes around e-learning within universities are such that they actively work against the formation of these connections. To have empathy-driven innovation you have to connect the folk involved in teaching, learning, technology and instructional design in ways that are meaningful and that actively strengthen the connections.

At the moment, at least in my institution, there is no easy way for an academic to talk to a technical person that actually knows anything about the system, i.e. someone who can actively modify the system. The technology person can’t easily talk with someone with educational knowledge to better inform technological change. Instead each group retreats to talking amongst themselves. The necessary connections are generally only there in spite of the system, not because of it.

The Webfuse Thesis

I’m somewhat engaged in this discussion because I have seen, for a small period of time, this type of empathy-driven innovation work in the context of e-learning within a University. This is the topic of my PhD Thesis, an attempt to describe an information systems design theory for e-learning that encapsulates this.

At it’s simplest, the Webfuse thesis embodies two aspects:

Process.
There are two broad types of process: teleological and ateleological. I describe the two types here. Empathy-driven innovation is an example of an ateleological process. The table in the blog post describing teleological and ateleological mentions Seely Brown’s and Hagels push and pull distinction.
University e-learning is always too teleological, it needs to be more ateleological.
Product.
Everyone focuses too much on the actual product or system when we talk about e-learning. With Webfuse the product was only important in terms of how flexible it could be. How much diversity could it support and how easy was it to support that diversity. Because the one thing I know about learning and teaching within universities, is that it is diverse. In addition, if you are to engage in ateleological (empathy-driven) design, you need to be able to respond to local needs.
Most importantly, the question of how flexible the product is, is not limited to just the characteristics of the product. Yes, Moodle as an open source LMS implemented with technology (PHP and MySQL) that has low entry barriers, can be very flexible. However, if the organisation implements with technology with high entry barriers and inflexibility (e.g. Oracle) or if it adopts a process that is more teleological than ateleological, it robs Moodle of its flexibility.

Phd Update #26 – Getting to first draft stage

By admin

On April 9, 2010

In phd, thesis, Uncategorized

Two weekly PhD updates in a row, will be interesting to see how long this keeps going.

What I did last week

The aim for this week was to get chapters 2 and 3 of the thesis into first draft stage and sent to the supervisor for comment. If possible, get chapter 4 into the same stage.

2 and 3 were sent on Tuesday. Chapter 4 is about 3/4 of the way into first draft stage. Hope to have that done over the weekend.

The week has been slower than normal due to school holidays and other work.

The aim for next week

The time factor is going to raise its ugly head. Found out today that a deadline for a grant application is now a month earlier than I thought – it’s now May 4. On the downside, that means that much of the time from now until May 4 will probably be spent working on that application. On the plus side, I’ll have some more free time after May 4 for the thesis.

For the next week, the plan is to:

get chapter 4 into first draft stage;
get a rough version of chapter 6 completed; and
if possible, get started on chapter 5.

My ultimate goal is for submission by the end of June, or as close to that as I can get.

Reducing the aggravation of student blogging: The story of BIM

By admin

On April 7, 2010

In bam, bim, Uncategorized

In a couple of weeks I’m off to Canberra to talk PhD, potential ALTC grants and promote the use of BIM. As part of the latter task, I’m giving a quick talk at the University of Canberra as part of their Stuff the works lunches. The title of the talk is “Reducing the aggravation of student blogging: The story of BIM”.

I have to send of a short abstract for the talk today, so thought I’d share it here. I’ll use this post as the home for all the resources associated with the talk. The slides should be up sometime just before the talk and I hope some audio/video will follow not long after.

The basic aim of the talk is to share the why, what and how of using individual student blogs in teaching. The premise is that there is value in using individual blogs, but that it can require a bit of work and that BIM can help.

Abstract

There are many good reasons (reflection and meta-cognition, reducing transactional distance, increasing sense of ownership and community, ICT literacy etc) to encourage or require students to use individual blogs as part of their learning. However, the use of individual student blogs is not without its problems, which include: limited quality of LMS blogging tools, difficulties of managing externally hosted blogs, the question of how to mark and comment on student posts, the novelty of blogging for many students and staff, increased workload etc.

This session will tell the story of BIM (BAM into Moodle). A Moodle module designed to reduce the aggravations of supporting individual student blogging. Since 2006 BIM, and its predecessor BAM, have been used to support 2800+ students in 26+ course offerings creating 20,000+ blog posts. The session will show how BIM works and describe one approach to why and how it was used in one course. It will include discussion of the challenges and benefits of using BIM.

More information about BIM can be found here https://djon.es/blog/research/bam-blog-aggregation-management/

Slides

The slides that will probably be used in the talk are below. If things go well, I’ll have some audio/video of the talk up in the next day or so.

What are good designs/design principles for Moodle

By admin

On April 3, 2010

In curriculumMapping, elearning, moodle, Uncategorized

My current institution has adopted Moodle as its institutional LMS as of 2010. Due to my role, I haven’t really had to think about how you best go about designing a Moodle course. Now, however, due to the curriculum mapping project it is likely that I am going to have to engage with this. Hence the question, what are the different principles, guidelines or approaches for designing a Moodle course site?

Why do you ask?

A part of the aim of the curriculum mapping project is to map the alignment of course activities, resources and assessment against graduate attributes, learning outcomes etc. At the moment, the project is at the stage of experimenting with existing Moodle courses – course sites that are live now – and seeing how well (or not) Moodle’s existing outcomes support can be used for this mapping purpose.

I’m only looking at a very small number of courses, however, these are courses put together by academics who care about their teaching, who want to make an effort. From this small sample it appears that, as they stand, the design of these course sites will not easily enable the clear mapping of the activities, resources and assessments against outcomes. It’s clear that the design of these courses is very different, and that’s in spite of the the institution paying some lip service to consistency of experience for the students (which is misguided I think as in the end it only results in superficial consistency and more importantly fails to engage with a key characteristic of learning and teaching – it’s diversity).

It appears that, in order for the curriculum mapping project to fully enable the mapping of activities, resources and assessments against outcomes etc, it will need to make recommendations for better course design.

Which raises the question, what is good course design in Moodle?

What are some of the possibilities?

From one perspective, it is important that this be specifically about Moodle, and not e-learning or LMS course site design in general. Moodle, like any technology, provides a set of affordances, a set of strengths and weaknesses. To get the most out of Moodle, like any technology, the design needs to be aware of the Moodle sweet spot, and its weak spots.

Moodle principles

There are some things that really work in Moodle. What are they?

An obvious place to turn is the principles underpinning Moodle itself, which are:

We are all potential teachers as well as learners
We learn a lot by watching others.
We learn well by creating and expressing for others
Understanding others transforms us.
We learn well when the learning environment is flexible and adaptable to suit our needs.

Aside: would be interesting to map the content of courses with these 5 principles and find out how many follow them in some way. I think there would be surprisingly few. Following this evolution over time might be interesting as well. Do people become more informed about Moodle course design over time? Or, do they simply follow the same path they established for their first course?

Course structure or organisation

It’s my perception that the design of Moodle course sites is intended to be a sequence of sections which contain activities for students to complete. It’s interesting that one of the major “innovations” at my current institution is a “course design” that, to at least some extent, breaks this structure.

Rather than a long vertical collection of sections for each week, there is one section which breaks the course site up into a course synopsis and 5 horizontal sections – The course, resources, discussions, assessment and enquiries (which probably change depending on the course).

The courses not taking this approach seem to follow the same approach. One section as “About the course” – usually with a banner and general administration stuff – followed by the weekly sections. The content of those weekly sections is wildly different.

In part, the difference here seems to be between having lots of scrolling or not. A more typical Moodle design ends up with a couple of pages of scrolling. I’m hearing some positive responses from staff/students about the scrolling.

Is there research to see how it is received? Does the scrolling thing cause problems or benefits?

Look and feel

We’re superficial, something that looks good will often result in more immediate positive feelings, even though it’s a pain to use. A fair bit of the Moodle promotion stuff seems focused on showing that Moodle can be good looking. Even though most institutional Moodles appear to focus on consistency, rather than quality.

Learning design

More abstractly, a good course design should obviously – following the theory of alignment – be driven by the learning outcomes. With activities, resources and assessments chosen and presented in a way that best achieves those outcomes.

So, where are the good examples of good, constructively aligned Moodle course sites? What were the problem in achieving those designs?

The blended kitchen sink

One important question for the curriculum mapping project is whether or not the course site captures everything that all students experience. Where face-to-face is possible, it would appear obvious that there may well be some experiences that students have which are not captured in the course site. This suggests it won’t be captured in the mapping.

Should/can a course site contain everything, or just the online stuff?

Suggestions?

So, what say you? What are the other principles? What is out there that can inform answers to this question?

Where are the design exemplars for Moodle course sites?

At this point I’ll include a quote attributed to Stephen Downes from here

What makes e-learning effective is, of course, typically in the eye of the beholder. One person’s toast and jam may be another person’s steak and kidney pie. This is what makes the drafting of a set of guidelines for effective e-learning so difficult.

Which is just one reason why I think “one ring to rule them all” corporate approaches to web course design is a big mistake.

It’s also the reason why input from many is needed.

Suggestions from Google

Some suggestions from Google follow. Only did a single, quick search.

Blog post describing some alternate course formats for Moodle – related to the course structure/organisation heading above.
Moodle course design – a word document with good coverage of the topic
Interesting, makes the point that a consistent theme “gives it robustness”. I like the Dave Snowden distinction between robustness and resilience. Robustness tries to prevent mistakes/failure – which with people is itself destined to fail. Whereas resilience makes it cheap to respond/solve mistakes/failure. I know which I prefer.
For similar reasons the document advises against messing with the standard Moodle design – which is what the local “innovation” does.

PhD Update #25: A return to discipline?

By admin

On April 2, 2010

In phd, thesis, Uncategorized

The last PhD update I posted here was in early November last year. It’s time to get back into the discipline of posting these updates, especially now I’m in the downhill stretch.

The rationale/excuse for not having posted in the last 3 or 4 months has been work and Christmas. For most of November, I was traveling to or working on conference presentations. I also visited Canberra in that time to work a bit on the thesis and received feedback from my supervisor – reduce content!. December was holidays and then, the great plan to spend January on holiday, working on the thesis went pear shape. For various silly, contextual issues I spent most of January and February working on BIM so folk could use it starting around March. This will hopefully bring some benefits, but it didn’t help thesis completion.

What I did last week

Early last week I had completed rough first drafts of chapters 2 (lit review) and 3 (research method). Since then, when time allows, I’ve been re-reading these drafts and making the slight modifications that are needed. A bit of space – it’s been almost 3 weeks since I finished the draft of chapter 2 – does wonders for perspective.

I’ve re-read all of chapter 2 and annotated it with suggested changes. I’m almost half way through making those changes in the soft copy.

What I’ll do in the next week

The aim for the next week is to have completed first drafts of chapters 2 and 3 sent off to my supervisor.

Time willing, the plan is then to get chapter 4 into the same state. I have until April 6 to get chapters in this state and sent to my supervisor.

Moodle curriculum mapping – Step 3

By admin

On April 2, 2010

In curriculumMapping, Uncategorized

This will be a brief extension of previous work around this project. The main aim is to start identifying some of the methods used by Moodle with its current outcomes approach and how those might be harnessed and modified to support curriculum mapping. In particular, some specific questions include: What’s necessary to

allow the outcomes to be grouped and displayed as such when showing an activity/resource? IDENTIFIED
include a “help” link for each outcome or other means to explain? IDENTIFIED
allow the outcome scale to be used on the activity/resource to indicate how well the activity/resource meets the outcome etc? IDENTIFIED
display the curriculum map for a course?IDENTIFIED
add “outcome mapping” to those elements that currently don’t have it? IDENTIFIED
prevent curriculum mapping outcomes showing up in the gradebook?IDENTIFIED

This is a work in progress and will be updated over the next couple of days.

The association – where in code and the database

Moodle tracks which outcomes apply to activities and resources, the question is where in the code does this happen and where in the database is this information stored?

The code

The association appears as part of the edit screen for an activity or resource. This is implemented by moodle/course/modedit.php. This script:

Is given various params, including section and course, including the module being used to “edit” the activity/resource.
Is fairly typical PHP spaghetti code with little or no comments.
Acts has a harness/factory getting the module code to generate part of the form.
Has a section of code that retrieves and display the outcomes, all embedded in this enormous file – ugly.

The outcomes code seems to consist of (this is actually the handling of submission of the form, not display of the form – more on this below)

Get all the outcomes for the course (whether or not to display them, is left till then)
if ($outcomes = grade_outcome::fetch_all_available($COURSE->id))

fetch_all_available is implemented in moodle/lib/grade/grade_outcome.php. And basically defines a class that represents a grade outcome. fetch_all_available gets all course related outcomes listed in grade_outcomes (the detail of the outcomes) and grade_outcomes_courses (which outcomes are being used in the course).
Build array of grade_items
It then loops through each outcome from above and uses moodle/lib/grade/grade_item.php to create a grade_item object for each outcome. This uses the grade_items table to store information. Am not 100% sure where this fits in.
The actual display is done using a “form” display…more on this below.

So the display is done using the form class defined by the module, which is an extension of moodleform_mod. As the specific modules won’t know about outcomes, the outcomes display would theoretically be done in course/moodleform_mod.php. Yep.
[sourcecode lang=”php”]
if ($this->_features->gradecat) {
$gradecat = false;
if (!empty($CFG->enableoutcomes) and $this->_features->outcomes) {
[/sourcecode]

The process seems to be:

Get grade outcomes for the course, again
Seems there is some duplication here, as it gets the grade outcomes for the course, all over again.
Get’s all grade items for the course, it any of them have an outcome set, then set this in the form?
A couple of other steps here, not immediately clear.

The above only seems to be preparatory. There’s a later section of code that adds the form elements for the outcomes. Again, there’s a fetch all available outcomes. This seems more directly related as it simply adds the elements.

Where does it store “mapping”

The next question is where does it store the fact that a particular activity/resource is using/assigned a particular set of outcomes?

This should be set in the code that processes the submission of the form. Which should be moodle/course/modedit.php. Ahh, this is done with grade_item as described above.

i.e. when you map an activity/resource in a course to an outcome or three, that mapping gets stored in the grade_items table. The fields in that table are (the descriptions are tentative):

id – the unique id for the mapping of activity/resource to a single outcome.
courseid – the id for the course that “owns” this mapping.
itemname – this is the actual name of the outcome assigned
itemtype – I believe this describes the type of object you’ve mapped the outcome to. Possible known values are currently mod, course.
itemmodule – the name of the specific module that implements the object. Possible values include: forum, bim (i.e. name of any module), assignment, resource.
iteminstance – I believe this is the id for this particular instance of the module. i.e. the id for the table course_modules. The pathway to more information about this instance.
itemnumber – for outcomes, this seems to start at 1000. It is used to give the sequence with which outcomes are assigned to the item. i.e. the first outcome assigned is 1000, the second 1001, the 3rd 1002 …. It appears that a value of 0, might indicate something important
iteminfo – currently set to NULL for all the entries I’ve seen so far. So, not currently sure what it is used for.
idnumber and calculation – also set to NULL or empty for the contents of my database – which doesn’t include a lot of real courses.
gradetype – integer, currently with value of 1 or 2. With outcomes I’ve set being 2.
grademax and grademin – fairly obvious. Seems to be set by scale and/or other stuff.
scaleid – the scale being used.
outcomeid – the id of the outcome
gradepass multfactor plusfactor aggregationcoef – various factors used for grade calculation, I believe.
sortorder – different integer values – purpose not immediately obvious.
display – big int, currently all set to 0. Not sure of purpose.
decimals hidden locked locktime needsupdate – various flags ?
timecreated timemodified – time stamps. Could be useful for identifying outcomes that need to be re-checked.

It appears that grade items and outcome items are treated the same, hence their use of the same table. The full view of categories and items give a good overview of this table.

There is the concept of categories of grades/items, this might be one avenue. i.e. a category for curriculum mapping.

What is the implication of this?

The next question is what are the implications to the rest of Moodle. If I map all the activites/resources within a course against a complex set of outcomes, does it have an effect on the gradebook? Any where else?

So, I’ve set outcomes for a number of activities/resources in a course. Does this show up anywhere else? Two ways of looking:

Check the gradebook from web interface.
Look for use of grade_item class/object.

Mmmm, not good. It appears that every time you add an outcome, it get’s added to the gradebook. In terms of curriculum mapping, not what is desired. This is perhaps the first obvious example that curriculum mapping and tracking student performance, while to some extent similar, serve different purposes.

The column in the gradebook for each outcome that is added, has a header that is a link. The link is to a script that shows some detail of the resource or activity that it the outcome is associated with.

Need to turn this off.

Now, you can hide an element in the gradebook. But that just greys it out, doesn’t remove it entirely from consideration, which is what is wanted here.

Adding a description/help

Problem

At least initially the outcomes etc shown are not going to make much sense to a teacher. Moodle currently only displays the name of the outcome. The teacher would have to somewhere else to read up on the outcome before they can determine if it applies. It would be helpful if additional assistance was provided there and then.

Some options in terms of what could be displayed, include:

The description of the outcome.
As it stands Moodle allows each outcome to have a textual description. Displaying this as a roll-over or in a new window could provide a minimal level of assistance.
Link to institutional area for discussion and description of outcomes.
The assumption being that most institutions would have a website in which institutional outcomes etc are discussed or described. Providing a link to this area, especially to the context specific to the a particular outcome might be useful.
Link to other examples.
Many of the forms of outcomes etc. are likely to be used in other courses. e.g. institutional graduate attributes. It might be useful to give the option to see other examples of how these attributes are used in other courses. Even to the extent of link directly to those courses and/or reflections/discussion from other teachers using this outome.

These ideas range from the simple and static through to something you’d want to have some curating.

Possible solutions

The outcomes are displayed around line 220 in moodle/course/moodleform_mod.php. This is where the change would have to happen. Some possibilities include:

Using a Moodle helpbutton.
Moodle forms have a function – setHelpButton – which associates a help button with an element. Very easy to make this modification. However, the problem is that the helpbutton is typically a call to open a new, small browser window to display HTML file.
This is problematic as the outcomes are added via the Moodle interface and doesn’t provide support for adding a help file. So, outcome specific would be difficult. However, an institutional area/approach could be possible. It would require the institution to create HTML files for each outcome.

Let’s do a simple test, put the Moodle code under git so I can manage this. And add a help button for each outcome. As expected, works easily. There is the question of how to create the filename for the HTML file. Most outcomes will have spaces and other characters that don’t necessarily play nicely with a filename. The language translation side of Moodle could help there, convert the complete outcome name into something more file system friendly.
More complicated HTML
Another approach would be to add roll overs, additional links etc to the outcomes. This would require a more radical modification of the Moodle core but not much more than the above. Not to mention the desire to separate attributes up into groups.

Groupings of outcomes

Problem

It is likely that a course may have multiple different types of outcomes etc to map against. e.g institutional graduate attributes, discipline graduate attributes, course learning outcomes, program learning outcomes etc. There are two possible solutions (possibly complementary):

Show outcomes grouped by category.
To allow the mapping of an activity/resource against all these different groupings, it would be useful to separate out the different outcomes by category. So you could have a visible separation.
Have a separate cross mapping.
Mapping against all of these different outcomes might be somewhat tiresome, especially given a large amount of overlap between them. An approach that has been used is to produce a mapping between the different outcomes and a single point, and then only map activities/resources against that single point. Which of the different outcomes applies, can then be derived from the single point.

Possible solutions

Showing outcomes by category is going to need:

Some way of specifying categories/groups of outcomes.
Which probably implies an additional database table and an additional interface or modification to an existing interface (e.g. the import outcome process) to specify which category an outcome belongs to.
A separate interface minimises changes to core Moodle code, modifying existing interfaces is probably a more user friendly approach, depending on how widespread this need is.
Modification to the form display to recognise the categories.
This should/would be a fairly simple thing to do, given the information above and Moodle’s form library.
Let’s try a simple test. Create two boxes of outcomes that contain a copy of the same outcomes. Mainly to test if nested header/boxes work. No, they don’t. You’d have to use a separate header label and then have separate boxes for each, perhaps a table? Though Moodle dislikes table for layout…..

Display a curriculum map for a course

Problem

One of the basic functions for curriculum mapping is to get a report that shows how widely (or not) the outcomes are represented within the course in terms of resources, activities and assessments. i.e. you want a visual representation of the outcome mappings.

Possible solutions

Well, it’s basically a report, but you might want it more interactive than that.

Well, the existing outcomes report can do this to some extent. So an extension of this, or the additional of a mapping report might fill the bill.

To a large extent this is a fairly standard web application. Get the data from the database and display it in an appropriate form.

You’ll be needing data from the following tables:

grade_items – given a course id, this will give you all the outcomes for the course that have been mapped to activities/resources and the ids of those activities and resources.
grade_outcomes – will give you details about the outcomes – name, description, and scale id.
scale – details about the scales
course_modules – more information about the module/activity, most importantly perhaps the section of the course in which it appears.
resource – for the same reason as modules

Show outcome scale on activity/resource

Problem

Rather than simply “mapping” an outcome to a particular activity/resource, it may be useful to indicate how well/to what level does the activity/resource map to the outcome. i.e. use a scale, rather than a simple check box.

This is a fairly major distinction between outcomes for curriculum mapping and outcomes for student progress.

Possible solutions

It’s looking like a separate set of “Mapping outcomes” might be the way to go. This would also get around the problem with the gradebook from above. This might mean duplicating the items table, or at least adding a flag to differentiate between mapping and progress outcomes.

Similarly, could probably still use the standard outcomes “creation/import” process for both purposes.

Adding separate support would also help make it a bit easier to add curriculum mapping to an instance of Moodle by minimising disruption to the Moodle core.

Elements that don’t have outcomes

Problem

As outlined earlier there are some elements of a Moodle course site to which you can’t map outcomes. The outcomes don’t appear on the “edit” page. Those identified so far are labels and sections.

Sections might be useful, if you wanted to map a course by weeks, rather than by item. But perhaps not, you can generate such a map by aggregating the mapping of the contents.

Labels are way of inserting HTML into sections. Currently they don’t have support for outcomes. I’ve already seen in one course how such labels can be used to specify tasks, such as reading.

Possible solutions

Well, labels are the only real problem. The form for labels is generated using moodle/course/modedit.php. The same for anything else. It is the place where outcomes are shown. So, perhaps it’s just a switch that needs setting. Perhaps, outcomes aren’t here as it isn’t expected that these will be used in grades – i.e. student progress tracking.

Nope. The mod_form.php file for label actively turns off outcomes in a setting. Yep, set that to true and outcomes are there.

In light of the above, you’d probably have a separate set of outcomes for mapping, have this defined as a feature that modules can turn on/off and go down that route.

Elevator pitch for "Moodle curriculum mapping"

By admin

On April 1, 2010

In curriculumMapping, Uncategorized

Over the next few weeks I am going to have a few meetings in which I’ll need to explain and justify the curriculum mapping project. This post is an attempt to further develop and share some of the perspectives and to develop an elevator pitch for the project.

It’s also reinforced the need to develop a name for the project. “Moodle curriculum mapping” doesn’t really send the right message.

Aside: I’m somewhat skeptical of the value and possible side effect of an aligned curriculum and curriculum mapping in general. (e.g. I’m not convinced institutional graduate attributes are possible or sensible, I’m not convinced that a mapped curriculum gives a true representation of what the student will actually learn/experience). However, I can see that this is becoming very important within Australian universities and that it will be done. I’m hoping this project can take the implementation of curriculum mapping in more interesting and useful directions. Time will tell.

Elevator pitch

The elevator pitch is formed by the sub-headings of the specific aims section. The “specific aims” section was written first.

At its simplest, the project aims to build on existing practices around curriculum mapping and fulfill existing needs, such as accreditation. More importantly, the project aims to make learning outcomes, graduate attributes etc a part of the everyday practice for a majority of academics in a way that is useful. As a result, it should help improve the validity of curriculum maps, encourage greater quantity and quality of use of the LMS and show how the institution is leading the sector. Most importantly, the project aims to provide a foundation that enables CDDU to more effectively engage with academics and, in broader terms, enable and encourage improvements in teaching and student learning outcomes.

It will do this through modifications to the Moodle LMS that aim make learning outcomes, graduate attributes and the alignment of those with learning activities, resources and assessment a “first class objects” within Moodle. These modifications, and more importantly, the processes used to roll them out will be designed to encourage and enable adoption and behaviour change.

Difficulties

The difficulties associated with doing this is that my overall focus for the project is around improving learning and teaching, which is a complex task with lots of connections. When people talk about curriculum mapping, they are typically talking about many different things, from many different perspectives.

Even more difficult is the fact that I am approaching this from the perspective of intervening in a complex adaptive system. This means that I don’t (and I believe, I can’t) have a firm idea of exactly where this project is going to go. This is because any fixed outcome is a waste of time, as the project proceeds we will learn more and the system (i.e. the university, its staff and their practice of L&T) will change around the project. What we think is a good thing to do in 6 months time, will be very different from where what we think would be good now.

And this is exactly the sort of “meta-discussion” that many of the people I’m going to talk to, will hate.

Some specific aims

So, let’s start with some specific aims.

Terminology:

“outcomes etc.” – used to encapsulate the broad array of “stuff” people want to map, including: university graduate attributes, course learning outcomes, discipline outcomes/attributes etc.
“course” – the smallest unit of study in which a student enrolls. Lasts for around 12 weeks.
“program” – a collection of courses that form a degree.
“majority” – the following often makes statements like “the majority of academic staff don’t teach well”. This is not a deficit model of academic staff. In the vast majority of cases the “don’t teach well” is due to contextual issues. I don’t blame the teacher.
There are a small number of academic staff who teach very well. They typically do this in spite of contextual issues and because they are intrinsically motivated. This group are in the minority.

My interest is in helping change the context so that the majority of academic staff are enabled and encouraged to improve their teaching.

Practical outcomes, reuse and a foundation

Just about every program that needs to be accredited has to generate some form of curriculum map, usually to meet the requirements of the external accrediting body. Increasingly, Australian universities are being required to demonstrate the presence and use of graduate attributes, typically illustrated through curriculum maps.

Proposition: This project will provide the functionality required to generate these and other curriculum maps.

There is existing work being done to generate these curriculum maps using established approaches (i.e. Word documents and spreadsheets). Rather than waste this work, these need to be used as inputs into this process.

Curriculum mapping is not a ends unto itself. It is typically part of a process used to increase understanding of a course and in particular its alignment and its relationships with other courses. That improved understanding informs subsequent action.

The intent is that this project is not simply about curriculum mapping, but it is focused on how the project can provide a foundation to enable and encourage subsequent action.

Make it part of everyday practice

Curriculum mapping is based around the idea that having alignment between the outcomes etc. and the learning activities, resources and assessments within a course is a good thing. The trouble is that as currently implemented outcomes etc are not part of the everyday practice for academic staff. Most academic staff, when planning a course, don’t think about outcomes etc. and alignment.

Support for this perspective comes from one of the few empirical examinations of academic practice that I’m aware of, the work of Stark, Lowther et al (1988), Stark et al (1990), Stark (2000), and Lattuca and Stark (2009). Some choice quotes from my thesis drawing on this work

How academics design their teaching is not described by a rational planning model (Lattuca and Stark 2009). In part, this is because the dominant setting for academics is teaching an existing course, generally one the academic has taught previously. In such a setting, academics spend most of their time fine tuning a course or making minor modifications to material or content (Stark 2000). Academics are usually not often required to engage in the development of new courses or major overhauls of existing courses (Stark and Lowther 1988). The practice of most academics does not separate planning from implementation, and rather than starting with explicit course objectives, starts with content (Lattuca and Stark 2009).

In part, this is because consideration of outcomes etc. are not a part of what many of them do around learning and teaching. That is, the LMS they are using doesn’t usually provide any support or recognition of outcomes etc. It’s not part of what they do in face-to-face L&T. To some extent, it’s a case of out of sight and out of mind.

This problem is made worse in institutions where an increasing number of courses are being taught multiple times a year. In my local context, this means that rather than an academic designing and teaching the course and its assessment. The design is done by the person currently teaching the course, and the next person teaching the course has to live with that design.

This becomes an increasing problem when many of these staff are contract staff employed to deliver the course. They don’t design the course, so the only place they see the outcomes etc. are in the small section of the course profile/syllabus.

Proposition: If alignment between outcomes etc, learning resources, activities and assessment is a good thing, then making outcomes etc and their relationship with learning resources, activities and assessment a highly visible and first class component of the LMS/learning environment is necessary to increase alignment, or at least consideration of it.

Improve the validity of the mappings

As argued above, outcomes etc and alignment is not a key component of the thinking of most academics. It has been widely recognised for some time that the validity of the mappings represented in static documents is somewhat questionable. Some more thesis quotes

In the absence of formally documented teaching goals, the actual teaching and learning that occurs is more in line with the teacher’s implicit internalised knowledge, than that described in published course descriptions (Levander and Mikkola 2009). Formal descriptions of the curriculum do not necessarily provide much understanding about how teachers put their curriculum ideas into action (Argyris and Schon 1974).

Curriculum mapping is typically done in a way divorced in time and space from the context within which academics teach. Subsequently, due to the known limitations of human memory – not to mention pressure to comply – it is unlikely that such curriculum maps capture the full complexity what occurs in a course. Given the static nature of such maps and their lack of use in everyday teaching, then over time the validity of their representation is only going to decrease.

Proposition: Curriculum maps that are generated, and continue to evolve, in the same time and space as everyday teaching and learning will have stronger validity in terms of capturing reality and consequently be of more value.

Enable and encourage improvements in teaching

The mapping process is fairly straight forward, if you are familiar and comfortable with a lot of the educational language involved in the process. A lot of academics aren’t. Some more thesis quotes

In the absence of formal qualifications or knowledge in learning and teaching, most academics teach in ways they have been taught (Phillips 2005) and/or which fit with disciplinary norms and their recent teaching experience (Entwistle 2003). Academic staff rarely read educational literature or call upon any available expert assistance when planning a course (Stark 2000).

A well designed mapping process would provide the scaffolding necessary for academic staff to be guided through the mapping process. It would provide pointers to similar maps, explanations of why it was done a certain way, reflections and insights from other teaching academics (not instructional designers) etc.

More importantly, the mapping process is the educationally easiest part of this process. What’s much more difficult is, once you have the map, using that to improve the alignment of the course learning resources, activities and assessments. Knowing how best, within the specific context, to provide students with the opportunity to practice and receive feedback on “critical thinking” is far more difficult than identifying where it is or isn’t provided.

The aim here is not just to help academics map the course, but then build on this to enable and encourage them to improve their courses.

Proposition: A learning environment that makes visible to all stakeholders the alignment (or not) of a course and then provides scaffolding necessary to improve that alignment will help improve teaching.

Encourage greater usage (quantity and quality) of institutional LMS

A lot is written about the poor quantity and quality of the learning and teaching that occurs within an institutional LMS. The vast majority of course sites are little more electronic photo-copiers, places to disseminate text. This is due to a variety of issues, most of them contextual and nothing to do with the LMS. However, some are due to the nature of the LMS and the types of tasks it makes easy (e.g. uploading some powerpoint slides) and the types of tasks for which it has no support (making visible and offering advice on how to improve course alignment).

Proposition: Well designed extensions to an LMS that encourage and enable improvement of course alignment will increase the quantity and quality of usage of the institutional LMS and subsequent student outcomes.

In a more institutional specific aspect of this aim, is the observation that Moodle (my institutions LMS) already offers support for tracking student progress against outcomes. However, this feature, which is optional, is not even enabled within the institutional instance of Moodle, and is currently not being used. Even though, there are parts of the institution that want to use this sort of feature.

Proposition: Building curriculum mapping around Moodle’s student “tracking” functionality will enable and encourage greater use of the student tracking functionality.

Demonstrate innovation and leadership

Within my own institution I have heard Deputy Vice-Chancellors ask “Where is all the innovative learning and teaching? We used to be at the fore-front.”
I have seen universities claim how innovative they are being around curriculum mapping because they are moving from using a Word document as the course curriculum map, to using an Excel spreadsheet. It’s not hard to be innovative in this area.

Proposition: The ideas described here are innovative and if successfully implemented can enable the institutions involved to demonstrate leadership within the sector.

Enable the CDDU to engage more effectively with academics

I work for the Curriculum Design and Development Unit (CDDU) at CQU. While I am not a curriculum designer, curriculum design is a key part of what the unit does. For me, the overall aim of curriculum design is to help academics improve the quality of their teaching and their students learning.

Current approaches to curriculum design have little impact. Arguably, this is because of a number of reasons, including – but not limited to:

Mismatch between the instructional design process and how academics plan courses.
As outlined above, the majority of academics don’t use a rational planning model for course design. Instructional design is typically guided by a rational planning model. This mismatch is incredibly difficult for most academics to bridge, or even understand. The mismatch, in many cases, limits outcomes.
Help arrives outside of the context of need.
An academic usually has a problem or question about teaching, while they are planning or teaching (which as argued above, is typically not separate tasks for academics). Instructional design assistance is typically not available within this context of need. Instead the academic must remember their need, at a separate time remember to ask for assistance, and then try to remember and explain the context of need to the instructional designer. Is it any wonder academics don’t draw on expert help?
Instructional designer as the police or fire brigade.
The only time you need the police or fire brigade is when you are in trouble. The association people build up of these services is, trouble. Academics can form a similar impression of instructional designers, we only see them when I have a problem. It’s made worse in cases when management directs the academic to see the instructional designer.

Proposition: Properly implemented, this approach can make it easier for curriculum designers to embed assistance into the context within which teaching is taking place. If this works well, relationships will develop.

Specific project stages

While the specifics of the project are up in the air, not the least because of contextual uncertainties, it is possible to identify a collection of likely project stages:

Explore what is possible with Moodle.
Where I am now. Playing with Moodle and its current outcomes functionality to see how it works and where the limits are. To find out what might need to be done, what are the limitations.
Talk with external partners.
In mid-April I’m going to Canberra to talk with folk at University of Canberra and find out what there interest is around this topic.
An initial local trial.
Half in this at the moment, but more progress once the “exploration” stage is complete. Work with a couple of programs to get some initial “mappings” done. This might involve a bit of rough coding to enable simple practices. Again, getting more of an idea of the project and what needs to happen. The purpose of this stage is to generate something concrete to show people what we’re talking about, rather than rely on abstract hand waving.
Some “innovative” applications.
Arising out of the last stage, the aim will be to generate some new applications out of the initial trial to illustrate what might be possible. To really show that this isn’t just about curriculum mapping.
Initial publications
Around about this stage we should be in a place for an initial publication or two, to get the word out.
An ALTC grant application.
This is by July this year.
Contributions to the Moodle community.
Eventually, when/if we produce something useful. It has to be given back to the broader community.

References

Lattuca, L. and J. Stark (2009). Shaping the college curriculum: Academic plans in context. San Francisco, John Wiley & Sons.

Stark, J., M. Lowther, et al. (1988). “Faculty reflect on course planning.” Research in Higher Education 29(3): 219-240.

Stark, J. and et al (1990). Planning introductory college courses: Influences on faculty. Ann Arbor, National Center for Research to Improve Postsecondary Teaching and Learning.

Stark, J. (2000). “Planning introductory college courses: Content, context and form.” Instructional Science 28(5): 413-438.