Month: February 2010

What follows is an attempt to develop some of the foundations of what is driving the work I do in my current position.

The main aim

My main aim is to help improve the quality of learning and teaching within the university I work for.

The fundamental flaw

It’s my suggestion that almost all of the current institutional attempts to improve learning and teaching suffer the same fundamental flaw. They attempt to herd cats, they focus on what management does, rather than focus on what the teachers and learners do every day and in particular the tremendous problems created by the systems within which they operate.

To be a little more concrete,

Senior management embark on vision statements etc. exclaiming the importance of teaching.
Senior management seek to identify broad-scale, one-size-fits-all solutions, e.g. we’ll have a common course template, a single system etc..
Institutions develop high-level policies about how courses/programs should be designed, but pay little attention to actually how these tasks are carried out leading to problems and compliance.
If everyone has a certificate in higher education, then they will be good teachers.

At the same time, there is ignorance of (though it’s probably a case of “don’t mention the war”) the complete and utter mismatch between the system/environment in which academic staff work and improving L&T. A few references/quotes from this post

Academic staff are trained, selected and evaluated on the discipline expertise and their ability to perform quality research. The experience and training of academic staff not only focuses on discipline and research expertise it can, and often does, socialise aspiring academics towards a vision of academic work that emphasises these tasks (Austin 2002). While universities promote the importance of teaching the create ambiguous, even contradictory expectations by rewarding academic staff primarily for research (Zellweger 2005) and creating environments where spending more time teaching is a negative influence on academic pay (Fairweather 2005).

But it’s more than that, I would argue that even if we changed the rewards aspect. For example, if a university effectively introduced a practice where good teaching staff could get promoted solely on the quality of their teaching, there would remain little change. The very nature of the system and the subsequent day to day experiences of teaching staff are blocking improvements in learning and teaching.

My point overlaps in part part with the point being made by George Siemens in this post

More than any other element, this systemic inertia is responsible for limited innovation in education.

The processes within most universities are equivalent to a group of obese people with chocolate cravings living and working in a chocolate factory. Day to day they are being exposed to influences and practices that actively work against them loosing weight.

The solutions being proposed for learning and teaching within universities are a bit like addressing the chocolate problem above by:

Allocating the each group of these big folk with a nutritionist that will explain to them the requirements of good nutrition.
Employing special expert nutritionists to come and talk to the big folk about various strategies for good nutrition.
Providing a single menu and exercise regime that all big folk are expected to follow.

You have to change the nature of the system so that it encourages and enables people to change what they do.

Nudging and choice architecture

I’ve been reading Nudge: Improving decisions about health, wealth and happiness for some time. A review of the book with some connections to higher education was recently posted on Tomorrow’s Professor.

The review suggests that the book is based on two key foundations:

There is no such thing as neutral design.
Whatever system, process or tool you are talking about it embodies a design. The design may not be intentional, it may be a result of unconnected decisions. The big point is, however, that the nature of that system will influence the likelihood of particular outcomes.
Obviously this connects with the point I’m trying to make above that the design of the current teaching system within universities are more likely to result in bad outcomes, rather than good.
People do not act rationally.
Most systems, processes and policies assume that people make perfectly rational decisions. They don’t. People have numerous, well-known flaws in their abilities to make rational decisions.

The book talks about how “choice architecture” can be used to nudge people towards doing good things. The book takes an approach called “libertarian paternalism” where people are not forced to do the right thing, they are free to do what they want. However, they assume that it is okay, and in fact a good thing, to design the choice architecture in order to influence people’s behaviour. Which brings up the definition of a nudge (Thaler and Sunstein, 2008; p6)

is any aspect of the choice architecture that alters people’s behaviour in a predictable way without forbidding any options or significantly changing their economic incentives.

The centrality of the pedagogue

When it comes to improving teaching and learning a, if not the, major role is played by the individual pedagogue/teacher and what they believe. At some stage, the teacher makes a decision to improve or change their practice, or not. My argument is that the “choice architecture” inherent in universities is set up to actively discourage people to make the decision to improve their teaching and learning.

If you don’t get them to make this decision, the outcome of whatever you do will be limited.

Principles of good choice architecture

Thaler and Sunstein (2008; pp102) give six principles for good choice architecture:

iNcentives;
They suggest four questions about incentives – who uses? who chooses? who pays? who profits – and outline some examples of how naive incentive setting fails.
For example, the question of salience or do the people making choice actually notice the incentives they face. A case could be made here between the “incentive” of teaching awards to improve learning and teaching not being noticed because of the broader incentive to do research.
Understand mappings;
A decision involves a mapping between the choice and the ultimate consumption experience. How do people making choices make rational decisions about the ultimate experience of their choice?
One example in L&T might be the decision to try something innovative. What if people map the attempt to try something new with the possibility of failure and then censure from the “standards focused” management? Are they likely to try something innovative?
Defaults;
People will usually take the path of least resistance, they will do the easiest thing, the default. The default in L&T within higher education is passive lectures, tutorials, a couple of assignments and exam. The entire system of L&T is setup to enshrine these defaults.
Give feedback;
Thaler and Sunstein (2008; p92)

The best way to help Humans improve their performance is to provide feedback. Well-designed systems tell people when they are doing well and when they are making mistakes.

In my experience feedback to academics about their L&T is minimal to non-existent.
Expect error;
People will make mistakes, the system should be designed to expect this and be as forgiving as possible. I would suggest this isn’t high on the agenda in current structured, corporate approaches to teaching.
Structure complex choices.
People adopt simplifying strategies when faced with complex choices (go with the default etc.). Those simplifying strategies are often flawed in some way and directly influence outcomes.
What assistance does the L&T system within a university provide to help academics make choices about L&T?

Using these principles

While not wishing to be exclusive about these principles, there would seem to be strong support for using these to help guide the design of changes within a university system for L&T. I’m hoping to do just this in the choice and design of projects I work upon. For example:

Indicators project and;
I’m particularly interested in how this project can be used to give feedback to teaching staff about what is happening within their online courses and how this feedback can be used expect error, structure complex choices, understand mappings and develop incentives through a collaborative process of reflection.
bottom up curriculum mapping
I’ve been thinking about a bottom up approach to curriculum mapping. My main interest is how to modify slightly existing practice to develop curriculum maps within a program that can give feedback to participants about the problems in a course. In addition, there are possibilities that such an approach can be used to structure complex choices – be developing “wizards” that help guide the solution to identified problems – and perhaps help change some of the defaults.
The old REACT idea.
This was intended to give feedback, help structure complex choices and understand mappings, while at the same time attempting to provide some incentive.

References

Thaler, R. and C. Sunstein (2008). Nudge: Improving decisions about health, wealth and happiness. New York, Penguin.

BIM – Tidy up #3

By admin

On February 1, 2010

In bim, Uncategorized

Next step in tidying up is to clean up the checking, calculation and setting of the blog and feed URLs as students register their blogs.

First, fix up some problem with markers without any students.

Unregistered students

Which highlights a problem with coordinator seeing unregistered students. Add this information to the Manage Marking page.

Process is: get all student details, call the appropriate functions.

Now there is a bim_get_all_students, but it currently relies on bim_markers_students. Need to replace that.

Let’s use it first and get that bit working. Done.

Now time to replace the kludge with something that will actually get all students in a course. I’m guessing that get_users_by_capbility is the one to use here. Done.

Checking of blog URLs

One of the biggest areas for work is students who make mistakes when registering their blog. BAM does a range of tests including:

Can the URL actually be retrieved.
Does the URL include a link to RSS feed – done via auto-discovery.
Can the feed (any one of them) be retrieved.
Can the feed be parsed as XML

At the very least these checks need to be made.

Additional checks that need to be made, based one experience:

The feed is not the wordpress.com feed
Students can choose to register something a bit too early in the blog creation process that returns the main feed from wordpress.com.
They register the edit screen of WordPress e.g. http://myblog.wordpress.com/wp-admin/edit.php.

The checking is one stage, the other is how to communicate these problems back to the students.

So, checks to implement:

~~Is it even a valid URL.~~
Can the URL be retrieved
Can we get some auto discovery feeds
If not, is the URL itself a feed?
If there are feeds, loop through each one, and work with the first one that
- We can retrieve
- Is a feed
- And doesn’t break any of the exclusions we know of

Interestingly, SimplePie makes this process much simpler in that it does much of the checking itself.

Should also look at client side checking i.e. make the form stop if the student hasn’t put in a URL. Do that later.

Testing SimplePie

User enters blog url (not RSS) – e.g. http://davidtjones.wordpress.com – then permalink is blog and subscribe url is feed for both WordPress and blogger
User enters feed URL – e.g. https://djon.es/blog/feed/ – permalink is blog and subscribe url is feed. WordPress and blogger.

So, it seems I can trust SimplePie to do this and simply check permlink/link and the subscribe link. Need to check that both URLs are gettable and that the subscribe url is actually RSS.

Simply getting a page that doesn’t have a feed, will generate an error.

So the logic appears to be:

Create SimplePie object with URL provided by user.
Any errors, crash out.
Compare the permalink and subscribe URLs
If the value passed in is not the subscribe URL, get it??? No, it looks like SimplePie automatically gets the RSS, so if it can’t it would generate an error. Will need to test that. Yes, that’s what happens

Client side validation of URL

The form for registration uses Moodles formslib which in turn is based on the Pear library which has some documentation around validation.

Client side validation is generally done (I think) with the addRule method. Pear doesn’t seem to support a URL type, but Moodle does.

That seems to have too much difficulty for now.

Processing and allocating the feed elements

The last major task is to process and allocate items in the feed. This needs to be called on an individual blog when it is first registered, but also via cron at the fixed time.

Given a single student feed, the process is:

Get the elements in the feed.
For each element in the feed
- Check to see if there is an existing feed in the database for this item, this is based on the permalink for the item stored in bim_marking
- If there isn’t an entry, prepare to put one in.
- Loop through each unallocated question for this student
  - If the post seems to match the question, modify the entry for the post.
- Insert the new entry

Implementation and a need to re-use this code for the cron job that mirrors/processes registered blogs has resulted in a bit a re-factor. bim_process_feed( $bim, $student_feed, $questions ) is up and going.

In terms of text similarity there appear to be a few in-built PHP functions that might be helpful later on. This Pear class could also be a great help.

Get and test the cron updating

Okay, the cron code has been updated with the new methods. Rather than run from cron I want to see what’s going on. So going to modify the default “config” page for a coordinator to run the cron function.

It seems to be working, but to be truly sure I need to start looking at some individual student entries and how they change. For that, I need to add the student search facility that is currently missing.

Delete a small number of entries from bim_marking for students.

Run the cron thing to see if it works. Done.

Student search facility

Coordinators need to be able to search through all students and view their details. A marker doesn’t need to do this as they have a list of all their students shown and they can simply use the browser find facility. A coordinator, theoretically, has their own students as well as a responsibility to look after all students. Hence they need to be able to enter a student number and see their details.

Implementation plan:

Add a “Find student” tab.
Implement a simple form that takes details about students (try to do something like MyCQU in terms of making it simple).
Display specific student details or a list of potentials to search from.
Use the standard “student details” function to show the details.

Done, which is now bringing us back to the question of the item titles which is problematic because of the changes in what’s being stored in the database…

Limiting use of xml files – all in database

A difference between BIM and BAM is that BIM will be storing all posts made to the student feed in the database. BAM only stored posts allocated to questions. This caused two issues:

The ability for markers to “unallocate” posts meant that there were some unallocated posts in the database. This starts to create a duplication/difference. Some unallocated posts are in the database, some are.
Feeds don’t always provide a history of all the posts. So some might be lost. Not really a problem with current use as most students didn’t make enough posts.

By making the decision to put all posts in the database, there’s a need to update some early design decisions that were made on the basis of the BAM design.

Add post title to bim_marking

Because all the posts are going into the database. Need to add the title into the database table bim_marking.

Done.

Week 10 – don't know why I'm testing ignore