Documenting the gap between "start of art" and "state of the actual"

Came across Perrotta et al (2013) in my morning random ramblings through my PLN and was particular struck by this

a rising awareness of a gap between ‘state of art’ experimental studies on learning and technology and the ‘state of the actual’ (Selwyn, 2011), that is, the messy realities of schooling where compromise, pragmatism and politics take centre stage, and where the technological transformation promised by enthusiasts over the last three decades failed to materialize. (pp. 261-262)

For my own selfish reasons (i.e. I have work within the “state of the actual”) my research interests are in understanding and figuring out how to improve the “state of the actual”. My Moodlemoot’AU 2013 presentation next week is an attempt to establish the rationale and map out one set of interventions I’m hoping to undertake. This post is about an attempt to make explicit some on-going thinking about this and related work. In particular, I’m trying to come up with a research project to document the “state of the actual” with the aim of trying to figure out how to intervene, but also, hopefully, to inform policy makers.

Some questions I need to think about

  1. What literature do I need to look at that documents the reality of working with current generation university information systems?
  2. What’s a good research method – especially data capture – to get the detail of the state of the actual?

Why this is important

A few observations can and have been made about the quality of institutional learning and teaching, especially university e-learning. These are

  1. It’s not that good.

    This is the core problem. It needs to be better.

  2. The current practices being adopted to remedy these problems aren’t working.

    Doing more of the same isn’t going to fix this problem. It’s time to look elsewhere.

  3. The workload for teaching staff is high and increasing.

    This is my personal problem, but I also think it’s indicative of a broader issue. i.e. much of the current practices aimed at improving quality assume a “blame the teacher” approach. Sure there are some pretty poor academics, but the most of the teachers I know are trying the best they can.

My proposition

Good TPACK == Good learning and teaching

Good quality learning and teaching requires good TPACK – Technological Pedagogical and Content Knowledge. The quote I use in the abstract for the Moodlemoot presentation offers a good summary (emphasis added)

Quality teaching requires developing a nuanced understanding of the complex relationships between technology, content, and pedagogy, and using this understanding to develop appropriate, context-specific strategies and representations. Productive technology integration in teaching needs to consider all three issues not in isolation, but rather within the complex relationships in the system defined by the three key elements. (Mishra & Koehler, 2006, p. 1029)

For some people the above is obvious. You can’t have quality teaching without a nuanced and context specific understanding of the complex relationships between technology, pedagogy and context. Beyond this simple statement there are a lot of different perspectives on the nature of this understanding, the nature of the three components and their relationships. For now, I’m not getting engaged in those. Instead, I’m simply arguing that

the better the quality of the TPACK, then the better the quality of the learning and teaching

Knowledge is not found (just) in the teacher

The current organisational responses to improving the quality of learning and teaching is almost entirely focused on increasing the level of TPACK held by the teacher. This is done by a variety of means

  1. Require formal teaching qualifications for all teachers.

    Because obviously, if you have a teaching qualification then you have better TPACK and the quality of your teaching will be better. Which is obviously way the online courses taught by folk from the Education disciplines are the best.

  2. Running training sessions introducing new tools.
  3. “Scaffolding” staff by requiring them to follow minimum standards and other policies.

This is where I quote Loveless (2011)

Our theoretical understandings of pedagogy have developed beyond Shulman’s early characteristics of teacher knowledge as static and located in the individual. They now incorporate understandings of the construction of knowledge through distributed cognition, design, interaction, integration, context, complexity, dialogue, conversation, concepts and relationships. (p. 304)

Better tools == Better TPACK == Better quality learning and teaching

TPACK isn’t just found in the head of the academic. It’s found in the tools, the interaction etc they engage in. The problem that interests me is that the quality of the tools etc found in the “state of the actual” within university e-learning is incredibly bad. Especially in terms of helping the generation of TPACK.

Norman (1993) argues “that technology can make us smart” (p. 3) through our ability to create artifacts that expand our capabilities. Due, however, to the “machine-centered view of the design of machines and, for that matter, the understanding of people” (Norman, 1993, p. 9) our artifacts, rather than aiding cognition, “more often interferes and confuses than aids and clarifies” (p. 9). Without appropriately designed artifacts “human beings perform poorly or cannot perform at all” (Dickelman, 1995, p. 24). Norman (1993) identifies the long history of tool/artifact making amongst human beings and suggests that

The technology of artifacts is essential for the growth in human knowledge and mental capabilities (p. 5)

Documenting the “state of the actual”

So, one of the questions I’m interested in is just how well are the current artifacts being used in institutional e-learning helping “the growth in human knowledge and mental capabilities”?

For a long time, I’ve talked with a range of people about a research project that would aim to capture the experiences of those at the coal face to answer this question. The hoops I am having to currently jump through in trying to bring together a raft of disparate information systems to finalise results for 300+ students has really got me thinking about this process.

As a first step, I’m thinking I’ll take the time to document this process. Not to mention my next task which is the creation/modification of three course sites for the courses I’m teaching next semester. The combination of both these tasks at the same time could be quite revealing.


Norman, D. A. (1993). Things that make us smart: defending human attributes in the age of the machine. Cambridge, Mass: Perseus. Reading, MA: Addison Wesley.

Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054.

Perrotta, C., & Evans, M. A. (2013). Instructional design or school politics? A discussion of “orchestration” in TEL research. Journal of Computer Assisted Learning, 29(3), 260–269. doi:10.1111/j.1365-2729.2012.00494.x

0 Replies to “Documenting the gap between "start of art" and "state of the actual"”

  1. A worthy task. In the end the rich data may come from things that are not shiny; things that do not make headlines–the smaller things that really do make a difference in the long run. Right now, a lot of the reporting I hear relates to acquisition of gear such as apple tablets and not to the teaching and learning they are supposed to promote. I think the real issue that people forget is that teaching is so demanding that just the professional equivalent of ‘keeping the lights on’ is more than enough to fill the day. Little time is really available to pursue large scale improvements and hard-pressed governments are reluctant to devote extra resources to help this along. So it’s therefore likely that the real success stories lie in the smaller, less hyped practices that focus on students and learning–things not likely to find their way into reports or news stories. They are still the real things.

  2. It’s a good post (I put on garlic for the TPACK comments 🙂 ) In all the hoopla around tech in schools and universities, it is the teachers/lecturers who are the heroes. Putting up with all the silly nonsense of someone higher up (the digitally homeless) and actually making something decent out of it for the kids with whom they work. The game (IT and Ed) was naturally immature when it all began way back when. I see no signs of maturation. I doubt there can be when you have “system” thinking folk trying to run/drive things. The folk at the pointy end have little room to be the professionals they would prefer to be (all that ballast that should have been jettisoned, aka managers). Instead of requiring the workers to demo they have done something, I think the very first question to ask is for each and every manager to point to the hard evidence that the policies they have put in place have actually improved anything. If any business ran their IT like this. They’d be out of business so quickly. Get rid of the bloated bureaucracies and you could support teachers be the professionals they are.

    1. I had wondered whether the TPACK stuff might spark a reaction. In this context, I think it can play a useful role. Time will tell.

      In terms of your comments re: managers I agree. Did you see the MOOA post? Not sure whether I came to it via you or someone else.

      Which connects to the link Taleb tweeted recently on The myth of intelligent design and his ideas that innovation arises out of experience and bricolage. The “moving beyond” part of my Moodlemoot presentation will be on enabling that. Trying to argue that all the systems folk need to make a space for coal-face bricolage rather than getting on the version train (e.g. USQ about to upgrade to Moodle 2.4, when Moodle 2.5 has just been released.

      A big part of that is having some idea about the state of the actual

      Time to work on that presentation.

  3. Yeah you know how to get me to bite! 🙂 Had not seen the MOOA post- what a gem! And yes Taleb no dill. The line running there echoes Johansson’s argument (Johansson, F. (2012). The click moment : seizing opportunity in an unpredictable world. New York: Portfolio/Penguin), i.e. [prepared mind, heaps of affordable experiments (Taleb) and you can luck into a biggie.] Stephen Johnson’s argument about where good idea come from is also resonant here as well. You could easily make a strong case that one of the last places to find innovation is in a university, well of the Oz over-managed variety. Skunk works rule ok? 😉

Leave a Reply

Your email address will not be published. Required fields are marked *