Assembling the heterogeneous elements for (digital) learning

Category: research

Bye, Bye Mendeley?

So, I have a problem. What was a wonderful open source product was bought by a big publisher. I’m an open kind of guy so this has always been a bit disquieting. It recently got a bit worse.

Time to start actively searching.

@matbury recommended the alternative I was most aware of

Then I stumbled across this comparison of citatation management software from the University of Toronto. What struck my particularly about this comparison was the following entry under the Zotero column

Also works with Google Docs

As it happens, I’m increasingly using Google docs for collaborative creation for both work and research. Working with Google docs is a big plus. The question here is how well does Zotero work with Google docs? How much time/agony will be involved in moving?

The following documents on-going experimentation in doing so. It reveals that it doesn’t look like it will be too painful and that as I’m in the early days of a couple of group writing projects, probably a good time to make the move and test it out on some authentic tasks.

Looking increasingly like bye bye Mendeley. Would probably prefer an approach that didn’t rely on a single server for syncing/group sharing, but Zotero looks good enough for now.

Install Zotero

First step is to install Zotero. Apparently there is now both a standalone (which works with browser extensions) and a Firefox version. Download the stand alone version for the Mac.

Installing the stand alone version it picked up an old Zotero data directory from the days of early experimentation in the noughties.

Installation also takes you back to the Zotero site to create an account on the service, because

It’s a free way to sync and access your library from anywhere, and it lets you join groups and back up all your attached files.

I’ll do that (not without some minor reservations, wonder if anyone is working on a distributed method for sharing citations/references?). Now time to configure browser extension and stand alone Zotero with that account (and restart Word for that integration to work).

That seems to be working. There is a surprising amount of material in the old Zotero directory, back from the thesis days.

How to import my Mendeley library? A Google search reveals this advice. Let’s try that.

Creates a 1.6 Mb file for 3808 documents. The advice is that

Unfortunately, Zotero won’t retain your Mendeley folders or groups, but it does import the PDFs of any attached journal articles.

Losing the folders is a bit of a bugger. But I suppose I can retain Mendeley for when I need to find something.

It’s probably taken somewhere around 10 minutes to import that collection into Zotero.

All seems to be there, but of course there are some things that have been lost, including

  • As mentioned the folder/group structure that I’d used in Mendeley to organise references into structures relevant to the work I was doing.
  • I’ve also lost the annotations and notes that I’d started making using Mendeley’s internal features. Had been worried about that. Appears Zotero doesn’t support annotations on the PDFs, but then there are the Adobe methods for that, which might be a better long term approach.

Citation management and Google docs

According to this page working with Google docs is via the same mechanism used for plain-text documents.

  • Add a bibliography, by selecting the list of items and drag them into the document.

    Suggesting a need to maybe create a folder or similar for every reference you want to use in a Google doc. Suggesting collaborative use would require all the collaborators to agree on processes, perhaps using a common Zotero group owned resource?

  • Add a citation, hold the shift key before dragging.

Apparently there are requests in to Google to allow a better integration.

Let’s see how the existing method (more detail here) works. The following image shows the result. The first line shows the results of adding a bibliography, the second adding a citation. The citation format is not what I’d prefer. So the question is whether it can be changed.

Google docs / Zotero usage

There is some other advice on how to do this a bit more effectively. But it doesn’t quite meet the expectations about what citation management really working with Google docs might be. But it is a step up. What about other features?

What’s it like with Word?

I still end up writing most my formal publications in Word, so that experience is important.

New Word document – Add-ins tab has now been modified to include icons for Zotero. Insert new citation, pops up a dialog box to configure document preferences. Assume this is because the first time I’ve used Zotero with this document.

Much like Mendeley another dialog box pops up, you enter author name or other, choose from some options and hey presto citation is inserted.

Insert bibliography and done.

All very similar. Might have a few wrinkles of difference, but I don’t imagine that (given the size of the Zotero community) wouldn’t be something that can’t be figured out.

Online library, groups etc.

With the Zotero account comes an online library, as shown in the following image. Seems the current situation is

  • 300Mb for free and never (big call) expires
  • 2Gb for $USD20 a year.
  • 6Gb for $USD60 a year.
  • Unlimited for $USD120 a year.

Zotero library online

There are options to control the privacy of the online library: public the entire library; public notes; hide from search engines. I’ve just changed those settings so that the library (not the notes) is public. You can see it here

The group functionality supports collaboration and explicitly mentions web-based bibliographies for classes. This could be interesting.

The group page mentions that there are no limits on groups and outlines a range of features and uses. One of the limits, I assume will be the amount of storage space online.

What is theory and why use theories?

The following is an edited version of something used in a course I teach that’s currently hidden away in the LMS. I’m adding it here because I’m using it with another group of students.

It’s a quick attempt to cover what I perceive to be a reasonable whole for many education students. i.e. what exactly is a theory and why the hell would I want to use them? My impression is that not many of them have developed an answer to these questions that they are comfortable with.

This is a complex and deeply contested pair of questions. I’m assuming that if you lined up 50 academics you’d get at least 50 different sets of answers. My hope that this is a useful start for some. Feel free to add your own pointers and answers to these questions.

If you want a more detailed look into the nature of theory then I can recommend Gregor (2006).

What is theory?

I take an inclusive and pragmatic view of theory.

An inclusive view, because there is a huge array of very different ideas that can be labelled theories. A pragmatic view is taken because the reason we use theories in this course is to make it easier to do something. To understand a particular situation, or for most reading this figure out how to design some use of digital technology to enhance or transform student learning.

Hirst (2012, p. 3) describes educational theory as

A domain of practical theory, concerned with formulating and justifying principles of action for a range of practical activities.

i.e. educational theory should help you teach and help your learners learn.

In the context of this particular course we touch on various ideas such as: the Computer
Practice Framework, TPACK, Backwards Design, the RAT framework, the SAMR model, The TIP Model, constructivism, and many more. For the purposes of this course, we’ll call these things theories. They help with “formulating and justifying principles of action”.

There is huge variability in the purpose, validity, and approaches used to formulate and describe these objects called theories. A theory isn’t inherently useful, important, or even appropriate. That’s a judgement that you need to make.

A theory is just a model and All models are wrong, but some are useful (Box, 1979).

Why use theories?

Thomas (1997, p. 78) cites Mouly (1978)

Theory is a convenience a necessity, really organizing a whole slough of facts, laws, concepts, constructs, principles into a meaningful and manageable form

These theories are useful because they help you understand, formulate and justify how and what to do. In this course, these theories will help you plan, implement, and evaluate/reflect upon the use of digital technologies to improve your teaching and your students’ learning.

Learning and teaching are difficult enough. When you add digital technologies to the mix even more complexity arises. The theories we introduce in this course should hopefully help you make sense of this complexity. Guide you in understanding, planning, implementing and evaluating of your use of ICTs.

References

Gregor, S. (2006). The nature of theory in information systems. MIS Quarterly, 30(3), 611–642.

Hirst, P. H. (2012). Educational theory. In P. H. Hirst (Ed.), Educational Theory and Its Foundation Disciplines (pp. 3-29). Milton Park, UK: Routledge.

Thomas, G. (1997). What’s the Use of Theory? Harvard educational review, 67(1), 75:105.

Learning about case study methodology to research higher education

The following is a summary and some thoughts on Harland (2014). The abstract for the paper is

Learning about teaching through inquiry is a sound practice for professional development and the university teacher-as-researcher is now commonplace. With the proliferation of inquiry-based postgraduate programmes in university teaching, more academics from across the disciplines are learning new ways of doing research. In this paper, I draw on 10 years’ experience of teaching research methods in higher education. I teach on a one-year part-time course that aims to help academics change their practice and contribute to the theories of teaching in higher education through publication of their work. The preferred research method is case study and there are four questions that both inexperienced and experienced participants can find challenging: what is the potential of case study; what forms of data are acceptable; when does analysis stop and what makes a quality case study? I conclude with a set of recommendations for the new researcher aimed at enhancing the quality of research. Suggestions include properly integrating existing theory into published work, avoiding positivist analogues to judge research, using multiple methods in research design and avoiding writing descriptively or without a critical audience in mind.

My interest

As outlined previously, I have to write more journal articles. The current paper idea I have goes under the following working title “BAM, BIM, Blogs and Breaking BAD: What does it take to create quality e-learning?”. It’s currently conceptualised as a case study of the development and use of BAM and BIM to support the use of individual student blogs from 2006 through 2015. The basic argument is that it’s no surprise that most e-learning is not that great, given the difficulty of doing anything decent within the current institutional mindset around e-learning. The idea is to draw on the Breaking BAD paper, a presentation to MoodleMoot and various other publications round BIM/BAM over the years.

Given it’s a case study and they have limitations, it’s probably a good idea to be able to write up the method in a way that ticks all the right boxes. Hence my interest in Harland (2014).

Thoughts

Case study is a dominant/well accepted method. Tick.

How often does Computers and Education accept case study work?

Points out some interesting points for consideration by early case study researchers in higher education. Not a “how to” guide such as Baxter and Jack (2008)

Summary

Introduction

Researching own teaching practice one way academics learn to teach. Teacher-as-researcher, link to high schools and Boyer’s SoTL. Defines “dual researchers” people rsearching both their discipline and the teaching of that discipline.

Author from hard sciences. Research in academic development didn’t follow the science rules.

cites Tight (2012) that qualitative inquiry remains dominant research method in higher ed journals.

As a reviewer, author finds most articles are sub-par. Case study used in most articles read.

# of case studies published in four higher education journals: 2007-2012
adapted from Harland (2014, p. 1114)
Journal Location Case study Conceptual study Total #
Higher Education Europe 344 181 525
Studies in Higher Education Europe 238 62 300
Teaching in Higher Education Europe 176 111 287
Higher Education Research & Development Australasia 146 101 247
Total: 904 455 1359

“Case study consists of empirical inquiries of single cases that are contextually unique (Stake, 1995)” (Harland, 2014, p. 1114) – my emphasis added – typically addressing something of interest to the authors professional practice. Has instrinic value to those that benefit from the professional practice, but can also contribute to “the theories and practices of higher education”

There are four sticking points in learning case study research methods

  1. What is the potential of the case study?
  2. What forms of data are acceptable?
  3. When does analysis stop?
  4. What makes a quality case study?

These are used to structure the rest of the paper.

Has a para which appears essentially the research method para. Autoethnography is used. The four sticking points are addressed through a personal narrative.

What is the potential of the case study?

Specificity of case study research seen to limit contribution to theory. But that type of certainty of knowledge is very techno-rational. Case study inquiry involves individuals/teams interpreting data. Requiring new standards of judgement (Flyvbjerg, 2006) who contrasts rule-based and case-based knowledge. Case-based is always context-dependent.

As I see it, no two practice contexts are ever genu- inely the same and so rules and deterministic models for guiding thinking and action are not that useful. (p. 1115-1116)

Case study research cannot be truly replicated given the uniqueness of context, but it can be learned from. What each reader may learn will differ.

While its possible to generalise from case studies (Denzin, 2009), it’s unusual. Though it is argued that cases provide an opportunity for generalisation.

Case study research seen as better for generating hypotheses than theory building (Flyvbjerg, 2006) – depending on the definition of theory. Options include

  1. Explanatory and predictive of cause & effect and thus can direct action
  2. pragmatist perspective that has theory/practice intertwined. People generate theories to seek meaning in practice. A form of personal theory building.

Theoretical relevance enhanced if existing theories are integrated. Through which contribution can be made through new interpretation of data. “Existing theory should be seen as an integral part of the case” (p. 1116)

What forms of data are acceptable?

“case study may rely on multiple sources of evidence and be practiced as multi-method research (Denzin & Lincoln, 1994)” (p. 1117) as long as it helps answer the research question. i.e. it can use quantitive research methods. Apparently a surprise to some.

Tight (2012) found only 5% of 440 published articles in 15 higher education journals used a mixed quantitative/qualitative methodology.

What does analysis stop?

Outcomes of any analytic technique will depend on intentions, background knowledge, cognitive processes, mindset etc. Hence analysis is recursive.

Minimise the time between collecting data and writing the research account. “disciplined writing seems to be the most essential part of the analytical process” (p. 1118).

No genuine endpoint.

What makes a quality case study?

Better to engage wider theories than just describe practice. But not sufficient

Quality case research:

  • requires imagination (Dewey, 1938)
  • requires creativity (Morse, 1995)
  • must bring the reader as close as possible to the experience (Fossey et al, 2002)
  • provide conceptual insight (Siggelkow, 2007)
  • should be believable, which requires coherence and provide new theory and instrumental utility (Eisner, 1991)
  • “potential to create an impact on the field of practice” (p. 1118)
  • have something important to say
  • well structured and clearly writte

Argues that a case study should enable someone to learn from it.

Author explores the impact (from this measure) of one of his case study publications. Somewhat sobering results.

Conclusion

Summarises the key points to be “attentive to” and “cautious of” against the four challenges and some more general comments e.g. research must fulfil its purpose and this needs to be known before time to help align process and outcomes.

Does make the point that case study is a form of learning and that this can be seen in daily practice, more so than in research articles.

Also points out that publications on case study methods are “often complex or underpinned by unstated assumptions, following a procedure is never straightforward” (p. 1121). Case study research does allow you to learn from experience, so these methods should be seen as guidelines.

References

Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report, 13(4), 544–559.

Harland, T. (2014). Learning about case study methodology to research higher education. Higher Education Research & Development, 33(6), 1113–1122. doi:10.1080/07294360.2014.911253

Choosing a research publication outlet: Part II

My name is David. I’m an academic. It’s been 8 years since my last journal paper (but what a paper it was, thanks Shirley).

In my defence, for three of those years I was finishing the PhD, 1 year finishing a GDL&T, three years getting used to a new discipline, and generally just cynical about the whole journal process, and not having much I considered useful to say. But that has to change, otherwise I’ll have some explaining to do. This year is about writing some journal papers. So where should I publish?

I asked and answered a similar question in 2009, hopefully this time I might actually go the next step and write something.

(Note: I start the writing process by getting to know the destination. The theory being that a paper is more likely to get accepted if it fits the requirements, aims, tone etc of the destination. I already have a few ideas re: papers, finding a suitable destination is the first task.)

What’s changed since 2009?

I’ve officially moved from the Information Systems discipline to Education. But given I was originally focused on educational technology, the journals identified in that original post are still suitable.

I’m at a new university that has recently introduced awards for journal articles “designed to encourage .. researchers to make strategic decisions about the journals in which they seek to publish, in order to help them get the best return on their research efforts and as a crucial step in improving the research performance of … as an institution”. Obviously linked to the on-going ERA. This is judged largely on the Scopus SNIP (Source-Normalized Impact per Paper) index.

Obviously there is more to research impact than the journal’s “citation potential”, but it’s the current measure of choice. At the very least I’ll have to consider it.

Updating the list

For the following list I’ve

  1. Taken a list of journals mentioned in the original post.

    Have excluded a couple I no longer deem interesting.

  2. Added any other journals I think potentially relevant.
  3. Identified/updated the following information about the journals
    • Ranking as per the Oz government approach.
      This was used in the 2009 ERA, but wasn’t used in the 2012 exercise.
    • SNIP
      Provided by this search form.
    • h5-index and h5-median.
      In my travels I’ve discovered various other types of rankings such as Google scholar’s method which generates the following for

    • Are the papers open or closed?

      This remains an important personal consideration.

    • Position on article copyright.

      i.e. is it ok for me to put a copy of the article on my personal site? A potential proxy for being open.

    • Max paper size.
    • # Issues per year.
    • Turnaround time on review.
    • ERA FoR
      Using John Lamp’s interface to the ERA data.

Once I have that information, the plan is to look more closely at the current editorial directions and the types of papers being accepted for publication in each of the different journals.

The list and observations

The complete list is available as a Google spreadsheet.

The top 10 journals ordered by SNIP are

  1. Computers and Education
  2. Internet and Higher Education
  3. Studies in Higher Education
  4. Educational Technology Research & Development
  5. IRRODL
  6. British Journal of Educational Technology
  7. Educational Technology & Society
  8. Higher Education Research & Development
  9. Teaching in Higher Education
  10. AJET

The top 10 journals ordered h5-index as per Google Scholar are

  1. Computers and Education
  2. British Journal of Educational Technology
  3. Educational Technology & Society
  4. Internet and Higher Education
  5. IRRODL
  6. Educational Technology Research & Development
  7. Studies in Higher Education
  8. AJET
  9. Higher Education Research & Development
  10. Teaching in Higher Education

Interesting to see BJET and ET&S climb the ladder in terms of h5-index.

Could I win an institutional award?

This is not the purpose for writing, but it’s an interesting exercise in exploring how level the playing field is between various disciplines (other than the sciences).

The only results I can find for the institutional publication awards had the following SNIP values

  • 1st – 3.154
  • 2nd – 3.01
  • 3rd – 2.181
  • Student award – 3.273
  • Special mention – 8.2

    An article in Nature that was in the wrong time frame.

Computers and Education is by far the highest ranked of these journals using either SNIP of h5-index. C&E’s SNIPP value was 3.29. Meaning a first place, at least for the above period.

Internet and Higher Education has a SNIP value of 2.55, making 3rd possible.

BJET, IRRODL and AJET (the most familiar journals to me) have SNIPP values of: 1.71, 1.77 and 1.16 respectively.

While a sample size of one isn’t great, it appears possible but only just if I focus on higher education or general educational technology journals.

The couple of teacher education related journals that get a mention either don’t have a SNIPP or have one just above 1. Not likely to please the “impact police”. Of course, I didn’t go searching for more general teacher education journals.

Other misc. observations on journals

Computers and Education

Apparently has a “liberal copyright policy”. Which appears to permit posting to open websites but doesn’t exactly trumpet that position, e.g. doesn’t appear to be mentioned on this page.

There is a choice to publish an article as open access, but there’s a fee required. $1800!!!!!! But as an author you do get a 30% discount on Elsevier books.

Recommends a clearly defined article structure. Wonder how many published articles follow that? Apparently not many.

Has a “turnaround time” of around 10 weeks.

Internet and Higher Education

Audience – “faculty, administrators, and librarians charged with the responsibility of fostering the use of information technology and the Internet on their respective campuses.” Potentially a good fit.

Seems Elsevier have a standard structure they like. Same here as Computers & Education.

About a 5 week turnaround time.

Educational Technology Research and Development

Two month turn around claimed.

The fee for your article being open access is $USD3000!!!!

BJET

$USD3000 cost for open. Not real clear about whether it’s okay to share versions of articles via personal websites.

So where might I publish?

No clear winner, but some thoughts include

  • Publishing in Computers and Education would satisfy the “impact police” but the initial topic I have planned may not fit well.
  • My preference is in open access journals and IRRODL is perhaps a closer fit for what I’m thinking of writing.
  • Internet and Higher Education is a closed journal, but also a reasonable fit and it promises the fastest decision time.

Time will tell.

Evaluating the use of blogs/reflective journals

The use of blogs in one of the courses I teach is now into it’s fourth semester. Well past time to do explore how it’s all going, evaluate some of the design decisions, and make some decisions about future developments. In preparation for that it’s time to look at some of the extant literature to look at findings and methods. The following is the first such summary and is focused on @spalm et al’s

Palmer, S., Holt, D., & Bray, S. (2008). The learning outcomes of an online reflective journal in engineering, 724–732.

Summary

Looks at the use of an “online reflective journal” (implemented using discussion forum in WebCT) in a 4th year engineering course. Combines a survey-based evaluation of student perceptions with “an analysis of student use of the journal…to investigate its contribution to unit learning outcomes”

Findings are

  • Most students understood the purpose and value the journal in their learning
  • Most read the entries of others and said this help their learning
  • Two most useful things about the journal
    • The need to continuously revise course material
    • Ability to check personal understanding against that of others
  • Least useful related to problems with using WebCT – the difficulty of the interface and “problems with CMS operation”.
  • Significant contributors to final mark
    • Prior academic performance
    • Number of journal postings
    • Mode of study

Thoughts

The quantitative nature of this is interesting as it’s something I need to do more of. If only to gain some experience of this approach and to tick the box (which is a great reason).

There is always going to be a bit of a “so what” issue with this type of thing as research. What really does a study in a single course reveal that’s new or easily transferable. But perhaps doing almost a replication addresses that somewhat.

Of course the real interest in doing this research is just to find out more about what’s going on in the course, its impact and what the students think so it can inform further development.

To do

  • Can I get a copy of the evaluation survey used?
  • Can I apply the evaluation survey to past students?
  • Do I need to get student permission to analyse the data around their use of their blogs and final results?

Comparison with EDC3100

My impression is that prior academic performance would be the most significant factor in EDC3100. Raising the question about how much value there is in what teachers do, if prior academic performance is a big contributing factor are we “failing” the weaker student?

Question: Could the students performance on the course be included in the evaluation survey in some form?

I think mode of study might play a role. My guess is that the online students would get see the value in the blogs more than some of the on-campus students.

Question: What impact does mode of study play in EDC3100?

Given that the blog posts were only marked based on number of posts, average word count and number of links it would be interesting to explore what impact that had on the final mark. Especially given the quote about “assessment” as a “strategic tool for creating student engagement”.

Question: Exactly what type of student engagement is the assessment of the blogs in EDC3100 creating?

Question: What patterns exist in the learning journal marks?

The journaling is contributes 5% of the total course mark for each of the 3 assignments. Is there a pattern in the marks for each assignment and the final outcome?

Question: Does the medium used make any difference?

In EDC3100 students are using their own blog hosted on their choice of external service. Very different from an LMS forum. Does this make a difference? Is it more their space?

Is it seen as too difficult or a waste of time creating a blog?

Question: Is the collection of technologies used to complete this task too difficult?

To get marks students have to create their blog (e.g. WordPress) and follow the blogs of others (Feedly, WordPress “follow” mechanism, WordPress reader etc). This leads to problems with creating links to posts. E.g. students link to the post in Feedly or the WordPress reader, not in the student’s blog.

But that said, the interface is “better” (subjective) and might be seen as more realistic – not a Uni tool, something broader.

Question: Does the time of posting make any difference? What are the different patterns of posting visible? Any patterns indicating task corruption?

Palmer et al (2008)’s journal is essentially a weekly task. In EDC3100 students need to post at least 3 posts a week to get full marks. There is no specific direction as to what or when to post. What patterns are there?

Question: How does the perception of reflection/journaling in the discipline impact thoughts?

Palmer et al (2008) describe how a work journal is a common practice for engineers. Hence doing this in the course can be linked to professional practice.

The same doesn’t apply to the teaching profession. While reflection is seen as important, there’s doesn’t appear to be the accepted practice of regularly keeping a work journal.

Leadership as defining what's successful

After spending a few days visiting friends and family in Central Queensland – not to mention enjoying the beach – a long 7+ hour drive home provided an opportunity for some thinking. I’ve long had significant qualms about the notion of leadership, especially as it is increasingly being understood and defined by the current corporatisation of universities and schools. The rhetoric is increasingly strong amongst schools with the current fashion for assuming that Principals can be the saviour of schools that have broken free from the evils of bureaucracy. I even work within an institution where a leadership research group is quite active amongst the education faculty.

On the whole, my experience of leadership in organisations has been negative. At the best the institution bumbles along through bad leadership. I’m wondering whether or not questioning this notion of leadership might form an interesting future research agenda. The following is an attempt to make concrete some thinking from the drive home, spark some comments, and set me up for some more (re-)reading. It’s an ill-informed mind dump sparked somewhat by some early experiences on return from leave.

Fisherman’s beach by David T Jones, on Flickr

In the current complex organisational environment, I’m thinking that “leadership” is essentially the power to define what success is, both prior to and after the fact. I wonder whether any apparent success attributed to the “great leader” is solely down to how they have defined success? I’m also wondering how much of that success is due to less than ethical or logical definitions of success?

The definition of success prior to the fact is embodied in the current model of process assumed by leaders, i.e. telological processes. Where the great leader must define some ideal future state (e.g. adoption of Moodle, Peoplesoft, or some other system; an organisational restructure that creates “one university”; or, perhaps even worse, a new 5 year strategic plan etc.) behind which the weight of the institution will then be thrown. All roads and work must lead to the defined point of success.

This is the Dave Snowden idea of giving up the evolutionary potential of the present for the promise of some ideal future state. A point he’ll often illustrate with this quote from Seneca

The greatest loss of time is delay and expectation, which depend upon the future. We let go the present, which we have in our power, and look forward to that which depends upon chance, and so relinquish a certainty for an uncertainty.

Snowden’s use of this quote comes from the observation that some systems/situations are examples of Complex Adaptive Systems (CAS). These are systems where traditional expectations of cause and effect don’t hold. When you intervene in such systems you cannot predict what will happen, only observe it in retrospect. In such systems the idea you can specify up front where you want to go is little more than wishful thinking. So defining success – in these systems – prior to the fact is a little silly. It questions the assumptions of such leadership, including that they can make a difference.

So when the Executive Dean of a Faculty – that includes programs in information technology and information systems – is awarded “ICT Educator of the Year” for the state because of the huge growth in student numbers, is it because of the changes he’s made? Or is it because he was lucky enough to be in power at (or just after) the peak of the IT boom? The assumption is that this leader (or perhaps his predecessor) made logical contributions and changes to the organisation to achieve this boom in student numbers. Or perhaps they made changes simply to enable the organisation to be better placed to handle and respond to the explosion in demand created by external changes.

But perhaps rather than this single reason for success (great leadership), it was instead there were simply a large number of small factors – with no central driving intelligence or purpose – that enabled this particular institution to achieve what it achieved. Similarly, when a few years later the same group of IT related programs had few if any students, it wasn’t because this “ICT Educator of the Year” had failed. Nor was it because of any other single factor, but instead hundreds and thousands of small factors both internally and externally (some larger than others).

The idea that there can be a single cause (or a single leader) for anything in a complex organisational environment seems to be faulty. But because it is demanded of them, leaders must spend more time attempting to define and convince people of their success. In essence then, successful leadership becomes more about your ability to define and promulgate widely acceptance of this definition of success.

KPIs and accountability galloping to help

This need to define and promulgate success is aided considerably by simple numeric measures. The number of student applications; DFW rates; numeric responses on student evaluation of courses – did you get 4.3?; journal impact factors and article citation metrics; and, many many more. These simple figures make it easy for leaders to define specific perspectives on success. This is problematic and it’s many problems are well known. For example,

  • Goodhart’s law – “When a measure becomes a target, it ceases to be a good measure.”
  • Campbell’s law – “The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
  • the Lucas critique.

For example, you have the problem identified by Tutty et al (2008) where rather than improve teaching, institutional quality measures “actually encourage inferior teaching approaches” (p. 182). It’s why you have the LMS migration project receiving an institutional award for quality etc, even though for the first few weeks of the first semester it was largely unavailable to students due to dumb technical decisions by the project team and required a large additional investment in consultants to fix.

Would this project have received the award if a senior leader in the institution (and the institutional itself) heavily reliant upon the project being seen as a success?

Would the people involved in giving the project the award have reasonable reasons for thinking it award winning? Is success of the project and of leadership all about who defines what perspective is important?

Some other quick questions

Some questions for me to consider.

  • Where does this perspective sit within the plethora of literature on leadership and organisational studies? Especially within the education literature? How much of this influenced by earlier reading of “Managing without Leadership: Towards a Theory of Organizational Functioning”
  • Given the limited likelihood of changing how leadership is practiced within the current organisational and societal context, how do you act upon any insights this perspective might provide? i.e. how the hell do I live (and heaven forbid thrive) in such a context?

References

Tutty, J., Sheard, J., & Avram, C. (2008). Teaching in the current higher education environment: perceptions of IT academics. Computer Science Education, 18(3), 171–185.

An ad hoc exploration ethnographic research

The following is an initial attempt to restart some earlier explorations of research methods that may prove useful in examining the “Story of BIM” for potential useful insights. The starting place is ethnography and auto ethnography and an exploration of some writings.

Rescuing Autoethnography

Atkinson, P. (2006). Rescuing Autoethnography. Journal of Contemporary Ethnography, 35(4), 400–404. doi:10.1177/0891241606286980

Apparently a response to or a continuation of an on-going debate about the value and problems of analytic autoethnography.

(Atkinson, 2006, p. 401)

These are not just matters affecting the choice of fieldwork site but a biographically grounded, experientially rich engagement with the social processes that are observable in the field, and that render those processes comprehensible in particular ways.

Comment arising from a range of examples where the particular skills and background of researchers enabled engagement/insight that would have previously been not possible.

This close connection need not be justified “exclusively on postmodernist rationales” but is indicative of a longer history of close relationships between the researcher and the informant. More broadly the idea of understanding a social life ethnographically depends on the “homology between the social actors who are being studied and the social actor who is making sense of their actions” (Atkinson, 2006, p. 402) This is linked to the fundamental principle of reflexivity – a much abused term – which is defined as

the ineluctable fact that the ethnographer is thoroughly implicated in the phenomena that he or she documents, that there can be no disengaged observation of a social scene that exists in a “state of nature” independent of the observer’s presence, that interview accounts are coconstructed with informants, that ethnographic texts have their own conventions of representation. In other words, “the ethnography” is a product of the interaction between the ethnographer and a social world, and the ethnographer’s interpretation of phenomena is always something that is crafted through an ethnographic imagination. (p. 402)

If it’s so embedded in ethnography, what is the problem with auto-ethnography? When the ethnographer becomes more memorable than the ethnography.

The solution to this is to insist on the analytic aspect of ethnography. The “experiential value, its evocative qualities and its personal commitments” should not be promoted at the expense of the “scholarly purpose, it’s theoretical bases and its disciplinary contributions”.

Obviously a time to look at the rest of the discussion.

Analytic autoethnography

Anderson, L. (2006). Analytic Autoethnography. Journal of Contemporary Ethnography, 35(4), 373–395. doi:10.1177/0891241605280449

Appears to be the origins of “analytic autoethnography”. An explicit attempt to distinguish it from “evocative autoethnography”. Analytic autoethnography is defined as, (Anderson, 2006, p. 373)

research in which the researcher is

  1. a full member in the research group or setting;
  2. visible as such a member in published texts, and
  3. committed to developing theoretical understandings of broader social phenomena.

Ellis, Bochner and Denzin said to be influential in rise of auto-ethnography. But Anderson does also outline a broader history of “an autoethnographic element i qualitative sociological research” (Anderson,2006, p. 375). Though much of this work “continued the earlier tendency to downplay or obscure the researcher as a social actor in the settings or groups under study” (Anderson, 2006, p. 376). They were “neither particularly self-observational in their method nor self-visible in their texts”. There were other strands of research but it is 1979 and an essay on autoethnography by David Hayano in 1979.

There is then the rise of “the descriptive literary approach of evocative autoethnography” (Anderson, 2006, p. 377). An approach that moves away from the analytical and toward an empistemology of emotion. Anderson (2006, p. 377)

Evocative autoethnographers have argued that narrative fidelity to and compelling description of subjective emotional experiences create an emotional resonance with the reader that is the key goal of their scholarship.

Within the paper five key features of analytic autoethnography are then proposed, these are

  1. Complete member researcher (CMR) status.

    quotes Merton (1988, p. 18) describing the research as “the ultimate participant in the dual participant-observer role”.

    Patricia and Adler (1987, p. 67-84) identifies 2 types

    1. opportunistic” – the more common, born into a group, thrown into it by chance or acquired familiarity through occupation, lifestyle etc. Group membership may precede the research decision.
    2. covert – begin with a purely data-oriented research interest but are converted into immersion and membership during the course of research.

    Membership gives close connections, but does not “imply a pnaoptical or nonproblematic positionality” (Anderson, 2006, p. 380). Auto-ethnographers are apart in that the spend time documenting and analysing action as well as engaging in action.

    Analysis raises the “Schutzian distinction” (Schutz, 1962) between practically oriented, first-order interpretations and the more “abstract, transcontextual , second-order constructs of social science analysis”. Then there is the problem of the variety of first-order interpretations within the social social groups. Different members see things differently and the researcher’s role in the group makes some of these more accessible than others. This leads to the question of how or even if it is possible for the auto-ethnographic research to achieve “becoming the phenomenon” (Mehan and Wood, 1975, p. 227).

    What the auto-ethnographer “knows”/learns emerge from engaged dialogue, rather than detached discovery

  2. Analytic reflexivity.

    reflexivity involves an awareness of reciprocal influence between ethnographers and their settings and informants. It entails self-conscious introspection guided by a desire to better understand both self and others through examining one’s actions and perceptions in reference to and dialogue with those others. (Anderson, 2006, p. 382)

    For auto-ethnographers it goes deeper than this. Their data arises from their own experience and sense marking. They are part of the representational process but are also partially formed by those processes through co-creation in conversation, action and text.

    While this is an important component, it’s not enough to engage in reflexive social analysis etc. There’s a need to be…

  3. Narrative visibility of the researcher’s self.

    In convention ethnography there is apparently a problem with the enthnographer being often invisible in the text, but omniscient. Even though this has not always been the case.

    Autoethnography “demands enhanced textual visibility of the researcher’s self. Demonstrate the researcher’s personal engagement in the social world. Illustrate analytic insights through recounting experiences and thoughts as well as those of others. Should also “openly discuss changes in their beliefs and relationships over the course of field work”. To show the grappling that occurs with issues in “fluid rather than static social worlds”.

    The goal of reflexive ethnography (and autoethnography) according to Davies (1999, 5) is to “seek to develop forms of research that fully acknowledge and utilize subjective experience as an intrinsic part of research” (Anderson, 2006, p. 385)

    The descent to self-absorption is where autoethnography loses its value. The visibility has to be more than “decorative flourish”. For analytic ethnography the aim “is to develop and refine generalised theoretical understandings of social processes”.

  4. Dialogue with informants beyond self.

    Quote from Rosaldo (1993, 7) “if classic ethnography’s vice was the slippage from the ideal of detachment to actual indifference, that of present-day reflexivity is the tendency for the self-absorbed Self to lose sight altogether of the cuturally different Other.”. There is a need to engage with others in the field. “No ethnographic work – not even autoethnography – is a warrant to generalise from an “N of one”. There is a need for dialogue with “data” or “others”

  5. Commitment to theoretical analysis.

    There must be some aim to use “empirical data to gain insight into some broader set of social phenomena than those provided by the data themselves” (Anderson, 2006, p. 387). Using evidence to formulate and refine theoretical understandings of social processes. This narrower definition of “analytic” is in line with Lofland (1970, 1975) and Snow et al (2003) points to “a broad set of data-transcending practices that are directed toward theoretical development, refinement and extension”.

    The aim is not to produce “undebatable conclusions”, but instead to contribute “to a spiraling refinement, elaboration, extension and revision of theoretical understanding”.

Virtues and limitations of analytic autoethnography

Analytic autoethnography is positioned as a sub-genre of analytic ethnography.

Virtues fall into

  • Methodological.

    Being a CMR makes data more available. There are multiple incentives to participate. But the multitaking also creates potential pitfalls. Research focus fading. Participation outweighing writing of field notes.

    There is also access to “insider meanings”.

    Personal involvement also provides access to data not normally available.

  • Analytic.

    Provides “grounded opportunities to pursue the connections between biography and social structure”.

Limitations. Most don’t find research interests that are deeply intwined with personal lives – as required by autoethnography. Analytic ethnography assumes a “professional stranger” role.

There is little more conversation of the limitations, beyond that all methods have limits. Perhaps this is taken up more by the rest of the articles in the issue.

Conclusions

Specific research method flourish in the absence of other well-articulated methods. This is one explanation given for the rise of evocative autoethnography and the paucity of analytic autoethnography.

An example closer to home

Time to explore autoethnography a bit closer to the context or type of application I’m interested in.

Clark, C., & Gruba, P. (2010). The use of social networking sites for foreign language learning : An autoethnographic study of Livemocha. In C. Steel, M. Keppell, P. Gerbic, & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010 (pp. 164–173).

Data collection – self-aware participation, learner diaries and peer debriefing. To investigate use of social networking sites in foreign language learning. A grounded, thematic analysis used.

In describing the method, starts by mentioning history of “large-scale diary studies” in a range of fields. “particularly useful to examine events in their natural context to obtain reliable, person-level information”. Autoethnography can be considered “unstructured, uncontrolled ….and necessarily subjective and anecdotal”. And a quote or two to justify.

One author recorded all language learning experiences with the chosen site in 3 phases: register as himself, a four week study of Korean starting from scratch. Detailed notes were taken. Using Bolger et al (2004) principles for event-based autoethnographic design – details/impressions/experiences were recorded during and after. Later collation of all into a learner journal.

Phase three – thematic analysis involving two analysts. Primary issue was return to the site and continued study – was the site “addictive and effective”.

In terms of continued use: three themes emerged – motivation, frustration and demotivation. These are explained in detailed, summarised in a table and linked to suggestions for pedagogical improvement.

And another

Duarte, F. (2007). Using Autoethnography in the Scholarship of Teaching and Learning: Reflective practice from “the Other Side of the Mirror”. International Journal for the Scholarship of Teaching and Learning, 1(2), 1–11.

Initially strikes me as potentially more evocative autoethnography.

this essay is based on my reflections and recollections of important events and insights that occurred during the Redevelopment Project, and on the notes of the reflective journal I kept to document my shifts of consciousness as I gained new pedagogical knowledge and skills.

The author has used autoethnography in prior research and argues that it has good links with SoTL – particularly due to the focus on reflection in both. Gives quotes from Bass around SoTL and reflection.

The story told reveals a great deal about the experience of the “early adopter” of blended learning in a system with technology and processes that isn’t set up well to handle it. In particular focuses on the “time pressures created by the shift to blended learning”. Includes a reference to a study by Lefoe and Hedberg (2006, p. 334)

Some other interesting quotes there. Not sure the paper offers a strong example of a method that might be accepted, but some good insights into blended learning and it’s implementation.

1000 blog posts – a time to look back

According to the WordPress dashboard for this blog this is the 1000th published post (I have 100 odd drafts that I never finished or thought better of posting). Given I’m about to mark first year in a new job in a new institution in a new region, undergo my annual performance review and commence a new academic year, it would seem time to reflect and think about the future.

But first, thanks to those folk who have read and contributed to the blog over the years. Much of the good from blogging has arisen from those connections.

Reflections on this process

A few reflections from writing the summary below.

The staying power of bad ideas

Many of the problems I see with institutional attempts to support quality learning and teaching remain. Mainly, I would propose, because it’s easier to accept the simple practice everyone else uses than try and address the known problems. Examples of this include strategic management of universities (perhaps the fundamental cause of the rest), attempts to make teaching conform to a standard (quality through consistency), the reliance on end of semester student evaluation to determine quality of teaching and the teachers.

Purgatory was the best time to blog

2008 through 2010 was the best time for blogging. There was more engagement with and reflection upon the literature. Need to get back to that.

Perhaps the workload associated with life as a contemporary university academic, lots of centralised hoops, poor systems etc.

Still think the quality of my blogging – in terms of the writing, the referencing, the depth, the contribution, the connections – could stand for some improvement.

Workload implications

My blogging last year wasn’t great. Certainly less (quantity and quality) than in 2008-2010. One reason for that is workload. The workload of a contemporary Australian academic in a regional university is high. In terms of the workload and the nature of the work there are certainly much worse jobs (but to some extent that evaluation is fairly subjective). But the problem with academia in this context is that the workload is increasingly be increased due to management mandates at the expense of research/thinking time. Subsequently academics are getting in trouble for not producing research outputs. To make it worse, some of that workload is being created by the poor quality of the institutional systems being put in place to help meet those mandates.

What to do

I’m not going to bother with grand plans. To some extent it’s wasted effort. You can’t predict what is going to happen, at best you can only create the capability to respond.

I need to blog more and to think about different approaches to improving the blogging. I need to focus more and stop retreating to some of the same topics. I need to start producing some work that moves some of these forward a bit. I need to connect and comment more, get out of my shell.

Perhaps I need to do more work to address the workload implications. Give myself some space by exploring solutions to the workload problem, especially around learning and teaching.

Origins

The first blog post was written in March 2006 when the blog was on a self-hosted version of WordPress (most of the links to blog posts within those early blog posts won’t work, they still point to the old host). So it’s taken almost seven years to clock up 1000 published posts.

At that stage I was thinking about using blogs in the teaching of C++ to first year Information Technology students. The first few posts are little more than experiments trying to find out how WordPress worked and what it felt as an author/commenter.

It’s not until the 4th and 5th posts (in August 2006 suggesting the initial plan died) that I start storing away a bit of knowledge. The 5th post has a couple of Drucker quotes which still resonate strongly with me and perhaps should inform the “thinking about the future” part of this post. e.g.

Planning’ as the term is commonly understood is actually incompatible with an entrepreneurial society and economy….innovation , almost by definition, has to be decentralized, ad hoc, autonomous, specific and microeconomic.

Not long after that is the first post about BAM (which I’ve just edited to point to the new home for BAM/BIM. It was actually being used at this stage by a Masters-level course in Information Systems, some of that story is told in this paper.

That experience led to the first mention of Web 2.0 course sites in September 2006. Given this 2013 post this appears an issue that remains open. Somewhat surprisingly a similar post from that time shows I was reading Steve Wheeler’s (@timbuckteeth) years ago.

It would seem that late 2006 saw me on quite the kick around this idea.

It’s also when I started work on the “The missing Ps” an attempt to develop a framework for thinking about technology adoption (and what was missing) in universities. This ended up with my first presentation of the idea which is also my first Slideshare presentation. That eventually morphed into the Ps Framework and became the lens for the analysis done in the literature review of my thesis.

The move to middle management

In February 2007 I started a new job. For various reasons this was not a long term role, nor was it from a number of perspectives a successful one. But it did commence my move away from being an Information Systems/Technology academic, which given shrinkage in those disciplines over recent years is probably a positive.

The group did some interesting things. I got my chance to help implement a Web 2.0 course site and I blogged a bit more. Time in that job reinforced the silliness of much of what central L&T organisations/management do to encourage quality learning and teaching. I also participated in the ASCILITE mentoring scheme in 2007.

I did propose the idea of extreme learning during 2007. We also got involved in PLEs. Both ideas never really went far at the institution.

We also engaged in a bit of an exploration of what students find useful.

Purgatory and the PhD

By the end of 2008 the writing was on the wall. It appeared likely I’d be made redundant and for other reasons decided it was time to move my website into a WordPress blog. The self-hosted website I’d started 14 years previously had to go away.

What perhaps irked me most about that move was the Google ranking. In 2008, the website was 7th on a Google for “david jones”. Given the commonality of the name, this was quite nice. I wonder where it sites now? (I realise that Google’s search algorithm etc has changed considerably over time). After 16 pages of Google results I won’t go any further. The “David Jones” chain of stores consumes most of those. “david jones -store” removes most of those, but still no me. Of course, the blog is hosted in the US and it’s hostname is davidtjones etc. A google of “david jones blog” returns my blog as the 3rd result.

Late 2008 saw us purchase a new bull.

Wandilla Zanzibar - Big Z

By the end of 2008 I did get to visit Paris with my wife. She presented a paper and I got an award which had more to do with my esteemed PhD supervisor (as it happens I’ll meet up with her again today) than me.

Eiffel from Trocadero

During the Paris trip I found out that I was getting a redundancy, but it wasn’t entirely clear what I would be employed as.

It was during this time that I first posted about the silliness of L&T evaluations, academic staff development, and minimum standards for course websites. Somethings which four/five years later has changed little.

During this time I did finally start working fairly consistently on the PhD which did eventually get finished. I also worked and thought more about BAM and BIM. Must get back to the idea of cooked feeds for BIM. Other vague ideas and interests from that time include: reflective alignment, task corruption, the myth of rationality, the fad cycle and management fashions, the grammar of school, nudging, the Chasm,

Early 2009 saw us become the breeder of race horses!!

Malina - the new money burner

July of 2009 also saw commencement of work on BIM

More importantly it was this period that really saw the growth of my PLN. e.g. this post which mentions Mark Smithers (@marksmithers) and Claire Brookes (@clairebrooks)

It also saw the start of the Indicators project our little foray into “learning analytics”.

By the end of July 2010 I was finally made redundant.

Student life

Initially I became a full-time PhD student and a Dad.

Kronosaurus corner

By the end of 2010 I’d just about finished the PhD and enrolled to become a high school teacher. By January 2011 the thesis was finished, by May it was accepted and graduation was July.

Dr Jones

While I kept blogging about educational technology stuff, most of the blog for 2011 was spent reflecting on what I was doing in the Graduate Diploma of Learning and Teaching. The experience as a student of institutional e-learning was interesting.

And then in November I had the opportunity to return to “life” as a University academic.

Academia redux

So almost a year ago today I started life as an education academic. Still feel like an immigrant, which is probably a good thing. A year of teaching other people’s courses will be redressed somewhat this year.

The blog posts last year focused on understanding the courses I was teaching, grappling with the current state of institutional e-learning, a bit of work on BIM, and sharing a bit of research thinking and writing.

Some statistics

  • 2012 – Visits: ~83,000; 92 posts
  • 2011 – Visits: ~70,000; 142 posts
  • 2010 – Visits: 59,880
  • 2009 – Visits: 48,482
  • 2008 – Visits: 3,181
    Moved to WordPress.com in October 2008.

Explorations of narrative research

For a long time I’ve had a vague interest in narrative research, i.e. it’s one of those things I always meant to learn more about. Here are some initial explorations.

Narrative approaches to education research

My google for “narrative research method education” turns up this site from the UK as the #1 hit. I’ll start there.

Connects strongly with me from the start due to this “Human beings are storying creatures. We make sense of the world and the things that happen to us by constructing narratives to explain and interpret events both to ourselves and to other people.”

Dave Snowden often uses the label Homo Narrans as an alternate label for the species. These folks have an academic reference for something similar

Indeed, somewhat playfully, it has been suggested that there is a case for revising the term homo sapiens to ‘homo fabulans – the tellers and interpreters of narrative’ (Currie, 1998: 2).

Lots of discussion here, particular liked this

Bruner has suggested that there are two basic ways in which human beings think about, make sense of, and tell about the world: narrative cognition and logico-scientific paradigmatic cognition (Bruner, 1986). Essentially, logico-scientific cognition is concerned with universals, empiricist reasoning and proof: and narrative cognition, with how the particular and specific contribute to the whole.

Especially the last point about the “particular and specific” have contributions to make for the whole.

Richardson’s (2000) criteria for evaluating narrative papers

  • Substantive contribution.
  • Aesthetic merit.
  • Reflexivity and participatory ethics.
  • Impact.
  • Experience – near

Deluze and rhizomes get a mention for a number of things, including structure.

List of narrative approaches

  • Autoethnography.
  • Ethnographic fiction
  • Poetry.
  • Performance ethnography.
  • Mixed genres.
  • Writing as a method of inquiry.
  • Narrative interviewing.

It would appear that autoethnography is approach currently most appropriate. Resources to follow up with include

  • Sparkes A (2001) Auto-ethnography: self indulgence or something more In: Bochner A and Ellis C (eds) Ethnographically Speaking Alta Mira Press CA
  • Bochner A (2000) Criteria Against Ourselves Qualitative Inquiry, Volume 6 Number 2, pp.266 – 272
  • Denzin, N. (2003) Performing (Auto)Ethnography: The Politics and Pedagogy of Culture (Thousand Oaks, Sage).
  • Ellis C and Bochner A (2000) Auto Ethnography, Personal Narrative, Reflexivity: Researcher As Subject, In Denzin N and Lincoln Y (eds) (2nd Ed) Handbook of Qualitative Research Sage Thousand Oaks
  • Etherington, K. (2004). Becoming a reflexive researcher. London: Jessica Kingsley

Autoethnography

So let’s explore this little thread a bit. Wikipedia is about as good a place as any to start.

Apparently I’m leaning towards analytic authoethnography, rather than evocative authoethnography, as per Ellingson and Ellis (2008, p 445) – as quoted on Wikipedia

Analytic autoethnographers focus on developing theoretical explanations of broader social phenomena, whereas evocative autoethnographers focus on narrative presentations that open up conversations and evoke emotional responses.

But perhaps not, in some other literature there appears to be some disquiet about the rationale of analytics autoethnography.

This captures an aspect/perspective interesting to me (again from Wikipedia)

According to Bochner and Ellis (2006), an autoethnographer is “first and foremost a communicator and a storyteller.” In other words, autoethnography “depicts people struggling to overcome adversity” and shows “people in the process of figuring out what to do, how to live, and the meaning of their struggles” (p. 111).

More resources

  • 35(4), August 2006 of the Journal of Contemporary Ethnography
  • 13(3), Summer 2007 of Culture and Organization
  • Humphreys, M. (2005). Getting Personal: Reflexivity and Autoethnograhic Vignettes, Qualitative Inquiry, 11, 840-860
  • Ellingson, Laura. L., & Ellis, Carolyn. (2008). Autoethnography as constructionist project. In J. A. Holstein & J. F. Gubrium (Eds.), Handbook of constructionist research (pp. 445-466). New York: Guilford Press.
  • Maréchal, G. (2010). Autoethnography. In A. J. Mills, G. Durepos & E. Wiebe (Eds.), Encyclopedia of case study research (Vol. 2, pp. 43–45). Thousand Oaks, CA: Sage Publications.

The clash of corporate IT and open source learning management system: a nascent research idea

Following up on a broader research idea the following paragraph summarises a nascent, more specific idea a colleague and I are stumbling toward. It fits within the broader agenda.

Suggestions, volunteers, pointers and criticisms welcome.

For now it’s just a single paragraph.

Open source education is seen by Davidson and Golberg (2009) as one of the principles key to the rethinking of learning institutions. Perhaps the most prevalent incursion of open source into Universities has been the adoption of open source Learning Management Systems (LMS) such as Moodle. Currently, at least 15 of the 38 Australian universities have adopted Moodle. The typical implementation of such systems appears to be informed more by best practice in enterprise systems implementation with its focus on scalability, consistency and cost savings (Jones, 2008). The governance and management of these open source LMSs seems likely to be an early focal point of the struggles between traditional organisational practices and policies and the requirements of the digital age. The aim of the research will be to investigate what is happening, what is changing and what might the future hold within the context of this struggle.

Defining the digital age and its influence on work integrated learning

If you hadn’t realised by now, this blog is often used as a notebook to record ideas in development. This is by no means complete, but the record is there.

Did you catch the high buzz word content in the title of this post? It can possibly mean only one thing, academics and grant writing. As a jobbing academic – saw that phrase used last week and like one definition, but then found this definition from professional wrestling which perhaps better fits some aspects of grant writing – I’ve fallen into writing a research grant on very short notice and with no real planning and forethought.

The perspective taken with this grant has evolved in an ad hoc way over the last week – which itself says something about the reality of this sort of work and how it is far from planned and relies much on chance – into something which may be looking at how the digital age might influence work integrated learning. This post is an attempt to quickly define what is meant by the digital age.

Don’t go looking for the definitive answer. Even if such a thing does exist, you won’t find it here. The following is hoping to be enough to given an idea and excite folk…..in the end I’m hoping it’s sufficient that I don’t need to do anything more.

If you have better definitions, resources etc to share, please do.

The following starts with the definition, following that is the rough working that went into forming the definition.

Defining the digital age, at least our interest

Davidson and Goldberg (2009, p. 15) paint a picture of the digital age bringing to an end the future of conventional learning institutions unless they recognise the urgent necessity of fundamental and foundational change. This research seeks to engage with, evaluate and respond to the type of fundamental and foundational change being created by the digital age. Changes so foundational that Lankshear (2003) describes four ways in which the digital age is challenging what it is to know and how this has potentially far-reaching implications for educational practice. Technology, however, will not transform learning by itself, instead it will be shaped and constrained by social, cultural, and economic factors (Warschauer, 2007). These changes and challenges should be properly considered, challenged and engaged with. After all, as Selwyn (2012) suggests there may be subtler and less disruptive methods through which educational institutions can effectively engage in the digital age, than radical alteration or disposal.

The future of learning institutions in a digital age

I’m starting my boning up on this topic with Davidson and Goldberg (2009) and some related resources – including Tony Bates’ review. The book was collaboratively/openly developed via a website. Practicing what they preach. I believe what I’m referencing (Davidson and Golberg, 2009) is a report, rather than the complete book.

Davidson and Goldberg (2010, p. 49)

This book advocates institutional change because our current formal educational institutions are not taking enough advantage of the modes of digital and participatory learning available to students today.

(this quote is from Bates).

So, what are the changes wrought (or to be wrought) by the digital age?

  • “Modes of learning have changed dramatically over the past two decades” (p. 8)
  • “How do collaborative, interdisciplinary, multi-institutional learning spaces help to transform traditional learning institutions and, specifically, universities?” (p. 11)
  • “A key term in thinking about these emergent shifts is participatory learning. Participatory learning includes the many ways that learners (of any age) use new technologies to participate in virtual communities” (p. 12)
    Which brings up echoes with connectivism and Downes’ views on what becoming a professional is (becoming part of the community of professionals).
  • “The concept of particpatory leanring is very different from “IT” (instructional technology)….IT tends to be top-down, designer determined, administratively
    driven, commercially fashioned. In participatory learning, outcomes are typically customizable by the participants.”. (pp. 12-13)
  • “This puts education and educators in the position of bringing up the rearguard, of holding desperately to the fragments of an educational system which, in its form, content, and assessments, is deeply rooted in an antiquated mode of learning” (p. 14)

The big quotes

  • “The future of conventional learning institutions is past—it’s over—unless those directing the course of our learning institutions realize, now and urgently, the necessity of fundamental and foundational change.” (p. 15)

In the end, not the sort of thing I was after, but will purloin a couple of quotes.

Rethinking the future of schools in the digital age

A bit of Selwyn (2012?)

(Selwyn 2012, p. 11)

Yet as with all debates about the “future” of education, it is important that we take time to properly consider and challenge these proposals and assumptions

(Selwyn 2012, p. 14)

We should be wary of giving up on the entire notion of the industrial-era school or university as it currently exists. Instead, it may be more productive – and certainly more practical to set about addressing the “problem” of formal education and technology in subtler and less disruptive ways than radically altering educational institutions or even disposing of them altogether.

References

Brown, J. S. (2001). Learning in the digital age. The Internet and the university: Forum (pp. 71–72). Retrieved from http://net.educause.edu/ir/library/pdf/FFPIU015.pdf

Davidson, C. N., & Goldberg, D. T. (2009). The future of learning institutions in a digital age. Digital Media. The MIT Press. Retrieved from http://mitpress.mit.edu/books/chapters/future_of_learning.pdf

Davidson, C and Goldberg, D. (2010) The Future of Thinking: Learning Institutions in a Digital Age Chicago: The John D. and Catherine T. MacArthur Foundation

Selwyn, N. (2012). School 2.0: Rethinking the future of schools in the digital age. In A. Jimoyiannis (Ed.), Research on e-Learning and ICT in Education (pp. 3-16). New York: Springer.

Warschauer, M. (2007). The paradoxical future of digital learning. Learning Inquiry, 1(1), 41-49.

Faculty-centered, peer-reviewed online course development models

So after three weeks at the new institution and busy thinking about how I’m going to do something interesting with my course design I find out that EOIs for a fairly large and beneficial set of research grants is due in a week or so. Bugger. No way I’m going to get my head around the new field, teaching and meet that deadline.

Which is when a fellow new staff member points me at Mears (2011). The following is an initial attempt to skim this and see what it means to me.

Summary

Seems to be a useful annotated bibliography of what I would class as the “systems” approach to course design. Heavily influenced by instructional designers/expert educational folk.

As I mention below and elsewhere I have grave concerns about this sort of approach in terms of its effectiveness, cost-effectiveness and sustainability. They all seem to end up being gamed.

I also wonder how much of these design models is influenced by the folk researching and proposing them either being digital visitors or having to deal primarily with teaching academics who are digital visitors. Or worse still, are working within the confines of a university that is a digital visitor. I wonder if you were able to work with digital residents or in a digital resident university, whether the necessary models would be radically different?

From the conclusions (Mears, 2011, p. 119)

Faculty engagement is shown to be the single most important arbiter of quality assurance in online instruction as reflected in the literature of this 30-study bibliography

This makes me wonder whether engaging in a mechanistic development process with appropriate quality assurance processes is the best way to engage faculty.

My prejudice here is that an approach where you actively seek to address the problems of faculty on their own terms leads to much better engagement.

The argument here seems to be that standards provide “offer a neutral point of discussion for transforming pedagogy” (Mears, 2011, p. 120). And that the academics ability to engage in self-directed discovery of how to meet the guidelines gives academic freedom etc.

And then (Mears, 2011, p. 122)

faculty engagement is dependent upon a perceived value that the institution designates for all the activities necessary to support interrelated features that scaffold quality in online instruction

This assumes throwing resources at the identified standards will be perceived as valuable by faculty. Too often these efforts become seen as wasteful or self-serving. wonder/believe that seeing the institutional actively trying to help address faculty problems. e.g. producing and disseminating coffee cups and coasters emblazoned with Chickering and Gamson’s (1987) 7 principles of good practice for undergraduate education.

And there is the four-phase teleological process (Mears, 2011, p 123)

  1. Chart: Resource and needs assessment, and long term, explicit strategic investment institutional plan for online infrastructure and faculty development
  2. Adopt: Collegial process selects competitive online mix of course offerings, integrates with department curriculum and gains institutional approvals; department seeks institutional incentives for faculty course development and delivery
  3. Build: Team-based faculty development concurrent with course development, supported by instructional design and technology team. Requires course online with homework.
  4. Launch and Assess: Plan, develop instruments and process, and apply periodic departmental assessment for program quality, student satisfaction and learning, and peer review of courses

My basic problem with this is the assumption that with something as young, diverse and fluid as online learning and teaching, it is sensible to identify before actually doing anything a set of standards by which to evaluate quality.

And here – with some comments from me – are the key characteristics of “quality, standards-driven course development”:

  • targeted faculty development in the use of teams to build and administer courses online;
    I agree, but the focus should be on building upon what is already there.
  • standardization focused on learning patterns with detailed assessment rubrics;
    Disagree, waste of time.
  • faculty-driven pedagogy;
    Agree in terms of working with what the academic wants and moving on from there.
  • encouraged student engagement through alignment with millennial preferences for
    social media and action-oriented projects;
    Really don’t think this is a millennial preference.
  • prompt and individualized feedback;
    Obviously, but the trouble is that an overly teleological process only provides feedback within the constaints of the identified purpose. A teleological process can modify itself based on the feedback it receives and changes it makes/sees.
  • soft skills of teamwork with substantive dialog emphasized in student project discourse;
    Yea, perhaps.
  • peer review and assessment.
    Yes.

There is much more here than I’ve covered in this. If you’re interested, I recommend taking a closer look.

Abstract

The abstract is (Mears, 2011, p. 3)

The engagement of faculty in development, course design, and peer review is central to quality online instruction. Thirty refereed case studies of standards-driven online course development in higher education since 2004 are annotated and analyzed for common principles, procedures, or recommended practices. Discussion explores strategic planning for faculty and online administrators, including four phases of implementation, faculty support needs, barriers to engagement, and instructional and technology characteristics faculty must weigh carefully in specific pedagogical designs

I like the first sentence – faculty engagement. From there it starts to lose me with “standards-driven” and extracting “common principles, procedures, or recommended practices” from case studies. Not to mention strategic planning and the idea of four phases.

Reminded of something from Papert (1993, p. xiii)

Intellectual activity does not progress, as logicians and the designers of school curricula might want us to believe, by going step-by-step from one clearly stated and well-confirmed truth to the next.

It sounds like this articles is taking a very teleological approach, while I’m an ateleological kind of guy – my prior summary of the difference between teleological/ateleogical.

But must allow biases to prejudice reading. Must be objective.

Purpose

This is an annotated bibliography that aims (Mears, p. 10)

to provide an overview of
faculty-centered, peer reviewed online higher education course development models

where “course development models”

may reflect a systems approach to structuring the production of teaching courseware and implementation processes by targeting specific learning objectives and aligning them with both learning and interactive media strategies, utilizing a range of expertise

This might be done by:

  • purchasing courseware;
  • using teams of faculty and instructional designers; or,
  • individual faculty using varying strategies.

The literature chosen must cover at least one of three aspects

  1. standards-driven faculty development for online instruction;
  2. faculty see a fit between the model and the pedagogy/curriculum they use; and,
  3. effects of uniform online delivery on quality.

I really do hate words like uniform and consistency when used to describe learning, teaching or course design. I much prefer Chris Dede’s comparison of learning to eating, sleeping and bonding (1m39s mp3 audio).

Ahh, it is perhaps not the first time this reaction has been formed. Mears (2011, p.11)

Embedded within this assumption is the notion that
a standards-driven model provides a framework for quality rather than similarity (Lewis et al., 2011; Olson & Shershneva, 2004; Seok, 2007; Shelton, 2010); quality standards may be shown to support the intellectual richness that is the promise of academic freedom in higher education (Lewis et al., 2011; Lam & McNaught, 2006; Olson & Shershneva, 2004; Shelton, 2010).

In extracting principles from these case studies, one of the aims is to address some of the concerns about “uniform delivery of standards-driven instruction”, including

  1. Compatibility between different pedagogies and computer-based instruction;
  2. Impact on academic freedom;
  3. Intellectual challenge of coursework; and,
  4. Competitive difference for institutions.

Uniform delivery

With the LMS, and print-based distance education before it, there was a tendency to what I called “quality through consistency”. It appears that “uniform delivery” is the more academic term and refers (Mears, 2011, p. 29)

course development models that follow “a common format for course structure” to “provide consistency for both instructors and students” (Knowles & Kalata, 2007, p. 5). Uniform delivery is not a proxy for quality, but standards applied uniformly can maintain the quality of content while gaining efficiencies in interoperability and re-use or collaboration among institutions (Seok, 2007).

Must remember that.

References

The following are some of the more interesting references from the annotated bibliography

Graham, L. & Thomas, L. (2011). Certification in distance learning for online instructors: exploration of the creation of an organic model for a research-based state institution. Online Journal of Distance Learning Administration, 4(1).

The mention of organic makes this sound interesting, maybe some ateleological type thinking.

Eib, B.J., & Miller, P. (2006). Faculty development as community building: An approach to professional development that supports communities of practice for online teaching. International Review of Research in Open and Distance Learning, 7(2), 1-15.

The community aspect sounds interesting.

Green, N., Edwards, H., Wolodko, B., Stewart, C., Brooks, M., & Littledyke, R. (2010). Reconceptualizing higher education pedagogy in online learning. Distance Education, 31(3), 257–273. Charles Stuart University, New South Wales Australia: Routledge. doi: 10.1080/01587919.2010.513951

I hear the lead author is a very talented and insightful academic, teacher and researcher. Plus some of the theoretically underpinnings appears potentially useful to my mumblings and bumblings around course design.

Ward, M., Peters, G, & Shelley, K. (2010). Student and faculty perceptions of the quality of online learning experiences. International Review of Research in Open and Distance Learning, 11(3), 58-77.

Might have some application locally in what I’m doing. Ahh, it’s focused on synchronous teaching. Still possibly useful as I dislike synchronous. Apparently the authors develop arguments around the weaknesses of asyncrhonous.

References

Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3-7.

Mears, L. (2011). An Overview of Faculty-Centered, Peer-Reviewed Online Course Development Models for Application within Accredited Institutions of Higher Education. Higher Education. AIM Capstone 2011; Linda Mears. Retrieved from https://scholarsbank.uoregon.edu/xmlui/bitstream/handle/1794/11398/Mears-2011.pdf?sequence=1

Papert, S. (1993). Mindstorms: Children, Computers and Powerful ideas (2nd ed.). New York, New York: Basic Books.

Social media in higher education – to read

One of these days I plan/hope to have the time to catch up on some reading that I have let slide over the last couple of years. To this end this post marks a new practice. I’m using a “to read” category on this blog to save some detail of things I need to come back to, read more carefully and follow up on some of the references.

First cab off the rank is this Neil Selwyn essay titled “Social Media in Higher Education”. As I’m skimming it now there seems to be more to come back to. Especially some of the more recent references to various suggestions about trends in e-learning. Including the expected, if slightly cringe worthy, “pedagogy 2.0”.

Powered by WordPress & Theme by Anders Norén

css.php