Assembling the heterogeneous elements for (digital) learning

Category: phd Page 1 of 11

One example of industrial e-learning as "on the web" not "of the web"

The following arises from some recent experiences with the idea of “minimum course sites” and this observation from @cogdog in this blog post

I have no idea if this is off base, but frankly it is a major (to me) difference of doing things ON the web (e.g. putting stuff inside LMSes) and doing things OF the web.

It’s also an query to see if anyone knows of an institution that has implemented a search engine across the institutional e-learning systems in a way that effectively allows users to search for resources in a course centric way.

The symptom

There’s a push on at my current institution’s central L&T folk to develop a minimum course site standard. Some minimum set of services, buttons etc. that will achieve the nirvana of consistency. Everything will be the same.

The main espoused reason as to why this is a good thing is that the students have been asking for it. There has been consistent feedback from students that none of the course sites are the same.

The problem

Of course, the real problem isn’t that students want everything to be the same. The real problem is that they can’t find what they are looking for. Sure, if everything was the same then they might have some ideas about where to find things, but that has problems including:

  • The idea that every course site at a university can be structured the same is a misunderstanding of the diversity inherent in course. Especially as people try to move away from the traditional models such as lecture/tutorial etc.
  • The idea that one particular structure will be understandable/appropriate to all people also is questionable.
  • Even if all the sites are consistent and this works, it won’t solve the problem of when the student is working on a question about “universal design” and wants to find where that was mentioned amongst the many artefacts in the course site.

The solution

The idea that the solution to this problem is to waste huge amounts of resources in the forlorn attempt to achieve some vaguely acceptable minimum standards that is broadly applicable seems to be a perfect example of “doing things ON the web, rather than doing things OF the web”.

I can’t remember the last time I visited a large website and attempted to find some important information by navigating through the site structure. Generally, I – like I expect most people – come to a large site almost directly to the content I am interested in either through a link provided by someone or via a search engine.

Broader implications

To me the idea of solving this problem through minimum standards is a rather large indication of the shortcomings of industrial e-learning. Industrial e-learning is the label I’ve applied to the current common paradigm of e-learning adopted by most universities. It’s techno-rational in its foundations and involves the planned management of large enterprise systems (be they open source or not). I propose that “industrial e-learning” is capable and concerned primarily with “doing things On the web, rather than doing things OF the web”.

Some potential contributing factors might include:

  1. Existing mindsets.
    At this institution, many of the central L&T folk come from a tradition of print-based distance education where consistency of appearance was a huge consideration. Many of these folk are perhaps not “of the web”.
  2. Limitations of the tools.
    It doesn’t appear that Moodle has a decent search engine, which is not surprising given the inspiration of its design and its stated intent of not being an information repository.
  3. The nature of industrial e-learning, its product and process.
    A key characteristic of industrial e-learning is a process that goes something like this
    1. Spend a long time objectively selecting an appropriate tool.
    2. Use that tool for along time to recoup the cost of moving to the new tool.
    3. Aim to keep the tool as vanilla as possible to reduce problems with upgrades from the vendor.
      This applies to open source systems as much as proprietary systems.
    4. Employ people to help others learn how to best use the system to achieve their ends.
      Importantly, the staff employed are generally not their to help others learn how to “best achieve their ends”, the focus definitely tends to be on ho to “best use the system to achieve their ends”.
    5. Any changes to the system have to be requested through a long-scale process that involves consensus amongst most people and the approval of the people employed in point d.

    This means that industrial e-learning is set up to do things the way the chosen systems work. If you have to do something that isn’t directly supported by the system, it’s very, very hard. e.g. add a search engine to Moodle.

All of these make it very hard for industrial e-learning to be “doing things OF the web”

People and e-learning – limitations and an alternative

So the last of three sections examining the limitations of industrial e-learning and suggesting an alternative. Time to write the conclusion, read the paper over again and cut it down to size.

People

The characteristics of the product and process of industrial e-learning (e.g. focus on long periods of stable use and the importance of efficient use of the chosen LMS) directly reinforced by and directly impact the people and roles involved with tertiary e-learning. This section briefly examines just four examples of this impact, including:

  1. The negative impact of organizational hierarchies on communication and knowledge sharing.
    The logical decomposition inherent in teleological design creates numerous, often significant, organizational boundaries between the people involved with e-learning. Such boundaries are seen as inhibiting the ability to integrate knowledge across the organization. The following comments from Rossi and Luck (2011, p. 68) partially illustrate this problem:

    During training sessions … several people made suggestions and raised issues with the structure and use of Moodle. As these suggestions and issues were not recorded and the trainers did not feed them back to the programmers … This resulted in frustration for academic staff when teaching with Moodle for the first time as the problems were not fixed before teaching started.

  2. Chinese whispers.
    Within an appropriate governance structure the need for changes to an LMS would typically need to flow up from the users to a central committee typically made up of senior leaders from the faculties, Information Technology and central learning and teaching. There would normally be some representation from teaching staff and students. The length of the communication chain for the original need becomes like a game of Chinese Whispers as it is interpreted through the experiences and biases of those involved. Leading to this impression reported by Rossi and Luck (2011, p. 69)

    The longer the communication chain, the less likely it was that academic users’ concerns would be communicated correctly to the people who could fix the problems.

    The cost of traversing this chain of communication means it is typically not worth the effort of raising small-scale changes.

    Not to mention killing creativity which just came through my Twitter feed thanks to @kyliebudge.

  3. Mixed purposes.
    Logical decomposition also encourages different organizational units to focus on their part of the problem and lose sight of the whole picture. An IT division evaluated on its ability to minimize cost and maximize availability is not likely to want to support technologies in which it has limited expertise. This is one explanation for why the leader of an IT division would direct the IT division’s representatives on an LMS selection panel to ensure that the panel selected the LMS implemented in Java. Or a decision to use the latest version of the Oracle DBMS – the DBMS supported by the IT division – to support the new Moodle installation even though it hasn’t been tested with Moodle and best practice advice is to avoid Oracle. A decision that leads to weeks at the start of the “go live” term where Moodle is largely unavailable.
  4. The perils of senior leadership.
    Having the support and engagement of a senior leader at an institution is often seen as a critical success factor for an LMS implementation. But when the successful completion of the project is tied to the leader’s progression within the leadership hierarchy it can create the situation where the project will be deemed a success, regardless of the outcome.

As an alternative, the Webfuse system relied on a multi-skilled, integrated development and support team. This meant that the small team was responsible for training, helpdesk support, and systems development. The helpdesk person handling the user’s problem was typically also a Webfuse developer who was empowered to make small changes without formal governance approval. Behrens (2009, p. 127) quotes a manager in CQU’s IT division describing the types of changes made to Webfuse as “not even on the priority radar” due to traditional IT management techniques. The developers were also located within the faculty, so they also interacted with academic staff in the corridors and the staff room. This context created an approach to the support of an e-learning system with all the hallmarks of a social constructivist, situated cognition, or community of practice. The type of collaborative and supportive environment identified by Tickle et al (2009) in which academics learn through attempts to solve genuine educational problems, rather than being shown how to adapt their needs to the constraints of the LMS.

References

Behrens, S. (2009). Shadow systems: the good, the bad and the ugly. Communications of the ACM, 52(2), 124-129.

Rossi, D., & Luck, J. (2011). Wrestling, wrangling and reaping: An exploration of educational practice and the transference of academic knowledge and skill in online learning contexts. Studies in Learning, Evaluation, Innovation and Development, 8(1), 60-75. Retrieved from http://www.sleid.cqu.edu.au/include/getdoc.php?id=1122&article=391&mode=pdf

Tickle, K., Muldoon, N., & Tennent, B. (2009). Moodle and the institutional repositioning of learning and teaching at CQUniversity. Auckland, NZ. Retrieved from http://www.ascilite.org.au/conferences/auckland09/procs/tickle.pdf

Introducing the alternative

The last couple of posts have attempted to (in the confines of an #ascilite12 paper) summarise some constraints with the dominant product and process models used in industrial e-learning and suggest an alternative. The following – which probably should have been posted first – describes how and where this alternative comes from.

As all this is meant to go into an academic paper, the following starts with a discussion about “research methods” before moving onto describing some of the reasons why this alternative approach might have some merit.

As with the prior posts, this is all still first draft stuff.

Research methods and limitations

From the initial stages of its design the Webfuse system was intended to be a vehicle for both practice (it hosted over 3000 course sites from 1997-2009) and research. Underpinning the evolution of Webfuse was an on-going process of cycle action research that sought to continually improve the system through insights from theory and observation of use. This commenced in 1996 and continued, at varying levels of intensity, through to 2009 when the system ceased directly supporting e-learning. This work has contributed in varying ways to over 25 peer-reviewed publications. Webfuse has also been studied by other researchers investigating institutional adoption of e-learning systems (Danaher, Luck, & McConachie, 2005) and shadow systems in the context of ERP implementation (Behrens, 2009; Behrens & Sedera, 2004).

Starting in 2001 the design of Webuse became the focus of a PhD thesis (Jones, 2011) that made two contributions towards understanding e-learning implementation within universities: the Ps Framework and an Information Systems Design Theory (ISDT). The Ps Framework arose out of an analysis of existing e-learning implementation practices and as a tool to enable the comparison of alternate approaches (Jones, Vallack, & Fitzgerald-Hood, 2008). The formulated ISDT – An ISDT for emergent university e-learning systems –offers guidance for e-learning implementation that brings a number of proposed advantages over industrial e-learing. These contributions to knowledge arose from an action research process that combined broad theoretical knowledge – the principles of the ISDT are supported by insights from a range of kernel theories – with empirical evidence arising from the design and support of a successful e-learning system. Rather than present the complete ISDT – due primarily to space constraints – this paper focuses on how three important components of e-learning can be re-conceptualised through the principles of the ISDT.

The ISDT – and the sub-set of principles presented in this paper – seek to provide theoretical guidance about how to develop and support information systems for university e-learning that are capable of responding to the dominant characteristics (diversity, uncertainty and rapid change) of university e-learning. This is achieved through a combination of product (principles of form and function) and process (principles of implementation) that focus on developing a deep and evolving understanding of the context and use of e-learning. It is through being able to use that understanding to make rapid changes to the system, which ultimately encourages and enables adoption and on-going adaptation. It suggests that any instantiation built following the ISDT will support e-learning in a way that: is specific to the institutional context; results in greater quality, quantity and variety of adoption; and, improves the differentiation and competitive advantage of the host institution.

As with all research, the study described within this study has a number of limitations that should be kept in mind when considering its findings. Through its use of action research, this work suffers the same limitations, to varying degrees, of all action research. Baskerville and Wood-Harper (1996) identify these limitations as: (1) lack of impartiality of the researcher; (2) lack of discipline; (3) mistaken for consulting; and (4) context-dependency leading to difficulty of generalizing findings. These limitations have been addressed within this study through a variety of means including: a history of peer-reviewed publications throughout the process; use of objective data sources; the generation of theory; and, an on-going process of testing. Consequently the resulting ISDT and the principles described here have not been “proven”. This was not the aim of this work. Instead, the intent was to gather sufficient empirical and theoretical support to build and propose a coherent and useful alternative to industrial e-learning. The question of proof and further testing of the ISDT in similar and different contexts provides – as in all research aiming to generate theory – an avenue for future research.

On the value of Webfuse

This section aims to show that there is some value in considering Webfuse. It seeks to summarise the empirical support for the ISDT and the principles described here by presenting evidence that the development of Webfuse led to a range of features specific to the institution and to greater levels of adoption. It is important to note that from 1997 through 2005 Webfuse was funded and controlled by one of five faculties at CQUniversity. Webfuse did not become a system controlled by a central IT division until 2005/2006 as a result of organizational restructures. During the life-span of Webfuse CQU adopted three different official, institutional LMS: WebCT (1999), Blackboard (2004), and Moodle (2010).

Specific to the context

During the period from 1999 through 2002 the “Webfuse faculty” saw a significant increase in the complexity of its teaching model including the addition of numerous international campuses situated within capital cities and a doubling in student numbers, primarily through full-fee paying overseas students. By 2002, the “Webfuse faculty” was teaching 30% of all students at the University. Due to the significant increased in complexity of teaching in this context, a range of teaching management and support services were integrated into Webfuse including: staff and student “portals”, an online assignment submission and management system, a results upload application, an informal review of grade system, a timetable generator, student photo gallery, academic misconduct database, email merge facility, and assignment extension systems.

The value of these systems to the faculty is illustrated by this quote from the Faculty annual report for 2003 cited by Danaher, Luck & McConachie (2005, p. 39)

[t]he best thing about teaching and learning in this faculty in 2003 would be the development of technologically progressive academic information systems that provide better service to our students and staff and make our teaching more effective. Webfuse and MyInfocom development has greatly assisted staff to cope with the complexities of delivering courses across a large multi-site operation.

By 2003 the faculties not using Webfuse were actively negotiating to enable their staff to have access to the services. In 2009 alone, over 12,000 students and 1100 staff made use of these services. Even though no longer officially supported, it is a few of these services that continue to be used by the university in the middle of 2012.

Quotes from staff using the Webfuse systems reported in various publications (Behrens, 2009; Behrens, Jamieson, Jones, & Cranston, 2005; Jones, Cranston, Behrens, & Jamieson, 2005) also provide some insights into how well Webfuse supported the specific context at CQUni.

my positive experience with other Infocom systems gives me confidence that OASIS would be no different. The systems team have a very good track record that inspires confidence

The key to easy use of OASIS is that it is not a off the shelf product that is sooooo generic that it has lost its way as a course delivery tool.

I remember talking to [a Webfuse developer] and saying how I was having these problems with uploading our final results into [the Enterprise Resource Planning (ERP) system] for the faculty. He basically said, “No problem, we can get our system to handle that”…and ‘Hey presto!’ there was this new piece of functionality added to the system … You felt really involved … You didn’t feel as though you had to jump through hoops to get something done.

Beyond context specific systems supporting the management of learning and teaching, Webfuse also included a number of context specific learning and teaching innovations. A short list of examples includes:

  • the course barometer;
    Based on an innovation (Svensson, Andersson, Gadd, & Johnsson, 1999) seen at a conference the barometer was designed to provide students a simple, anonymous method for providing informal, formative feedback about a course (Jones, 2002). Initially intended only for the authors courses, the barometer became a required part of all Webfuse course sites from 2001 through 2005. In 2007/2008 the barometers were used as part of a whole of institution attempt to encourage formative feedback in both Webfuse and Blackboard.
  • Blog Aggregation Management (BAM); and
    BAM allowed students to create individual, externally hosted web-logs (blog) and use them as reflective journals. Students registered their external blog with BAM, which then mirrored all of the students’ blog posts on an institutional server and provided a management and marking interface for teaching staff. Created by the author for use in his own teaching in 2006, BAM was subsequently used in 26 course offerings by 2050+ students and ported to Moodle as BIM (Jones & Luck, 2009). In reviewing BAM, the ELI guide to blogging (Coghlan et al., 2007) identified as
    One of the most compelling aspects of the project was the simple way it married Web 2.0 applications with institutional systems. This approach has the potential to give institutional teaching and learning systems greater efficacy and agility by making use of the many free or inexpensive—but useful—tools like blogs proliferating on the Internet and to liberate institutional computing staff and resources for other efforts.
  • A Web 2.0 course site.
    While it looked like a normal course website, none of the functionality – including discussion, wiki, blog, portfolio and resource sharing – was implemented by Webfuse. Instead, freely available and externally hosted Web 2.0 tools and services provided all of the functionality. For example, each student had a portfolio and a weblog provided by the site http://redbubble.com. The content of the default course site was populated by using BAM to aggregate RSS feeds (generated by the external tools) which were then parsed and displayed by Javascript functions within the course site pages. Typically students and staff did not visit the default course site, as they could access all content by using a course OPML file and an appropriate reader application.

Even within the constraints placed on the development of Webfuse it was able to develop an array of e-learning applications that are either not present in industrial LMSes, were added much later than the Webfuse services, or had significantly reduced functionality.

Greater levels of adoption

Encouraging staff adoption of the Webfuse system was one of the main issues raised in the original Webfuse paper (Jones & Buchanan, 1996). Difficulties in encouraging high levels of quality use of e-learning within universities has remained a theme throughout the literature. Initial use of Webfuse in 1997 and 1998 was not all that successful in achieving that goal, with only five – including the designer of Webfuse who made 50% of all edits using the system – of 60 academic staff making any significant use of Webfuse by early 1999 (Jones & Lynch, 1999). These limitations were addressed from 1999 onwards by a range of changes to the system, how it was supported and the organizational context. The following illustrates the success of these changes by comparing Webfuse adoption with that of the official LMS (WebCT 1999-2003/4; Blackboard 2004-2009) used primarily by the non-Webfuse faculties. It first examines the number of course sites and then examines feature adoption.

From 1997 Webfuse automatically created a default course site for all Faculty courses by drawing on a range of existing course related information. For the official institutional LMS course sites were typically created on request and had to be populated by the academics. By the end of 2003 – 4 years after the initial introduction of WebCT as the official institutional LMS – only 15% (141) of courses from the non-Webfuse faculties had WebCT course sites. At the same time, 100% (302) of the courses from the Webfuse faculty had course sites. Due to the need for academics to populate WebCT and Blackboard courses sites, the presence of a course website doesn’t necessarily imply use. For example, Tickle et al (2009) report that 21% of the 417 Blackboard courses being migrated to Moodle in 2010 contained no documents.

Research examining the adoption of specific categories of LMS features provides a more useful insight into LMS usage. Figures 1 through 4 use the research model proposed by Malikowski, Thompson, & Thies (2007) to compare the adoption of LMS features between Webfuse (the thick continuous lines in each figure), CQUni’s version of Blackboard (the dashed lines), and range of adoption rates found in the literature by Malikowski et al (2007) (the two dotted lines in each figure). This is done for four of the five LMS feature categories identified by Malikowski et al (2007): content transmission (Figure 1), class interaction (Figure 2), student assessment (Figure 3), and course evaluation (Figure 4).

(Click on the graphs to see large versions)

Content Transmission Interactions
Figure 1: Adoption of content transmission features: Webfuse, Blackboard and Malikowski Figure 2: Adoption of class interactions features: Webfuse, Blackboard and Malikowski
(missing archives of most pre-2002 course mailing lists)
Evaluate Students Evaluate Courses
Figure 3: Adoption of student assessment features: Webfuse, Blackboard and Malikowski Figure 4: Adoption of course evaluation features: Webfuse, Blackboard and Malikowski

The Webfuse usage data included in Figures 1 through 4 only include actual feature use by academics or students. For example, from 2001 through 2005 100% of Webfuse courses contained a course evaluation feature called a course barometer, only courses where the course barometer was actually used by students are included in Figure 4. Similarly, all Webfuse default course sites contained content (either automatically added from existing data repositories or copied across from a previous term). Figure 1 only includes data for those Webfuse course sites where teaching staff modified or added content.

Figures 2 and 3 indicate Webfuse adoption rates of greater than 100%. This is possible because a number of Webfuse features – including the EmailMerge and online assignment submission and management applications – were being used in course sites hosted on Blackboard. Webfuse was seen as providing services that Blackboard did not provide, or that were significantly better than what Blackboard did provide. Similarly, the spike in Webfuse course evaluation feature adoption in 2008 to 51.6% is due to a CQU wide push to improve formative feedback across all courses that relied on the Webfuse course barometer feature.

Excluding use by non-Webfuse courses and focusing on the time period 2003-2006, Figures 2 and 3 show that adoption of Webfuse class interaction and student assessment features significantly higher than the equivalent Blackboard features at CQU. It is also significantly higher than the adoption rates found by Malikowski et al (2007) in the broader literature. It also shows adoption rates that appear to be somewhat higher than that found amongst 2008, Semester 1 courses at the University of Western Sydney and Griffith University by Rankine et al (2009). Though it should be noted that Rankine et al (2009) used different sampling and feature categorization strategies that make this comparison tentative.

References

Behrens, S. (2009). Shadow systems: the good, the bad and the ugly. Communications of the ACM, 52(2), 124-129.

Behrens, S., Jamieson, K., Jones, D., & Cranston, M. (2005). Predicting system success using the Technology Acceptance Model: A case study. 16th Australasian Conference on Information Systems. Sydney. Retrieved from http://cgit.nutn.edu.tw:8080/cgit/PaperDL/tkw_090717140108.pdf

Behrens, S., & Sedera, W. (2004). Why do shadow systems exist after an ERP implementation? Lessons from a case study. In C.-P. Wei (Ed.), (pp. 1713-1726). Shanghai, China.

Coghlan, E., Crawford, J., Little, J., Lomas, C., Lombardi, M., Oblinger, D., & Windham, C. (2007). ELI Discovery Tool: Guide to Blogging. EDUCAUSE. Retrieved from http://www-cdn.educause.edu/eli/GuideToBlogging/13552

Danaher, P. A., Luck, J., & McConachie, J. (2005). The stories that documents tell: Changing technology options from Blackboard, Webfuse and the Content Management System at Central Queensland University. Studies in Learning, Evaluation, Innovation and Development, 2(1), 34-43.

Jones, D. (2002). Student Feedback, Anonymity, Observable Change and Course Barometers. In S. R. Philip Barker (Ed.), (pp. 884-889). Denver, Colorado: AACE.

Jones, D. (2011). An Information Systems Design Theory for E-learning. Philosophy. Australian National University. Retrieved from https://djon.es/blog/research/phd-thesis/

Jones, D., & Buchanan, R. (1996). The design of an integrated online learning environment. In P. J. Allan Christie Beverley Vaughan (Ed.), (pp. 331-345). Adelaide.

Jones, D., Cranston, M., Behrens, S., & Jamieson, K. (2005). What makes ICT implementation successful: A case study of online assignment submission. Adelaide.

Jones, D., & Luck, J. (2009). Blog Aggregation Management: Reducing the Aggravation of Managing Student Blogging. In G. Siemns & C. Fulford (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications 2009 (pp. 398-406). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/31530

Jones, D., & Lynch, T. (1999). A Model for the Design of Web-based Systems that supports Adoption, Appropriation and Evolution. In Y. D. San Murugesan (Ed.), (pp. 47-56). Los Angeles.

Jones, D., Vallack, J., & Fitzgerald-Hood, N. (2008). The Ps Framework: Mapping the landscape for the PLEs@CQUni project. Hello! Where are you in the landscape of educational technology? ASCILITE’2008. Melbourne.

Malikowski, S., Thompson, M., & Theis, J. (2007). A model for research into course management systems: bridging technology and learning theory. Journal of Educational Computing Research, 36(2), 149-173.

Rankine, L., Stevenson, L., Malfroy, J., & Ashford-Rowe, K. (2009). Benchmarking across universities: A framework for LMS analysis. Ascilite 2009. Same places, different spaces (pp. 815-819). Auckland. Retrieved from http://www.ascilite.org.au/conferences/auckland09/procs/rankine.pdf

Svensson, L., Andersson, R., Gadd, M., & Johnsson, A. (1999). Course-Barometer: Compensating for the loss of informal feedback in distance education (pp. 1612-1613). Seattle, Washington: AACE.

Tickle, K., Muldoon, N., & Tennent, B. (2009). Moodle and the institutional repositioning of learning and teaching at CQUniversity. Auckland, NZ. Retrieved from http://www.ascilite.org.au/conferences/auckland09/procs/tickle.pdf

The e-learning process – limitations and an alternative

And here’s the followup to the well received “LMS Product” post. This is the second section looking at the limitations of how industrial e-learning is implemented, this time focusing on the process used. Not really happy with this one, space limitations are making it difficult to do a good job of description.

Process

It has become a maxim of modern society that without objectives, without purpose there can be no success, the setting of goals and achieving them has become the essence of “success” (Introna, 1996). Many, if not most, universities follow, or at least profess to follow, a purpose driven approach to setting strategic directions (Jones, Luck, McConachie, & Danaher, 2005). This is how institutional leaders demonstrate their strategic insight, their rationality and leadership. This is not a great surprise since such purpose driven processes – labeled as teleological processes by Introna (1996) – has dominated theory and practice to such an extent that it has become ingrained. Even though the debate between the “planning school” of process thought and the “learning school” of process thought has been one of the most pervasive debates in management (Clegg, 2002).

Prior papers (Jones et al., 2005; Jones & Muldoon, 2007) have used the nine attributes of a design process formulated by Introna (1996) to argue that purpose driven processes are particularly inappropriate to the practice of tertiary e-learning. The same papers have presented and illustrated the alternative, ateleological processes. The limitations of teleological processes can be illustrated by examining Introna’s (1996) three necessary requirements for teleological design processes

  1. The system’s behaviour must be relatively stable and predictable.
    As mentioned in the previous section, stability and predictability do not sound like appropriate adjectives for e-learning, especially into the future. Especially given the popular rhetoric about organizations in the present era no longer being stable, and instead are continuously adapting to shifting environments that places them in a state of constantly seeking stability while never achieving it (Truex, Baskerville, & Klein, 1999).
  2. The designers must be able to manipulate the system’s behaviour directly.
    Social systems cannot be “designed” in the same way as technical systems, at best they can be indirectly influenced (Introna, 1996). Technology development and diffusion needs cooperation, however, it takes place in a competitive and conflictual atmosphere where different social groups – each with their own interpretation of the technology and the problem to be solved – are inevitably involved and seek to shape outcomes (Allen, 2000). Academics are trained not to accept propositions uncritically and subsequently cannot be expected to adopt strategies without question or adaptation (Gibbs, Habeshaw, & Yorke, 2000).
  3. The designers must be able to determine accurately the goals or criteria for success.
    The uncertain and confused arena of social behaviour and autonomous human action make predetermination impossible (Truex, Baskerville et al. 2000). Allen (2000) argues that change in organizational and social setting involving technology is by nature undetermined.

For example, Tickle et al (2009) offer one description of the teleological process used to transition CQUni to the Moodle LMS in 2009. One of the institutional policies introduced as part of this process was the adoption of Minimum Service Standards for course delivery (Tickle et al., 2009, p. 1047). Intended to act as a starting point for “integrating learning and teaching strategies that could influence students study habits” and to “encourage academic staff to look beyond existing practices and consider the useful features of the new LMS” (Tickle et al., 2009, p. 1042). In order to assure the quality of this process a web-based checklist was implemented in another institutional system with the expectation that the course coordinator and moderator would actively check the course site met the minimum standards. A senior lecturer widely recognized as a quality teacher described the process for dealing with the minimum standards checklist as

I go in and tick all the boxes, the moderator goes in and ticks all the boxes and the school secretary does the same thing. It’s just like the exam check list.

The minimum standards checklist was removed in 2011.

A teleological process is not interested in learning and changing, only in achieving the established purpose. The philosophical assumptions of teleological processes – modernism and rationality – are in direct contradiction to views of learning meant to underpin the best learning and teaching. Rossi and Luck (2011, p. 62) talk about how “[c]onstructivist views of learning pervade contemporary educational literature, represent the dominant learning theory and are frequently associated with online learning”. Wise and Quealy (2006, p. 899) argue, however, that

while a social constructivist framework may be ideal for understanding the way people learn, it is at odds not only with the implicit instructional design agenda, but also with current university elearning governance and infrastructure.

Staff development sessions become focused on helping the institution achieve the efficient and effective use of the LMS, rather than quality learning and teaching. This leads to staff developers being “seen as the university’s ‘agent’” (Pettit, 2005, p. 253). There is a reason why Clegg (2002) references to teleological approaches as the “planning school” of process thought and the alternative ateological approach the “learning school” of process.

The ISDT abstracted from the Webfuse work includes 11 principles of implementation (i.e. process) divided into 3 groups. The first and second groupings refer more to people and will be covered in the next section. The second grouping focused explicitly on the process and was titled “An adopter-focused, emergent development process”. Webfuse achieved this by using an information systems development processes based on principles of emergent development (Truex et al., 1999) and ateleological design (Introna, 1996). The Webfuse development team was employed and located within the faculty. This allowed for a much more in-depth knowledge of the individual and organizational needs and an explicit focus on responding to those needs. The quote early in this paper about the origins of the results uploading system is indicative of this. Lastly, at its best Webfuse was able to seek a balance between teleological and ateleological processes due to a Faculty Dean who recognized the significant limitations of a top-down approach.

This process, when combined with a flexible and responsive product, better enabled the Webfuse team to work with the academics and students using the system to actively modify and construct the system in response to what was learned while using the system. It was an approach much more inline with a social constructivist philosophy.

References

Allen, J. (2000). Information systems as technological innovation. Information Technology & People, 13(3), 210-221.

Clegg, S. (2002). Management and organization paradoxes. Philadelphia, PA: John Benjamins Publishing.

Gibbs, G., Habeshaw, T., & Yorke, M. (2000). Institutional learning and teaching strategies in English higher education. Higher Education, 40(3), 351-372.

Introna, L. (1996). Notes on ateleological information systems development. Information Technology & People, 9(4), 20-39.

Jones, D., Luck, J., McConachie, J., & Danaher, P. A. (2005). The teleological brake on ICTs in open and distance learning. Adelaide.

Jones, D., & Muldoon, N. (2007). The teleological reason why ICTs limit choice for university learners and learning. In R. J. Atkinson, C. McBeath, S. K. A. Soong, & C. Cheers (Eds.), (pp. 450-459). Singapore. Retrieved from http://www.ascilite.org.au/conferences/singapore07/procs/jones-d.pdf

Pettit, J. (2005). Conferencing and Workshops: a blend for staff development. Education, Communication & Information, 5(3), 251-263. doi:10.1080/14636310500350505

Rossi, D., & Luck, J. (2011). Wrestling, wrangling and reaping: An exploration of educational practice and the transference of academic knowledge and skill in online learning contexts. Studies in Learning, Evaluation, Innovation and Development, 8(1), 60-75. Retrieved from http://www.sleid.cqu.edu.au/include/getdoc.php?id=1122&article=391&mode=pdf

Tickle, K., Muldoon, N., & Tennent, B. (2009). Moodle and the institutional repositioning of learning and teaching at CQUniversity. Auckland, NZ. Retrieved from http://www.ascilite.org.au/conferences/auckland09/procs/tickle.pdf

Truex, D., Baskerville, R., & Klein, H. (1999). Growing systems in emergent organizations. Communications of the ACM, 42(8), 117-123.

Wise, L., & Quealy, J. (2006). LMS Governance Project Report. Melbourne, Australia: Melbourne-Monash Collaboration in Education Technologies. Retrieved from http://www.infodiv.unimelb.edu.au/telars/talmet/melbmonash/media/LMSGovernanceFinalReport.pdf

Ateleological travels in a teleological world: Past and future journeys around ICTs in education

In my previous academic life, I never really saw the point of book chapters as a publication form. For a variety of reasons, however, my next phase in academia appears likely to involve an increasing number of book chapters. The need for the first such chapter has arisen this week and the first draft is due by February next year, which is a timeline to give me just a little pause for thought. (There is a chance that this book might end up as a special edition of a journal)

What’s you perception of book chapters as a form of academic publication? Am particularly interested in the view from the education field.

What follows is a first stab at an abstract for the book chapter. The title for the book/special edition is “Meanings for in and of education research”. The current working title for my contribution is the title to this post: “Ateleological travels in a teleological world: Past and future journeys around ICTs in education”.

Abstract

The Australian Federal Government are just one of a gaggle of global stakeholders suggesting that Information and Communication Technologies are contributing to the creation a brave, new, digital world. Such a digital world is seen as being radically different to what has gone before and consequently demanding a radically different education system to prepare the next generation of learners. A task that is easier said than done. This chapter argues that the difficulties associated with this task arise because the meanings underpinning the design of education systems for the digital world are decidedly inappropriate and ill-suited for the nature of the digital world. The chapter draws upon 15+ years of research formulating an Information Systems Design Theory for emergent e-learning systems for universities to critically examine these commonly accepted meanings, suggest alternate and more appropriate meanings, and discuss the potential implications that these alternate meanings hold for the practice of education and education research.

The plan

The plan is that this chapter/paper will reflect on the primary focus of my research over recent years and encourage me to think of future research directions and approaches. Obviously it will draw on the PhD research and in particular the Ps Framework and the presentation I gave at EdMedia a couple of years ago. It will also draw on the presentation I gave analysing the Digital Education Revolution as part of my GDLT studies this year.

And the thesis is complete, what's next?

Just after 9:30pm last Wednesday I read an email from the Dean and Director of the ANU College of Business and Economics congratulating me on the fact that my thesis had been accepted without revision by the examiners and the institution. Needless to say that it was good news.

http://twitter.com/#!/djplaner/status/70817513038036993

Celebrations, however, were somewhat muted and restricted to the above tweet and a couple of drinks the following night. Celebrations, like blogging, had become victim of circumstance. Circumstances that included: being a student teacher placed at a local high school 4 days a week and having to prepare and deliver an increasing number of lessons, at the same time having to complete University assignments, spend time with my family, and most recently recover from the flu.

Most of these are on-going tasks, but I thought I’d take a bit of time to reflect. After all it is Friday and I currently feel like I’m getting a little ahead on tasks, which I fear is more a false dawn.

The value of a thesis

Over recent years, especially the last couple, I’ve seen quite a bit written about the value of a PhD thesis. Leigh Blackall has embarked on a PhD his way after identifying several limitations with more traditional practice. Sarah Thorneycroft has been doing work around traditional academic publishing and then there’s the more recent news (mostly out of the US) about there being an over-supply of PhD graduates. And that’s just the few that I’ve gleaned while I’ve been doing an ostrich impression and focusing on getting the thesis done and thinking about teaching. I really should make the time at some stage to follow what these and other folk are doing.

So, while it’s finished, is there still value in a PhD? Especially since I’m moving away from academia into high school teaching?

This was a question that struck me really quite hard early in my high school prac teaching experience. During the first mathematics class I’d taught with one group of students who are largely disengaged. One of the students said, “I’ve never passed a math class.” She then proceeded to quite comfortably complete a set of fairly abstract algebra exercises with a minimum of assistance. It became obvious that there was a lot of room for value generation in this class. Value that could have some significant impact on the lives of students.

What value is there in a PhD? Certainly I don’t see mine ever having the same sort of impact as a good high school teacher. And that’s with a thesis that generated a journal paper that has a Google Scholar citation count of 197 and over 2000 visits to the thesis page on my blog.

The common refrain I hear in Academia is that the PhD is just the entry ticket into Academia. That it’s your on-going work that will make the contribution and have the impact. But frankly, my experiences and observations of academia and its recent trends are such that it is becoming harder and harder to have an impact beyond the pointless ticking of boxes, meeting of targets, mouthing of slogans, and the mounting of projects that are meant to look good at the time (i.e. in terms of fulfilling all of the previous important tasks of academia) while failing to have any lasting impact.

Add to this the observation that my thesis is within the Information Systems discipline which appears to me to be a dying discipline. A discipline suffering from the growing persuasiveness of information technology in turn reducing the importance and relevance of specialist IS researchers. A disease that seems to be infecting many IT specialist disciplines, but is especially difficulty for IS and its attempt to distinguish itself from other business disciplines and also the IT discipline. That said, being actively engaged with attempts to increase the relevance of the Information Systems/Technology disciplines would be an interesting and challenging project.

In the end, the value of my PhD comes down to a purely personal value. After taking so long to complete the thesis, I have indeed completed it. I’ve proven that I could finish it.

High school teaching

While, as described above, I can see the great impact a quality high school teacher can make, I can also see how difficult it might be. I wonder about whether or not I have the energy required to make the impact. Even though my experience is limited, I can already see the mismatch between the nature of schools, their curriculum and the needs of the students. NAPLAN and QCS testing is driving an increased focus on intellectual pursuits, somewhat like the point Sir Ken makes in the well known video below. Interestingly, this video was shown at the weekly staff meeting at the school I’m currently placed at.

Yes, there is some moves to broadening school with the offering of vocational education as part of high school. But the pressure of NAPLAN seems to be particularly limiting on mathematics. Especially within the constraints of existing curriculum and resources such as textbooks. The kids that are prepared to fit within the expectations of school are a joy to work with and get a lot out of this approach. But there are other kids who, for a variety of reasons, don’t fit and subsequently are ill-served by the system. Trying to help those students within the constraints of the system sounds like a recipe for frustration and burn out.

Seems like I’ve found the challenge for what’s next.

On the potential flexibility of open source LMS and its limits

Today a mate posted to his blog about a small project he’s involved with. The context of this project seems to be a good opportunity to comment on the potential flexibility of open source LMS and the limits of that flexibility within an institutional context. It’s also an attempt to link it back to the design theory described in my thesis (if you want more of the theory behind the following, look at the thesis).

The following uses Moodle as an example, but I believe that similar limitations exist regardless of the open source LMS. This is in part because a significant limit on the flexibility is not the LMS, but instead the institutional governance processes and associated factors..

The need

In this case, the need is to send students an email with a link to a survey. The link has been personalised based on the students’ membership of Moodle groups. They survey asks them to answer questions based on their experience of a group task.

My initial thought would be that this sounds like something Moodle should be able to do. Given the increasing emphasis on group related work I doubt that this is a novel requirement. So, there might be something in Moodle that can do this, however, based on my limited knowledge of Moodle I can’t think of anything off the top of my head.

I believe that there might be the functionality within Moodle to do each of the components of this task. There is probably a way to send emails to members of a group. There might even be a way to customise that email to some extent (there is a bulk email facility in Moodle 1.9, but, from memory, it seems somewhat limited). There is also probably a way to do a group-based survey (a MCQ might be the obvious solution).

But I doubt that there is an easy way to combine these separate functions so that the group email can automatically include the link to the group’s MCQ/survey.

Doing it outside of Moodle

There is another interesting and related comment in post describing this project

hile not ideal in that it is a separate system from the LMS, it is hoped that this trial will help inform the development of a Moodle module that will perform the same function albeit in a more integrated and seamless way

Over 6 months ago, I used to work at the institution being described. Based on that out-of-date experience, my initial guess is that “doing it outside of Moodle” is deemed to be easier than engaging with Moodle and the institutional IT department.

Two limits of open source LMS flexibility

Drawing on the above examples, I’d like to propose at least two, somewhat related limits on the flexibility of open source LMS:

  1. Inflexible institutional structures and processes.
  2. The difficulty of producing/the absence of scaffolding conglomerations.

Inflexible institutional structures and processes

Modifying an enterprise implementation of Moodle effectively and efficiently is hard. You don’t want your institution’s Moodle instance to be unavailable to students and staff because a code change has broken something drastically. The Moodle code-base is itself quite difficult to get a handle on. Not overly difficult, but a non-Moodle developer can’t simply front-up and start making changes quickly. They need to be enculturated into the Moodle way, to learn what works and what doesn’t. Such a requirements means that someone who is able to modify Moodle is often a scarce and expensive resources. Especially within most universities who often don’t have someone dedicated as a Moodle developer.

To address this difficulty and also to CYA (some might argue that CYA is the major reason) institution’s spend a lot of time and effort setting up appropriate governance structures. The theory being that these are objective and rational ways to manage the difficult process and the expensive and scarce resource.

The trouble is that the difficulty and expense involved means that it becomes difficult for such processes to effectively engage in “small” problems like this one. i.e. problems that don’t actually require development of any significantly new functionality or large-scale modules. It just needs a few minor changes or wrappers around existing functionality. For example, the requirement above could possibly be solved (the following is an example description given off the top of my head without any investigation as to whether this would work) by

  • Modifying the Moodle quiz function to populate a database table linking groups to URLs for group specific quizzes.
  • Modifying the existing Moodle bulk-email facility (or perhaps adding a wrapper around it like I did with bim) to use this database table to send personalised emails to group members.
  • Perhaps add a new quiz report that allows viewing/comparing within/across groups.

For a variety of reasons traditional institutional LMS policies and processes are too heavy-weight to respond to this sort of need. Instead, in order for something like this to be considered, it has to be blown up into some institutional priority. e.g. a system to support peer and group-based assessment for the entire institution. A project that will require a significant amount of time doing a needs analysis,……..

A big project that requires lots of resources is expensive enough to be efficiently considered by the governance and related processes. Small projects are too cheap to be efficiently considered by the expensive institutional processes.

In the hardware/operating systems field, this is a problem known as starvation or indefinite postponement. The situation where a task is forever ignored because of a flaw in the priority mechanism.

So, I’m proposing that the institutional implementation of open source LMS end up suffer from the “starvation limit” on flexibility.

The need for rapid development of scaffolding conglomerations

The need in this case, at least to me, sounds like an example of what I termed scaffolding, context-sensitive conglomerations. Rather than necessarily requiring a brand new Moodle module or block, this problem sounds like something that actually needs to combine the functionality from a number of existing Moodle services. Something that conglomerates the lower-level functionality provided by Moodle into something that better meets this higher-level need.

A large part of the popularity of Moodle arises from its modularity. A feature that allows for the easy development of lots of new functionality. Something that increases the flexibility of Moodle.

The problem is that this flexibility arises, in part, from keeping these different modules separate. It’s the separation that makes it easy to add a new function without (theoretically) worrying about how it will effect the other modules. They are meant to be independent. The current problems moving to Moodle 2.0 is an example of the problems that arise from dependency. All the third-party modules depend on the Moodle core, so when the Moodle core changes all the third-party modules have to change.

A strict separation between modules makes it more difficult to combine parts of these different modules into a scaffolding conglomeration.

So, I’m proposing that open source LMS have an “over reliance on module independence” that limits their flexibility.

It’s really all about balance

I can already here proponents of traditional institutional IT governance processes or strict software engineers bemoaning the problems of not having institutional governance or of module dependence. And I do agree. There are dangers and problems. I’m not suggesting that they should necessarily be done away with.

I do, however, think that too often the balance has gone too far one way. There needs to be more recognition of a need for balance the other way. A bit less of a focus on the objective, best ways of technical implementation, and a bit more on the subjective, best ways to improve learning and teaching.

My god, is it done?

After more years than I care to count, almost as many structures, and many, many more plans and timetables, the thesis is just about done. I have just finished stuffing around with Word and have produced a single PDF that will almost certainly be the version that is submitted.

All that is left is to figure out how to submit from a distance and sit back and wait for the judgement of the examiners.

Not entirely certain what to make of this milestone, it just seems to be yet another step in the on-going denouement of the thesis and all its intricacies. There is a sense of palpable relief in having reached this stage. There remains, however, an on-going niggle of uncertainty about whether or not Word (or my own carelessness) has inserted some enormous blunder in the middle of the thesis. There’s the small bit of fear that one of the examiners will turn out to be a mongrel. But mostly there is relief and a need to go have a good lie down.

There is also some recognition that I should knuckle down and publish from the thesis and its contents. There are at least two good journal articles waiting to be written. Two good journal articles that are likely going to have to continue waiting. Mostly because there is also significant anticipation arising from all the (non-academic) activities that now become possible as the Sword of Damocles that was the PhD has been removed.

First step, a couple of days with the better half in Melbourne next week. With the sole intent of eating, drinking and being merry.

A command for organisations? Program or be programmed

I’ve just finished the Douglas Rushkoff book Program or be Programmed: Ten commands for a digital age. As the title suggests the author provides ten “commands” for living well with digital technologies. This post arises from the titular and last command examined in the book, Program or be programmed.

Dougls Rushkoff

This particular command was of interest to me for two reasons. First, it suggests that learning to program is important and that more should be doing it. As I’m likely to become a information technology high school teacher there is some significant self-interest in there being a widely accepted importance to learning ot program. Second, and the main connection for this post, is that my experience with and observation of universities is that they are tending “to be programmed”, rather than program. In particular when it comes to e-learning.

This post is some thinking out loud about that experience and the Ruskoff command. In particular, it’s my argument that universities are being programmed by the technology they are using. I’m wondering why? Am hoping this will be my last post on these topics, I think I’ve pushed the barrow for all its worth. Onto new things next.

Program or be programmed

Rushkoff’s (p 128) point is that

Digital technology is programmed. This makes it biased toward those with the capacity to write the code.

This also gives a bit of a taste for the other commands. i.e. that there are inherent biases in digital technology that can be good or bad. To get the best out of the technology there are certain behaviours that seem best suited for encouraging the good, rather than the bad.

One of the negative outcomes of not being able to program, of not being able to take advantage of this bias of digital technology is (p 15)

…instead of optimizing our machines for humanity – or even the benefit of some particular group – we are optimizing humans for machinery.

But is all digital technology programmed?

In terms of software, yes, it is all generally created by people programming. But not all digital technology is programmable. The majority of the time, money and resources being invested by universities (I’ll stick to unis, however, much of what I say may be applicable more broadly to organisations) is in “enterprise” systems. Originally this was in the form of Enterprise Resource Planning system (ERPs) like Peoplesoft. It is broadly recognised that modifications to ERPs are not a good idea, and that instead the ERP should be implemented in “vanilla” form (Robey et al, 2002).

That is, rather than modify the ERP system to respond to the needs of the university. The university should modify its practices to match the operation of the ERP system. This appears to be exactly what Rushkoff warn’s against “we are optimizing humans for machinery”.

This is important for e-learning because, I would argue, the Learning Management System (LMS) is essentially an ERP for learning. And I would suggest that much of what goes on around the implementation and support of an LMS within a university is the optimization of humans for machinery. In some specific instances that I’m aware of, it doesn’t matter whether the LMS is open source or not. Why?

Software remains hard to modify

Glass (2001), describing one of the frequently forgotten fundamental facts about software engineering, suggested that maintenance consumes about 40 to 80 percent of software costs, with 60% of the maintenance cost is due to enhancement. i.e. a significant proportion of the cost of any software system is adding new features to it. You need to remember that this is a general statement. If the software you are talking about is part of a system that operates within a continually changing context, then the figure is going to be much, much higher.

Most software engineering remains focused on creation. On the design and implementation of the software. There hasn’t been enough focus on on-going modification, evolution or co-emergence of the software and local needs.

Take Moodle. It’s an LMS. Good and bad like other LMS. But it’s open source. It is meant to be easy to modify. That’s one of the arguments wheeled out by proponents when institutions are having to select a new LMS. And Moodle and its development processes are fairly flexible. It’s not that hard to add a new activity module to perform some task you want that isn’t supported by the core.

The trouble is that Moodle is currently entering a phase which suggests it suffers much the same problems as most large enterprise software applications. The transition from Moodle 1.x to Moodle 2.0 is highlighting the problems with modification. Some folk are reporting difficulties with the upgrade process, others are deciding to delay the upgrade as some of the third-party modules they use haven’t been converted to Moodle 2. There are even suggestions from some that mirror the “implement vanilla” advice for ERPs.

It appears that “we are optimizing humans for machinery”.

I’m wondering if there is anyone doing research how to make systems like Moodle more readily modifiable for local contexts. At the very least, looking at how/if the version upgrade problem can be improved. But also, the ability to modify the core to better suit local requirements. There are aspects there already. One of the difficulties is that to achieve this you would have to cross boundaries between the original developers, service providers (Moodle partners) and the practices of internal IT divisions.

Not everyone wants to program

One reason this will be hard is that not everyone wants to program. Recently, D’Arcy Norman wrote a post talking about the difference between the geeks and folk like his dad. His dad doesn’t want to bother with this techy stuff, he doesn’t want to “program”.

This sort of problem is made worse if you have an IT division that has senior management with backgrounds in non-IT work. For example, an IT director with a background in facilities management isn’t going to understand that IT is protean, that it can be programmed. Familiar with the relative permanence of physical buildings and infrastructure such a person isn’t going to understand that IT can be changed, that it should be optimized for the human beings using the system.

Organisational structures and processes prevent programming

One of the key arguments in my EDUCAUSE presentation (and my thesis) is that the structures and processes that universities are using to support e-learning are biased away from modification of the system. They are biased towards vanilla implementation.

First, helpdesk provision is treated as a generic task. The folk on the helpdesk are seen as low-level, interchangeable cogs in a machine that provides support for all an organisation’s applications. The responsibility of the helpdesk is to fix known problems quickly. They don’t/can’t become experts in the needs of the users. The systems within which they work don’t encourage, or possibly even allow, the development of deep understanding.

For the more complex software applications there will be an escalation process. If the front-line helpdesk can’t solve the problem it gets handed up to application experts. These are experts in using the application. They are trained and required to help the user figure out how to use the application to achieve their aims. These application experts are expert in optimizing the humans for the machinery. For example, if an academic says they want students to have an individual journal, a Moodle 1.9 application expert will come back with suggestions about how this might be done with the Moodle wiki or some other kludge with some other Moodle tool. If Moodle 1.9 doesn’t provide a direct match, they figure out how to kludge together functionality it does have. The application expert usually can’t suggest using something else.

By this stage, an academic has either given up on the idea, accepted the kludge, gone and done it themselves, or (bravely) decided to escalate the problem further by entering into the application governance process. This is the heavy weight, apparently rational process through which requests for additional functionality are weighed against the needs of the organisation and the available resources. If it’s deemed important enough the new functionality might get scheduled for implementation at some point in the future.

There are many problems with this process

  • Non-users making the decisions;
    Most of the folk involved in the governance process are not front-line users. They are managers, both IT and organisational. They might include a couple of experts – e-learning and technology. And they might include a couple of token end-users/academics. Though these are typically going to be innovators. They are not going to be representative of the majority of users.

    What these people see as important or necessary, is not going to be representative of what the majority of academic staff/users think is important. In fact, these groups can quickly become biased against the users. I attended one such meeting where the first 10/15 minutes was spent complaining about foibles of academic staff.

  • Chinese whispers;
    The argument/information presented to such a group will have had to go through chinese whispers like game. An analyst is sent to talk to a few users asking for a new feature. The analyst talks to the developers and other folk expert in the application. The analysts recommendations will be “vetted” by their manager and possibly other interested parties. The analysts recommendation is then described at the governance meeting by someone else.

    All along this line, vested interests, cognitive biases, different frames of references, initial confusion, limited expertise and experience, and a variety of other factors contribute to the original need being morphed into something completely different.

  • Up-front decision making; and
    Finally, many of these requests will have to battle against already set priorities. As part of the budgeting process, the organisation will already have decided what projects and changes it will be implementing this year. The decisions has been made. Any new requirements have to compete for whatever is left.
  • Competing priorities.
    Last in this list, but not last overall, are competing priorities. The academic attempting to implement individual student journals has as their priority improving the learning experience of the student. They are trying to get the students to engage in reflection and other good practices. This priority has to battle with other priorities.

    The head of the IT division will have as a priority of staying in budget and keeping the other senior managers happy with the performance of the IT division. Most of the IT folk will have a priority, or will be told that their priority is, to make the IT division and the head of IT look good. Similarly, and more broadly, the other senior managers on 5 year contracts will have as a priority making sure that the aims of their immediate supervisor are being seen to be achieved……..

These and other factors lead me to believe that as currently practiced, the nature of most large organisations is to be programmed. That is, when it comes to using digital technologies they are more likely to optimize the humans within the organisation for the needs of the technology.

Achieving the alternate path, optimizing the machinery for the needs of the humans and the organisation is not a simple task. It is very difficult. However, by either ignoring or being unaware of the bias of their processes, organisations are sacrificing much of the potential of digital techology. If they can’t figure out how to start programming, such organisations will end up being programmed.

References

Robey, D., Ross, W., & Boudreau, M.-C. (2002). Learning to implement enterprise systems: An exploratory study of the dialectics of change. Journal of Management Information Systems, 19(1), 17-46.

Scaffolding, context-sensitive conglomerations – v2.0

This is an excerpt from Chapter 5 of the thesis. It’s a re-do of an earlier post, I’m a bit happier with it than v1.0. I’m posting it here because it connects with some work I’m doing preparing to work on BIM v2.0 and also because it summarises what I think may areas of further research in e-learning would be, if I were to do any.

First, there is a definition of what I mean by scaffolding conglomerations, then there is some brief justificatory knowledge on which this very early, rough idea is based.

The last sentence in the following, summarises why I think this idea has some value. To put it another way, I’d probably argue the following:

  • The vast majority of university e-learning is not very good.
  • The common response is based on a deficit model of the academic.
    i.e. the academic doesn’t know enough about learning and teaching, so let’s force them to do a formal education qualification and all will be good. Alternatively, lets send them on some LMS training or require them to complete checklists. At best, let’s employ educational developers to work with the academics to design their courses and course sites.
  • This type of approach fails to connect with academics and their intrinsic motivation and subsequently is very unlikely to work.
  • This type of approach fails to recognise that much of the deficit in the university e-learning system arises not from the academic, but from the information systems used to implement it and the general university environment.
  • The proposition is that adding well-designed scaffolding, context-sensitive conglomerations addresses the limitations in the information systems and improves the level of knowledge within the system in a way that may be more effective.
  • However, adding the conglomerations is only one step. Fixing the limitations in the university environment (e.g. research is more important than teaching; bad management robbing academics of their passion etc.) is also required.

What are scaffolding conglomerations?

The design of e-learning in universities requires the combination of skills from a variety of different professions (e.g. instructional design, web design etc), and yet is often most performed by academics with limited knowledge of any of these professions. This limited knowledge creates significant workload for the academics and contributes to the limited quality of much e-learning. Adding experts in these fields to help course design is expensive and somewhat counter to the traditional practice of learning and teaching within universities. This suggests that e-learning in universities has a need for approaches that allow the effective capture and re-use of expertise in a form that can be re-used by non-experts without repeated direct interaction with experts. Such an approach could aim to reduce perceived workload and increase the quality of e-learning.

An emergent university e-learning information system should:

  • Provide the ability to easily develop, including end user development, larger conglomerations of packaged services.
    A conglomeration is not simply an e-learning service such as a discussion forum. Instead it provides additional scaffolding around such services, possibly combining multiple services, to achieve a higher-level task. The scaffolding should generally embody and provide easy access to forms of expert knowledge that help encourage and enable effective use of the service. On the other hand, while many conglomerations would be expert designed and developed, offering support for end-user development would increase system flexibility and serve to embody and enable the re-use of contextual knowledge. The Webfuse default course site approach (Section 5.3.5) is one example of a conglomeration. A default course site combines a number of separate page types (services), specific graphical and instructional designs, and existing institutional content into a course website with a minimum of human input. Another form of conglomeration that developed with Webfuse was Staff MyCQU. This “portal” grew to become a conglomeration of integrated Wf applications designed to package a range of services academics required for learning and teaching.
  • Design conglomerations to provide a range of scaffolding to aid users, increase adoption and increase quality.
    There is likely to be some distance between the knowledge of the user and that required to effectively use e-learning services. Scaffolding provided by the conglomerations should seek to bridge this distance, encourage good practice, and help the user develop additional skills. For example, over time an “outstanding tasks” element was added to Staff MyCQU to remind staff of unfinished work in a range of Wf applications. The BAM Wf application was designed to support the workload involved in tracking and marking individual student reflective journals (Jones & Luck, 2009). A more recent example focused more on instructional design is the instructional design wizard included in the new version of the Desire2Learn LMS. This wizard guides academics through course creation via course objectives.
  • Embed opportunities for collaboration and interaction into conglomerations.
    An essential aim of scaffolding conglomerations is enabling and encouraging academics to learn more about how to effectively use e-learning. While the importance of community and social interaction to learning is widely recognised, most professional development opportunities occur in isolation (Bransford, Brown, & Cocking, 2000). Conglomerations should aim to provide opportunities for academics to observe, question and discuss use of the technology. Examples from Webfuse are limited to the ability to observe. For example, all Webfuse course sites were, by default, open for all to see. The CourseHistory Wf application allowed staff to see the grade breakdown for all offerings of any course. A better example would have been if the CourseHistory application encouraged and enabled discussions about grade breakdowns.
  • Encourage and support conglomerations that are context-sensitive.
    Effective integration with the specific institutional context enables conglomerations to leverage existing resources and reduce cognitive dissonance. For example, the Webfuse default course site conglomeration was integrated with a range of CQU specific systems, processes and resources. The Webfuse online assignment submission system evolved a number of CQU specific features that significantly increased perceptions of usefulness and ease-of-use (Behrens et al., 2005).

Some justificatory knowledge

The concept of constructive templates (Catlin, Garret, & Launhardt, 1991; Nanard et al., 1998) was developed in response to the difficulty faced by content providers in developing hypermedia structures that followed the known principles of interface and hypermedia design. Constructive templates helped content experts to create well designed hypermedia (Catlin et al., 1991). The “conglomeration” principles build on the constructive template idea through insights from distributed cognition and related ideas. Amongst other important aspects, Hollan et al (2000) describe how distributed cognition expands what is considered cognitive beyond an individual to encompass interactions between people, their environment and the tools therein. Boland et al (1994, p. 459) define a distributed cognition system as one that “supports interpretation and dialogue among a set of inquirers by providing richer forms of self-reflection and communication”. Scaffolding, context-sensitive conglomerations aim to improve or increase the quality and quantity of cognition within an e-learning system and support self-reflection and communication.

References

Behrens, S., Jamieson, K., Jones, D., & Cranston, M. (2005). Predicting system success using the Technology Acceptance Model: A case study. Paper presented at the Australasian Conference on Information Systems’2005, Sydney.

Boland, R., Ramkrishnan, V., & Te’eni, D. (1994). Designing information technology to support distributed cognition. Organization Science, 5(3), 456-475.

Bransford, J., Brown, A., & Cocking, R. (2000). How people learn: brain, mind, experience, and school. Washington, D.C.: National Academy Press.

Catlin, K., Garret, L. N., & Launhardt, J. (1991). Hypermedia Templates: An Author’s Tool. Paper presented at the Proceedings of Hypertext’91.

Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition: Toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction, 7(2), 174-196.

Jones, D., & Luck, J. (2009). Blog Aggregation Management: Reducing the Aggravation of Managing Student Blogging. Paper presented at the World Conference on Education Multimedia, Hypermedia and Telecommunications 2009. from http://www.editlib.org/p/31530.

Nanard, M., Nanard, J., & Kahn, P. (1998). Pushing Reuse in Hypermedia Design: Golden Rules, Design Patterns and Constructive Templates. Paper presented at the Proceedings of the 9th ACM Conference on Hypertext and Hypermedia.

Making the LMS more like the Globe Theatre: distributed cognition, the extended mind and Moodle

The following draws together some work on distributed cognition in the Globe Theatre and some early thoughts on how that might be useful in improving the design of learning management systems such as Moodle (there’s also an argument to be made about university teaching and learning, but I’ll mostly leave that alone). While primarily focusing on helping teachers teach and thus improve the student learning experience, the ideas could equally be applied to learners (and probably have been).

At the core of the following seems to be the question of whether o not the design of LMS, and e-learning tools in general, have been limited due to the following observation made by Tribble (2005)

consistently distorted by a tendency to view cognition as individual rather than social, which has caused us to imagine the workings of complex group structures in mechanistic terms. In other words, we have mistakenly assumed that properties of the system as a whole must be possessed by each individual within it.

Shakespeare's - Globe Theatre

The post continues to build some recent thoughts about this issue and was sparked by listening to this episode from ABC Radio National show – The Philosophers Zone. This led me to the article Distributing Cognition in the Globe by Tribble (2005).

After finishing the post, the idea of adding “plots” to LMS tools might be the most interesting outcome. Will need to do more reading and thinking. Must be other folk looking at this, pointers?

Cognitive demands of the Elizabethan repertory system

Imagine you are a member of an Elizabethan repertory theatre company in the late 16th century. Just how cognitively demanding is your work? Tribble (2005) draws on a number of sources to give the following examples:

  • As a leading player you would have to remember 71 different roles, around 52 of which were new.
  • Your company would be putting on 6 plays a week, there would be infrequent repetition, and would have a new play every fortnight.

This appears to be quite different, and cognitively a lot more demanding, than more modern theatre companies.

So how did they do it? Tribble (2005) summarises some literature that suggests it was achieved through “mechanistic” approaches such as: actors performing the same type of role, learning roles by imitation, and having little or not interest in creative interpretation.

How did the Globe and distributed cognition help?

Tribble’s (2005) argument is that the design of theatre system – embedded in the theatres, the plots, actors’ roles, the plays’ verbal structures, the apprentice system, and the organisational practices of the companies – provided elements of cognitive structure that helped individuals and the company to fulfill their cognitive demands. In particular, she argues that the design of the theatres played a part

The more thinking that can be off-loaded onto the environment, the more mental energy remains available for those tasks that are primarily internal (memory for the spoken lines, for instance).

For example, it is argued that “plots” – summaries/directions of when, who enters the scene – hanging back stage from pegs by stage entrances provided players with reminders of the order of scenes. Especially helpful for actors playing multiple small roles. Another is that rather than all players having the full script, they instead have their lines in detail and various cues to indicate when they are to be performed. Similarly, it’s argued that some of the physical spaces within the theatre (e.g. a balcony) provided cognitive hints as to what type of scene you were meant to perform.

The podcast that brought me to Tribble and her work includes a better description starting at about 13m48s. Tribble’s summary

The productive constraint of the stripped down part reduces the need to filter signal (one’s own part) from noise (everyone else’s); the plot provides a schematic diagram of the shape of the play as a whole to supplement the part; the physical space of the theater and the conventions of movement it supports enable the transition from the two-dimensional maps of plot and part to its three-dimensional embodiment onstage; and the structures and protocols of the theatrical company pass on its practices to new members. Such a theater can best be understood, in other words, through a framework that takes group practices seriously, that assumes that systems can work well, and that sees individual agency as constrained but not contained by these practices.

Tribble (2005) draws inspiration from Hutchins (1995) and describes on of his contentions as

that the lines between “inside” and “outside” are frequently misdrawn or misidentified, “creat[ing] the impression that individual minds operate in isolation and encourag[ing] us to mistake the properties of complex sociocultural systems for the properties of individual minds.”

and that when the design of these systems are done well

“Novice[s]” can thus be “embed[ded]” in the system and perform at a level far above that which they could attain by means of their individual cognitive powers alone.

The design of the LMS and the tyranny of the individual mind

It is my suggestion that the design of the LMS (and probably the assumptions underpinning much of the actions around improving university teaching) is limited by the tyranny of the individual mind. Sturgess and Nouwens (2004) describe one universities rationale for an LMS as being

to enable teaching staff to develop and
manage online courses with little professional support

. This fits with the broader practice of university teaching being essentially a solo act. It’s my course, I’ll teach it my way. Give me control of the course on the LMS and let me do my thing.

It is assumed that the academic (or their proxy who has been employed to create the course) has the skills and knowledge to effectively use the features of the LMS to prepare the course site. When adopting a new LMS, a focus for most institutions is on training sessions to provide these skills and knowledge to staff.

But what additional scaffolding or “distributed cognition” does the design of most LMS provide? In my experience it is generally limited to either some canned “help” document or the contact details of a support group. The “help” resources are usually fairly limited and it’s not unusual for them to be out of date. The support group are generally keen and willing to help (often above and beyond the call of duty) but are often limited to knowing where the academic might find a query, or about how to use the LMS features as they exist.

I’m not sure that most LMS are designed in a way such that

“Novice[s]” can thus be “embed[ded]” in the system and perform at a level far above that which they could attain by means of their individual cognitive powers alone.

. The evidence from past experience with LMS seems to indicate that most e-learning is not all that good, perhaps indicative of how well the LMS support distributed cognition.

Some ideas for BIM

BIM is a Moodle module that I’ve written. It’s probably going to be the first place I experiment with some of these ideas. Some initial thoughts:

  • Improve the help documentation provided with BIM to include the videos that have been produced and also provide links and summaries of some of the literature around the use of blogs in higher education.
    This is a simple first step that arguably should have been done already.
  • Link the help documentation with some the BIM community.
    Rather than just static help documentation, provide mechanisms for asking questions of and seeing answers/resources from a broader BIM community.
  • Design a scaffolded “BIM configuration” process informed by common problems and linked to the help documentation, especially the literature.
    As it stands, the BIM configuration process is your typical IT artifact. It is a better representation of the data requirements of BIM than a process that helps academics create and design a BIM activity that is effective.
  • Figure out a way to develop standard BIM “plots”.
    There are common events in the use of a BIM activity (e.g. not all students have registered a blog, not all the students within the course have been allocated to groups and markers, etc.). These can be picked up from the BIM interface, but the academic has to apply a fair bit of cognition in understanding how to do this.

    A better approach might be to implement some “plots” (in the sense described above) that suggest when and how to handle a particular event. For example, when BIM sees that not all students in a course are allocated to groups, generate a “plot” entry that includes a list of the unallocated students and links to the Moodle group allocation process and the help documentation associated with groups explaining this problem.

Ideas for Moodle

Some of the ideas for BIM could potentially be applied to any complex process within the use of an LMS/Moodle such as the design of a course site or the management of a discussion forum. This links to some extent to the earlier suggestion of combining analytics and discussion forums.

Misc reflections

Beginning to wonder how much of this is related to the design of computer systems being done by programmers more interested in representing the models underpinning the system, rather than providing cognitive support to the folk using the system?

Wondering how much the type of support given to academics using an LMS within a University context in terms of the skills of the support staff and the organisational structures within which they work limits the provision of appropriate cognitive support?

Wondering if an increasingly hierarchical organisational structure can effectively provide the type of distributed cognition that might actually help improve the LMS and university teaching?

Wondering if the hierarchical organisational structure can effectively grok the need for this type of approach? Can it provide the environment in which this type of work can grow?

References

Hutchins, E. (1995). Cognition in the Wild. Cambridge, MA, MIT Press.

Sturgess, P. and F. Nouwens (2004). “Evaluation of online learning management systems.” Turkish Online Journal of Distance Education 5(3).

Tribble, E. B. (2005). “Distributing cognition in the Globe.” Shakespeare Quarterly 56(2): 135-155.

Dilbert as an expository instantiation

A few recent posts have been first draft excerpts from my Information Systems Design Theory (ISDT) from emergent university e-learning systems. Being academics and hence somewhat pedantic about these things there are meant to be a number of specific components of an ISDT. One of these is the expository instantiation that is meant to act as both an explanatory device and a platform for testing (Gregor and Jones, 2007) i.e. it’s meant to help explain the theory and also examples of testing the theory.

The trouble is that today’s Dilbert cartoon is probably as good an explanation of what is currently the third principle of implementation for my ISDT.

Dilbert.com

I’m sure that most folk working in a context where they’ve had to use a corporate information system have experienced something like this. A small change – either to fix a problem or improve the system – simply can’t be made because of the nature of the technology or the processes used to make the changes. The inability to make these changes is a major problem for enterprise systems.

The idea from the ISDT is that the development and support team for an emergent university e-learning system should be able to make small scale changes quickly without having to push them up the governance hierarchy. Where possible the team should have the skills, insight, judgement and trust so that “small scale” is actually quite large.

An example

The Webfuse e-learning system that informed much the ISDT provides one example. Behrens (2009) quotes a user of Webfuse about one example of how it was responsive

I remember talking to [a Webfuse developer] and saying how I was having these problems with uploading our final results into [the Enterprise Resource Planning (ERP) system] for the faculty. He basically said, “No problem, we can get our system to handle that”… and ‘Hey presto!’ there was this new piece of functionality added to the system… You felt really involved… You didn’t feel as though you had to jump through hoops to get something done.

Then this is compared with a quote from one of the managers responsible for the enterprise system

We just can’t react in the same way that the Webfuse system can, we are dealing with a really large and complex ERP system. We also have to keep any changes to a minimum because of the fact that it is an ERP. I can see why users get so frustrated with the central system and our support of it. Sometimes, with all the processes we deal with it can take weeks, months, years and sometimes never to get a response back to the user.

Is that Dilbert or what?

The problem with LMS

Fulfilling this requirement is one of the areas where most LMS create problems. For most universities/orgnaisations it is getting into the situation where the LMS (even Moodle) is approaching the “complex ERP system” problem used in the last quote above. Changing the LMS is to fraught with potential dangers that these changes can’t be made quickly. Most organisations don’t try, so we’re back to a Dilbert moment.

Hence, I think there are two problems facing universities trying to fulfil principle #3:

  1. Having the right people in the support and development team with the right experience, insight and judgement is not a simple thing and is directly opposed to the current common practice which is seeking to minimise having these people. Instead there’s reliance on helpdesk staff and trainers.
  2. The product problem. i.e. it’s too large and difficult to change quickly and safely. I think there’s some interesting work to be done here within Moodle and other open source LMS. How do you balance the “flexibility” of open source with the complexity of maintaining a stable institutional implementation?

References

Behrens, S. (2009). Shadow systems: the good, the bad and the ugly. Communications of the ACM, 52(2), 124-129.

Gregor, S., & Jones, D. (2007). The anatomy of a design theory. Journal of the Association for Information Systems, 8(5), 312-335.

Justificatory knowledge

The following is a first version of the justificatory knowledge section of my ISDT for emergent university e-learning systems. Still fairly uncertain just how information is meant to go in here and also just how far I should go with the reference to other theories (there are lots) and how much time should be spent looking at the interactions between them.

If you have some literature/theories which support or contradict this approach, will be really happy to hear about it.

Justificatory knowledge

The purpose of the justificatory knowledge component is to provide an explanation of why the ISDT is structured as it is and why it should be expected to work appropriately. Much of the justificatory knowledge that underpins this ISDT has been described previously within the literature review (Chapter 2), the first Webfuse action research cycle (Chapter 4), and earlier in this chapter. To avoid repetition this section provides a summary and brief discussion of the justificatory knowledge underpinning the ISDT for emergent university e-learning systems. This summary is linked specifically to the ISDT’s principles for form and function, and principles of implementation.

The justificatory knowledge described below arose from the experiences obtained and literature read during the design and support of Webfuse. This knowledge described below is not necessarily complete or the only established knowledge – theoretical or otherwise – that could be used to justify the principles of the ISDT. Hovorka and Germonprez (2009) identify as a weakness of design research, the lack of guidance around the interaction between the various kernel theories that make up justificatory knowledge and how the influence of these kernel theories may change during use. To some extent, the context-sensitive, emergent nature of the approach embodied by this ISDT – and its kernel theories – means that such advice is embedded in the justificatory knowledge that supports the ISDT.

Justificatory knowledge for principles for form and function

Table 5.20 provides a summary of the justificatory knowledge and is followed by a brief description. For each the three categories of principles of form and function for this ISDT, Table 5.20 provides pointers to sections of this thesis and references to literature that describe the justificatory knowledge in more detail.

Table 5.20 – Summary of justificatory knowledge for principles of form and function
Principle Justificatory knowledge
Integrated and independent services Section 2.3.2 – Software wrappers (Bass, Clements et al. 1998; Sneed 2000)
Adaptive and inclusive architecture Systems of Systems (Perrochon and Mann 1999)
Section 2.3.2 – Best of breed (Light, Holland et al. 2001; Lowe and Locke 2008), Service Oriented Architectures (Chen, Chen et al. 2003; Weller, Pegler et al. 2005), End-user development (Eriksson and Dittrich 2007)
Section 4.4.4 – Micro-kernel architecture (Liedtke 1995)
Scaffolding, context-sensitive conglomerations Constructive templates (Nanard, Nanard et al. 1998) , End-user development (Eriksson and Dittrich 2007)

As summarised in chapter 4, a software wrapper is a type of encapsulation that enables software components to be encased in an alternative abstraction that enables clients, often in a new context, to access the wrapped components services (Bass, Clements et al. 1998; Sneed 2000). As such software wrappers are one example of an approach that provides integrated and independent software services.

Some of the relative advantages and limitations more tightly integrated systems is provided by the enterprise software literature. In this literature, comparisons between tightly integrated systems and best-of-breed approaches have argued that integration involves centralisation of processes and a consequently a tendency to reduce autonomy, increase rigidity, and reduce competitiveness (Light, Holland et al. 2001; Lowe and Locke 2008). The best-of-breed approach, focusing on a more inclusive integration of appropriate packages, increases system flexibility while at the same time requires greater time, skills and resources to integrate diverse applications (Light, Holland et al. 2001). Perrochon and Mann (1999) argue that traditional approaches to system architecture, even those with a focus on adaptivity, are appropriate for greenfield developments due to their reliance on the assumption of design (specify architecture) and then implement. The rise of component-oriented software has created the problem of systems of systems that require the combination of well-engineered components or systems into an overall system they were never, and could never be, designed for (Perrochon and Mann 1999).

The concept of constructive templates (Catlin, Garret et al. 1991; Nanard, Nanard et al. 1998) was developed in response to the difficulty faced by content providers in developing hypermedia structures that followed the known principles of interface and hypermedia design. Constructive templates helped content experts to create well designed hypermedia (Catlin, Garret et al. 1991).

Justificatory knowledge for principles of implementation

The justificatory knowledge for this ISDT’s principles of implementation – summarised in Table 5.21 and briefly described below – draws heavily on what is down about alternatives to traditional, plan-driven software development methodologies as discussed in Section 2.4 and Sections 5.3.1 and 5.3.2 of this chapter.

Table 5.21 – Summary of justificatory knowledge for principles of implementation
Principle Justificatory knowledge
Multi-skilled, integrated development and support team Job rotation, multi-skilling etc (Faegri, Dyba et al. 2010), Organisational learning (Seely Brown and Duguid 1991), Situated learning, Situated action, Communities of practice (Seely Brown and Duguid 1991), Knowledge-based theory of organizational capability (Grant 1996)
An adopter-focused, emergent development process Section 2.4 examines the topic of processes, including a comparison of traditional plan-driven processes (e.g. the SDLC) and learning-focused processes such as emergent development. Additional discussion occurs in Section 5.3.2
Section 5.3.1 introduces the conception of adopter-focused development.
A supportive organisational context Organisational fit (Hong and Kim 2002), Strategic alignment (Henderson and Venkatraman 1993), Bricolage (Chae and Lanzara 2006), Mindful innovation (Swanson and Ramiller 2004)

Seely Brown and Duguid (1991) argue that the tendency for education, training and technology design to focus on abstract representations that are detached from practice actually distort the intricacies of practice and consequently hinder how well practice can be understood, engendered, or enhanced. The idea of the development team integrated and embedded in the everyday practice of e-learning seeks to improve the learning of both academics and students about how to harness e-learning, and also improve the learning of the development team (and the organisation) about how e-learning is being used. The ISDT seeks to establish a process for supporting and developing e-learning which is situated in shared practice with a joint, collective purpose.

Faegri, Dyba et al (2010) argue that turbulent environments increase the importance of employee skills and competences and that having employees cycle through different jobs – such as developers being on helpdesk – can improve knowledge redundancy, organizational knowledge creation, and other benefits. Faegri, Dyba et al (2010) also cite Keil-Slawik (1992) as arguing that full understanding of software requires experience developing the software. The traditional hierarchical structures associated with the division of labour around the e-learning within universities – e.g. helpdesk and developers organized into separate units within an IT division; learning and teaching experts located in another division focused on learning and teaching; and, faculty academics located academic units – are seen by Grant (1996) to inhibit the ability to integrate knowledge from members of an organisation. Such integration is seen as fundamental to the organisation’s ability to create and sustain competitive advantage (Grant 1996).

There is significant literature (March 1991; Baskerville, Travis et al. 1992; Mintzberg 1994; Bamford and Forrester 2003) in a variety of disciplines that identifies plan-driven processes as the dominant approach in most organizations. This and related literature also examines the limitations this over-emphasis suffers, especially in contexts with rapid change or significant diversity (see Section 2.4). Consequently there is significant literature identifying both the theoretical basis and guidance (Introna 1996; Truex, Baskerville et al. 1999) and practical implementation methods (Beck 2000; Schwaber and Beedle 2002) for more emergent or adopter-based development processes.

An emergent, university e-learning information system is a large-scale information system. In this context, “large-scale” is used in the sense adopted by Chae and Lanzara (2006), as referring to systems that involve both organisational technologies and technological innovations that “comprise and connect multiple communities of practice within an organisation or between organistions”. Literature examining success factors with information systems development (e.g. Ewusi-Mensah 1997; Scott and Vessey 2002) has long considered it vital for senior management to be supportive of and committed to systems development. Brown et al (2007) identify commitment – defined as the resources dedicated to IT, organisational dedication to change procedures, and top management support – as one of two most cited problems in the IS projects they examined and identified it as the factor most cited within the literature.

Organisational fit (Hong and Kim 2002) and strategic alignment (Henderson and Venkatraman 1993) between various aspects of an organisation and its information technology systems and processes have long been argued as critical success factors. A similar importance on having an organisational context that is committed and appropriate to information systems development is also found in approaches that are less traditional or teleological (e.g. bricolage and mindful innovation) and have more in common with the emergent, adopter-focused approach advocate in this ISDT. Collective or organisational bricolage requires the combined effort of several individuals and groups (Chae and Lanzara 2006). An organisation which is mindful in innovating with IT, uses reasoning grounded in its own organisational facts and specifics when thinking about the innovation, the organisation recognises that context matters (Swanson and Ramiller 2004). Within mindful innovation, management have a responsibility to foster conditions that prompt collective mindfulness (Swanson and Ramiller 2004).

References

Bamford, D., & Forrester, P. (2003). Managing planned and emergent change within an operations management environment. International Journal of Operations and Production Management, 23(5), 546-564.

Baskerville, R., Travis, J., & Truex, D. (1992). Systems without method: the impact of new technologies on information systems development projects. In K. E. Kendall (Ed.), The Impact of Computer Supported Technologies on Information Systems Development (pp. 241-251). Amsterdam: North-Holland.

Bass, L., Clements, P., & Kazman, R. (1998). Software Architecture in Practice. Boston: Addison-Wesley.

Beck, K. (2000). Extreme Programming Explained: Embrace Change: Addison-Wesley.

Brown, S., Chervany, N., & Reinicke, B. (2007). What matters when introducing new information technology. Communications of the ACM, 50(9), 91-96.

Catlin, K., Garret, L. N., & Launhardt, J. (1991). Hypermedia Templates: An Author’s Tool. Paper presented at the Proceedings of Hypertext’91.

Chae, B., & Lanzara, G. F. (2006). Self-destructive dyamics in large-scale technochange and some ways of conteracting it. Information Technology & People, 19(1), 74-97.

Chen, M., Chen, A., & Shao, B. (2003). The implications and impacts of web services to electronic commerce research and practices. Journal of Electronic Commerce Reseaerch, 4(4), 128-139.

Eriksson, J., & Dittrich, Y. (2007). Combining tailoring and evolutionary software development for rapidly changing business systems. Journal of Organizational and End User Computing, 19(2), 47-64.

Ewusi-Mensah, K. (1997). Critical Issues in Abandonded Information Systems Development Projects. Communications of the ACM, 40(9), 74-80.

Faegri, T. E., Dyba, T., & Dingsoyr, T. (2010). Introducing knowledge redundancy practice in software development: Experiences with job rotation in support work. Information and Software Technology, 52(10), 1118-1132.

Grant, R. (1996). Prospering in dynamically competitive environments: organizational capability as knowledge integration. Organization Science, 7(4), 357-387.

Henderson, J., & Venkatraman, N. (1993). Strategic alignment: Leveraging information technology for transforming organizations. IBM Systems Journal, 32(1), 4-16.

Hong, K.-K., & Kim, Y.-G. (2002). The critical success factors for ERP implementation: an organizational fit perspective. Information & Management, 40(1), 25-40.

Hovorka, D., & Germonprez, M. (2009). Tinkering, tailoring and bricolage: Implications for theories of design. Paper presented at the AMCIS 2009.

Introna, L. (1996). Notes on ateleological information systems development. Information Technology & People, 9(4), 20-39.

Keil-Slawik, R. (1992). Artifacts in software design. In C. Floyd, H. Zullighoven, R. Budde & R. Keil-Slawik (Eds.), Software Development and Reality Construction (pp. 168-188). Berlin: Springer-Verlag.

Liedtke, J. (1995). On micro-kernel construction. Operating Systems Review, 29(5), 237-250.

Light, B., Holland, C., & Wills, K. (2001). ERP and best of breed: a comparative analysis. Business Process Management Journal, 7(3), 216-224.

Lowe, A., & Locke, J. (2008). Enterprise resource planning and the post bureaucratic organization. Information Technology & People, 21(4), 375-400.

March, J. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1), 71-87.

Mintzberg, H. (1994). The rise and fall of strategic planning: Reconceiving roles for planning, plans, planners. New York: Free Press.

Nanard, M., Nanard, J., & Kahn, P. (1998). Pushing Reuse in Hypermedia Design: Golden Rules, Design Patterns and Constructive Templates. Paper presented at the Proceedings of the 9th ACM Conference on Hypertext and Hypermedia.

Perrochon, L., & Mann, W. (1999). Inferred Designs. IEEE Software, 16(5), 46-51.

Schwaber, K., & Beedle, M. (2002). Agile Software Development with Scrum. Upper Saddle River, NJ: Prentice-Hall.

Scott, J., & Vessey, I. (2002). Managing risks in enterprise systems implementations. Communications of the ACM, 45(4), 74-81.

Seely Brown, J., & Duguid, P. (1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40-57.

Sneed, H. (2000). Encapsulation of legacy software: A technique for reusing legacy software components. Annals of Software Engineering, 9(1-4), 293-313.

Swanson, E. B., & Ramiller, N. C. (2004). Innovating mindfully with information technology. MIS Quarterly, 28(4), 553-583.

Truex, D., Baskerville, R., & Klein, H. (1999). Growing systems in emergent organizations. Communications of the ACM, 42(8), 117-123.

Weller, M., Pegler, C., & Mason, R. (2005). Students’ experience of component versus integrated virtual learning environments. Journal of Computer Assisted Learning, 21(4), 253-259.

Principles of form and function

The aim of my thesis is to formulate an information systems design theory for e-learning. Even though I have a publication or two that have described early versions of the ISDT, I’ve never been really happy with them. However, I’m getting close to the end of this process, at least for the purposes of getting the thesis submitted.

The following is a first draft of the “Principles of form and function”, one of the primary components of an ISDT as identified by Gregory and Jones (2007). I’ll be putting up a draft of the principles of implementation in a little while (UPDATE principles of implementation now up). These are still just approaching first draft stage, they need a bit more reflection and some comments from my esteemed supervisor. Happy to hear thoughts.

By the way, the working title for this ISDT is now “An ISDT for emergent university e-learning systems”.

Principles of form and function

Gregor and Jones (2007) describe the aim of the principles of form and function as defining the structure, organisation, and functioning of the design product or design method. The ISDT described in Chapter 4 was specifically aimed at the World-Wide Web as shown in its title, “An ISDT for web-based learning systems”. Such technology specific assumptions are missing from the ISDT described in this chapter to avoid technological obsolescence. By not relying on a specific technology the ISDT can avoid a common problem with design research – the perishability of findings – and, enable the on-going evolution of any instantiation to continue regardless of the technology.

The principles of form and function for this ISDT are presented here as divided into three groupings: integrated and independent services; adaptive and inclusive architecture; and, scaffolding, context-sensitive conglomerations. Each of these groupings and the related principles are described in the following sub-sections and illustrated through examples from Webfuse. The underlying aim of the following principles of form and function is to provide a system that is easy to modify and focused on providing context-specific services. The ISDT’s principles of implementation (Section 5.6.4) are designed to work with the principles of form and function in order to enable the design of an emergent university e-learning information system.

Integrated and independent services

The emergent nature of this ISDT means that, rather than prescribe a specific set of services that an instantiation should provide, the focus here is on providing mechanisms to quickly add and modify new services in response to local need. It is assumed that an instantiation would provide an initial set of services (see principle 4) with which system use could begin. Subsequent services would be added in response to observed need.

An emergent university e-learning system should:

  1. Provide a method or methods for packaging and using necessary e-learning services from a variety of sources and of a variety of types.
    For example, Webfuse provided two methods for user-level packaging services: – page types and Wf applications – and also used design patterns and object-oriented design for packging of implementation level services. The types of services packaged through these means included: information stored in databases; various operations on that data; external services such as enterprise authentication services; open source COTS; and, remote applications such as blogging tools.
  2. Provide numerous ways to enable different packages to interact and integrate.
    Webfuse provided a number of methods through which the packaging mechanisms described in the previous point could be integrated. For example, Wf applications provided a simple, consistent interface that enabled easy integration from numerous sources. It was through this approach that Wf applications such as email merge, course list, and course photo album were integrated into numerous other services. To allow staff experience what students say on StudentMyCQU, the ViewStudentMyCQU application was implemented as a wrapper around the StudentMyCQU application.
  3. Provide a packaging mechanism that allows for a level of independence and duplication.
    Within Webfuse, modifications to page types could be made with little or no effect on other page types. It was also possible to have multiple page types of the same type. For example, there were three different web-based discussion forums with slightly different functionality preferred by different users. Similarly, the use of the Model-View-Controller design pattern in Wf applications enabled the same data to be represented in many different forms. For example, class lists could be viewed by campus, with or without student photos, as a CSV file, as a HTML page etc.
  4. Provide an initial collection of services that provide a necessary minimum of common e-learning functionality covering: information distribution, communication, assessment, and administration.
    The initial collection of services for Webfuse in 2000 included the existing page types and a range of support services (see Section 4.4.3). These provided an initial collection of services that provided sufficient services for academics to begin using e-learning. It was this use that provided the opportunity to observe, learn and subsequently add, remove and modify available services (see Section 5.3).
  5. Focus on packaging existing software or services for integration into the system, rather than developing custom-built versions of existing functionality.
    With Webfuse this was mostly done through the use of the page types as software wrappers around existing open source software as described in Chapter 4. The BAM Wf application (see 5.3.6) integrated student use of existing blog engines (e.g. http://wordpress.com) into Webfuse via standardised XML formats.
  6. Present this collection of services in a way that for staff and students resembles a single system.
    With Webfuse, whether users were managing incidents of academic misconduct, finding the phone number of a student, responding to a student query on a discussion forum, or uploading a Word document they believed they were using a single system. Via Staff MyCQU they could access all services in a way that fit with their requirements.
  7. Minimise disruption to the user experience of the system.
    From 1997 through 2009, the authentication mechanism used by Webfuse changed at least four times. Users of Webfuse saw no visible change. Similarly, Webfuse page types were re-designed from purely procedural code to being heavily object-oriented. The only changes in the user interface for page types were where new services were added.

Adaptive and inclusive architecture

Sommerville (2001) defines software architecture as the collection of sub-systems within the software and the framework that provides the necessary control and communication mechanisms for these sub-systems. The principles for integrated and independent services described in the previous section are the “sub-systems” for an emergent university e-learning system. Such as a system, like all large information systems, needs some form of system architecture. The major difference for this ISDT is that traditional architectural concerns such as consistency and efficiency are not as important as being adaptive and inclusive.

The system architecture for an emergent university e-learning system should:

  1. Be inclusive by supporting the integration and control of the broadest possible collection of services.
    The approach to software wrappers adopted as part of the Webfuse page types, was to enable the integration of any external service at the expense of ease of implementation. Consequently, the Webfuse page types architecture integrated a range of applications using very different software technologies including a chat room that was a Java application; a page counter implemented in the C programming language; a lecture page type that combined numerous different applications; and, three different discussion forums implemented in Perl. In addition to the page types, Webfuse also relied heavily on the architecture provided by the Apache web server for access control, authentication, and other services. The BAM Wf application (Section 5.3.6) used RSS and Atom feeds as a method for integrating disparate blog applications. Each of these different approaches embody very different architectural models which increase the cost of implementation, but also increase the breadth of services that can be integrated and controlled.
  2. Provide an architecture that is adaptive to changes in requirements and context.
    One approach is the use of an architectural model that provides high levels of maintainability through fine-grained, self-contained components (Sommerville 2001). This was initially achieved in Webfuse through the page types architecture. However, in order to achieve a long-lived information system there is a need for more than this. Sommerville (2001) suggests that major architectural changes are not a normal part of software maintenance. As a system that operated for 13 years in a Web-environment, Webfuse had to undergo major architectural changes. In early 2000, performance problems arose due to increased demand for dynamic web applications (student quizzes) resulting in a significant change in Webfuse architecture. This change was aided through Webfuse’s reliance on the Apache web server and its continual evolution that provided the scaffolding for this architectural change.

The perspective for this ISDT is that traditional homogenous approaches to software architecture (e.g. component architectures) offer numerous advantages. However, there are some drawbacks. For example, a component architecture can only integrate components that have been written to meet the specifications of the component architecture. Any functionality not available within that component architecture, is not available to the system. To some extent such a limitation closes off possibilities for diversity – which this ISDT views as inherent in university learning and teaching – and future emergent development. This does not rule out the use of component architectures within an emergent university e-learning system, but it does mean that such a system would also be using other architectural models at the same time to ensure it was adaptive and inclusive.

Scaffolding, context-sensitive conglomerations

The design of e-learning in universities requires the combination of skills from a variety of different professions (e.g. instructional design, web design etc), and yet is often most performed by academics with limited knowledge of any of these professions. This limited knowledge creates significant workload for the academics and contributes to the limited quality of much e-learning. Adding experts in these fields to help course design is expensive and somewhat counter to the traditional practice of learning and teaching within universities. This suggests that e-learning in universities has a need for approaches that allow the effective capture and re-use of expertise in a form that can be re-used by non-experts without repeated direct interaction with experts. Such an approach could aim to reduce perceived workload and increase the quality of e-learning.

An emergent university e-learning information system should:

  1. Provide the ability to easily develop, including end user development, larger conglomerations of packaged services.
    A conglomeration is not simply an e-learning service such as a discussion forum. Instead it provides additional scaffolding around such services, possibly combining multiple services, to achieve a higher-level task. While many conglomerations would be expert designed and development, offering support for end-user development would increase system flexibility. The Webfuse default course site approach (Section 5.3.5) is one example of a conglomeration. A default course site combines a number of separate page types (services), specific graphical and instructional designs, and existing institutional content into a course website with a minimum of human input. Another form of conglomeration that developed with Webfuse was Staff MyCQU. This “portal” grew to become a conglomeration of integrated Wf applications designed to package a range of services academics required for learning and teaching.
  2. Ensure that conglomerations provide a range of scaffolding to aid users, increase adoption and increase quality.
    There is likely to be some distance between the knowledge of the user and that required to effectively use e-learning services. Scaffolding provided by the conglomerations should seek to bridge this distance, encourage good practice, and help the user develop additional skills. For example, over time an “outstanding tasks” element was added to Staff MyCQU to remind staff of unfinished work in a range of Wf applications. The BAM Wf application was designed to support the workload involved in tracking and marking individual student reflective journals (Jones and Luck 2009). A more recent example focused more on instructional design is the instructional design wizard included in the new version of the Desire2Learn LMS. This wizard guides academics through course creation via course objectives.
  3. Embed opportunities for collaboration and interaction into conglomerations.
    An essential aim of scaffolding conglomerations is enabling and encouraging academics to learn more about how to effectively use e-learning. While the importance of community and social interaction to learning is widely recognised, most professional development opportunities occur in isolation (Bransford, Brown et al. 2000). Conglomerations should aim to provide opportunities for academics to observe, question and discuss use of the technology. Examples from Webfuse are limited to the ability to observe. For example, all Webfuse course sites were, by default, open for all to see. The CourseHistory Wf application allowed staff to see the grade breakdown for all offerings of any course. A better example would have been if the CourseHistory application encouraged and enabled discussions about grade breakdowns.
  4. Ensure that conglomerations are context-sensitive.
    Effective integration with the specific institutional context enables conglomerations to leverage existing resources and reduce cognitive dissonance. For example, the Webfuse default course site conglomeration was integrated with a range of CQU specific systems, processes and resources. The Webfuse online assignment submission system evolved a number of CQU specific features that significantly increased perceptions of usefulness and ease-of-use (Behrens, Jamieson et al. 2005).

References

Behrens, S., Jamieson, K., Jones, D., & Cranston, M. (2005). Predicting system success using the Technology Acceptance Model: A case study. Paper presented at the Australasian Conference on Information Systems’2005, Sydney.

Bransford, J., Brown, A., & Cocking, R. (2000). How people learn: brain, mind, experience, and school. Washington, D.C.: National Academy Press.

Gregor, S., & Jones, D. (2007). The anatomy of a design theory. Journal of the Association for Information Systems, 8(5), 312-335.

Jones, D., & Luck, J. (2009). Blog Aggregation Management: Reducing the Aggravation of Managing Student Blogging. Paper presented at the World Conference on Education Multimedia, Hypermedia and Telecommunications 2009. from http://www.editlib.org/p/31530.

Sommerville, I. (2001). Software Engineering (6th ed.): Addison-Wesley.

Back into the thesis

After a week away traveling with my sons in Western Queensland, I have to get back into the thesis. This post is essentially an attempt to capture where I’m up to and plan out what I need to do for the next week or so. As with the last update, I’ll start scratching these out as I complete.

The outstanding work can essentially be broken into two parts:

  1. Make modifications to existing chapters; and
    The last PhD update has a good summary of much of this work.
  2. Complete the last chapter.
    Chapter 5 is the last chapter not yet in first draft stage, and it’s not far from it with only two sections not really there yet. So, the focus has to be on finalising those: the ISDT and Evaluation.

The ISDT

This is the major product of the thesis. Much of it is written, what’s remaining to doincludes:

  • Clean up the intro section to use the summary table.
  • Develop a section outlining that the ISDT is more a guide than a hard blue print – what other literature is there on this?
  • Define the purpose of constructs and double check the list in the table
  • Principles of form and function
    • Get a few broader definitions of architecture, especially leaning towards the inclusive idea.
    • Explain the scaffolding idea a bit more, perhaps borrow from this post.
    • Better explanation of human support as an aspect of scaffolding.
  • Principles of implemenation
    • Work in a connection with learning.
  • Artifact mutability
    • Work in the ideas proposed by Hovorka and Germonprez (2009) around artifact mutability.
    • Connect it with learning for academics, students and supporters.
  • Justificatory knowledge
    • Expand support for scaffolding conglomerations.
    • Problems and references for best of breed.
    • MOre work on principles of implementation – including learning
    • distill some of the references and double check outstanding references.
    • Cioborra and related work around the move away from the teleological and analytical.
  • Put more on BAM/BIM into the expository instantiation section.
  • Link the summary table of Webfuse testing with the results of the evaluation.

Evaluation

  • Finalise table 5.6.
  • Decide on which time frame to limit evaluation to and argue why.
    Use all of Webfuse post 2000. Rationale: While Webfuse did not at all times follow the ISDT, it did use aspects of it and even with just aspects showed some benefits. If all had been followed it may have been better again.
  • Table 5.14
    • Calculate number of students in courses with webfuse sites
      • Fix up 1998 entries in stats::courses to put in right term and also add 3rd term.1997, 1999, 2000, 2001 counts.
    • Re-examine course site hits, page updates, unique authors from 2002 onwards?
  • Table 5.15
    • Add standard deviation for updates per staff – based on updates from Table 5.14
    • Update accesses per enrolled student – based on data from Table 5.14
  • Complete the table “usage of webfuse courses”
  • Do power laws get a mention? — NO
  • % of BAM requests of overall staff requests for 2009.
  • Rethink the adoption section and draw on summaries of previous sections and also comparisons with similar section from chapter 4. Also incorporate work from the TAM papers.
  • Modify Figure 5.15 so that it uses a year where data is not missing.

Misc to dos

In working on the above I’ll almost certainly identify additional tasks that need to be performed, they are captured below.

  • Add a summary table of the ISDT in chapter 4.
  • Come up with a title for the chapter 4 ISDT.

References

Hovorka, D., & Germonprez, M. (2009). Tinkering, tailoring and bricolage: Implications for theories of design. Paper presented at the AMCIS 2009.

Light-weight analytics tools as part of scaffolding, context-sensitive conglomerations

A couple of days ago I floated the idea of scaffolding, context-sensitive conglomerations as one idea/model/suggestion for how e-learning systems (currently mostly LMS, but hopefully other models will arise).

George Siemens has posted about light-weight analytics tools such as SNAPP. Both the comments on that post are, to my current somewhat focused/biased perspective, suggestions for the need for scaffolding conglomerations. Both comments are from practitioners who talk about how the supplement their use of discussion forums with other forms of representation. It would appear obvious that these combinations of tools are useful. I’m pretty sure you could find quite a few talented and motivated academics across the world that are using this combination. I’m also pretty sure that few of them would be located within the same institution.

Are there any discussion forum tools in e-learning systems that already provide this sort of scaffolding for users? Are there any IT departments in universities that have recognised this need and are helping academics make this connection?

I’m not aware of any, and this suggests to me that there are some fundamental problems with the way these systems are being supported and structured. i.e. current approaches mean it is unlikely for these sorts of networks of tools/conglomerations to arise.

SNAPP has used a good approach that makes it simpler to create these conglomerations, through the use of browser plugins. But the advantages of that approach come with a negative. i.e. I don’t believe you can currently generate a SNAPP visualisation for a group of courses (e.g. to see how the students/staff in a program are interacting), or compare visualisations between different courses. You also can’t easily combine SNAPP with other context specific data sources such as student records system etc.

Consequently, SNAPP is a great example of a tool that enables a “scaffolding conglomeration” when combined with an LMS discussion forum. But it still remains difficult to add the “context-sensitive” component.

I think there’s value in exploring how SNAPP and similar tools can be used as both scaffolding and context-sensitive conglomerations, and more importantly, what impact it can have on the practice of teachers, and subsequently the quality of learning.

Adding advice

While I remember, there’s a next step that I’d like to see a “scaffolding, context-sensitive conglomeration” take. Advice, examples and connections.

For example, assume I’m teaching a course, I use a discussion forum and I’ve designed its use for a specific pedagogic purpose. I’ve installed SNAPP and to my horror discover a problem. What do I do? What strategies can I employ to address this problem? What strategies have other academics in similar (or even different situations) used? What happened? Can I get their contact details so I can have a chat?

I would imagine that this addition could also be implemented for students who discovered a “bad” pattern in their own practice. They could receive advice, examples and connections from other students.

The learning analytics group seems to include a leaning towards “intelligent”/”adaptive” software to provide this sort of service. I’m more interested in how we can use these tools to connect people and provide the scaffolding that enables and encourages them to take some action.

Page 1 of 11

Powered by WordPress & Theme by Anders Norén

css.php