Assembling the heterogeneous elements for (digital) learning

Month: September 2010

The nature of user involvement in LMS selection and implementation

Given what know (see the below) about the importance of people to the implementation of information systems and also to learning and teaching, how would you characterise the involvement of uses in the selection and implementation of an LMS at most universities? What impact does it have?

The importance of people

There has been significant research within the information systems discipline – a small subset includes user participation and involvement (Ives and Olson 1984); technology acceptance and use (Davis 1989; Venkatesh, Morris et al. 2003); decision-making around system selection and implementation (Bannister and Remenyi 1999; Jamieson, Hyland et al. 2007); system success (DeLone and McLean 1992; Myers 1994); development methods (Mumford 1981); and, the social shaping of technology (Kling 2000)- around the importance and impact of people on information systems and their success. In terms of user participation and involvement, Lynch and Gregor (2004) found that previous studies were inconclusive in terms of links with system success, however, they suggest that the level of influence users have on the development process is a better indicator of system outcomes. The perceptions of the people who may potentially use an information and communication technology play a significant role in their adoption and use of that technology (Jones, Cranston et al. 2005). Information systems are designed and used by people operating in complex social contexts, consequently such a system is understood differently by different people and given meaning by the shared understanding that arises out of social interaction (Doolin 1998).

Similar findings and suggestions are evident in the educational and e-learning literature. John and La Velle (2004) argue that new technologies at most enable rather than dictate change. Dodds (2007) suggests that any excellence demonstrated by a University is not a product of technology, it is a product of the faculty, students and staff who play differing roles in the pursuit of scholarship and learning. For Morgan (2003), teaching and learning are two of the most highly personalised processes. Numerous authors (e.g. Alexander 2001; Oblinger 2003) identify understanding learners, and particularly their learning styles, attitudes, and approaches as essential to the effective facilitation of learning. For Watson (2006), it is clear that consideration of the human dimension is critical to education. Since, as Stewart (2008) observes, the beliefs held by those involved in the educational process, regardless of how ill-informed, can have a tremendous impact on the performance of both students and teachers and how effectively technology may be utilised. Personal characteristics have been found to influence e-learning implementation (Siritongthaworn, Krairit et al. 2006) and most universities are still struggling to engage a significant percentage of students and staff in e-learning (Salmon 2005). While technology may be the stimulus, the essential matters are complex and will be the purview of academics (Oblinger, Barone et al. 2001).


Alexander, S. (2001). E-learning developments and experiences. Education and Training, 43(4/5), 240-248.

Bannister, F., & Remenyi, D. (1999). Value perception in IT investment decisions. Electronic Journal of Information Systems Evaluation, 2(2).

Davis, F. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13(3), 319.

DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60-95.

Dodds, T. (2007). Information Technology: A Contributor to Innovation in Higher Education. New Directions for Higher Education, 2007(137), 85-95.

Doolin, B. (1998). Information technology as disciplinary technology: being critical in interpretive research on information systems. Journal of Information Technology, 13(4), 301-311.

Ives, B., & Olson, M. (1984). User involvement and MIS success: a review of research. Management Science, 30(5), 586-603.

Jamieson, K., Hyland, P., & Soosay, C. (2007). An exploration of a proposed balanced decision model for the selection of enterprise resource planning systems. International Journal of Integrated Supply Management, 3(4), 345-363.

John, P. D., & La Velle, L. B. (2004). Devices and Desires: subject subcultures, pedagogical identity and the challenge of information and communications technology. Technology, Pedagogy and Education, 13(3), 307-326.

Jones, D., Cranston, M., Behrens, S., & Jamieson, K. (2005). What makes ICT implementation successful: A case study of online assignment submission. Paper presented at the ODLAA’2005, Adelaide.

Kling, R. (2000). Learning about information technologies and social change: The contribution of social informatics. The Information Society, 16(3), 217-232.

Lynch, T., & Gregor, S. (2004). User participation in decision support systems development: Influencing system outcomes. European Journal of Information Systems, 13(4), 286-301.

Morgan, G. (2003). Faculty use of course management systems: Educause Centre for Applied Research.

Mumford, E. (1981). Participative systems design: Structure and method. Syst. Objectives solutions, 1(1), 5-19.

Myers, M. D. (1994). Dialectical hermeneutics: a theoretical framework for the implementation of information systems. Information Systems Journal, 5, 51-70.

Oblinger, D. (2003). Boomers, gen-Xers and millennials: Understanding the new students. EDUCAUSE Review, 37 – 47.

Oblinger, D., Barone, C., & Hawkins, B. (2001). Distributed education and its challenge: An overview. Washington DC: American Council on Education.

Salmon, G. (2005). Flying not flapping: a strategic framework for e-learning and pedagogical innovation in higher education institutions. ALT-J, Research in Learning Technology, 13(3), 201-218.

Siritongthaworn, S., Krairit, D., Dimmitt, N., & Paul, H. (2006). The study of e-learning technology implementation: A preliminary investigation of universities in Thailand. Education and Information Technologies, 11(2), 137-160.

Stewart, D. P. (2008). Technology as a management tool in the Community College classroom: Challenges and Benefits. Journal of Online Learning and Teaching, 4(4).

Venkatesh, V., Morris, M., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.

Watson, D. (2006). Understanding the relationship between ICT and education means exploring innovation and change. Education and Information Technologies, 11(3-4), 199-216.

BIM, missing students and Moodle groups

The following is a description of a common “problem” with BIM.


The problem manifests itself along the lines of this:

  • There are students enrolled in the Moodle course you are using BIM for.
  • You’ve set up BIM, allocated student groups to markers.
  • However, some or all of the students aren’t showing up on the “Manage Marking” or “Your students” screens.
  • This might only be for one of the teaching staff, it might be for all.
  • The lead/coordinating teacher can probably find them using “Find student”.


The “Manage marking” and “Your students” tabs in BIM rely on students being members of Moodle groups. If the students aren’t in the Moodle groups that are allocated to the staff, then the staff can’t easily see them.


Some of the potential steps in a solution include:

  • Check that all students are in groups.
    Go to the Moodle “Groups” functionality for your course and make sure that all the students have been allocated to the groups appropriately.

    In an institutional setting this should be done automatically, but if this process is broken you might be having problems.

  • Check that groups have been allocated to staff.
    Go (as the coordinating/lead teacher) into BIM and use the “Allocate markers” tab to allocate the groups to staff as necessary.

Dilbert as an expository instantiation

A few recent posts have been first draft excerpts from my Information Systems Design Theory (ISDT) from emergent university e-learning systems. Being academics and hence somewhat pedantic about these things there are meant to be a number of specific components of an ISDT. One of these is the expository instantiation that is meant to act as both an explanatory device and a platform for testing (Gregor and Jones, 2007) i.e. it’s meant to help explain the theory and also examples of testing the theory.

The trouble is that today’s Dilbert cartoon is probably as good an explanation of what is currently the third principle of implementation for my ISDT.

I’m sure that most folk working in a context where they’ve had to use a corporate information system have experienced something like this. A small change – either to fix a problem or improve the system – simply can’t be made because of the nature of the technology or the processes used to make the changes. The inability to make these changes is a major problem for enterprise systems.

The idea from the ISDT is that the development and support team for an emergent university e-learning system should be able to make small scale changes quickly without having to push them up the governance hierarchy. Where possible the team should have the skills, insight, judgement and trust so that “small scale” is actually quite large.

An example

The Webfuse e-learning system that informed much the ISDT provides one example. Behrens (2009) quotes a user of Webfuse about one example of how it was responsive

I remember talking to [a Webfuse developer] and saying how I was having these problems with uploading our final results into [the Enterprise Resource Planning (ERP) system] for the faculty. He basically said, “No problem, we can get our system to handle that”… and ‘Hey presto!’ there was this new piece of functionality added to the system… You felt really involved… You didn’t feel as though you had to jump through hoops to get something done.

Then this is compared with a quote from one of the managers responsible for the enterprise system

We just can’t react in the same way that the Webfuse system can, we are dealing with a really large and complex ERP system. We also have to keep any changes to a minimum because of the fact that it is an ERP. I can see why users get so frustrated with the central system and our support of it. Sometimes, with all the processes we deal with it can take weeks, months, years and sometimes never to get a response back to the user.

Is that Dilbert or what?

The problem with LMS

Fulfilling this requirement is one of the areas where most LMS create problems. For most universities/orgnaisations it is getting into the situation where the LMS (even Moodle) is approaching the “complex ERP system” problem used in the last quote above. Changing the LMS is to fraught with potential dangers that these changes can’t be made quickly. Most organisations don’t try, so we’re back to a Dilbert moment.

Hence, I think there are two problems facing universities trying to fulfil principle #3:

  1. Having the right people in the support and development team with the right experience, insight and judgement is not a simple thing and is directly opposed to the current common practice which is seeking to minimise having these people. Instead there’s reliance on helpdesk staff and trainers.
  2. The product problem. i.e. it’s too large and difficult to change quickly and safely. I think there’s some interesting work to be done here within Moodle and other open source LMS. How do you balance the “flexibility” of open source with the complexity of maintaining a stable institutional implementation?


Behrens, S. (2009). Shadow systems: the good, the bad and the ugly. Communications of the ACM, 52(2), 124-129.

Gregor, S., & Jones, D. (2007). The anatomy of a design theory. Journal of the Association for Information Systems, 8(5), 312-335.

The office and more procrastination

Today has not been a good day for progress on the thesis. Have posted a few things I’ve already written and there are some folks reading bits of it thanks to Stephen Downes and OLDaily. That these folk find the work vaguely interesting enough to click on the link is a comforting thing, if not a motivating one for today.

Entering a nascent meme?

So, instead of writing, I thought I’d procrastinate a bit more and attempt to enter what might be a nascent meme. Lisa Lane started it by posting this annotated photo of her office. I came across this, via Mike Bogle’s post showing off his more spartan office. Not sure if there are other examples, but it’s enough of an excuse for me to waste some time.

So here it is, a photo of where I’m sitting at the moment not working on the thesis. Unlike Mike and Lisa, I’m in the very lucky position of not having an office at work, as I’m not currently working. An experience I’m increasingly enjoying, but realise will have to end soon. Which is a bit like the PhD only that I’m not really enjoying it at all.

The Office

Of course one of the reasons for doing this was to play with mbedr, but it won’t work on this blog as strips object and iframe tags. You’ll have to go here to see the mbedr version or the Flickr original.

Justificatory knowledge

The following is a first version of the justificatory knowledge section of my ISDT for emergent university e-learning systems. Still fairly uncertain just how information is meant to go in here and also just how far I should go with the reference to other theories (there are lots) and how much time should be spent looking at the interactions between them.

If you have some literature/theories which support or contradict this approach, will be really happy to hear about it.

Justificatory knowledge

The purpose of the justificatory knowledge component is to provide an explanation of why the ISDT is structured as it is and why it should be expected to work appropriately. Much of the justificatory knowledge that underpins this ISDT has been described previously within the literature review (Chapter 2), the first Webfuse action research cycle (Chapter 4), and earlier in this chapter. To avoid repetition this section provides a summary and brief discussion of the justificatory knowledge underpinning the ISDT for emergent university e-learning systems. This summary is linked specifically to the ISDT’s principles for form and function, and principles of implementation.

The justificatory knowledge described below arose from the experiences obtained and literature read during the design and support of Webfuse. This knowledge described below is not necessarily complete or the only established knowledge – theoretical or otherwise – that could be used to justify the principles of the ISDT. Hovorka and Germonprez (2009) identify as a weakness of design research, the lack of guidance around the interaction between the various kernel theories that make up justificatory knowledge and how the influence of these kernel theories may change during use. To some extent, the context-sensitive, emergent nature of the approach embodied by this ISDT – and its kernel theories – means that such advice is embedded in the justificatory knowledge that supports the ISDT.

Justificatory knowledge for principles for form and function

Table 5.20 provides a summary of the justificatory knowledge and is followed by a brief description. For each the three categories of principles of form and function for this ISDT, Table 5.20 provides pointers to sections of this thesis and references to literature that describe the justificatory knowledge in more detail.

Table 5.20 – Summary of justificatory knowledge for principles of form and function
Principle Justificatory knowledge
Integrated and independent services Section 2.3.2 – Software wrappers (Bass, Clements et al. 1998; Sneed 2000)
Adaptive and inclusive architecture Systems of Systems (Perrochon and Mann 1999)
Section 2.3.2 – Best of breed (Light, Holland et al. 2001; Lowe and Locke 2008), Service Oriented Architectures (Chen, Chen et al. 2003; Weller, Pegler et al. 2005), End-user development (Eriksson and Dittrich 2007)
Section 4.4.4 – Micro-kernel architecture (Liedtke 1995)
Scaffolding, context-sensitive conglomerations Constructive templates (Nanard, Nanard et al. 1998) , End-user development (Eriksson and Dittrich 2007)

As summarised in chapter 4, a software wrapper is a type of encapsulation that enables software components to be encased in an alternative abstraction that enables clients, often in a new context, to access the wrapped components services (Bass, Clements et al. 1998; Sneed 2000). As such software wrappers are one example of an approach that provides integrated and independent software services.

Some of the relative advantages and limitations more tightly integrated systems is provided by the enterprise software literature. In this literature, comparisons between tightly integrated systems and best-of-breed approaches have argued that integration involves centralisation of processes and a consequently a tendency to reduce autonomy, increase rigidity, and reduce competitiveness (Light, Holland et al. 2001; Lowe and Locke 2008). The best-of-breed approach, focusing on a more inclusive integration of appropriate packages, increases system flexibility while at the same time requires greater time, skills and resources to integrate diverse applications (Light, Holland et al. 2001). Perrochon and Mann (1999) argue that traditional approaches to system architecture, even those with a focus on adaptivity, are appropriate for greenfield developments due to their reliance on the assumption of design (specify architecture) and then implement. The rise of component-oriented software has created the problem of systems of systems that require the combination of well-engineered components or systems into an overall system they were never, and could never be, designed for (Perrochon and Mann 1999).

The concept of constructive templates (Catlin, Garret et al. 1991; Nanard, Nanard et al. 1998) was developed in response to the difficulty faced by content providers in developing hypermedia structures that followed the known principles of interface and hypermedia design. Constructive templates helped content experts to create well designed hypermedia (Catlin, Garret et al. 1991).

Justificatory knowledge for principles of implementation

The justificatory knowledge for this ISDT’s principles of implementation – summarised in Table 5.21 and briefly described below – draws heavily on what is down about alternatives to traditional, plan-driven software development methodologies as discussed in Section 2.4 and Sections 5.3.1 and 5.3.2 of this chapter.

Table 5.21 – Summary of justificatory knowledge for principles of implementation
Principle Justificatory knowledge
Multi-skilled, integrated development and support team Job rotation, multi-skilling etc (Faegri, Dyba et al. 2010), Organisational learning (Seely Brown and Duguid 1991), Situated learning, Situated action, Communities of practice (Seely Brown and Duguid 1991), Knowledge-based theory of organizational capability (Grant 1996)
An adopter-focused, emergent development process Section 2.4 examines the topic of processes, including a comparison of traditional plan-driven processes (e.g. the SDLC) and learning-focused processes such as emergent development. Additional discussion occurs in Section 5.3.2
Section 5.3.1 introduces the conception of adopter-focused development.
A supportive organisational context Organisational fit (Hong and Kim 2002), Strategic alignment (Henderson and Venkatraman 1993), Bricolage (Chae and Lanzara 2006), Mindful innovation (Swanson and Ramiller 2004)

Seely Brown and Duguid (1991) argue that the tendency for education, training and technology design to focus on abstract representations that are detached from practice actually distort the intricacies of practice and consequently hinder how well practice can be understood, engendered, or enhanced. The idea of the development team integrated and embedded in the everyday practice of e-learning seeks to improve the learning of both academics and students about how to harness e-learning, and also improve the learning of the development team (and the organisation) about how e-learning is being used. The ISDT seeks to establish a process for supporting and developing e-learning which is situated in shared practice with a joint, collective purpose.

Faegri, Dyba et al (2010) argue that turbulent environments increase the importance of employee skills and competences and that having employees cycle through different jobs – such as developers being on helpdesk – can improve knowledge redundancy, organizational knowledge creation, and other benefits. Faegri, Dyba et al (2010) also cite Keil-Slawik (1992) as arguing that full understanding of software requires experience developing the software. The traditional hierarchical structures associated with the division of labour around the e-learning within universities – e.g. helpdesk and developers organized into separate units within an IT division; learning and teaching experts located in another division focused on learning and teaching; and, faculty academics located academic units – are seen by Grant (1996) to inhibit the ability to integrate knowledge from members of an organisation. Such integration is seen as fundamental to the organisation’s ability to create and sustain competitive advantage (Grant 1996).

There is significant literature (March 1991; Baskerville, Travis et al. 1992; Mintzberg 1994; Bamford and Forrester 2003) in a variety of disciplines that identifies plan-driven processes as the dominant approach in most organizations. This and related literature also examines the limitations this over-emphasis suffers, especially in contexts with rapid change or significant diversity (see Section 2.4). Consequently there is significant literature identifying both the theoretical basis and guidance (Introna 1996; Truex, Baskerville et al. 1999) and practical implementation methods (Beck 2000; Schwaber and Beedle 2002) for more emergent or adopter-based development processes.

An emergent, university e-learning information system is a large-scale information system. In this context, “large-scale” is used in the sense adopted by Chae and Lanzara (2006), as referring to systems that involve both organisational technologies and technological innovations that “comprise and connect multiple communities of practice within an organisation or between organistions”. Literature examining success factors with information systems development (e.g. Ewusi-Mensah 1997; Scott and Vessey 2002) has long considered it vital for senior management to be supportive of and committed to systems development. Brown et al (2007) identify commitment – defined as the resources dedicated to IT, organisational dedication to change procedures, and top management support – as one of two most cited problems in the IS projects they examined and identified it as the factor most cited within the literature.

Organisational fit (Hong and Kim 2002) and strategic alignment (Henderson and Venkatraman 1993) between various aspects of an organisation and its information technology systems and processes have long been argued as critical success factors. A similar importance on having an organisational context that is committed and appropriate to information systems development is also found in approaches that are less traditional or teleological (e.g. bricolage and mindful innovation) and have more in common with the emergent, adopter-focused approach advocate in this ISDT. Collective or organisational bricolage requires the combined effort of several individuals and groups (Chae and Lanzara 2006). An organisation which is mindful in innovating with IT, uses reasoning grounded in its own organisational facts and specifics when thinking about the innovation, the organisation recognises that context matters (Swanson and Ramiller 2004). Within mindful innovation, management have a responsibility to foster conditions that prompt collective mindfulness (Swanson and Ramiller 2004).


Bamford, D., & Forrester, P. (2003). Managing planned and emergent change within an operations management environment. International Journal of Operations and Production Management, 23(5), 546-564.

Baskerville, R., Travis, J., & Truex, D. (1992). Systems without method: the impact of new technologies on information systems development projects. In K. E. Kendall (Ed.), The Impact of Computer Supported Technologies on Information Systems Development (pp. 241-251). Amsterdam: North-Holland.

Bass, L., Clements, P., & Kazman, R. (1998). Software Architecture in Practice. Boston: Addison-Wesley.

Beck, K. (2000). Extreme Programming Explained: Embrace Change: Addison-Wesley.

Brown, S., Chervany, N., & Reinicke, B. (2007). What matters when introducing new information technology. Communications of the ACM, 50(9), 91-96.

Catlin, K., Garret, L. N., & Launhardt, J. (1991). Hypermedia Templates: An Author’s Tool. Paper presented at the Proceedings of Hypertext’91.

Chae, B., & Lanzara, G. F. (2006). Self-destructive dyamics in large-scale technochange and some ways of conteracting it. Information Technology & People, 19(1), 74-97.

Chen, M., Chen, A., & Shao, B. (2003). The implications and impacts of web services to electronic commerce research and practices. Journal of Electronic Commerce Reseaerch, 4(4), 128-139.

Eriksson, J., & Dittrich, Y. (2007). Combining tailoring and evolutionary software development for rapidly changing business systems. Journal of Organizational and End User Computing, 19(2), 47-64.

Ewusi-Mensah, K. (1997). Critical Issues in Abandonded Information Systems Development Projects. Communications of the ACM, 40(9), 74-80.

Faegri, T. E., Dyba, T., & Dingsoyr, T. (2010). Introducing knowledge redundancy practice in software development: Experiences with job rotation in support work. Information and Software Technology, 52(10), 1118-1132.

Grant, R. (1996). Prospering in dynamically competitive environments: organizational capability as knowledge integration. Organization Science, 7(4), 357-387.

Henderson, J., & Venkatraman, N. (1993). Strategic alignment: Leveraging information technology for transforming organizations. IBM Systems Journal, 32(1), 4-16.

Hong, K.-K., & Kim, Y.-G. (2002). The critical success factors for ERP implementation: an organizational fit perspective. Information & Management, 40(1), 25-40.

Hovorka, D., & Germonprez, M. (2009). Tinkering, tailoring and bricolage: Implications for theories of design. Paper presented at the AMCIS 2009.

Introna, L. (1996). Notes on ateleological information systems development. Information Technology & People, 9(4), 20-39.

Keil-Slawik, R. (1992). Artifacts in software design. In C. Floyd, H. Zullighoven, R. Budde & R. Keil-Slawik (Eds.), Software Development and Reality Construction (pp. 168-188). Berlin: Springer-Verlag.

Liedtke, J. (1995). On micro-kernel construction. Operating Systems Review, 29(5), 237-250.

Light, B., Holland, C., & Wills, K. (2001). ERP and best of breed: a comparative analysis. Business Process Management Journal, 7(3), 216-224.

Lowe, A., & Locke, J. (2008). Enterprise resource planning and the post bureaucratic organization. Information Technology & People, 21(4), 375-400.

March, J. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1), 71-87.

Mintzberg, H. (1994). The rise and fall of strategic planning: Reconceiving roles for planning, plans, planners. New York: Free Press.

Nanard, M., Nanard, J., & Kahn, P. (1998). Pushing Reuse in Hypermedia Design: Golden Rules, Design Patterns and Constructive Templates. Paper presented at the Proceedings of the 9th ACM Conference on Hypertext and Hypermedia.

Perrochon, L., & Mann, W. (1999). Inferred Designs. IEEE Software, 16(5), 46-51.

Schwaber, K., & Beedle, M. (2002). Agile Software Development with Scrum. Upper Saddle River, NJ: Prentice-Hall.

Scott, J., & Vessey, I. (2002). Managing risks in enterprise systems implementations. Communications of the ACM, 45(4), 74-81.

Seely Brown, J., & Duguid, P. (1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40-57.

Sneed, H. (2000). Encapsulation of legacy software: A technique for reusing legacy software components. Annals of Software Engineering, 9(1-4), 293-313.

Swanson, E. B., & Ramiller, N. C. (2004). Innovating mindfully with information technology. MIS Quarterly, 28(4), 553-583.

Truex, D., Baskerville, R., & Klein, H. (1999). Growing systems in emergent organizations. Communications of the ACM, 42(8), 117-123.

Weller, M., Pegler, C., & Mason, R. (2005). Students’ experience of component versus integrated virtual learning environments. Journal of Computer Assisted Learning, 21(4), 253-259.

Principles of implementation

The following is the other main component of the ISDT for e-learning I’m working on. A prior post focused on the principles of form and function (i.e. the structure of an information system, the product), the following focuses the principles of implementation (i.e. the process).

While both these posts are based on a bit of prior work, the current formulation of the principles are very much a first draft (not only the principles themselves, but the expression as well). I’ll be coming back to them and welcome feedback.

The biggest worry I have at the moment with the following is that it doesn’t express strongly enough the need for “end-user development” (i.e. enabling academics/students to do much of this themselves), while balancing that with a need (or perhaps the point) of having a specific support team. Need to work on this.

Much of the following is based on experience, but there are also theoretical justifications, that’s another section that is coming.

Principles of implementation

As defined by Gregor and Jones (2007) the principles of implementation specify “the means by which the design is brought into being”. The previous iteration of this ISDT described in Chapter 4 was extremely limited in terms of principles of implementation. This represented the implicit and naïve use of traditional software development methodologies during the initial implementation of Webfuse from 1997 through 1999. As described in this chapter a significant amount of work associated with Webfuse after 1999 has been aimed at developing more appropriate principles of implementation for university e-learning. The result is a rejection of more traditional, plan-driven approaches to information systems implementation, and, instead a set of implementation principles that are founded heavily on the ideas of adopter-focused and emergent approaches to information systems development. The following principles of implementation for the “ISDT for emergent university e-learning systems” are grouped and described in the following three sub-sections.

Multi-skilled, integrated development and support team

An emergent university e-learning information system should have a team of people that:

  1. Is responsible for performing the necessary training, development, helpdesk, and other support tasks required by system use within the institution and contains an appropriate combination of technical, training, media design and production, institutional, and learning and teaching skills and knowledge.
    The merging of these tasks into a single integrated unit avoids problems that arise when such responsibilities are spread across different organisational units. For example, the Infocom web team (the group supporting Webfuse from 2000 through 2004) was responsible for all these tasks and were co-located within a single building. This enabled significant opportunities for knowledge sharing and team building. It also meant that people providing training and support for the system were generally developers of the system with a much greater understanding of what was possible and what could be changed.
  2. Integrated into the everyday practice of learning and teaching within the institution and cultivates relationships with system users, especially teaching staff.
    Being regularly involved with teaching staff in their daily practice enables the building of trust through shared problem-solving and greater insight into the problems and experience of system users. A necessary foundation on which to implement an adopter-focused and emergent development approach. For example, the Infocom web team provided services, including helpdesk, that were used throughout the learning process and, as a result, they interacted with academic staff and students everyday.
  3. Are empowered to make small-scale changes to the system in response to problems, observations, and lessons learned during system support and training tasks rapidly without needing formal governance approval.
    The ability to make visible changes to systems in response to user problems or requests that provide a sense of user-involvement and lead to feelings of trust. Traditional governance processes can slow down and even prevent small-scale changes ever happening. The Infocom web team was free to make minor changes to Webfuse as soon as possible given other workload constraints. In many cases the developers would make these changes in order to prevent repeated helpdesk queries about the issues. Behrens (2009) quotes a manager in CQU’s IT division describing the types of changes made to Webfuse as “not even on the priority radar” due to traditional IT management techniques and quotes a Webfuse user as saying “You felt really involved”.
  4. Actively examines and reflects on system use and non-use – with a particular emphasis on identifying and examining what early innovators – to identify areas for system improvement and extension;
    While the Webfuse default course site approach provided an initial structure for course sites, this structure could be significantly modified through use of the page types or real course sites (Section 5.3.5). Observing what changes were being made, typically by innovative users, was a useful way of identifying improvements. For example, the addition of a study schedule page to the default course site arose from observing its use in non-default course sites.
  5. Plays a significant part of the governance process.
    This enables the governance process to harness the detailed insight into user experience with the system to inform decisions about system evolution.

    For example, governance of Webfuse was, for a short period of time, implemented using a representative committee of faculty staff that met each month. At those meetings, the Infocom web team would present a summary of what it had done in the previous month and, based on directions given by senior management (top-down) and insights gained from observation of system use (bottom-up), present a draft plan for the next month. Members of the governance group would comment and suggest changes to the plan.

An adopter-focused, emergent development process

Software development performed as part of an emergent university e-learning information system should:

  1. Adopt the goals, perspectives and techniques associated with alternate information systems development perspectives and methodologies such as emergent development (Truex, Baskerville et al. 1999), ateleological design (Introna 1996), and agile development methodologies (Highsmith and Cockburn 2001).
    As described in Sections 5.3.2, from 2000 onwards Webfuse development was increasingly informed by an emergent development approach using practices associated with eXtreme programming (Beck 1999).
  2. Use in-depth knowledge of the human, social and interpersonal aspects of the institutional context to inform the design and dissemination of new system features with the intent of encouraging greater levels of adoption by users.
    As outlined in Section 5.3.1, Webfuse development was informed by an adopter-focused approach.
  3. Maximise the ability of the system to be tailored for and by the users of the system.
    End-user development is perhaps the ultimate adopter-based development process as end-users develop applications in response to their own needs and perspectives. However, even with systems that enable end-user development there remains a need for effective and appropriate interaction between end-users, system owners and system developers (Eriksson and Dittrich 2007).
  4. Seek to establish a balance between the internal emergent process and external plan-driven processes.
    The title and the majority of the principles of this ISDT suggest that e-learning should adopt an emergent process. This ISDT seems to be heavily in what Clegg (2002) describes as the “learning school” of thought around process (see Section 2.4.1 for a discussion of the two predominant views on process). However, as shown in Section 2.4.2 the majority of processes within universities continue to adopt, or at least espouse, a heavy emphasis on plan-driven processes. While these perspectives represent divergent ways of understanding process, there are risky extremes inherent in both approaches that need to be avoided (Jones, Luck et al. 2005). An instantiation should seek to achieve an appropriate synthesis between the two approaches that enables adaptability and inclusiveness within an appropriately efficient and sufficiently resourced framework that is moving towards institutional goals.

A supportive organisational context

The organisational context in which an emergent university e-learning information system is used should:

  1. Have senior management and an organisational culture that understands, accepts, and actively enables and encourages an emergent approach to e-learning.
    As briefly described in Section 5.5 the emergent development of Webfuse was most effective when the faculty Dean recognised and supported and the benefits of emergent development and what is means and requires. A leader or organisational culture that places significant value on consistency, efficiency and more plan-based processes will not value the characteristics of emergent development.
  2. Use an approach to governance that encourages decentralised control while maintaining an appropriate, but minimal, level of top-down control.
    A description of the governance process used by Webfuse is provided in


Beck, K. (1999). Embracing change with extreme programming. IEEE Computer, 32, 70-77.

Clegg, S. (2002). Management and organization paradoxes. Philadelphia, PA: John Benjamins Publishing.

Eriksson, J., & Dittrich, Y. (2007). Combining tailoring and evolutionary software development for rapidly changing business systems. Journal of Organizational and End User Computing, 19(2), 47-64.

Gregor, S., & Jones, D. (2007). The anatomy of a design theory. Journal of the Association for Information Systems, 8(5), 312-335.

Highsmith, J., & Cockburn, A. (2001). Agile software development: Business of innovation. IEEE Computer, 34(9), 120-122.

Introna, L. (1996). Notes on ateleological information systems development. Information Technology & People, 9(4), 20-39.

Jones, D., Luck, J., McConachie, J., & Danaher, P. A. (2005). The teleological brake on ICTs in open and distance learning. Paper presented at the Conference of the Open and Distance Learning Association of Australia’2005, Adelaide.

Truex, D., Baskerville, R., & Klein, H. (1999). Growing systems in emergent organizations. Communications of the ACM, 42(8), 117-123.

Alan Kay and some reasons why the educational technology revolution hasn't happened

While reading a recent post from Gardner Campbell I was taken by a quote from Alan Kay

The computer is simply an instrument whose music is ideas

A google search later and I came across this interview with Kay for the Scholastic Administrator magazine. The article is titled “Alan Kay still waiting for the revolution” and there are some, for me, interesting perspectives. A smattering below.

The difficult part is helping the helpers

Kay identifies the greatest obstacle to his work as being “helping the helpers”. i.e. the teachers. In talking about Logo, Kay a key failure being that the second and third waves of teachers were not interesting in Logo and didn’t have the math skills to teach well with Logo.

I see this as the biggest problem around e-learning (or blended, flexible, personal etc learning if that’s your buzz word of the moment) within universities, helping the helpers.

The tokenism of computers

On computers and tokenism

But I think the big problem is that schools have very few ideas about what to do with the computers once the kids have them. It’s basically just tokenism, and schools just won’t face up to what the actual problems of education are, whether you have technology or not.

Again there’s some resonance with universities. For a lot of senior and IT management in universities there’s an idea that we must have an LMS, but there’s not always a good idea of what the organisation should do with it once it has it. The most important part of that “idea”, is being able to identify what about the policies and practices of the institution needs to change to best achieve that idea.

For example, with the LMS the institution can increase interaction between staff and students via discussion forums, e-portfolios etc. But we won’t change the workload or funding model for teaching, or recognise the need to change the timetable to remove the traditional 2 hour lecture, 2 hour tutorial model.

The difference between music and instruments

In talking about some of the limits or potential problems associated with the trend to one-to-one computer

Think about it: How many books do schools have—and how well are children doing at reading? How many pencils do schools have—and how well are kids doing at math? It’s like missing the difference between music and instruments. You can put a piano in every classroom, but that won’t give you a developed music culture, because the music culture is embodied in people……The important thing here is that the music is not in the piano. And knowledge and edification is not in the computer. The computer is simply an instrument whose music is ideas.

The provision of the LMS or some other “instrument” is the simple task. Helping the people figure out what you want to do with it and how it can be done well, is the hard part.

Helping everyone find their inner musician

Why educational computing hasn’t lived up to the potential?

So computers are actually irrelevant at this level of discussion—they are just musical instruments. The real question is this: What is the prospect of turning every elementary school teacher in America into a musician? That’s what we’re talking about here. Afterward we can worry about the instruments.

How do you encourage and enable university academics to become musicians? I don’t think you can forget about computers, e-learning or the LMS. They are already in universities. There’s a need to look out how you can change how academics experience these technologies so that they can start developing their musical ability. Sending them to “band camp” (e.g. Grad Cert in Higher Education) isn’t enough if they return to a non-musical family. The environment they live in has to be musical in every aspect.

Principles of form and function

The aim of my thesis is to formulate an information systems design theory for e-learning. Even though I have a publication or two that have described early versions of the ISDT, I’ve never been really happy with them. However, I’m getting close to the end of this process, at least for the purposes of getting the thesis submitted.

The following is a first draft of the “Principles of form and function”, one of the primary components of an ISDT as identified by Gregory and Jones (2007). I’ll be putting up a draft of the principles of implementation in a little while (UPDATE principles of implementation now up). These are still just approaching first draft stage, they need a bit more reflection and some comments from my esteemed supervisor. Happy to hear thoughts.

By the way, the working title for this ISDT is now “An ISDT for emergent university e-learning systems”.

Principles of form and function

Gregor and Jones (2007) describe the aim of the principles of form and function as defining the structure, organisation, and functioning of the design product or design method. The ISDT described in Chapter 4 was specifically aimed at the World-Wide Web as shown in its title, “An ISDT for web-based learning systems”. Such technology specific assumptions are missing from the ISDT described in this chapter to avoid technological obsolescence. By not relying on a specific technology the ISDT can avoid a common problem with design research – the perishability of findings – and, enable the on-going evolution of any instantiation to continue regardless of the technology.

The principles of form and function for this ISDT are presented here as divided into three groupings: integrated and independent services; adaptive and inclusive architecture; and, scaffolding, context-sensitive conglomerations. Each of these groupings and the related principles are described in the following sub-sections and illustrated through examples from Webfuse. The underlying aim of the following principles of form and function is to provide a system that is easy to modify and focused on providing context-specific services. The ISDT’s principles of implementation (Section 5.6.4) are designed to work with the principles of form and function in order to enable the design of an emergent university e-learning information system.

Integrated and independent services

The emergent nature of this ISDT means that, rather than prescribe a specific set of services that an instantiation should provide, the focus here is on providing mechanisms to quickly add and modify new services in response to local need. It is assumed that an instantiation would provide an initial set of services (see principle 4) with which system use could begin. Subsequent services would be added in response to observed need.

An emergent university e-learning system should:

  1. Provide a method or methods for packaging and using necessary e-learning services from a variety of sources and of a variety of types.
    For example, Webfuse provided two methods for user-level packaging services: – page types and Wf applications – and also used design patterns and object-oriented design for packging of implementation level services. The types of services packaged through these means included: information stored in databases; various operations on that data; external services such as enterprise authentication services; open source COTS; and, remote applications such as blogging tools.
  2. Provide numerous ways to enable different packages to interact and integrate.
    Webfuse provided a number of methods through which the packaging mechanisms described in the previous point could be integrated. For example, Wf applications provided a simple, consistent interface that enabled easy integration from numerous sources. It was through this approach that Wf applications such as email merge, course list, and course photo album were integrated into numerous other services. To allow staff experience what students say on StudentMyCQU, the ViewStudentMyCQU application was implemented as a wrapper around the StudentMyCQU application.
  3. Provide a packaging mechanism that allows for a level of independence and duplication.
    Within Webfuse, modifications to page types could be made with little or no effect on other page types. It was also possible to have multiple page types of the same type. For example, there were three different web-based discussion forums with slightly different functionality preferred by different users. Similarly, the use of the Model-View-Controller design pattern in Wf applications enabled the same data to be represented in many different forms. For example, class lists could be viewed by campus, with or without student photos, as a CSV file, as a HTML page etc.
  4. Provide an initial collection of services that provide a necessary minimum of common e-learning functionality covering: information distribution, communication, assessment, and administration.
    The initial collection of services for Webfuse in 2000 included the existing page types and a range of support services (see Section 4.4.3). These provided an initial collection of services that provided sufficient services for academics to begin using e-learning. It was this use that provided the opportunity to observe, learn and subsequently add, remove and modify available services (see Section 5.3).
  5. Focus on packaging existing software or services for integration into the system, rather than developing custom-built versions of existing functionality.
    With Webfuse this was mostly done through the use of the page types as software wrappers around existing open source software as described in Chapter 4. The BAM Wf application (see 5.3.6) integrated student use of existing blog engines (e.g. into Webfuse via standardised XML formats.
  6. Present this collection of services in a way that for staff and students resembles a single system.
    With Webfuse, whether users were managing incidents of academic misconduct, finding the phone number of a student, responding to a student query on a discussion forum, or uploading a Word document they believed they were using a single system. Via Staff MyCQU they could access all services in a way that fit with their requirements.
  7. Minimise disruption to the user experience of the system.
    From 1997 through 2009, the authentication mechanism used by Webfuse changed at least four times. Users of Webfuse saw no visible change. Similarly, Webfuse page types were re-designed from purely procedural code to being heavily object-oriented. The only changes in the user interface for page types were where new services were added.

Adaptive and inclusive architecture

Sommerville (2001) defines software architecture as the collection of sub-systems within the software and the framework that provides the necessary control and communication mechanisms for these sub-systems. The principles for integrated and independent services described in the previous section are the “sub-systems” for an emergent university e-learning system. Such as a system, like all large information systems, needs some form of system architecture. The major difference for this ISDT is that traditional architectural concerns such as consistency and efficiency are not as important as being adaptive and inclusive.

The system architecture for an emergent university e-learning system should:

  1. Be inclusive by supporting the integration and control of the broadest possible collection of services.
    The approach to software wrappers adopted as part of the Webfuse page types, was to enable the integration of any external service at the expense of ease of implementation. Consequently, the Webfuse page types architecture integrated a range of applications using very different software technologies including a chat room that was a Java application; a page counter implemented in the C programming language; a lecture page type that combined numerous different applications; and, three different discussion forums implemented in Perl. In addition to the page types, Webfuse also relied heavily on the architecture provided by the Apache web server for access control, authentication, and other services. The BAM Wf application (Section 5.3.6) used RSS and Atom feeds as a method for integrating disparate blog applications. Each of these different approaches embody very different architectural models which increase the cost of implementation, but also increase the breadth of services that can be integrated and controlled.
  2. Provide an architecture that is adaptive to changes in requirements and context.
    One approach is the use of an architectural model that provides high levels of maintainability through fine-grained, self-contained components (Sommerville 2001). This was initially achieved in Webfuse through the page types architecture. However, in order to achieve a long-lived information system there is a need for more than this. Sommerville (2001) suggests that major architectural changes are not a normal part of software maintenance. As a system that operated for 13 years in a Web-environment, Webfuse had to undergo major architectural changes. In early 2000, performance problems arose due to increased demand for dynamic web applications (student quizzes) resulting in a significant change in Webfuse architecture. This change was aided through Webfuse’s reliance on the Apache web server and its continual evolution that provided the scaffolding for this architectural change.

The perspective for this ISDT is that traditional homogenous approaches to software architecture (e.g. component architectures) offer numerous advantages. However, there are some drawbacks. For example, a component architecture can only integrate components that have been written to meet the specifications of the component architecture. Any functionality not available within that component architecture, is not available to the system. To some extent such a limitation closes off possibilities for diversity – which this ISDT views as inherent in university learning and teaching – and future emergent development. This does not rule out the use of component architectures within an emergent university e-learning system, but it does mean that such a system would also be using other architectural models at the same time to ensure it was adaptive and inclusive.

Scaffolding, context-sensitive conglomerations

The design of e-learning in universities requires the combination of skills from a variety of different professions (e.g. instructional design, web design etc), and yet is often most performed by academics with limited knowledge of any of these professions. This limited knowledge creates significant workload for the academics and contributes to the limited quality of much e-learning. Adding experts in these fields to help course design is expensive and somewhat counter to the traditional practice of learning and teaching within universities. This suggests that e-learning in universities has a need for approaches that allow the effective capture and re-use of expertise in a form that can be re-used by non-experts without repeated direct interaction with experts. Such an approach could aim to reduce perceived workload and increase the quality of e-learning.

An emergent university e-learning information system should:

  1. Provide the ability to easily develop, including end user development, larger conglomerations of packaged services.
    A conglomeration is not simply an e-learning service such as a discussion forum. Instead it provides additional scaffolding around such services, possibly combining multiple services, to achieve a higher-level task. While many conglomerations would be expert designed and development, offering support for end-user development would increase system flexibility. The Webfuse default course site approach (Section 5.3.5) is one example of a conglomeration. A default course site combines a number of separate page types (services), specific graphical and instructional designs, and existing institutional content into a course website with a minimum of human input. Another form of conglomeration that developed with Webfuse was Staff MyCQU. This “portal” grew to become a conglomeration of integrated Wf applications designed to package a range of services academics required for learning and teaching.
  2. Ensure that conglomerations provide a range of scaffolding to aid users, increase adoption and increase quality.
    There is likely to be some distance between the knowledge of the user and that required to effectively use e-learning services. Scaffolding provided by the conglomerations should seek to bridge this distance, encourage good practice, and help the user develop additional skills. For example, over time an “outstanding tasks” element was added to Staff MyCQU to remind staff of unfinished work in a range of Wf applications. The BAM Wf application was designed to support the workload involved in tracking and marking individual student reflective journals (Jones and Luck 2009). A more recent example focused more on instructional design is the instructional design wizard included in the new version of the Desire2Learn LMS. This wizard guides academics through course creation via course objectives.
  3. Embed opportunities for collaboration and interaction into conglomerations.
    An essential aim of scaffolding conglomerations is enabling and encouraging academics to learn more about how to effectively use e-learning. While the importance of community and social interaction to learning is widely recognised, most professional development opportunities occur in isolation (Bransford, Brown et al. 2000). Conglomerations should aim to provide opportunities for academics to observe, question and discuss use of the technology. Examples from Webfuse are limited to the ability to observe. For example, all Webfuse course sites were, by default, open for all to see. The CourseHistory Wf application allowed staff to see the grade breakdown for all offerings of any course. A better example would have been if the CourseHistory application encouraged and enabled discussions about grade breakdowns.
  4. Ensure that conglomerations are context-sensitive.
    Effective integration with the specific institutional context enables conglomerations to leverage existing resources and reduce cognitive dissonance. For example, the Webfuse default course site conglomeration was integrated with a range of CQU specific systems, processes and resources. The Webfuse online assignment submission system evolved a number of CQU specific features that significantly increased perceptions of usefulness and ease-of-use (Behrens, Jamieson et al. 2005).


Behrens, S., Jamieson, K., Jones, D., & Cranston, M. (2005). Predicting system success using the Technology Acceptance Model: A case study. Paper presented at the Australasian Conference on Information Systems’2005, Sydney.

Bransford, J., Brown, A., & Cocking, R. (2000). How people learn: brain, mind, experience, and school. Washington, D.C.: National Academy Press.

Gregor, S., & Jones, D. (2007). The anatomy of a design theory. Journal of the Association for Information Systems, 8(5), 312-335.

Jones, D., & Luck, J. (2009). Blog Aggregation Management: Reducing the Aggravation of Managing Student Blogging. Paper presented at the World Conference on Education Multimedia, Hypermedia and Telecommunications 2009. from

Sommerville, I. (2001). Software Engineering (6th ed.): Addison-Wesley.

Portfolios often implemented poorly

I’m slowly reading through How people learn and thinking about how it applies to my own practice and what I see within higher education. This post arises from this paragraph

Portofolio assessments are another method of formative assessment. They provide a format for keeping records of students’ work as they progress throughout the year and, most importantly, for allowing students to discuss their achievements and difficulties with their teachers, parents and fellow students (e.g. Wiske, 1997; Wolf, 1988). They take time to implement and they are often implemented poorly – portfolios often become simply another place to store student work but no discussion of the work takes place – but used properly, they provide students and others with valuable information about their learning progress over time.

Technology to the rescue

So, non-technology based portfolio assessments are “often implemented poorly”. Well, the obvious solution to this poor implementation is to add an “e-” on the front. Technology, in the form of e-portfolio systems, will solve these poor implementation problems.

Yea, right.

On-going poor implementation through absence of contextual change

I’m a long-term e-portfolio skeptic. Not because I don’t like the idea. After all, there’s a lot of similarity between how BIM can be used an e-portfolio and increasing formative assessment to students has been a key focus of mine.

It’s also not because I think the e-portfolio systems are horrible. Though I have heard some rumblings from folk using them that there are problems.

It’s also not because Mike Bogle’s concerns about the institutional provision of what could/should be a student owned tool resonates with me.

I’m an e-portfolio skeptic because they are becoming just another fad for higher education being pushed as a great saviour without any real engagement with the culture and environment within higher education that will ensure that most applications of e-portfolios in higher education “are often implemented poorly”.

The case of BIM

The presentation I originally envisioned giving at MoodleMoot’AU 2010 was going to use the story of BIM to talk about the inherent limits on innovative pedagogy that exist within higher education. (The Moot seemed the wrong place for that talk so it became more a show and tell for BIM). But here’s just one example.

The original purpose for developing BIM was to increase the level of discussion and formative assessment between staff and students. I was sick of seeing student assignments with basic mistakes that could of and should of been picked up earlier. So, when I used BAM (the BIM pre-cursor) with my students I interacted heavily with them and it worked well.

Trouble was that I was only responsible for about 5% of the students in the course. The remaining 95% of students were the responsibility of staff at other campuses. Most of these staff were part-time, casual staff employed on contracts that were very specific about what they had to do. These contract did not include time for interacting with students via their blog posts, so the staff didn’t. Attempts to put in place changes to address this problem were not successful, in fact they couldn’t even be understood. They were to far out of accepted organisational practice.

And this isn’t limited to me. The academic that has used BIM/BAM the most, has faced exactly the same problems. Her express purpose for using BIM has been to make student thinking and learning more visible to enable more discussion and feedback, but she has also been hamstrung by the organisational practices that are set up for the traditional lecture/tutorial practice.

A research project

I’d love to do some research into institutional e-portfolio systems with the aim of determining the quantity and quality of feedback students have received on the items in their e-portfolio and compare that with traditional assessment methods. I wonder how much of a difference there would be?

My prediction is that there would be maybe 5-10% of courses using e-portfolios that have lots of feedback and discussion. With the remainder becoming simply “another place to store student work”.

I wonder if there’s been this sort of research done already?

How strict a blueprint do ISDTs provide?

Am working on the final ISDT for the thesis. An Information Systems Design Theory (ISDT) is a theory for design and action. It is meant to aim to provide general principles that help practitioners design information systems. Design theory provides guidance about how to build an artifact (process) and what the artifact should look like when built (product/design principles) (Walls, Widmeyer et al. 1992; Gregor 2002). Walls et al (1992) see an ISDT as an integrated set of prescriptions consisting of a particular class of user requirements (meta-requirements), a type of system solution with distinctive features (meta-design) and a set of effective development practices (meta-design). Each of these components of an ISDT can be informed by kernel theories, either academic or practitioner theory-in-use (Sarker and Lee 2002), that enable the formulation of empirically testable predictions relating the design theory to outcomes (Markus, Majchrzak et al. 2002).

My question

I’ve just about happy with the “ISDT for emergent university e-learning systems” that I’ve developed. A key feature of the ISDT is the “emergent” bit. This implies that the specific context within which the ISDT might be applied is going to heavily influence the final system. To some extent there is a chance that aspects of the ISDT should be ignored based on the nature of the specific context. Which brings me to my questions:

  1. How far can the ISDT go in saying, “ignore principle X” if it doesn’t make sense?
  2. How much of the ISDT has to be followed for the resulting system to be informed by the ISDT?
  3. If most of the ISDT is optional based on contextual factors, how much use is the ISDT?
  4. How much and what sort of specific guidance does an ISDT have to give to be useful and/or worthwhile?

Class of systems

One potential line of response to this is based on the “class of systems” idea. The original definition provided by Walls et al (1992) for the meta-design component indicates that it “Describes a class of artefacts hypothesized to meet the meta-requirements” and not a specific instantiation. van Aken (2004) suggests that rather than a specific prescription for a specific situation (an instantiation), the intent should be for a general prescription for a class of problems. van Aken (2004) arrives at this idea through the use of Bunge’s idea of a technological rule.

van Aken (2004) goes onto explain the role of the practitioner in the use of a technological rule/ISDT

Choosing the right solution concept and using it as a design exemplar to design a specific variant of it presumes considerable competence on the part of practitioners. They need a thorough understanding both of the rule and of the particulars of the specific case and they need the skills to translate the general into the specific. Much of the training of students in the design sciences is devoted to learning technological rules and to developing skills in their application. In medicine and engineering, technological rules are not developed for laymen, but for competent professionals.

This seems to offer some support for the idea that this problem, is not really a problem.


It appears that the idea of “emergent” is then just an increase in emphasis on context than is generally the case in practice. There is, I believe, a significant difference between emergent/agile development and traditional approaches, it’s probably worthwhile making the distinction in a mild way when introducing the ISDT and then reinforcing this in the artifact mutability and principles of implementation section.

The first stab

The following paragraph is a first draft of the last paragraph in the introduction to the ISDT. It starts alright, but I’m not sure I’ve really captured (or understand) what I’m trying to get at with this. Is it just an attempt to signpost perspectives included below? Need to be able to make this clearer I think.

It is widely accepted that an ISDT – or the related concept of technological rule – are not meant to describe a specific instantiation, but instead to provide a general prescription for a class of problems (Walls, Widmeyer et al. 1992; van Aken 2004). The ISDT presented here is intended to offer a prescription for e-learning information systems for universities. In addition to this general class of problems, the ISDT presented here also includes in its prescription specific advice – provided in the principles of implementation, and artifact mutability components of the ISDT – to be more somewhat more general again. This is captured in the use of the word “emergent” in the title of the ISDT and intended in the sense adopted by Truex et al (1999) where “organisational features are products of constant social negotiation and consensus building….never arriving but always in transition”. This suggests the possibility that aspects of this ISDT may also be subject to negotiation within specific social contexts and subsequently not always seen as relevant.


Gregor, S. (2002). Design Theory in Information Systems. Australian Journal of Information Systems, 14-22.

Markus, M. L., Majchrzak, A., & Gasser, L. (2002). A Design Theory for Systems that Support Emergent Knowledge Processes. MIS Quarterly, 26(3), 179-212.

van Aken, J. (2004). Management research based on the paradigm of the design sciences: The quest for field-tested and grounded technological rules. Journal of Management Studies, 41(2), 219-246.

Walls, J., Widmeyer, G., & El Sawy, O. A. (1992). Building an Information System Design Theory for Vigilant EIS. Information Systems Research, 3(1), 36-58.

Back into the thesis

After a week away traveling with my sons in Western Queensland, I have to get back into the thesis. This post is essentially an attempt to capture where I’m up to and plan out what I need to do for the next week or so. As with the last update, I’ll start scratching these out as I complete.

The outstanding work can essentially be broken into two parts:

  1. Make modifications to existing chapters; and
    The last PhD update has a good summary of much of this work.
  2. Complete the last chapter.
    Chapter 5 is the last chapter not yet in first draft stage, and it’s not far from it with only two sections not really there yet. So, the focus has to be on finalising those: the ISDT and Evaluation.


This is the major product of the thesis. Much of it is written, what’s remaining to doincludes:

  • Clean up the intro section to use the summary table.
  • Develop a section outlining that the ISDT is more a guide than a hard blue print – what other literature is there on this?
  • Define the purpose of constructs and double check the list in the table
  • Principles of form and function
    • Get a few broader definitions of architecture, especially leaning towards the inclusive idea.
    • Explain the scaffolding idea a bit more, perhaps borrow from this post.
    • Better explanation of human support as an aspect of scaffolding.
  • Principles of implemenation
    • Work in a connection with learning.
  • Artifact mutability
    • Work in the ideas proposed by Hovorka and Germonprez (2009) around artifact mutability.
    • Connect it with learning for academics, students and supporters.
  • Justificatory knowledge
    • Expand support for scaffolding conglomerations.
    • Problems and references for best of breed.
    • MOre work on principles of implementation – including learning
    • distill some of the references and double check outstanding references.
    • Cioborra and related work around the move away from the teleological and analytical.
  • Put more on BAM/BIM into the expository instantiation section.
  • Link the summary table of Webfuse testing with the results of the evaluation.


  • Finalise table 5.6.
  • Decide on which time frame to limit evaluation to and argue why.
    Use all of Webfuse post 2000. Rationale: While Webfuse did not at all times follow the ISDT, it did use aspects of it and even with just aspects showed some benefits. If all had been followed it may have been better again.
  • Table 5.14
    • Calculate number of students in courses with webfuse sites
      • Fix up 1998 entries in stats::courses to put in right term and also add 3rd term.1997, 1999, 2000, 2001 counts.
    • Re-examine course site hits, page updates, unique authors from 2002 onwards?
  • Table 5.15
    • Add standard deviation for updates per staff – based on updates from Table 5.14
    • Update accesses per enrolled student – based on data from Table 5.14
  • Complete the table “usage of webfuse courses”
  • Do power laws get a mention? — NO
  • % of BAM requests of overall staff requests for 2009.
  • Rethink the adoption section and draw on summaries of previous sections and also comparisons with similar section from chapter 4. Also incorporate work from the TAM papers.
  • Modify Figure 5.15 so that it uses a year where data is not missing.

Misc to dos

In working on the above I’ll almost certainly identify additional tasks that need to be performed, they are captured below.

  • Add a summary table of the ISDT in chapter 4.
  • Come up with a title for the chapter 4 ISDT.


Hovorka, D., & Germonprez, M. (2009). Tinkering, tailoring and bricolage: Implications for theories of design. Paper presented at the AMCIS 2009.

More problems with BIM and special characters

The following is a record of some work to investigate some more apparent problem with BIM mirroring blog posts that contain “special” characters due to a bit of copying and pasting from Word into WordPress.

An aside on supporting a tool like BIM

All of my previous software support has been around software that I (or the team I work for) have developed and for which we are also responsible for hosting and supporting the people using it. i.e. we knew exactly what was going on where.

With BIM the development, support and hosting are all separated. I write BIM, but I don’t host of support it. Which means I don’t know what the folk who are doing the hosting/supporting have done (or haven’t done). Which adds all sorts of complexity to the process.

This sort of separation is increasingly common and is often aimed at saving resources/money. But I do wonder whether or not that if viewed from an overall perspective it is adding more cost (in the broadest possible sense i.e. including the hassle and inefficiency caused by the difficulty but which doesn’t typically show up in anyone’s budget bottom line) into the whole task of supporting systems.

The problem

There are at least two, almost invariably related, problems:

  • Special characters that aren’t being translated safely.
  • Some situations where Moodle/BIM on Postgres is not able to insert student posts.

The postgres problem

There appears to a problem with Postgres/Moodle/PHP/BIM falling over when attempting to insert some posts. Maybe because of special characters. The only way I have of testing this at the moment is black box, i.e. re-creating it on a Postgres-based Moodle install. First step is to identify where this is happening and see if something can be done to make it not so catastrophic a failure.

Fail all posts, if one post fails

The error I’m seeing is

bim_process_feed: inserting bim_marking **url here**

Ok, that seems to match this bit of code

[sourcecode lang=”php”]
if ( ! insert_record( "bim_marking", $safe ) ) {
mtrace( get_string( ‘bim_process_feed_error’, ‘bim’, $entry->link ) );
return false;

Ok, the first thing here is that the “return false” should go. This breaks out of the whole insert process. What should happen is simply move onto the next entry and try to insert that.

Ok, that’s updated and into github.

Why are posts/inserts failing

The above was causing a problem because Postgres would report a failure when blog posts with certain characters were being inserted. The same posts are not causing a problem with MySQL.

Is this a known problem with Postgres? Are there solutions from the Postgres community that might help out here?

Doesn’t seem to be too much via Google, at least with what I was searching for. Guess I turn to exploring more the special characters in the posts that are causing problems.

More special character problems

The process here is to repeat what I did last time, modify my version of BIM to be somewhat explicit about the characters in posts it is inserting, find out which aren’t being handled, and then add some code to handle them.

The list of potentially problematic characters are:

  • a bullet point
    Turns out that this is a middot (search for 183. So, I’ve added a translation for this. This works, however, it does highlight a problem with “bad handling” described in the next section.

Bad handling

At least on MySQL, some of the changes seem to be introducing some rather weird translations. For example, a middot and some white space is translated into similar looking characters but with each surrounded by a single quote. This needs to be fixed.

Ahh, some code I lifted is using ereg_replace, which according to this is not multi-byte safe. Replace it with mb_ereg_replace, and all is good.

This seems to be the major problem.

Will see how it goes when testing with other problem data.

Light-weight analytics tools as part of scaffolding, context-sensitive conglomerations

A couple of days ago I floated the idea of scaffolding, context-sensitive conglomerations as one idea/model/suggestion for how e-learning systems (currently mostly LMS, but hopefully other models will arise).

George Siemens has posted about light-weight analytics tools such as SNAPP. Both the comments on that post are, to my current somewhat focused/biased perspective, suggestions for the need for scaffolding conglomerations. Both comments are from practitioners who talk about how the supplement their use of discussion forums with other forms of representation. It would appear obvious that these combinations of tools are useful. I’m pretty sure you could find quite a few talented and motivated academics across the world that are using this combination. I’m also pretty sure that few of them would be located within the same institution.

Are there any discussion forum tools in e-learning systems that already provide this sort of scaffolding for users? Are there any IT departments in universities that have recognised this need and are helping academics make this connection?

I’m not aware of any, and this suggests to me that there are some fundamental problems with the way these systems are being supported and structured. i.e. current approaches mean it is unlikely for these sorts of networks of tools/conglomerations to arise.

SNAPP has used a good approach that makes it simpler to create these conglomerations, through the use of browser plugins. But the advantages of that approach come with a negative. i.e. I don’t believe you can currently generate a SNAPP visualisation for a group of courses (e.g. to see how the students/staff in a program are interacting), or compare visualisations between different courses. You also can’t easily combine SNAPP with other context specific data sources such as student records system etc.

Consequently, SNAPP is a great example of a tool that enables a “scaffolding conglomeration” when combined with an LMS discussion forum. But it still remains difficult to add the “context-sensitive” component.

I think there’s value in exploring how SNAPP and similar tools can be used as both scaffolding and context-sensitive conglomerations, and more importantly, what impact it can have on the practice of teachers, and subsequently the quality of learning.

Adding advice

While I remember, there’s a next step that I’d like to see a “scaffolding, context-sensitive conglomeration” take. Advice, examples and connections.

For example, assume I’m teaching a course, I use a discussion forum and I’ve designed its use for a specific pedagogic purpose. I’ve installed SNAPP and to my horror discover a problem. What do I do? What strategies can I employ to address this problem? What strategies have other academics in similar (or even different situations) used? What happened? Can I get their contact details so I can have a chat?

I would imagine that this addition could also be implemented for students who discovered a “bad” pattern in their own practice. They could receive advice, examples and connections from other students.

The learning analytics group seems to include a leaning towards “intelligent”/”adaptive” software to provide this sort of service. I’m more interested in how we can use these tools to connect people and provide the scaffolding that enables and encourages them to take some action.

Misc. reflections on reading about situated cognition

For various reasons, mostly PhD related (and somewhat related to procrastination), I’m taking the time to read a bit more about situated cognition. Not sure how far it will go. The following are some ad hoc reflections and essentially a diary of what I’m reading. Not aiming for this post to fulfil any purpose beyond being a place to dump observations.

The wikipedia page

So far, the Wikipedia page on situated cognition seems fairly extensive and a reasonable place to start.

Misapplication of community of practice?

The wikipedia page has the following definition of community of practice

The concept of a **community of practice** (often abbreviated as CoP) refers to the process of social learning that occurs and shared sociocultural practices that emerge and evolve when people who have common goals interact as they strive towards those goals.

I find this somewhat interesting in that my experience with CoPs around university learning and teaching has been with special groups set up for specific purposes above and beyond normal teaching. i.e. rather than have a CoP around teaching at university X, where the common purpose is to teach. A CoP is set up around attrition, graduate attributes etc and focuses on that as the goal, rather than the teaching.

I do wonder whether this on-going ad hoc creation of CoPs around special topics that are important, but haven’t been embedded into common institutional practice is a symptom of CoP misuse. i.e. if normal practice of teaching within an institution was more like a CoP, would you really need a separate CoP on retention etc? Does the need for separate CoPs indicate that the normal practice of teaching within an institution isn’t like a CoP and hence, perhaps there isn’t a sense of a common purpose amongst those involved in the process of normal teaching? Instead of a common purpose, do the actors within an institution’s normal practice of teaching and learning have their own different purposes?

The glossary from which this definition comes highlights the amount of though and subsequently special language that has arisen around situated cognition. Also has some interesting resonance with the need for design theories to have a constructs section.


Interesting to see some of the origins of the concept of affordances beyond Norman and HCI/usability. The idea that affordances are the individual’s interpretations about what action is possible within the given environment through their perception of the environment connects strongly with some of the problems I have around the stereotypical university environment around teaching and the nature of e-learning systems. i.e. I think the affordances seen by many teaching staff aren’t good in terms of improving learning and teaching.

The relationship between affordances and schemata seems an area of some disquiet and more reading – Glenberg and Robertson (1999)


This section was a little disappointing, it only mentions visual perception. While this is apparently an importent influence on situated cognition, even from my limited reading and knowledge there appears to have been a lot more work done on perception.


Again I feel there could be more here. But it does pose the interesting question of how situated cognition is downplaying the importance of stored, symbolic representations in memory. Instead having a belief that perception and action are co-determined by effectivities and affordances… Raises the question about what is
learning and knowing (which are covered next). Also links back to the disquiet about the link between affordances and schemata.

Knowing and learning

So knowing is not a think, a memory, but a verb, it is action/participation of an agent in an environment. This is where the idea that knowledge cannot be separate from context. It gives rise to the importance of context.

This is interesting, challenging and somewhat comforting. It is comforting because to some aspect it represents an idea embedded in how Webfuse worked. For Webfuse, context was important. Webfuse wasn’t a general purpose tool that could be used elsewhere, Webfuse could not be separate from its context. Mmm, situated cognition as a kernel theory for the ISDT and Webfuse?

It does seem that some of these sections have been made specifically very narrow and focussing on language/literacy learning.

Pedagogical implications

So, situated cognition is the theory of “mind/knowing/learning etc”, while cognitive apprenticeship etc are instructional design theories drawing on that theory, it appears.


A small section on critiques close off the page. On the face of it, the critiques do a reasonable job of removing many of the assumptions on which situated cognition is based. Need to have a look further into Anderson et al (2000).

Of course the criticisms arise from cognitivists – more information processing types – who hold a perspective that has also been challenged. Interesting that it appears that Herbert Simon is one of the critiques (last author on Anderson et al).


Anderson, J. R., Greeno, J. G., Reder, L. M., & Simon, H. A. (2000). Perspectives on learning, thinking, and activity. Educational Researcher, 29, 11-13.

Scaffolding, context-sensitive conglomerations in e-learning systems

For the last week or so I’ve been attempting to bring together the principles that underpin the design theory for e-learning that will be the main contribution of my thesis. This post summarises one of the principles of form and function that is emerging from the last week or so. I using this blog post to escape the confines of PhD-ese and see if writing about it can generate some further refinement.

Am aiming for brevity and clarity in the following.

A design theory contains principles for form and function; and for implementation. The following is more form and function, while the previous post is more implementation. There is more to the ISDT than just these two posts.

The problem

The functionality provided by most e-learning systems are in the form of low-level primitive tasks (e.g. upload a document, add a discussion forum, create a quiz). Most e-learning systems provide a fairly large collection of these low-level features. Many of these features, because they are designed for the general case, come with large numbers of options and configuration settings so that the task can be tweaked to the broadest collection of uses.

Consequently, achieving a high-level task (e.g. supporting a discussion forum that encourages student-to-student collaboration, or creating an educationally well-designed course website) requires a significant amount of knowledge, skills and time on the part of teaching academics. Some (perhaps much) of the time spent by teachers in leveraging these low-level, very general features is expended on bridging the gap between the generality of the feature set and the specifics of the context. Much of the time is spent trying to translate and wrangle what is known about good practice into the specifics required to implement it with the low-level feature set of the e-learning system.

I suggest that these difficulties limit the quality and quantity of how these feature sets are used. These difficulties appear to be one of the factors that contribute to the on-going fairly poor quality of most e-learning within formal education settings, especially universities.

Scaffolding, context-sensitive conglomerations as a solution?

The idea is that an e-learning system should provide the ability to create “scaffolding, context-sensitive, conglomerations” of its low-level feature set.


An example of this is the default course site “conglomeration” used in the Webfuse system. This is the system I designed and forms the basis for my PhD. So you can see where the idea has come from. The default course sites conglomeration was aimed at making it very simple for academics to create a default course site. So simple, in fact, that if they wanted to, they didn’t have to do anything.

Another possible example of a “conglomeration” from my own work could be the Moodle module BIM. BIM is a conglomeration of external blog engines (e.g. and various Moodle features, especially the gradebook. The BIM “conglomeration” is designed to make it simpler for teachers to manage students using individual blogs for assessment and other purposes. It provides a bridge between the external blogs and Moodle’s making/management features.

Moving further afield, the work being done by Desire2Learn with its instructional design wizard provide another approach to the provision of a conglomeration. The “Wizard” approach is used.


In this context, a conglomeration is essentially a way to group together the existing low-level functionality provided by the e-learning system into something at a much higher level of abstraction. The term conglomeration has been chosen specifically to avoid terms like component, framework, package etc that, at least with some technical systems, refer to fairly specific ways of combining technology features. I’m trying to avoid these terms for a range of reasons, including:

  • Non-specificity;
    The aim of the ISDT (information systems design theory) is to provide broad guidance that is not specific to a particular approach or technology.
  • Inclusivity;
    Most of the technical approaches to grouping functional (e.g. component-based architectures) are not inclusive. i.e. if you’re using architecture X, you can only group features that are implemented using architecture X. While recognising there’s always going to be a practical need for something like this, with my ISDT I am trying to argue that for e-learning systems, the conglomeration mechanism needs to be as inclusive as possible (e.g. BIM’s reliance on RSS/Atom feeds – loose coupling – rather than a specific API for a particular blog service).
  • User focus.
    As part of all this the conglomerations only have to group these features from the perspective of the user. It doesn’t actually mean that they are technically grouped through the use of the same architecture. What’s important is that the user experience with the conglomeration leads them to believe the different bits of low-level functionality are integrated and working together.


Context-sensitive implies that the conglomerations know something about and offer support for factors or knowledge that is specific to the context of the people using the conglomeration. i.e. it’s not such a general purpose tool as to require the user to bridge the gap between their context and the tool.

For example, the default course site approach within Webfuse was designed to know about and integrate with as much of the local context as possible. The course synopsis, details about the assessment, staff details etc were automatically drawn from institutional databases and available within the conglomeration. In a more able and educationally enlightened context, the default course site conglomeration might have known about and offered specific support for the use of institutional graduate attributes or course learning outcomes.


This aspect is still a work in progress and builds somewhat on the context-sensitive attributed. The term scaffolding was chosen because of the connection between this idea and situated/distributed cognition, cognitive apprenticeship etc. The idea is that the e-learning system should be designed in a way to help the teachers (and perhaps students) learn about using the system effectively.

The assumption is that for most teachers using the e-learning system to improve and/or change their teaching practice is a learning process. They haven’t done this before. Rather than treat learning about how to make this change a separate professional development activity, the e-learning embodies/embeds the learning into its conglomerations. In some way, the system adopts an approach that includes situated modelling, coaching and fading.

Collins et al (1991) define it this way

When scaffolding is provided by a teacher, it involves the teacher in executing parts of the task that the student cannot yet manage. A requisite to such scaffolding is accurate diagnosis of the student’s current skill level or difficulty and the availability of an intermediate step at the appropriate level of difficulty in carrying out the target activity. Fading involves the gradual removal of supports until students are on their own.

Am still considering the implications of this addition and how far to take it/interpret it. Potentially this could suggest that the system have a model of the teachers’ abilities and be “intelligent”. I don’t see this as being a requirement. An equally plausible, and probably more likely, approach would be for this diagnosis to be performed by people.

Need to think this through some more and see how much more of the insights offered by the learning theories this idea is based on could/should be used.


Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American Educator(Winter), 6-11, 38-46.

Powered by WordPress & Theme by Anders Norén