Emergent Development and the Virtual University

David Jones, Emergent Development and the Virtual University, Paper presented at Learning’2000, Roanoke, Virginia

Abstract

At some level the development of a Virtual University is a large-scale project involving change and information technology development. What the literature says about the chances of such projects is fairly consistent: 75% of all large systems may be considered operating failures (Laudon and Laudon, 1996). Blame for this apparent failure has been allocated to the development methodologies, the participants, the tools and their builders, and organizations in which they are implemented. Emergent development (Truex, Baskerville and Klein, 1999) suggests that the problem is with the mismatch between the aims of traditional information systems development and the requirements of modern organisations. This paper suggests that an organisation that must cope with continual change, such as a virtual university, requires a development methodology based on completely different assumptions. One such methodology, emergent development, is described.

Introduction

Since the number of Web-based courses appears to doubling every 11 months (Robson, 1999) it seems that the question of whether or not to use online technologies to support teaching and learning has been answered. This leaves many individuals and organisations, both old and new, having to decide how they will implement and support online technologies and the resulting "Virtual University". This paper ignores the questions of why and if online learning should be used and concentrates on how it might be supported. In particular, it draws on the idea of emergent development to suggest that many existing approaches to this problem are based on incorrect assumptions that may contribute to long-term problems.

The paper begins by offering an explanation of the significance of the "how" question before providing an overview of the common, existing approaches used to implement online teaching and learning. The paper then proposes that many of the problems that these approaches can generate are related to the problems caused by traditional information systems development. Based on the assumption that an existing University attempting to become a virtual university is an emergent organisation it is suggested that a development methodology with different assumptions than traditional approaches is required. The paper offers a description of emergent organisations, of emergent information systems development and of a variety of approaches, both technical and non-technical, which may be used to support the emergent development of a virtual university.

Significance of the "how" question

Is the question of how to implement and support online teaching and learning within the virtual university worthy of further examination? After all many institutions have for a number of years been using online teaching and learning, either as a supplementary or primary learning medium. The following section suggests that the "how" question is important due to a number of factors including:

• The limited quality of much current online teaching and learning.
• The limited widespread adoption of innovative approaches to teaching and learning.
• A limited perspective on the possibilities of information technologies.
• The likelihood that large-scale information systems development projects will fail.
• The ability for infrastructure to limit future possibilities.

Surveys of websites for teaching and learning (LaRose & Whitten, 1999; Mioduser, Nachmias, Oren & Lahav, 1999) show that the majority of sites designed to support online teaching and learning (OT&L) make less than effective use of the medium. In particular, one survey by Mioduser, Nachmias, Oren & Lahav (1999) shows that most current educational websites are not making use of the pedagogical approaches favoured by educational researchers. Many, if not most, applications of online learning continue to reuse teaching strategies from traditional forms of teaching and learning (Jones, Stewart and Power, 1999).

In part this can be described as a transition period during which practitioners’ replicate known models by means of the new technology (Mioduser, Nachmias, Oren and Lahav, 1999). It can also be said that the existing infrastructure and policies, e.g. workload policies, equipment supply and buildings, within Universities encourage staff to think in terms of traditional teaching and learning strategies. The existing infrastructure and policies are actually restricting future innovation. Most of the innovation in OT&L is the result of the efforts of individual staff members, the lone rangers, often in spite of a lack of institutional interest and support (Taylor, 1998).

An infrastructure that gets it wrong can fore close the possibilities that it was supposed to open up (Agre 99). This can be seen where existing infrastructures encourage the repetition of previous experience. Additionally, the new infrastructures and policies institutions develop to support OT&L will likely create new environments that may restrict future innovation. Information systems infrastructure that is flexible and adaptable can be powerful enablers of innovation, but rigid, inflexible systems are serious obstacles to organizational effectiveness and success (De Michelis et al, 1998).

It can be demonstrated that the use of new technologies can provide benefits by improving the efficiency of existing practice (McCormack and Jones, 1997). However, Sproull and Kiesler (1991) argue that the most significant effects of new technologies come from enabling previously impossible practices. A possible problem with achieving this is that management, the people often responsible for creating the environment in which an organization operates, tends to concentrate on the efficiency effects of new technology (Sproull and Kiesler, 1991; Hirschiem and Lacity, 1999). For example, the following quote is taken from a review of higher education initiated by the Australian Government

The consensus was that the impact of information technology on the back office will be more important than its impact on the student teacher interface, as information technology offers the potential for large savings and the opportunity to prune internal bureaucracies. (CRHEFP, 1997)

The adoption of communication and information technologies at Universities is often badly done and based on ignorant optimisms (Taylor 1998). There is a long history of failed technology-based innovations in education (Reeves, 1999). The full possibilities of a new technology are difficult to predict (Sproull and Kiesler, 1991) and plan for. Subsequently, many such projects fail due to the innovators underestimating the consequences of new technologies (Sproull and Kiesler, 1991) and failing to accommodate environmental and contextual factors affecting implementation (Jonassen, 1998). The unique interaction between the technology, the organisation, its policies, infrastructure, individuals and context all contribute to a continual feedback loop which generates new contexts and outcomes. The influence of a system does not stop with its construction and initial use. It is up to the educators and other participants within the organization to affect the technology design and educational policy that in the end influence the outcomes (Luck, 1999).

This is not to suggest that interesting and effective innovations in OT&L do not occur. On the contrary, every year the OT&L literature consists of hundreds, if not thousands, of good ideas associated with online learning (Jones and Stewart, 1999). However, there is a failure to institutionalize the outcomes (Gibbs, 1996) within their home institution and, even less likely, widespread adoption amongst other institutions. Collis and Oliver (1999) report in their analysis of the papers submitted to EdMedia’99 that the majority of papers report on prototype development and evaluation with few ideas going beyond this stage.

The implementation of online teaching and learning in an existing University is a process of involving change and information technology. According to the literature the success rates of such projects is not good. A study in the United Kingdom (OASIG, 1996) found that around 40% of information technology systems developments failed or were abandoned. Johnson (1995) surveying 3,682 such projects in 365 companies found that: 31% were canceled before completion, 53% had cost overruns and had impaired functionality, and only 12% were on-time and on-budget.

Addressing the "how to implement" question with any technology is often driven by limited, naive understandings of the implications and a concentration on achieving efficiency improvements. The possible social and organizational changes that emerge from the interaction between the technology, participants, context and organisation are often ignored. Subsequently the successful implementation and innovative use of OT&L is often less than effective and limited to repeating previous practice.

Common approaches

The development of online teaching and learning at many Universities has two branches: the lone ranger and institutional. Lone rangers are energetic, early adopters who implement online teaching and learning with the main aim of improving the accessibility and quality of their own teaching (Taylor 1998). They are often the first to make use of OT&L and the source of most of the innovation (Taylor 1998). At some stage there is institutional recognition of the need for OT&L that results in an organization-level program.

The lone ranger

Initially online teaching and learning was (is) such a different medium that no existing organizational unit had demonstrable ownership of, or responsibility for supporting it. Consequently, lone rangers perform the majority of early innovation. Many of these lone rangers operate with little or no institutional interest, in some cases these individuals have to battle the constraints and self-interest of existing institutional infrastructures and fiefdoms.

The lone ranger approach to online teaching and learning provides innovations that directly meet the unique characteristics of the individual class, teacher and students. The lone ranger has direct control over the innovation further reinforcing the principle of professional autonomy (Taylor, 1998). It is the innovators that lay the foundation for new teaching methods based on application of communication and information technologies (Taylor, 1998).

Implementation costs of the lone ranger approach are relatively cheap for the organisation but can be quite large for the individual, especially in terms of time. The lone ranger approach to online teaching and learning has a number of other possible problems:

• Variable quality and features;
  The quality and type of innovation depends heavily on the desires, knowledge, preferences and capabilities of the individual lone ranger. Different innovators will produce different levels of quality.
• Toys for the "boys";
  In some cases the lone ranger’s drive to innovate can be driven more by the desire to play with the new technology than improving educational quality.
• Limited organizational support and appropriation of the innovations;
  Lone ranging innovation often occurs in spite of a lack of institutional support and interest (Taylor, 1998). Which often means that the innovations do not find widespread use within an institution and often die when the lone ranger leaves the organisation.
• Repetition of common mistakes;
  Lone-ranger innovations often occur in isolation and consequently the different lone rangers discover the same problems and repeat the common mistakes. Additionally the lack of institutional appropriation of lone-ranger innovations often means that initial institutional approaches often make the same mistakes.
• Duplication of infrastructure;
  Due to the isolation and institutional indifference lone rangers often have to duplicate infrastructure such as web servers or user authentication.
• The "run over by a bus" problem;
  Heavy reliance on the individual lone ranger means that the absence of the lone ranger usually results in the failure of the innovation.
• Limited scalability.
  The lone-ranger approach to OT&L does not scale to an organizational level since very few staff can demonstrate the unique characteristics and desires of a lone ranger.

The settling of the Wild West

At some stage an organisation will realize the importance of OT&L and implement some sort of institutional approach to online teaching and learning. Institutional approaches to OT&L commonly fall into one of three approaches:

1. Buy a package.
   One of the many available learning management systems (e.g. WebCT, Blackboard etc) is chosen and the necessary infrastructure is setup to handle the conversion of existing practice to its use.
2. Build a package.
   Some organisations choose to leverage existing organizational experience with OT&L, often in the form of a successful lone ranger, and build an institutional specific package. Some of the existing learning management systems (e.g. WebCT and Prometheus) grew out of this situation.
3. Out source it.
   The development and production of OT&L is out sourced to an external organisation.

Most commercial Web-based learning management systems offer support for more or less the same pedagogy (Robson 1999). There are a number of websites that compare the functionality of common learning management systems (e.g. http://multimedia.marshall.edu/cit/webct/compare/comparison.html). Examining these comparisons reveal widespread commonality in the features provided by these systems. Learning management systems provide many common features but none support student critical thinking, generation of knowledge and collaborative teamwork (Bonk and Dennen, 1999).

The incredible variation of the needs, requirements, and tastes of students and teachers means that there is no one correct method for implementing a Web-based course (McCormack and Jones, 1997). The ability for teachers to customize an online course to identify their character, personality and teaching commitment can be vital to self-esteem and commitment (Brown 1999). An online course and the learning management system that enables it may need to handle change in response to changes in technology, significant differences across disciplines (Brown 1999), cultural, pedagogical and organizational issues.

The capabilities of any individual package, purchased or built, will be constrained by the initial analysis and specification produced by the creators. Reusing a package outside the existing constraints of the package is difficult. These constraints can include the computer platform supported by the package, the features, the supported pedagogies and the data model. However, the centralization and standardization enabled by these packages does help reduce costs, provide consistent quality, simplify support and enable scaling to an institutional level.

An organisation building its own package must be prepared to make the initial large investment as well as face problems associated with divergent convergence (Jones, Stewart and Power, 1999). The Internet and computer revolution has contributed to the convergence of media into a digital form. However, the historic development of organizational support divisions (e.g. computer support, video and audio production, multimedia development and support for distance education) within many institutions has led to the development of competing "empires" (Jones, Stewart and Power, 1999). The development of any institutional system for OT&L will have to address the divergent agendas of these competing empires. This is in addition to having to address the same problems as those associated with purchased systems such as WebCT etc.

Outsourcing is most likely to occur in companies with poor financial situations, poor IT functions, or IT functions with little status (Lacity and Hirschheim, 1993). The hope that outsourcing will provide the needed solution is often driven by the frustration caused by the differing stakeholder expectations and perceptions of IT performance (Hirschheim and Lacity, 1999). Senior management often perceive the infrastructure as a cost to be minimized while the users see it as a service to be customized for their idiosyncratic requirements (Hirschheim and Lacity, 1999). The internal organizational unit responsible for implementing the service is caught in the middle since the best practices associated with reducing cost are in direct conflict with the best practices necessary to maximize service levels.

While outsourcing can lead to cost reduction it is often at the cost of reduced service. Most cost savings arising out of out-sourcing come from following known cost reduction strategies (Hirschheim and Lacity, 1999). These strategies could be implemented internally without the need to pay for the profit which out-sourcing companies need to make. However, the absence of local skills and, often more importantly, internal politics may prevent local implementation.

Outsourcing does offer an approach that scales well to the institutional level usually via the use of a consistent, standard approach that can be easily repeated. This is at the cost of customization and compatibility with existing approaches. Since any development work must occur via the out-sourcing partner they become the bottleneck for development and evolution.

Information Systems Development and Emergent Organisations

The common institutional approaches to OT&L (buy it, build it or outsource it) generally demonstrate the characteristics of traditional information systems development projects. Such projects generally follow a common process with steps similar to the following

1. An analysis and design period intended to identify abstract requirements and generate a formal specification of the required system.
2. A period of construction where the abstract requirements are implemented as an information system. In some cases the construction process is replaced with the purchase of a commercial system followed by modifications to fit local conditions.
3. A sign-off stage where the capabilities of the implemented/purchased system are ticked off against the formal specification.
4. A long period of low-cost, widespread operation and maintenance where the cost of the previous stages is recouped. There may be some low-cost modifications but in some cases this may not be possible.
5. At some stage the features of a system no longer match or support the needs of the organisation at which stage there is a return to step 1.

Traditional information systems development methodologies are particularly ill suited to emergent organisations. In an emergent organisation all features of the organisation are continually undergoing social negotiation and consensus building and are never fully formed (Truex, et al, 1999). The rapid development of technology and global markets contributes to a need for constant change where organisations are no longer stable and must continuously adapt to their shifting environments (Truex et al, 1999).

Universities, traditional, virtual, and those in-between, increasingly find themselves in an ever-changing environment where they must deal with change.

These broad developments, and the ultimate uncertainty of the future, underline the importance of a policy framework which gives as much flexibility a possible to universities and students, and highlights the importance of building institutions that are responsive to change. (CRHEFP, 1997)

You are probably sick of me talking about the constancy of change. Well, I am not going to talk about that anymore – I am simply going to assume it. (Marshall, 2000)

Traditional development projects are hard to manage and even harder to change making it difficult to align them with changing organizational reality (De Michelis, et al, 1998). Having low-maintenance, stable systems means the organization is continuously battling against its constraining information systems (Truex et al, 1998).

If an organisation and its processes are stable then precisely designed systems can recoup costs via long-term use. However, if an organisation is operating in a continually changing context the large investment in up front analysis is a poor investment as requirements change before the end of the analysis stage (Truex et al, 1998). Additionally the abstract requirements for such systems are often little more than a history lesson in past organizational states or abstractions of obscure guesswork about future organizational states (Truex et al, 1998).

Due to the mismatch between emergent organisations and traditional information systems development methodologies, emergent development rejects the goals of traditional information systems development and replaces them with new goals. The new goals of emergent development include (Truex et al, 1998):

• Continual analysis;
  Since an emergent organization is always changing it is necessary to be continually analyzing the organization and its requirements. The results of this analysis can then be continually fed into the on-going maintenance and development activities within the organisation.
• Dynamic requirements negotiation;
  In an emergent organisation the users will always be in conflict with the capabilities of an information system. Requirements can no longer be pre-determined. Instead they become a negotiated outcome of the on-going change of the organisation.
• Useful, incomplete specifications;
  Complete and unambiguous specifications are only possible for stable organisations (Truex et al, 1998). Achieving a complete set of unambiguous specifications for an information system in an emergent organisation is difficult. Systems maintenance and modification must be based on usefully incomplete specifications and those specifications must be easily adaptable.
• Continuous redevelopment;
  Emergent development aims to ensure the on-going presence of existing application through continuous modification and enhancement. Truex et al (1998) use the railroad system as an example of a system that has undergone continuous redevelopment and enhancement to adapt to changing conditions, requirements and capabilities. There has never been wholesale replacement in the railroad system.
• The ability to adapt.
  In emergent information systems development the information systems infrastructure must undergo continuous redevelopment. In order to support this continuous redevelopment the information systems infrastructure, development approaches and organisation must be able to be easily adapted to dynamic requirements.

An emergent organisation is continuously re-negotiating how, why and what it does. As a result traditional information systems development approaches, with the emphasis on developing systems with long period of stable use and maintenance, are a bad fit. Emergent information systems development recognizes and aims to support and encourage the on-going change of an emergent organisation by revoking the goals of traditional information systems development and planning for continual adaptation.

Implementing Emergent Development

Implementing and supporting the development of an emergent organisation, like a virtual university, should itself be an emergent concern designed to match the continually changing context of the organisation. The approaches used should influence and be influenced by the emerging nature of the particular organisation. What makes sense now may not make sense in the future. What makes sense for one organisation may not make sense for another.

Since this makes it very difficult to offer a prescribed list of approaches this section instead offers a collection of approaches that may be useful in implementing the emergent development of a virtual university depending on the local context. The collection is divided into non-technical and technical approaches. It is suggested that implementing the non-technical approaches will be considerably more difficult than the technical approaches.

Non-technical approaches

The reality of any social organisation is, to all intents and purposes, a product of what the people in an organisation believe to be the case (Truex et al, 1998). When multiple versions of reality exist in an organisation the result is conflict. It is the discussion of and around this conflict that generates organizational emergence. Increasing participation increases the conflicting views of the organisation that, if handled well, can increase the emergent nature of the organisation.

In the case of implementing new teaching technologies within existing Universities staff engagement is the most complex and important success factor (Collis, 1998). Communication channels that encourage and allow participants to establish versions of organizational identity that conflict with other versions are important for emergence (Truex et al, 1998). These channels should include all participants, including the information systems development group, and not be restricted to "upper management".

An organizational culture that provides appropriate rewards is another success factor when implementing change in teaching and learning (Collis, 1998). Emergent development requires a shift of values particularly for information systems staff. Such a shift should be recognized in with a rewards structure that encourages innovation and continual change. Leaving behind conventional approaches is often indistinguishable from error and in many cases will lead to problems. It is possible to think of evolution as systematic error management (Kelly, 1994). An emergent organisation should use errors to encourage on-going emergence. It should ensure that "bad" errors are not repeated and reward "good" errors.

A homogeneous organisation will require large revolutions to adapt to change while a heterogeneous organisation can often adapt via small revolutions drawing on the strengths of existing organizational members. A healthy collection of lone rangers operating on the fringe of a virtual university will generally be the source of innovations (Taylor, 1998). An organisation that promotes and learns from these lone rangers will be more likely to be able to adapt to change.

One approach to harnessing the experience and expertise of the lone rangers and making it available to the rest of the organisation are patterns and pattern mining. Patterns, simple and elegant explanations which capture solutions that have developed and evolved over time, are abstractions being used to increase reuse and quality in a variety of fields including architecture, software engineering, hypermedia, and teaching/learning (Jones and Stewart, 1999). A development process using patterns in OT&L offers a bottom-up development process that places the emphasis on using and sharing the experience and expertise of practitioners. The patterning process also enables skill and design evolution, increases reuse of ideas and appears to address a number of the other issues associated with the development of OT&L (Jones, Stewart and Power, 1999).

In emergent systems wide and fast moving problems will route around any central authority (Kelly, 1994). In the context of rapid change only distributed control at the lowest levels will be effective. In traditional teaching institutions the key decisions about the infrastructure and implementation of teaching and learning can be the responsibility of key administrators with limited recent classroom practice. Recent changes in the context and practice of teaching at many institutions can lead to these administrators having significantly different goals and understandings of the teaching situation than the teachers involved (Jones, Stewart and Power, 1999). In these situations centralized decision-making can result in less than effective solutions.

Implementing OT&L within an existing organisation often involves innovations that are perceived to be complex and incompatible with previous practice. Tornatzkey and Klein (1982) found that relative advantage, compatibility and complexity are the most significant factors in explaining relationships across abroad range of innovations. Surry and Farquhar (1997) report on a number of studies that confirm the links between relative advantage, complexity and compatibility and the adoption of innovations in education. Attempting to quickly install complex systems that require significant change in practice is likely to fail. Complexity can be created by incrementally assembling simple modules that can operate independently (Kelly, 1994). Evolutionary growth via small, simple steps is liable to be more effective.

In summary some suggested non-technical approaches for supporting emergent development include:

• Fully engage the members of an organisation and encourage shared construction and positive debate about the identity of the organisation. Encourage alternative views.
• Provide an appropriate organizational culture where innovation is encouraged and errors honored.
• Ensure a heterogeneous organization by supporting and encouraging those operating at the fringes. Allow multiple approaches.
• Put in place an approach to capture and disseminate knowledge and lessons learned by practitioners.
• Enable control to occur from the bottom up.
• Grow in simple steps with an emphasis on high relative advantage, low complexity and high compatibility with existing practice.

Technical approaches

The ability to adapt is seen as an important goal for emergent information systems development (Truex et al, 1998). There are a number of technical approaches intended to encourage adaptability. This section offers a brief overview of some of them.

The need for emergent information systems development to support continual analysis, dynamic requirements negotiation and usefully ambiguous specifications means that the ability to easily maintain and share specifications is required. Approaches such as object-oriented design and design patterns (Gamma, Helm, Johnson and Vlissides, 1994) simplify the process of learning and modifying an information system (Truex et al, 1998).

The use of design patterns provides a number of benefits including: easier reuse of successful designs or ideas, proven techniques are more accessible to developers, enables choice between alternatives, improves the documentation and maintenance of existing systems (Gamma, Helm, Johnson and Vlissides, 1994). Design patterns also allow piecemeal growth. Piecemeal growth is an approach that emphasizes design for repair, not replacement. As the environment changes new patterns are selected and applied continually moving the design from one context to another, replacing older designs with more appropriate designs (Jones and Stewart, 1999).

Open systems architectures, based on established standards, enable the incorporation and rearrangement of existing and new components (Truex et al, 1998). Component architectures, such as JavaBeans, CORBA and DCOM, enable the integration of heterogeneous objects development by a diverse community of developers that can help an infrastructure grow quickly. However, these architectures do require a heavy investment to set up and maintain as a common, organisation wide infrastructure (De Michelis et al, 1998).

The flexibility provided by component architectures such as JavaBeans are limited by their requirement that all components be based on a single component model or associated platform. Additionally the power of component frameworks can be lost without a good scripting language (Ousterhoust, 1998). Scripting languages allow very rapid development of applications via the gluing together of existing applications which can be 5 to 10 times faster than through the use of traditional systems programming languages (Ousterhoust, 1998).

Webfuse (Jones, 1999) is a web-authoring tool that integrates a heterogeneous collection of support tools behind a common interface. The collection of support tools are hidden behind hypermedia templates (Catlin and Garret, 1991) which simplify the authoring process while ensuring good design principles, aiding in reuse. Additional functionality can be added as requirements change by developing new hypermedia templates. The use of a scripting language means that Webfuse is not restricted by a single platform or component model.

Providing a tool of any sort to support OT&L creates a framework that can provide useful features that fulfill important requirements. However, lone rangers are often able to and desire to innovate outside of this framework while still taking advantage of the advantages of the framework. An emergent development should allow the lone rangers to pick and choose the features of the framework they wish to use and the freedom to use it, as they deem appropriate. For example, lone rangers should be able to use "end-user development tools" (e.g. Visual Basic, Access, FrontPage etc) to develop inexpensive, essentially disposable systems which address particular needs (Truex et al, 1998). These developments, if appropriately managed, can help identify directions that can then be incorporated back into the framework.

Information systems development is usually guided by a methodology that specifies the stages of a project and the specific tasks and aims for each stage. Most of these methodologies are project based and demonstrate the characteristics of traditional information systems development. However, there has recently been wide spread interest in lightweight development methodologies. One such lightweight development methodology is Extreme Programming (Beck, 2000). Extreme programming uses many traditional programming practices but rejects many of the aims of traditional software development and consequently fits well with emergent information systems development. For example, extreme programming rejects large-scale, up-front analysis and design in favor of continual analysis. Extreme programming suggests starting with a simple design and constantly evolving that design through the continual involvement of the appropriate users, testing first and pair programming.

Conclusions

It is suggested that the virtual university, whether a new organisation or an existing university moving into the new medium, is an emergent organisation. In an emergent organisation all the features and processes within the organisation are never fully formed and are always undergoing social negotiation and consensus building. The aims and assumptions of traditional information systems development methodologies, the production of systems with long periods of stable operation, are ill suited to an emergent organisation and will contribute to long term problems. Emergent systems development rejects the creation-based approach of traditional information systems and assumes the need for continual analysis and requirement negotiations to drive unceasing redevelopment of the information systems that support an organization and help encourage its emergence. It is suggested that applying such an approach to the implementation of OT&L will provide a system that better meets the needs of the organization.

References

Committee of Review of Higher Education Financing and Policy (CRHEFP), . (1997). Learning for Life: Review of Higher Education Financing and Policy. Canberra, Australian Government Printing Services.

Agre, P. (May, 1999). Infrastructure and Institutional Change in the Networked University. CENIC’99, Monterey.

Beck, K. (2000). Extreme Programming Explained: Embrace Change, Addison-Wesley.

Bonk, C., Dennen, Vanessa (1999). "Teaching on the Web: With a little help from my pedagogical friends." Journal of Computing in Higher Education 11(1): 3-28.

Brown, C. (1999). From the what and why to the how of course support systems – the value of the teacher’s perspective provided by these systems. WWW-Based Course-Support Systems Seminar, EdMedia’99, Seattle, Washington.

Catlin, K. S., Garret, L.N. (1991). Hypermedia Templates: An Authors Tool. Hypertext’91, ACM.

Collis, B., Oliver, Ron (1999). Preface. EdMedia’99, Seattle, Washington.

Collis, B. (July, 1998). Implementing Change Involving WWW-Based Course Support Across the Faculty. ACEC’98.

De Michelis, G., Dubois, Eric., Jarke, Matthias., Matthes, Florian., Mylopoulos, John., Schmidt, Joachim W., Woo, Carson., Yu, Eric. (December, 1998). "A Three-Faceted View of Information Systems." Communications of the ACM 41(12): 64-70.

Duane Truex, B., Klein (1999). "Growing Systems in Emergent Organisations." Communications of the ACM 42(8): 117-123.

Gamma, E., Helm, R., Johnson, R., Vlissides, J. (1994). Design Patterns: Elements of Reusable Object-Oriented Software. Reading, Massachusetts, Addision Wesley.

Gibbs, G. (1996). Institutional Strategies for Implementing Resource-based learning. Resource-based Learning. S. Brown, Smith, B. London, Kogan Page: 10-21.

Johnson, J. (1995). Chaos: The Dollar Drain of IT Project Failures. Application Development Trends: 41-47.

Jonassen, D. (1998). Designing Constructivist Learning Environments. Instructional Theories and Models. C. M. Reigeluth. Mahwah, NJ, Lawrence Erlbaum.

Jones, D., Stewart, Sharonn., Power, Leonie (1999). Patterns: using proven experience to develop online learning. ASCILITE’99, Brisbane, Australia.

Jones, D. (1999). Webfuse: An Integrated, Eclectic Web Authoring Tool. EdMedia’99, Seattle, Washington.

Kelly, K. (1994). Out of Control: The New Biology of Machines, Social Systems and the Economic World. New York, Addison-Wesley.

Lacity, M., Hirschheim, Rudy. (1993). Information Systems Outsourcing: Myths, Metaphors and Realities. Chichester, Wiley.

LaRose, R., Whitten, P. (1999). Building Web Courses with Instructional Immediacy. EdMedia’99, Seattle, Washington, AACE.

Laudon, K. C., Laudon, J.P. (1996). Management Information Systems. New Jersey, Prentice-Hall.

Luck, J. (1999). Teaching and Learning using Interactive Videoconferencing: Screen-based Classrooms Require the Development of New Ways of Working. AARE-NZARE, Melbourne, Australia.

Marshall, S. (2000). Accepting Change. Email to the Faculty of Informatics and Communication, Central Queensland Univeristy

McCormack, C., Jones, David., (1997). Building a Web-based Education System. New York, John Wiley & Sons.

Mioduser, D., Nachmias, R., Oren A., Lahav, O. (1999). Web-Based Learning Environments (WBLE): Current State and Emerging Trends. Ed-Media, Seattle, Washington.

OASIG (1996). The performance of information technology and the role of human and organisational factors, University of Sheffield.

Ousterhoust, J. (1998). "Scripting: Higher Level Programming for the 21st Century." IEEE Computer.

Reeves, T. (1999). A Research Agenda for Interactive Learning in the New Millennium. Proceedings of EdMedia’99, Seattle, Washington.

Robson, R. (1999). WWW-Based Course-Support Systems: The First Generation. WWW-Based Course-Support Systems Seminar, Seattle, Washington.

Sproull, L., Kiesler, Sara (1991). Connections: new ways of working in the networked organization. Cambridge, MIT Press.

Surry D.W, F. J. D. (1997). "Diffusion Theory and Instructional Technology." Journal of Instructional Science and Technology 2(1).

Taylor, P. (1998). "Institutional change in uncertain times: lone ranging is not enough." Studies in Higher Education 23(3): 269-278.

Tornatzky L.G, K., K.J (1982). "Innovation Characteristics and Innovation Adoption-Implementation." IEEE Transations on Engineering Management 29(1): 28-45.