Assembling the heterogeneous elements for (digital) learning

Year: 2019

Is training the barrier to quality online learning in higher ed?

TL;DR

Recently there have been various suggestions that the biggest barrier to quality online learning in higher education is lack of knowledge held by teaching staff (Johnson, 2019; Mathes, 2019; Roberts, 2018). More or better training, faculty development and requirements for formal teaching qualifications are proposed as the solution.

The following argues that this is just a symptom of the real barrier. i.e. that Universities actually don’t know how to implement quality online learning. Specific evidence drawn from one of the clarion calls for more training/formal qualifications is offered. A pointer to possible solutions to the actual barrier is provided.

Note: I started writing this in late November. A week before ASCILITE’2019. Just finishing it now.

Why?

The following is sparked by personal experience (bias?) and a post on OLDaily from Stephen Downes titled More needs to be done to support teaching online in Canada. Downes’ post reports on results from a research report (Johnson, 2019) surveying Canadian, publicly funded, post-secondary institutions and reactions from Tony Bates and Clint Lalonde. Downes wonders how, after 25+ years of online learning, 79% of institutions surveyed can report that a major barrier to online learning is inadequate training for faculty? Lalonde finds “this number staggering, and a sobering wake up call”. And its not just Canada. This topic echoes the findings from a 2019 ICDE report (Mathes, 2019) that was also featured on OLDaily. That report – Global quality in online, open, flexible and technology enhanced education: An analysis of strengths, weaknesses, opportunities and threats – draw on interviews with senior leaders from ICDE member institutions from across the world from which three themes emerged. Theme #2 was professional development

Appropriate training is not always available to build the expertise and skills of faculty and staff responsible for developing and/or teaching courses in these modalities. This can result in a poor teaching experience for faculty and a poor learning environment for students. (Mathes, 2019, p. 10)

Lalonde isn’t certain why there is this apparent “massive skills gap among instructors to teach online”. Downes suggests that higher education’s problem isn’t a training problem, but a culture problem. He wonders “about the apparent inability or unwillingness of today’s professors to teach themselves how to use a computer to teach”. Bates identifies the lack of willingness amongst institutions to “make training in teaching mandatory” as a major contributor and suggests (in another post) Should all lecturers have to have a teaching certificate? Why the answer is a resounding ‘yes’. Lalonde wonders if institutional focus on training for online has become less of a priority as a feeling of “been there done that” is combined with more attention being paid to broader issues such as accessibility, inclusiveness etc. He then picks up on Bates’ formal certification solution, but expands it to include not just online learning, but any learning modality.

Is training really the barrier?

The image below is from an ASCILITE’2019 presentation I’m working on for next week (slides, paper and source code available from here). The image is from a Blackboard course site. It is not a site for a real course. However, each design decision present in this “course site” is inspired by a practice from an actual course site that was developed by someone with a formal teaching qualification. In some cases, they had more than one formal teaching qualification. Since it’s for a presentation, this example focused on limitations that were visual. A similar example could be generated focusing on design for learning.

The point is that these practices were taken from courses developed by people with formal teaching qualifications. Exactly the solution being suggested. Given we are now 20+ years into the digital/online learning revolution, this seems to suggest that more training and formal qualifications in (online) learning are not likely to help improve the quality of online learning. Suggesting that the barrier is not (just) a lack of training.

What might be the barrier?

To answer this question, the following digs a bit deeper into the report from the Canadian Digital Learning Research Association. It reveals a few more possible barriers, but each really appear to be symptoms – just like the need for more faculty training – of the real barrier to quality online learning and teaching.

An absence of strategic planning?

The following is graph won’t be found in the report, but it is drawn from data presented in the report.

It shows that 94% of the institutions responding to the survey identified online learning as being of strategic importance of the institution. A variety of reasons is given. Growing continuing/professional education, increasing student access, and attracting students from outside traditional catchment areas are reported as the most important.

However, the graph also shows that only 12% of the institutions responding to the survey had a fully implemented plan for online learning!!!!! 59% reported being in the process of developing a plan with 26% reporting that they don’t have one, but really should develop one.

Might this not create some issues? How is this the case after 25+ years of online learning?

The free text from one institution suggests the potential source of problems that this absence might create

By creating a strategy, we are hoping to provide a frame for blended learning at our institute that will help guide processes, policies, and systems that align. At a course level, we are creating more supports for instructors to create their own digital learning objects for curriculum.

In a organisation espousing a strategic management approach the absence of strategic plans creates problems all the way down.

Workload

It was interesting to note that training was NOT “the most significant barrier to the adoption of online learning” (p. 40) from institutions responding to the survey. The following graph shows the top two responses from responding institutions to the barreirs to online education (from both the 2018 and 2019 surveys)

Much of the discussion mentions the need for more support for teaching staff, but most of the discussion appears to have focused on the training. Rather than pondering if the absence of appropriate supports might be exacerbating the training problem. i.e. more training is being called for because the systems and supports currently available to teaching staff are insufficient and inappropriate for the task being asked.

The focus on the new shiny thing

Based on the responses so far, it appears that many institutions recognise the strategic importance of online learning, know they don’t have a plan, and are aware that there are significant barriers in terms of workload and preparation for our teaching staff. So, obviously institutions are focused on taking action to address these problems, right?

From the report

institutions are experimenting with different delivery methods to better meet the needs of students. A variety of strategies are being employed: new technologies, OER, blended/hybrid learning, and alternative credentials. (p. 54)

Oh dear.

The need for careful implementation

To break the iron triangle of access, cost and quality Ryan et al (2019) propose the following “practical and pedagogical techniques”

  • High-quality large group teaching and learning;
  • Alternative curriculum structures;
  • Automation of assessment and feedback;
  • Personalising feedback at scale;
  • Peer-based learning; and,
  • Offloading administrative and technical support.

Arguably, each of these are shiny new things. The last suggestion includes one of the recent “poster boy” shiny new things – teacher bots. But the authors recognise the problem of shiny new things (arguably) with the recognition that the shiny new things “are surely part of the solution, they are by no means the entire solution”. The understand that it is important that shiny new things are

…implemented carefully and with a clear purpose…(and)…used to support good teachers, teaching practice and learning and assessment designs

As established above, it appears that many of the responders to the survey haven’t gotten there just yet. The question is whether they ever will. After all, we are 25+ years into this online learning fad.

The biggest barrier to quality online learning is actually…

Western universities don’t know how to do online learning

The real barrier to quality online learning and teaching in higher education is that they don’t know how to do this. Universities are good at the shiny new thing, but not so much at figuring out how the shiny new thing can be “implemented carefully and with a clear purpose…(and)…used to support good teachers, teaching practice and learning and assessment designs”. In the last pages of their book Ellis and Goodyear (2019) describe it this way

Over recent decades, Western universities have been very good at picking up and reproducing modish language about their purposes and methods – engaged enquiry, T-shaped graduates, being and becoming, and so on. They have been less good at ‘tooling up’ to deal with the complexity of analysing how their educational ecosystems actually function and of systematically redesigning for sustainable improvement. (p. 242)

For me, the lack of training for teaching staff is just a symptom of this broader problem. Universities are full of good people with a lot of knowledge about aspects (e.g. technical, pedagogical, content etc) of the challenge of online learning. But they are (and have been for some time) operating within organisations underpinned by a mindset that actively prevents those people from working effectively together to achieve “careful implementation”.

The fact that we are 25+ years into this online learning thing and its possible to make observations like the above seems to provide some support for this perspective.

What’s the solution?

That’s the (significantly more than) $64K question. More training, better (or even some) strategic plans, more project managers, and more shiny new things won’t provide a solution.

Ellis and Goodyear (Ellis & Goodyear, 2019) offer a research-based book that offers both diagnosis and remedy. The (second) best top-level answer to the question I’ve seen so far.

The work I describe in this year’s ASCILITE paper/presentation describes one meso-level practitioners attempt at a possible solution derived by combining some of ideas from Ellis and Goodyear (2019) with some other ideas.

Though whether these theoretical answers are good answers is waiting further work.

References

Ellis, R. A., & Goodyear, P. (2019). The Education Ecology of Universities: Integrating Learning, Strategy and the Academy. Routledge.

Johnson, N. (2019). National Survey of Online and Digital Learning 2019 National Report (p. 67). Retrieved from Canadian Digital Learning Research Association website: https://onlinelearningsurveycanada.ca/publications-2019/

Mathes, J. (2019). Global quality in online, open, flexible and technology enhanced education: An analysis of strengths, weaknesses, opportunities and threats. Retrieved from International Council for Open and Distance Education website: https://www.icde.org/knowledge-hub/report-global-quality-in-online-education

Roberts, J. (2018). Future and changing roles of staff in distance education: A study to identify training and professional development needs. Distance Education, 39(1), 37–53. https://doi.org/10.1080/01587919.2017.1419818

Ryan, T., French, S., & Kennedy, G. (2019). Beyond the Iron Triangle: Improving the quality of teaching and learning at scale. Studies in Higher Education, 0(0), 1–12. https://doi.org/10.1080/03075079.2019.1679763

 

Theory of workarounds

Introduction

The following is a summary of the paper Theory of Workarounds.

Alter, S. (2014). Theory of Workarounds. Communications of the Association for Information Systems, 34(1). https://doi.org/10.17705/1CAIS.03455

The paper provides “an integrated theory of workarounds that describes how and why” they are created. It is a process theory “driven by the interaction of key factors that determine whether possible workarounds are considerd and how they are executed” and is seen as useful for

  • classifying workarounds and analysing how they occur;
  • understanding compliance and noncompliance to management mandates;
  • figuring how to consider possible workarounds as part of systems development;
  • studying how workarounds may lead to larger planned changes.

My interest – digital learning and teaching

I’m interested in workarounds as a way to better understanding what’s happening around higher education’s use of digital technology to support learning and teaching, and identifying ways to improve it.

Definition and theory of workarounds

Alter (2014) offers the following definition of workarounds

A workaround is a goal-driven adaptation, improvisation, or other change to one or more aspects of an existing work system in order to overcome, bypass, or minimize the impact of obstacles, exceptions, anomalies, mishaps, established practices, management expectations, or structural constraints that are perceived as preventing that work system or its participants from achieving a desired level of efficiency, effectiveness, or other organizational or personal goals. (p. 1044)Comparisons between this and related definitions suggest this is a broader definition, including additional factors such as

  • workarounds don’t need to use digital technology;
  • workarounds may include work not formally recognised by the organisation;
  • workarounds don’t always compensate for or bypass system deficiencies;
  • workarounds may not be temporary;
  • workarounds are not necessarily examples of noncompliance;

Alter’s (2014) definition of workarounds does rely on it occuring within a work system. Another theoretical concept developed by Alter (2002). See this section from an old paper of mine for a summary of the Work System Framework.

It is argued that this reliance on the work system framework provides a “broader and more comprehensive view of the changes that can be included in workarounds” (Alter, 2014, p. 1046)

Figure 1 is a representation of Alter’s (2014) theory of workarounds. It is positioned as a process theory that describes how and why workarounds are created. A brief description follows the figure.

Alter’s theory of workarounds draws on a number of theories and concepts, including:

  • Theory of planned behaviour;
  • Improvisation and bricolage;
  • Agency theory;
  • Work system theory

Figure 1 – Alter’s (2014) Theory of Workarounds (p. 1056)

Workarounds arise from a context that includes each work system participant’s personal goals, interests and values. Communication and sharing of these goals/values between participants may be flaws or incomplete leading to misalignment in the work system. It also includes the structure of the work system that includes architecture, characteristics, performance goals and also emergent change.

From this context arises the perceived need for a work around.

Which triggers a process of trying to identify possible workarounds. Often starting with the obstacles in the current situation and the perceived need combined with consideration of the costs, benefits, risks of being identifed, and possible ramifications. An essential component is the knowledge available to those involved.

Eventually this leads to a decision to select a workaround to pursue, if any.

If going ahead, then development and execution of the workaround is driven by factors such as attention to current conditions, intuition guiding action, testing of intuitive understanding, and situational decision making.

Subsequently, there are local consequences and broader consequences. Locally, may lead to elimination of the obstacles that initiated the process, but may also include failure of the workaround or various unintendended consquences. More broadly, these types of consquences might be felt or pushed into other locations.

Temporality of workarounds

Alter also makes a point of outlining the temporality of workarounds as outlined in Figure 2.


Figure 2: Temporality of Workarounds (adapted from Alter, 2014, p. 1058)

Five voices in the workarounds literature

Alter performed a literature review on the workarounds literature. He gather 289 papers and used that to derive his theory. He summarises that work by using five “voices” which in turn include topics, including:

Phenomena associated with workarounds;

  • Obstacles, exceptions, anomalies, mishaps and structural constraints
  • Agency
  • Improvisation and bricolage
  • Routines, processes and methods
  • Articulation work and loose coupling
  • Technology misfits
  • Design and emergence
  • Technology usage and adaptation
  • Motives and control systems
  • Knowledge
  • Temporality

Types of workarounds;

  • Overcome inadequate IT functionality
  • Bypass an obstacle built into processes or practices
  • Respond to a mishap or anomaly with a quick fix
  • Substitute for unavailable resources
  • Design and implement new resources
  • Prevent future mishaps
  • Pretent to comply
  • Lie, cheat, steal for personal benefit
  • Colluse for mutual benefit

direct effects of workarounds;

  • Continuation of work despite obstacles, mishaps or anomalies
  • Creation of hazards, inefficiencies or errors
  • Impact on subsequent activities
  • Compliance or non-compliance with management intentions

perspectives on workarounds; and,

  • Workarounds as necessary activities in everyday life
  • Workarounds as sources for future improvements
  • Workarounds as creative acts
  • Workarounds as add-ons or shadow systems
  • Workarounds as quick fixes that won’t go away
  • Workarounds as facades of compliance
  • Workarounds as inefficiencies of hazards
  • Workarounds as resistance
  • Workarounds as distortions or subterfuge

organisational challenges and dilemmas related to workarounds.

  • Ability to operate despite obstacles
  • Enactment of interpretive flexibility
  • Balance of personal, group and organisational interests
  • Permitting and learning from emergent change

He uses these 5 voices then to group and establish some sense of causality within the “breadth of ideas and examples that were found in the literature” (p. 1047).

Usefulness and Further research

Since the theory is developed based on a literature search, it is limited by anything that hasn’t made it into the literature. e.g. accounts of workarounds that were considered, but never attempted.

Each step in the process theory could inform survey and/or case study research to explore how well the theory maps onto reality and lead to discoveries of factors/relationships not currently in the theory.

The workaround literature identifies fundamental limtiations in assumptions underpinning traditional approaches to organisational and system analysis and design (e.g. that prescribed business processes will be followed consistently). Theory of workarounds can be used to analyse systems in organisations, reveal conditions that lead to workarounds, provide opportunities to incorproate learning from workaround into emergent/planned change. Helping reveal insights into whether or not intended methods are followed; how systems in organisations evolve over time, how implementation evolves over time..

Starting from alternate theoretical foundations (e.g. Actor Network Theory, activity theory, socio-materiality etc) might lead to different outcomes and insights.

The conceptualisation of e-learning: Lessons and implications

The following is the content of my first solo journal article.

Jones, D. (2004). The conceptualisation of e-learning: Lessons and implications. Best Practice in University Learning and Teaching: Learning from Our Challenges.  Theme Issue of Studies in Learning, Evaluation, Innovation and Development, 1(1), 47–55.

I’m posting this because: sadly the journal is no more; the only version I had on my blog was the original PDF; this paper is the only public thing I’ve written on Alter’s Work System Framework; and, I wanted to reference that in another post. PDF isn’t accessible hence the HTML version.

Reading your old work is painful and occasionally surprising

As with many people reading my prior work is painful. Not sure I understood the notion of a paragraph. These issues continue into the short Lessons and Implications section but there are some bits that seem to have stood the test of time, including:

  1. Great diversity and continual change is common across the entire e-learning work system.
  2. No one single system can integrate all requirements and existing attempts to do so sacrifice contextual capabilities that contribute to strategic advantage.
  3. The penalty for not catering for future change and diversity…are limiting innovation, transformation and adoption of new approaches.
  4. Some of this is due to simplistic top-down approaches that don’t effectively harness bottom-up potential.
  5. IT has been viewed as fixed, neutral and independent of context rather than as one of a number of components that contribute to an emergent process of change.

The NGDLE link

Strong echoes of Point #2 can be seen in EDUCAUSE’s 2015 proposal of the NGDLE (Brown, Dehoney, & Millichap, 2015) in particular the following quote

although the NGDLE might include a traditional LMS as a component, it will not itself be a single application like the current LMS or other enterprise applications. Rather, the NGDLE will be an ecosystem of sorts

For more on the NGDLE idea, take a look the 2015 responses from Tony Bates and Jon Dron. Both critique the use of Lego as a metaphor for the NGDLE, which resonates strongly with me. As does Dron’s recommendation of glue or Velcro as better analogies. Back in the day, the “LMS” I (and many others) built and maintained was called Webfuse because

Webfuse was positioned as the glue used to “fuse” together widely different services and tools into an integrated whole (Jones, 2012)

Abstract

The implementation of e-learning within an institution of higher learning can be seen as a design problem. A first step in a design problem is the generation of an appropriate representation of the initial state. The nature of this representation has significant influence over the characteristics and suitability of any solution derived from it. This paper uses the work systems framework to present a shortened conceptualisation of this initial state. This conceptualisation reveals lessons and implications to improve future institutional approaches to e-learning.

Introduction

There is no question that institutions of higher education will make some use of e- learning1 The questions about e-learning have become how, why and with what outcomes (Hitt & Hartman, 2002). There is evidence that existing answers are not particularly innovative, demonstrate limited quality, tend to limit future possibilities and have a high likelihood of failure (Alexander, 2001; Jones, 2000).

Implementing an e-learning approach is a design problem. Design is the core of all professional training (Simon, 1996). There is growing interest in design research in fields such as management (Boland, 2002), information systems (Hevner, March, Park & Ram, in press) and education (Collective, 2003). Design can be seen as a transformation from some known situation, deemed to be problematic by some interested parties, to a more desirable target state (Jarvinen, 2001).

The formulation of the initial state into an effective representation is crucial to finding an effective design solution (Weber, 2003). Representation has a profound impact on design work (Hevner et al., 2004), particularly on the way in which tasks and problems are conceived (Boland, 2002). How an organisation conceptualises the e-learning problem will significantly influence how it answers the questions of how, why and with what outcomes.

The paper starts by offering an introduction to the work systems framework (Alter, 2002). This framework and the e-learning literature are used to develop a conceptualisation of the existing state facing institutions of higher education adopting e-learning. This conceptualisation is then used to identify lessons and implications that could be used to improve the design of e-learning.

The aim of this paper is not to generate the best and only conceptualisation of the initial e-learning state. Instead it seeks to demonstrate that existing conceptualisations of the e-learning problem are limited, that consequently so are many of the organisational approaches to e-learning and that the work system framework provides a useful approach to generating a deeper understanding.

The work system framework

A work system is a system, not necessarily computer-based, in which human participants perform business processes using information, technologies and other resources to produce products and services for customers (Alter, 2002). The work systems method includes both a static and dynamic view of the work system. This paper focuses on the static view of the e-learning work system.

The static view of a work system, the work system framework, identifies the basic elements for understanding and evaluating a work system and is useful in describing the system, describing possible changes, identifying problems and opportunities and tracing the likely impacts of changes to the system (Alter, 2002). Figure 1 provides a representation of the structure and components of the work system framework. Table 1 offers a brief description of the components.

Figure 1 – The Work System Framework (Alter, 2002)

Table 1: Components of the work system framework

Component

Description

Participants

People who perform the work

Information

Created and used by the participants

Technologies

Tools and techniques

Business processes

Steps used to perform the work

Products and services

Physical objects, intangibles, and services produced

Customers

Those who receive direct benefit from the products

Environment

Organisational, cultural, competitive, technical and regulatory environment within which work takes place

Infrastructure

Resources relied upon but managed from outside the work system

Strategies

Plans used to achieve the goals

The e-learning work system

The aim here is not to generate the most complete conceptualisation of the initial e- learning state. The aim instead is to introduce the use of the work system framework and to demonstrate how this can help generate a more complex and suitable conceptualisation. It is hoped that future work might lead to the generation of a more complete conceptualisation in both general and institutionally specific forms.

Such conceptualisations, even if targeted at a specific institution, would need to draw upon multiple perspectives in order to approach completeness. Any description authored by an individual cannot hope to achieve completeness.

However, in the generation of the following example conceptualisation, an attempt to include multiple perspectives has been made by drawing on a broad range of literature. While it is recognised that this is still somewhat limited, owing to its reliance on a single author and his knowledge of the literature, it is believed that this is sufficient to achieve the aims of this paper.

Participants

Teacher-centred, classroom education is the predominant form of learning within universities (Piccoli, Ahmad & Ives, 2000). Teachers’ conceptions of learning are a major influence on the planning of courses and on the development of teaching strategies (Alexander, 2001). Technology serves whichever goals motivate the people guiding its design and use (Lian, 2000). Teacher customisation of an online course can be vital to self-esteem and commitment (Brown, 1999).

Teaching is increasingly a team-based activity, with great diversity in the backgrounds, perspectives and roles of the team members. Course development with teams can be difficult in cultures with a different emphasis on peer review or entrenched academic hierarchies (Calder, 2000). Team members often have limited understanding or appreciation of the fields represented by other members.

Information

Academics can spend upwards of 90% of their planning and development creating information (Oliver, 1999) as the primary focus of learning. Contemporary views of teaching require information that is authentic, provides multiple perspectives and is not seen as the focus (Oliver, 1999).

Contemporary learning environments should integrate academic and administrative support services directly into the students’ environment (Segrave & Holt, 2003).

Information supporting these services is generated and consumed by people, from different fields, performing different roles and belonging to different parts of the organisation, and this information is stored in a variety of computer- and non- computer-based systems. This diversity leads to problems such as information ownership, version control, limited reuse and many others.

Technologies

The selection of a Course Management System (CMS), such as WebCT, is a common institutional response to e-learning. The primary use of CMSs tends to be as an administrative tool to facilitate classroom tasks and not as a tool anchored in pedagogy (Morgan, 2003). No CMS supports student critical thinking, generation of knowledge and collaborative teamwork (Bonk & Dennen, 1999). Most CMSs support more or less the same pedagogy (Robson, 1999). CMSs provide little support for usage monitoring and reporting at an institutional level across multiple courses (Morgan, 2003). CMSs alone may not be sufficiently conducive to supporting the design, development and operation required within contemporary learning environments (Segrave & Holt, 2003). CMSs are structured with little capability for customisation (Morgan, 2003).

CMSs are claimed to encapsulate a view of ‘best practice’, as defined by the vendor, that may not match an institution’s interests. The dominance within Australia of a small number of CMSs raises concerns that these systems are becoming embedded in the operational culture of higher education institutions and that uncompetitive pricing structures could evolve (Paulsen, 2002).

A typical university makes use of a large number of software applications, partly because creating a single application to run a complete business is virtually impossible. There is a general lack of integration amongst these systems (Paulsen, 2002). It appears, however, that maximum benefit from e-business is obtained when it is integrated throughout applications within the organisation (Marshall & Gregor, 2002). It is hypothesised that institutions implementing integrated systems will improve their chances of becoming successful, large-scale e-learning providers (Paulsen, 2002).

E-learning technologies are undergoing a continual process of change, presenting an ongoing challenge to management (Huynh, Umesh & Valacich, 2003). Any frozen definition of ‘best’ technology is likely to be temporary (Haywood, 2002). Increasing consumer technological sophistication adds to demand for sustained technological and pedagogical innovations (Huynh et al., 2003).

One view is that technology can help with automation and provide efficiency effects (Sproull & Kiesler, 1991) which may improve existing practices (Hannafin & Kim, 2003). Alternatively, technology is seen as a source of strategic advantage and as an enabler of previously impossible practices (Sproull & Kiesler, 1991) and of significant transformation (Hannafin & Kim, 2003). Managers tend to concentrate on the efficiency effects (Lacity & Hirschheim, 1993; Sproull & Kiesler, 1991).

It is most likely that technology will reinforce the old systems rather than the new paths (Lian, 2000). Educators are likely to use the technology to do things the way they have always been done, but with new and more expensive equipment (Dutton & Loader, 2002). Universities have not, despite many obvious exceptions, employed technology to the same degree or effect as the business community (Piccoli et al., 2000).

Business processes

Universities’ teaching processes include content production, packaging content, credentialing programs, presentation to students, marketing, registration, payment, record keeping and assessment (Marshall & Gregor, 2002). Using e-learning to facilitate routine transactions and services can be critically important both to the efficiency of services and in shaping the choices of students (Dutton & Loader, 2002).

Teaching and learning are two of the most highly personalised processes (Morgan, 2003). Academics, as knowledge workers, have considerable autonomy about how they perform tasks (Jones, Gregor & Lynch, 2003), a fact that encourages diversity. Many universities seem reluctant to cease using learning processes that predate the information revolution by centuries (Piccoli et al., 2000). Increasing commercialisation in higher education is leading to certain processes being enshrined in business contracts.

Products and services

The teaching products of a university include diverse products such as career counselling, athletic and social facilities, library services (Agre, 2000), credentialing, curriculum development, instructional delivery and student evaluation. Responsibility for these products is often distributed amongst the organisation without a single point of delivery and often with separate information technology (IT) systems.

Customers

Customers for the ‘teaching products’ of a university can be seen as including students, the general community, government, business and professional bodies. The following examines only students. The term ‘customer’ is used, as defined in Table 1, to mean someone who receives benefit from the product. Any deeper implications about what this term implies for the student–teacher relationship is not considered in this paper.

An essential component of facilitating learning is understanding learners, and particularly their learning styles, attitudes and approaches (Alexander, 2001; Oblinger, 2003). Students are no longer insulated from external pressures and they have to deal with real world concerns, including student loans, poor accommodation and part-time-working, yet many students still aspire to the assumed richness of campus-based education (Haywood, 2002). A large group of students, with significantly different characteristics, find asynchronous e-learning highly suited to their lifestyles and requirements (Hitt & Hartman, 2002).

Variety means that there is no one student experience of e-learning (Alexander, 2001). How to deal with the variety of backgrounds and expectations of students is one of the greatest challenges facing higher education today (Oblinger, 2003).

Environment

There has been inadequate recognition of the inherent differences in organisational cultures, academic cultures, education and training philosophies, and teaching and learning values and traditions within different cultural groups (Calder, 2000). A critical strategy for effective e-learning is to recognise the different cultures of learning among and within organizations (Lea, 2003). The many parties involved magnify traditional problems of politics, management expectations, hidden agendas, disruption to the balance of power, technical concerns and differences in cultural values (Gregor, Jones, Lynch & Plummer, 1999).

The higher education industry is subject to the same pressures as other industries, including market, technological and societal pressures (Marshall & Gregor, 2002). Ultimately, the enterprises that are able to adapt to changes in the environment, while keeping the costs under control, will be successful (Huynh et al., 2003).

Uncertainty about the future and other developments highlight the importance of building institutions that are responsive to change. Innovation within higher education is constrained by a lack of competition, habits, values and traditions, and institutional arrangements and policies such as incentive structures, copyright and intellectual property rights (Dutton & Loader, 2002).

Infrastructure

Existing technological infrastructure requires modification to fulfil the performance, scalability and availability requirements of e-learning (Hitt & Hartman, 2002). Institutions need to reorient their infrastructures from their existing state, designed to support departments, to a user-centric state (Moul, 2003). Information systems infrastructure that is flexible and adaptable can be a powerful enabler of innovation, but rigid, inflexible systems are serious obstacles to organisational effectiveness and success.

Senior management often perceive infrastructure and IT as costs to be minimised. Users see it as a service to be customised for their idiosyncratic requirements. IT organisations are caught in the middle, since best practices associated with reducing costs are in direct conflict with best practices necessary to maximise service levels (Jones, 2000).

Strategies

The adoption of e-learning requires a revisiting of existing strategies, especially those associated with program development and instructional technology (Hitt & Hartman, 2002). An organisational culture that provides appropriate rewards is a success factor when implementing change in teaching and learning (Collis, 1998). While 74.5% of campuses have IT development programs, and 66.6% have campus support centres, only 13.4% have a formal, institutional program to recognise and reward the use of IT as part of the formal faculty review process (Green, 1999).

Too much of e-learning staff development focuses on the level of teachers’ strategies – how to use a particular tool – rather than on their conceptions of learning (Alexander, 2001).

Transformational change through e-learning requires institutional leaders to articulate a clear, bold vision, demonstrate a broad understanding and acceptance of that view, apply the focused use of resources and encourage widespread collaboration throughout the institution (Hitt & Hartman, 2002). Top–down planning often falters at the operational level because implementation throws up a range of messy human factors that need to be addressed with the same resolution that fuelled the initial policy enthusiasm (Haywood, 2002).

Technology projects fail because the innovators underestimate the consequences of new technologies (Sproull & Kiesler, 1991) and fail to accommodate environmental and contextual factors affecting implementation. The realities and subtleties of a comprehensive conception of a learning environment, which integrates virtual and physical and also academic and administrative, have eluded education policy-makers and designers (Segrave & Holt, 2003). Those who support a social shaping perspective of e-learning in universities emphasise that there are many paths that can be followed by the developers of e-learning (Dutton & Loader, 2002).

Alternate conceptualisations of the world raise doubts about whether it is possible to make sensible strategic and policy decisions in the traditional sense. This is particularly true when the world is seen as non-deterministic, evolutionary and highly complex – a world where the most desirable outcomes are unknown but there may be many possible acceptable outcomes, where change is characterised by both path dependence and unpredictability and where there are many diverse components, interaction and feedback among components, and multiple dimensions of each problem (Carlsson, 2002).

Lessons and implications

Great diversity and continual change are common characteristics across most of the nine components of the e-learning work system. As such, any response should be designed to cope with diversity and ongoing change. This has not always been the case. The most important policy objective is to remove obstacles to creativity and to foster entrepreneurship, rather than to take new initiatives: systematic planning should not replace the imaginative spark that creates innovation (Carlsson, 2002).

Diversity without integration creates problems. Integration across diverse systems, processes and organisational units is important in generating a customer-focused approach and an efficient operation. No one single system can integrate all requirements and existing attempts to do so sacrifice contextual capabilities that contribute to strategic advantage. An approach that enables diversity but achieves a customer-focused interface should provide greater strategic advantage.

The penalty for not catering for future change and diversity can be seen in the examples provided of how existing conceptions, infrastructure and organisational structures are limiting innovation, transformation and the adoption of new approaches. Some of this is due to simplistic top–down approaches that don’t effectively harness bottom–up potential.

IT has been taken for granted or assumed to be unproblematic. This results in a narrow conceptualisation of what technology is, how it has effects and how and why it is implicated in social change (Orlikowski & Iacono, 2001). Such limited conceptualisations often view IT as fixed, neutral and independent of context.

Alternatively, IT is one of a number of components of an emergent process of change where the outcomes are indeterminate because they are situationally and dynamically contingent (Markus & Robey, 1988). Ongoing change is not solely “technology led” or solely “organisational/agency driven”, instead change arises from a complex interaction among technology, people and the organization (Marshall & Gregor, 2002).

Conclusion

This paper has generated a shortened representation of the initial state within universities wishing to adopt e-learning. Even this limited representation has helped identify a number of lessons and implications that could be used to improve the future design of e-learning. A deeper, broader and more institutionally specific representation could enable even greater improvements.

Notes

1 ‘e-learning’ is one of many terms currently used to describe the use of information technology to support teaching and learning. Rather than argue about the ambiguities and differences among the various terms, this paper will use ‘e-learning’.

References

Agre, P. (2000). Commodity and community: Institutional design for the networked university. Planning for Higher Education, 29(2), 5-14.

Alexander, S. (2001). E-learning developments and experiences. Education and Training, 43(4/5), 240-248.

Alter, S. (2002). The work system method for understanding information systems and information system research. Communications of the Association for Information Systems, 9(6), 90-104.

Boland, R. J. (2002, June 14-15). Design in the punctuation of management action.

Paper presented at the Frontiers of Management workshop, Weatherhead School of Management, Case Western Reserve University, Cleveland, OH.

Bonk, C., & Dennen, V. (1999). Teaching on the Web: With a little help from my pedagogical friends. Journal of Computing in Higher Education, 11(1), 3-28. Brown, C. (1999). From the what and why to the how of course support systems –

the value of the teacher’s perspective provided by these systems. Paper presented at the WWW-Based Course Support Systems seminar, EdMedia ’99, Seattle, WA.

Calder, J. (2000). Beauty lies in the eye of the beholder. International Review of Research in Open and Distance Learning, 1(1).

Carlsson, B. (2002, June 14-15). Public policy as a form of design. Paper presented at the Frontiers of Management workshop, Weatherhead School of Management, Case Western Reserve University, Cleveland, OH.

Collective, T. D.-B. R. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.

Collis, B. (1998, July 8). Implementing change involving WWW-based course support across the faculty. Keynote address presented at the annual conference of the Australian National Computers in Education, Adelaide, SA.

Dutton, W., & Loader, B. (2002). Introduction. In W. Dutton & B. Loader (Eds.), Digital academe: The new media and institutions of higher education and learning (pp. 1-32). London: Routledge.

Green, K. (1999). The continuing challenge of instructional integration and user support: 1999 national survey of information technology in higher education (summary). Encino, CA: The Campus Computing Project.

Gregor, S., Jones, D., Lynch, T., & Plummer, A. A. (1999). Web information systems development: Some neglected aspects. Paper presented at the International Business Association conference, Cancun, Mexico.

Hannafin, M., & Kim, M. (2003). In search of a future: A critical analysis of research on web-based teaching and learning. Instructional Science, 31, 347- 351.

Haywood, T. (2002). Defining moments: Tension between richness and reach. In

W. Dutton & B. Loader (Eds.), Digital academe: The new media and institutions of higher education and learning (pp. 39-49). London: Routledge.

Hevner, A., March, S., Park, J., & Ram, S. (in press). Design science in information systems research. Management Information Systems Quarterly.

Hitt, J., & Hartman, J. (2002). Distributed learning: New challenges and opportunities for institutional leadership. Washington, DC: American Council on Education.

Huynh, M., Umesh, U. N., & Valacich, J. S. (2003). E-Learning as an emerging entrepreneurial enterprise in universities and firms. Communications of the Association for Information Systems, 12, 48-68.

Jarvinen, P. (2001). On research methods. Tampere, Finland: Opinpajan Kirja.

Jones, D. (2000). Emergent development and the virtual university. Paper presented at the Learning 2000 conference, Roanoke, VI.

Jones, D., Gregor, S., & Lynch, T. (2003). An information systems design theory for web-based education. Paper presented at the Web-Based Education symposium at the Computers and Advanced Technology in Education conference, Rhodes, Greece.

Lacity, M., & Hirschheim, R. (1993). Information systems outsourcing: Myths, metaphors and realities. Chichester, UK: John Wiley & Sons.

Lea, P. (2003). Understanding the culture of e-learning. Industrial and Commercial Training, 35(4/5), 217-219.

Lian, A. (2000, July). Knowledge transfer and technology in education: Toward a complete learning environment. Educational Technology & Society, 3(3).

Markus, M. L., & Robey, D. (1988). Information technology and organizational change: Causal structure in theory and research. Management Science, 34(5), 583-598.

Marshall, S., & Gregor, S. (2002). Distance education in the online world: Implications for higher education. In M. Khosrow-Pour (Ed.), Web-based instructional learning (pp. 110-124). Hershey, PA: IRM Press.

Morgan, G. (2003). Faculty use of course management systems. Educause Center for Applied Research.

Moul, B. (2003, March-April). Creating a unified digital campus to satisfy the needs of 21st century learners. The Technology Source.

Oblinger, D. (2003, July-August). Boomers, Gen-Xers and Millennials: Understanding the new students. EDUCAUSE Review, 38(4), 37-47.

Oliver, R. (1999). Exploring strategies for on-line teaching and learning. Distance Education, 20(2), 240-254.

Orlikowski, W., & Iacono, C. S. (2001). Research commentary: Desperately seeking the IT in IT research: A call to theorizing the IT artifact. Information Systems Research, 12(2), 121-134.

Paulsen, M. F. (2002, October). Online education systems in Scandinavian and Australian universities: A comparative study. International Review of Research in Open and Distance Learning, 3(2).

Piccoli, G., Ahmad, R., & Ives, B. (2000). Knowledge Management in Academia: A Proposed Framework. Information Technology and Management, 1, 229- 245.

Robson, R. (1999). WWW-based course-support systems: The first generation.

Paper presented at the WWW-Based Course-Support Systems seminar, Seattle, WA.

Segrave, S., & Holt, D. (2003). Contemporary learning environments: Designing e- learning for education in the professions. Distance Education, 24(1), 7-24.

Simon, H. (1996). The sciences of the artificial (3rd ed.). Cambridge, MA: MIT Press.

Sproull, L., & Kiesler, S. (1991). Connections: New ways of working in the networked organization. Cambridge, MA: MIT Press.

Weber, R. (2003). Still desperately seeking the IT artifact. Management Information Systems Quarterly, 27(2), iii-xi.

It’s more than how you use the technology – authoring online content

At the recent ASCILITE’2019 there was a common refrain throughout the conference, which started with the original keynote as captured in the following tweet

https://twitter.com/volkov_michael/status/1201679376225468417?s=20

While I get the intent – the ultimate goal is good learning, not good technology – “how you use it” is much more than pedagogy. How this is translated into action within higher education has always troubled me. The following is an attempt to explain some of that and the “more” that is required. It’s also a reason to show off a screen capture video that I couldn’t show at the conference.

Higher education has a long-term (and growing) interest in “improving and measuring quality in higher education” (Ryan, French, & Kennedy, 2019), but it has “an under-developed capacity to analyse and explain” (Ellis & Goodyear, 2019, p. 239) the processes used to achieve it. Much of what currently happens in higher education is focused on identifying principles etc. of good pedagogy, disseminating knowledge of those principles, and measuring whether or not they’ve been adopted. Far less attention is paid to how to implement said principles in a sustainable way. In particular, little attention appears to be paid just how difficult and time-consuming implementing said principles can be.

My argument is that the activity system – the complex combination of learner, teachers, curriculum materials, software tools and the physical environment (Greeno, 2005, p. 79) – involved in “using it” requires much more attention. I’d argue that most of these activity systems are incredibly broken and subsequently it’s no surprise that there are concerns about the quality of learning environments, experiences and outcomes. My belief is that improving these activity systems will lead to improvements in learning environments, experiences and outcomes.

Authoring online course modules in Blackboard – Integrating Word & OneDrive/Sharepoint

(The title I’ve given this section could be easily seen as focusing on the technology. Boo! Hiss!. The important word is integrating. Integrating into these technologies into the stupid existing activity system for producing online course modules in Blackboard. Figuring out how to use the technology to make it easier for others to use the technology to effectively achieve the desired “pedagogy”.)

Part of my work for the last year has been paying attention to the activity system within which online course modules are created and maintained. This work has been seen in earlier blog posts and in the ASCILITE paper. Part of this was embedding the production of online content for Blackboard within the broader authoring practices of academics. i.e. allowing them to write content in Word – thereby drawing on facilities such as citation management, track changes, grammar checking etc. – and import it into Blackboard.

Until recently, the importing into Blackboard was a bit too manual. The video below illustrates the new process. It’s an example of an on-going improvement to the activity system.

It shows:

  1. Original Blackboard module content.
    The video starts showing a Blackboard module. The top part of the page shows the module content within an accordion interface. This is what students interact with. At the bottom of the page is an item titled Tweak code. Only teaching staff see this section. This item includes a link that will open up an online version of a Word document containing the module content. Ready for editing. It also includes a green button that will update the Blackboard module content.
  2. Converting Word document to HTML.
    Clicking the green button opens up a new web page that retrieves the Word document from OneDrive/SharePoint and converts it to HTML. If successful another green button will appear that when clicked will copy the HTML content into the clipboard and open up the appropriate Blackboard edit page.
  3. Editing the Blackboard content.
    At this stage, the normal Blackboard edit process for HTML commences. The content is updated by clearing out the old HTML and pasting in the newly converted content. Save these changes and the module has been updated.

What’s the impact?

Don’t know. What’s shown above has only been completed in the last couple of weeks and is currently being rolled out. Anecdotal feedback from people working with the more manual Word/Blackboard integration has been largely positive. Though there have been comments made about the extra steps involved in the manual integration. This new and more direct integration will hopefully help.

Given the following types of comments about authoring in Blackboard

relying primarily on the Blackboard Content Editor to post materials in the course shell as HTML is a time relatively consuming process (sic) …maintaining these courses too technically challenging

There’s some hope and expectation that this will have some positive impact.

Adding more context by focusing on pedagogical model activity systems

As it stands, the above is talking about a more general online content authoring activity system. But we can do more. It’s possible to adapt the above approach to customise the Word template and the HTML conversion process in response to specific contextual needs. For example, if a particular pedagogical model (Conole, 2010) has been adopted in a course, we could develop a specific Word template/HTML conversion process to offer explicit support for elements of that pedagogical model. Moving beyond the generic content authoring activity system and to focus on an activity system specific to the chosen pedagogical model.

References

Conole, G. (2010). Review of Pedagogical Models and their use in e-learning. Retrieved from Open University website: http://cloudworks.ac.uk/cloud/view/2982

Ellis, R. A., & Goodyear, P. (2019). The Education Ecology of Universities: Integrating Learning, Strategy and the Academy. Routledge.

Greeno, G. J. (2005). Learning in Activity. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 79–96). https://doi.org/10.1017/CBO9780511816833.007

Ryan, T., French, S., & Kennedy, G. (2019). Beyond the Iron Triangle: Improving the quality of teaching and learning at scale. Studies in Higher Education, 0(0), 1–12. https://doi.org/10.1080/03075079.2019.1679763

How to share design knowledge in design for digital learning?

Using Blackboard Learn to design and manage a quality learning environment is hard. Managing the web content alone is hard to do well. To do so requires significant HTML, CSS and related web design knowledge. Knowledge that not everyone has, or should have, or can have. That’s why there are so many ugly Blackboard sites.

The difficulty of sharing this specific type of design knowledge is just one small example of the question facing institutions of higher education (especially in these pandemic times): How to share design knowledge in design for digital learning?

The following resources are associated with an ASCILITE’2019 paper (or conference proceedings PDF) that attempted to explain one possible way of answering that question.

Not the answer, but an answer that has since been successfully used to share design knowledge across hundreds of courses across multiple institutions (but mostly at my current institution).


Early versions of the work are described in blog posts (Card Interface and Content Interface) and are currently (mid-2021) open source software ready to use on any Blackboard Learn instance (Card Interface or Content Interface)

Working Software

This paper draws on the experience developing the Card and Content Interface. Two collections of Javascript & CSS that can be embedded into any instance of Blackboard Learn.

The Card Interface is simplest to install and use and has been used at Universities in Australia, Ireland, and New Zealand.

Both tools have

Abstract

Higher education is being challenged to improve the quality of learning and teaching while at the same time dealing with challenges such as reduced funding and increasing complexity. Design for learning has been proposed as one way to address this challenge, but a question remains around how to sustainably harness all the diverse knowledge required for effective design for digital learning. This paper proposes some initial design principles embodied in the idea of Context-Appropriate Scaffolding Assemblages (CASA) as one potential answer. These principles arose out of prior theory and work, contemporary digital learning practices and the early cycles of an Action Design Research process that developed two digital ensemble artefacts for 7 courses (units, subjects) and in less than a year been used in over 60 sites. Experience with this approach suggests it can successfully increase the level of design knowledge embedded in digital learning experiences, identify and address shortcomings with current practice, and have a positive impact on the quality of the learning environment.

Presentation

The PowerPoint animations don’t translate 100% correctly to Google Presentation but close enough.

References

Aitchison, C., Harper, R., Mirriahi, N., & Guerin, C. (2019). Tensions for educational developers in the digital university: Developing the person, developing the product. Higher Education Research & Development, 0(0), 1–14. https://doi.org/10.1080/07294360.2019.1663155

Arthur, W. B. (2009). The Nature of Technology: what it is and how it evolves. New York, USA: Free Press.

Behnke, J. (2018). Content editor HTML vs. PDF? Retrieved February 24, 2019, from Blackboard Community website: https://community.blackboard.com/thread/6523-content-editor-html-vs-pdf

Bennett, S., Agostinho, S., & Lockyer, L. (2017). The process of designing for learning: understanding university teachers’ design work. Educational Technology Research & Development, 65(1), 125–145. https://doi.org/10.1007/s11423-016-9469-y

DeLanda, M. (2006). A New Philosophy of Society: Assemblage Theory and Social Complexity. A&C Black.

Dimitriadis, Y., & Goodyear, P. (2013). Forward-oriented design for learning : illustrating the approach. Research in Learning Technology, 21, 1–13.

Ellis, R. A., & Goodyear, P. (2019). The Education Ecology of Universities: Integrating Learning, Strategy and the Academy. Routledge.

Fischer, G., & Girgensohn, A. (1990). End-user Modifiability in Design Environments. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 183–192. https://doi.org/10.1145/97243.97272

Goodyear, P. (2015). Teaching As Design. HERDSA Review of Higher Education, 2, 27–59.

Greeno, G. J. (2005). Learning in Activity. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 79–96). https://doi.org/10.1017/CBO9780511816833.007

Introna, L. (2013). Epilogue: Performativity and the Becoming of Sociomaterial Assemblages. In F.-X. de Vaujany & N. Mitev (Eds.), Materiality and Space: Organizations, Artefacts and Practices (pp. 330–342).

Mathes, J. (2019). Global quality in online, open, flexible and technology enhanced education: An analysis of strengths, weaknesses, opportunities and threats. Retrieved from International Council for Open and Distance Education website: https://www.icde.org/knowledge-hub/report-global-quality-in-online-education

Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054.

Mor, Y., Craft, B., & Maina, M. (2015). Introduction – Learning Design: Definitions, Current Issues and Grand Challenges. In M. Maina, B. Craft, & Y. Mor (Eds.), The Art & Science of Learning Design (pp. ix–xxvi). Rotterdam: Sense Publishers.

Nanard, M., Nanard, J., & Kahn, P. (1998). Pushing Reuse in Hypermedia Design: Golden Rules, Design Patterns and Constructive Templates. 11–20. ACM.

Roberts, J. (2018). Future and changing roles of staff in distance education: A study to identify training and professional development needs. Distance Education, 39(1), 37–53. https://doi.org/10.1080/01587919.2017.1419818

Ryan, T., French, S., & Kennedy, G. (2019). Beyond the Iron Triangle: Improving the quality of teaching and learning at scale. Studies in Higher Education, 0(0), 1–12. https://doi.org/10.1080/03075079.2019.1679763

Sein, M. K., Henfridsson, O., Purao, S., & Rossi, M. (2011). Action Design Research. MIS Quarterly, 35(1), 37–56.

Sun, S. Y. H. (2017). Design for CALL – possible synergies between CALL and design for learning. Computer Assisted Language Learning, 30(6), 575–599. https://doi.org/10.1080/09588221.2017.1329216

Zittrain, J. (2008). The Future of the Internet–And How to Stop It. Yale University Press.

Image attributions

Why did the chicken cross the road? flickr photo by bortescristian shared under a Creative Commons (BY) license

what flickr photo by Vikki-Lea shared under a Creative Commons (BY) license

How flickr photo by Infrogmation shared under a Creative Commons (BY) license

Why flickr photo by Jeremiah Vandermeer shared under a Creative Commons (BY-NC) license

KNOWLEDGE flickr photo by Troels Myrup shared under a Creative Commons (BY-NC-ND) license

Diamond road sign steep incline By User:Stannereden:Image:Diamond road sign steep incline.png, CC BY-SA 3.0, Link

King playing cards
By Enoch LauOwn work (photo), CC BY-SA 3.0, Link

Warning on the French Cigarettes Pack
By The original uploader was Arolga at English Wikipedia.
(Original text: Encyclopedia of Cigarettes) – http://www.cigarettespedia.com/index.php/B_Basic_%28Fine_Flavor%29_KS-30-H_%28grey%29_-_France_and_Germany, CC BY-SA 3.0, Link

Context is King flickr photo by _rebeccajackson shared under a Creative Commons (BY-NC-ND) license

Scaffold flickr photo by OiMax shared under a Creative Commons (BY) license

Sarah Ionnides – Conducting
By Izabel.zambrzyckiOwn work, CC BY-SA 4.0, Link

Exploring knowledge reuse in design for digital learning: tweaks, H5P, constructive templates and CASA

The following has been accepted for presentation at ASCILITE’2019. It’s based on work described in earlier blog posts.

Click on the images below to see full size.

Abstract

Higher education is being challenged to improve the quality of learning and teaching while at the same time dealing with challenges such as reduced funding and increasing complexity. Design for learning has been proposed as one way to address this challenge, but a question remains around how to sustainably harness all the diverse knowledge required for effective design for digital learning. This paper proposes some initial design principles embodied in the idea of Context-Appropriate Scaffolding Assemblages (CASA) as one potential answer. These principles arose out of prior theory and work, contemporary digital learning practices and the early cycles of an Action Design Research process that has developed two digital ensemble artefacts employed in over 30 courses (units, subjects). Early experience with this approach suggests it can successfully increase the level of design knowledge embedded in digital learning experiences, identify and address shortcomings with current practice, and have a positive impact on the quality of the learning environment.

Keywords: Design for Learning, Digital learning, NGDLE.

Introduction

Learning and teaching within higher education continues to be faced with significant, diverse and on-going challenges. Challenges that increase the difficulty of providing the high-quality learning experiences necessary to produce graduates of the standard society is expecting (Bennett, Lockyer, & Agostinho, 2018). Goodyear (2015) groups these challenges into four categories: massification and the subsequent diversification of needs and expectations; growing expectations of producing work-ready graduates; rapidly changing technologies, creating risk and uncertainty; and, dwindling public funding and competing demands on time. Reconceptualising teaching as design for learning has been identified as a key strategy to sustainably, and at scale, respond to these challenges in a way that offers improvements in learning and teaching (Bennett et al., 2018; Goodyear, 2015). Design for learning aims to improve learning processes and outcomes through the creation of tasks, environments, and social structures that are conducive to effective learning (Goodyear, 2015; Goodyear & Dimitriadis, 2013). The ability of universities to develop the capacity of teaching staff to enhance student learning through design for learning is of increasing financial and strategic importance (Alhadad, Thompson, Knight, Lewis, & Lodge, 2018).

Designing learning experiences that successfully integrate digital tools is a wicked problem. A problem that requires the utilisation of expert knowledge across numerous fields to design solutions that respond appropriately to the unique, incomplete, contextual, and complex nature of learning (Mishra & Koehler, 2008). The shift to teaching as design for learning requires different skills and knowledge, but also brings shifts in the conception of teaching and the identity of the teacher (Gregory & Lodge, 2015). Effective implementation of design for learning requires detailed understanding of pedagogy and design and places cognitive, emotional and social demands on teachers (Alhadad et al., 2018). The ability of teachers to deal with this load has significant impact on learners, learning, and outcomes (Bezuidenhout, 2018). Academic staff report perceptions that expertise in digital technology and instructional design will be increasingly important to their future work, but that these are also the areas where they have the least competency and the highest need for training (Roberts, 2018). Helping teachers integrate digital technology effectively into learning and teaching has been at or near the top of issues facing higher education over several years (Dahlstrom, 2015). However, the nature of this required knowledge is often underestimated by common conceptions of the knowledge required by university teachers (Goodyear, 2015). Responding effectively will not be achieved through a single institutional technology, structure, or design, but instead will require an “amalgamation of strategies and supportive resources” (Alhadad et al., 2018, pp. 427-429). Approaches that do not pay enough attention to the impact on teacher workload run the risk of less than optimal learner outcomes (Gregory & Lodge, 2015).

Universities have adopted several different strategies to ameliorate the difficulty of successfully engaging in design for digital learning. For decades a common solution has been that course design, especially involving the adoption of new methods and technologies, should involve systematic planning by a team of people with appropriate expertise in content, education, technology and other required areas (Dekkers & Andrews, 2000). The use of collaborative design teams with an appropriate, complementary mix of skills, knowledge and experience mirrors the practice in other design fields (Alhadad et al., 2018). However, the prevalence of this practice in higher education has been low, both then (Dekkers & Andrews, 2000) and now. The combination of the high demand and limited availability of people with the necessary knowledge mean that many teaching staff miss out (Bennett, Agostinho, & Lockyer, 2017). A complementary approach is professional development that provides teaching staff with the necessary knowledge of digital technology and instructional design (Roberts, 2018). However, access to professional development is not always possible and funding for professional development and training has rarely kept up with the funding for hardware and infrastructure (Mathes, 2019). There has been work focused on developing methods, tools and repositories to help analyse, capture and encourage reuse of learning designs across disciplines and sectors (Bennett et al., 2017). However, it appears that design for learning continues to struggle to enter mainstream practice (Mor, Craft, & Maina, 2015) with design work undertaken by teachers apparently not including the use of formal methods or systematic representations (Bennett et al., 2017). There does, however, remain on-going demand from academic staff for customisable and reusable ideas for design (Goodyear, 2005). Approaches that respond to academic concerns about workload and time (Gregory & Lodge, 2015) and do not require radical changes to existing work practices nor the development of complex knowledge and skills (Goodyear, 2005).

If there are limitations with current common approaches, what other approaches might exist? Leading to the research question of this study:

How might the diverse knowledge required for effective design for digital learning be shared and used sustainably and at scale?

An Action Design Research (ADR) process is being applied to develop one answer to this question. ADR is used to describe the design, development and evaluation of two digital artefacts – the Card Interface and the Content Interface – and the subsequent formulation of initial design principles that offer a potential answer to the research question. The paper starts by describing the research context and research method. The evolution of each of the two digital artefacts is then described. This experience is then abstracted into six design principles encapsulated in the concept of Context-Appropriate Scaffolding Assemblages (CASA). Finally, the conclusions and implications of this work are discussed.

Research context and method

This research project started in late 2018 within the Learning and Teaching (L&T) section of the Arts, Education and Law (AEL) Group at Griffith University. Staff within the AEL L&T section work with the AEL’s teachers to improve the quality of learning and teaching across about 1300 courses (units, subjects) and 68 programs (degrees). This work seeks to bridge the gaps between the macro-level institutional and technological vision and the practical, coal-face realities of teaching and learning (micro-level). In late 2018 the macro-level vision at Griffith University consisted of current and long-term usage of the Blackboard Learn Learning Management System (LMS) along with a recent decision to move to the Blackboard Ultra LMS. In this context, a challenge was balancing the need to help teaching staff continue to improve learning and teaching within the existing learning environment while at the same time helping the institution develop, refine, and achieve its new macro-level vision. It is within this context that the first offering of Griffith University’s Bachelor of Creative Industries (BCI) program would occur in 2019. The BCI is a future-focused program designed to attract creatives who aspire to a career in the creative industries by instilling an entrepreneurial mindset to engage and challenge the practice and business of the creative industries. Implementation of the program was supported through a year-long strategic project including a project manager and educational developer from the AEL L&T section working with a Program Director and other academic staff. This study starts in late 2018 with a focus on developing the course sites for the seven first year BCI courses. A focus of this work was to develop a striking and innovative design that mirrored the program’s aims and approach. A design that could be maintained by the relevant teaching staff beyond the project’s protected niche. This raised the question of how to ensure that the design knowledge required to maintain a digital learning environment into the future would be available within the teaching team?

To answer this question an Action Design Research (Sein, Henfridsson, Purao, & Rossi, 2011) process was adopted. ADR is a merging of Action Research with Design Research developed within the Information Systems discipline. ADR aims to use the analysis of the continuing emergence of theory-ingrained, digital artefacts within a context as the basis for developing generalised outcomes, including design principles (Sein et al., 2011). A key assumption of ADR is that digital artefacts are not established or fixed. Instead, digital artefacts are ensembles that arise within a context and continue to emerge through development, use and refinement (Sein et al., 2011). A critical element of ADR is that the specific problem being addressed – design of online learning environment for courses within the BCI program – is established as an example of a broader class of problems – how to sustainably and at scale share and reuse the diverse knowledge required for effective design for digital learning (Sein et al., 2011). This shift moves ADR work beyond design – as practised by any learning designer – to research intending to provide guidance to how others might address similar challenges in other contexts that belong to the broader class of design problems.

Figure 1 provides a representation of the ADR four-stage process and the seven principles on which ADR is based. Stages 1 through 3 represent the process through which ensemble digital artefacts are developed, used and evolved within a specific context. The next two sections of this paper describe the emergence of two artefacts developed for the BCI program as they cycled through the first three ADR stages numerous times. The fourth stage of ADR – Formalisation of Learning – aims to abstract the situated knowledge gained during the emergence of digital artefacts into design principles that provide guidance for addressing a class of field problems (Sein et al., 2011). The third section of this paper formalizes the learning gained in the form of six initial design principles structured around the concept of Contextually Appropriate Scaffolding Assemblages (CASA).

Action Design Research Method: Stages and Pinciples

Figure 1 – ADR Method: Stages and Principles (adapted from Sein et al., 2011, p. 41)

Card Interface (artefact 1, ADR stages 1-3)

In response to the adoption of a trimester academic calendar, Griffith University encourages the adoption of a modular approach to course design. It is recommended that course profiles use modules to group and describe the teaching and learning activities. Subsequently, it has become common practice for this modular structure to be used within the course site using the Blackboard Learn content area functionality. To do this well, is not straight forward. Blackboard Learn has several functional limitations in legibility, design consistency, content arrangement and content adjustment that make it difficult to achieve quality visual design (Bartuskova, Krejcar, & Soukal, 2015). Usability analysis has also found that the Blackboard content area is inflexible, inefficient to use, and creates confusion for teaching staff regardless of their level of user experience (Kunene & Petrides, 2017). Overcoming these limitations requires levels of technical and design knowledge not typically held by teaching staff. Without this knowledge the resulting designs typically range from purely textual (e.g. the left-hand side of Figure 2) through to exemplars of poor design choices including the likes of blinking text, poor layout, questionable colour choices, and inconsistent design. While specialist design staff can and have been used to provide the necessary design knowledge to implement contextually-appropriate, effective designs, such an approach does not scale. For example, any subsequent modification typically requires the re-engagement of the design staff.

To overcome this challenge the Blackboard Learn user community has developed a collection of related solutions (Abhrahamson & Hillman, 2016; Plaisted & Tkachov, 2011) that use Javascript to package the necessary design knowledge into a form that can be used by teachers. Griffith University has for some time used one of these solutions, the Blackboard Tweaks building block (Plaisted & Tkachov, 2011) developed at the Queensland University of Technology. One of the tweaks offered by this building block – the Themed Course Table – has been widely used by teaching staff to generate a tabular representation of course modules (e.g. the right-hand side of Figure 2). However, experience has shown that the level of knowledge required to maintain and update the Themed Course Table can challenge some teaching staff. For example, re-ordering modules can be difficult for some, and the dates commonly used within the table must be manually added and then modified when copied from one offering to another. Finally, the inherently text-based and tabular design of the Themed Course Table is also increasingly dated. This was an important limitation for the Bachelor of Creative Industries. An alternative was required.

Example blackboard content area Themed course table
Figure 2 – Example Blackboard Learn Content Areas: Textual versus Themed Course Table

That alternative would use the same approach as the Themed Course Table to achieve a more appropriate outcome. The approach used by the Themed Course Table, other related examples from the Blackboard community, and the H5P authoring tool (Singh & Scholz, 2017) are contemporary examples of constructive templates (Nanard, Nanard, & Kahn, 1998). Constructive templates arose from the hypermedia discipline to encourage the reuse of design knowledge and have been found to reduce cost and improve consistency, reliability and quality while enabling content experts to author and maintain hypermedia systems (Nanard et al., 1998). Constructive templates encapsulate a specific collection of design knowledge required to scaffold the structured provision of necessary data and generate design instances. For example, the Themed Course Table supports the provision of data through the Blackboard content area interface. It then uses design knowledge embedded within the tweak to transform that data into a table. Given these examples and the author’s prior positive experience with the use of constructive templates within digital learning (Jones, 2011), the initial plan for the BCI Course Content area was to replace the Course Theme Table “template” to adopt both a more contemporary visual design, and a forward-oriented view of design for learning. Dimitriadis and Goodyear (2013) argue that design for learning needs to be more forward-oriented and consider what features will be required in each of the lifecycle stages of a learning activity. That is, as the Course Theme Table replacement is being designed, consider what specific features will be required during configuration, orchestration, and reflection and re-design.

The first step in developing a replacement was to explore contemporary web interface practices for a table replacement. Due to its responsiveness to different devices, highly visual presentation, and widespread use amongst Internet and social media services, a card-based interface was chosen. Based on the metaphor of a paper card, this interface brings together all data for a particular object with an option to add contextual information. Common practice with card-based interfaces is to embed into a card memorable images related to the card content (see Figure 3). Within the context of a course module overview such a practice has the potential to positively impact student cognition, emotions, interest, and motivation (Leutner, 2014; Mayer, 2017). A practical advantage of card-based interfaces is that its widespread use means there are numerous widely available resources to aid implementation. This was especially important to the BCI project team, as it did not have significant graphical and client-side design knowledge to draw upon.

Next, a prototype was developed to test how effectively a card-based interface would represent a course’s learning modules. An iterative process was used to translate features and existing practice from the Course Theme Table to a card-based interface. Feedback from other design staff influenced the evolution of the prototype. It also highlighted differences of opinion about some of the visual elements such as the size of the cards, the number of cards per row, and the inclusion of the date in the top left-hand corner. Eventually the prototype card interface was shown to the BCI teaching team for input and approval. With approval given, a collection of Javascript and HTML was created to transform a specifically formatted Blackboard content area into a card interface.

Figure 3 shows just two of the six different styles of card-based interface currently supported by the Card Interface. This illustrates a key feature of the original conception of constructive templates – separation of content from presentation (Nanard et al., 1998) – allowing for different representations of the same content. The left-hand image in Figure 3 and the inclusion of dates on some cards illustrates one way the Card Interface supports a forward-oriented approach to design. Initially, the module dates are specified during the configuration of a course site. However, the dates typically only apply to the initial offering of the course and will need to be manually changed for subsequent offerings. To address this the Card Interface knows the trimester weekly dates from the university academic calendar. Dates to be included on the Card Interface can then be provided using the week number (e.g. Week 1, Week 5 etc.). The Card Interface identifies the trimester a course offering belongs to and translates all week numbers into the appropriate calendar dates.

view ANother card interface
Figure 3 – Two early visualisations of the Card Interface

Despite being designed for the BCI program, the first use of the Card Interface was not in the BCI program. Instead, in late 2018 a librarian working on a Study Skills site learned of the Card Interface from a colleague. Working without any additional support, the librarian was able to use the Card Interface to represent 28 modules spread over 12 content areas. Implementation of the Card Interface in the BCI courses started by drawing on existing learning module content from course profiles. Google Image Search was used to identify visually striking images that could be associated with each module (e.g. the left-hand side of Figure 3). The Card Interface was also used on the BCI program’s Blackboard site. However, the program site had a broader purpose leading to different design decisions and the adoption of a different style of card-based interface (see the right-hand image in Figure 3).

Anecdotal feedback from BCI staff and students suggest that the initial implementation and use of the Card Interface was positive. In addition, the visual improvements offered by the Card Interface over both the standard Blackboard Content Area and the Course Theme Table tweak led to interest from other courses and programs. As of late July 2019, the Card Interface has been used in over 55 content areas in over 30 Blackboard sites. Adoption has occurred at both the program and individual course level led by exposure within the AEL L&T team or by academics seeing it and wanting it. Widespread use has generated different requirements leading to creative uses of the Card Interface (e.g. the use of animated GIFs as card images) and the addition of new functionality (e.g. the ability to embed a video, instead of an image). Requirements from another strategic project led to a customisation of the Card Interface to provide an overview of assessment items, rather than modules.

With its adoption in multiple courses and use for different purposes the Card Interface appears to have successfully encapsulated a collection of design knowledge into a form that can be readily adopted and adapted. Use of that knowledge has improved the resulting design. Contributing factors to this success include: building on existing practice; providing advantages above and beyond existing practice; and, the capability for both teaching and support staff to rapidly customise the Card Interface. Further work is required to gain greater and more objective insight into the impact of the Card Interface on the student experience and outcomes of learning and teaching.

Content Interface (artefact 2, ADR stages 1-3)

The Card Interface provides a visual overview of course modules. The next challenge for the BCI project was the design, implementation and support of the learning activities and resources that form the content of those course modules. A task that is inherently more creative, important and typically involves significantly more content. Also, a task that must be completed using the same, problematic Blackboard interface. This requirement is known to encourage teaching staff to avoid the interface by using offline documents and slides (Bartuskova et al., 2015). This is despite evidence that failing to leverage affordances of the online environment can create a disengaging student experience (Stone & O’Shea, 2019) and that course content is a significant influence on students’ perceptions of course quality (Peltier, Schibrowsky, & Drago, 2007). Adding to the difficulty, the BCI teaching staff either had limited, none, or little recent experience with Blackboard. In the case of contracted staff, they did not have access to Blackboard. This raised the question of how to support the design, implementation and re-design of effective modular, online learning resources and activities for the BCI?

Observation of, and experience with, the Blackboard interface identified three main issues. First, staff did not know how or have access to the Blackboard content interface. Second, the Blackboard authoring interface provides limited authoring functionality. For example, beyond issues identified in the literature (Bartuskova et al., 2015; Kunene & Petrides, 2017) there is no support for standard authoring functionality such as grammar checking, reference management, commenting, and version control. Lastly, once the content is placed within Blackboard the user interface is limited and quite dated. On the plus side, the Blackboard interface does provide the ability to integrate a variety of different activities such as discussion forums, quizzes etc. The intent was to address the issues while at the same time retaining the ability to use the Blackboard activities.

For better or worse, the most common content creation tool for most University staff is Microsoft Word. Anecdotal observation suggests that many staff have adopted the practice of drafting content in Word before copying and pasting it into Blackboard. The Content Interface is designed to transform Word documents into good quality online learning activities and resources (see Figure 4). This is done by using an open source converter to semantically transform Word to HTML that is then copied and pasted into Blackboard. A collection of design knowledge embedded into Javascript then transforms the HTML in several ways. Semantic elements such as activities and readings are visually transformed. All external web links are modified to open in a new tab to avoid a common Blackboard error. The document is transformed into an accordion interface with vertical list of headings that be clicked on to display associated content. This progressive reveal: allows readers to get an overall picture of the module before focusing on the details; provides greater control over how they engage with the content; and is particularly useful on mobile platforms (Budiu, 2015; Loranger, 2014).

Word Content Interface
Figure 4 – Example Module as a Word document and in the Content Interface in Blackboard

To date, the Content Interface has been used to develop over 75 modules in 13 different Blackboard sites, most of these within the seven BCI courses. Experience using the still incomplete Content Interface suggests that there are significant advantages. For example, Library staff have adopted it to create research skills modules that are used in multiple course sites. Experience in the BCI shows that sharing documents through OneDrive and using comments and track changes enables the Word documents to become boundary objects helping the course development team co-create the module learning activities and resources. Where staff are comfortable with Word as an authoring environment, the authoring process is more efficient. The resulting accordion interface offers an improvement over the standard Blackboard interface. However, creating documents with Word is not without its challenges, especially the use of Word styles and templates. Also, the extra steps required can be perceived as problematic when minor edits need to be made, and when direct editing within Blackboard is perceived to be easier and quicker, especially for time-poor teaching staff. Better integration between Blackboard and OneDrive will help. More advantage is possible when the Content Interface is further contextually customized to offer forward-oriented functionality specific to the module learning design.

Initial Design Principles (ADR stage 4)

This section engages with the final stage of the ADR process – formalisation of learning – to produce design principles that help provide actionable insight for practitioners. The following six design principles help guide the development of Contextually-Appropriate Scaffolding Assemblages (CASA) that help to sustainably and at scale share and reuse the design knowledge necessary for effective design for digital learning. The design principles are grouped using the three components of the CASA acronym.

Contextually-Appropriate

1. A CASA should address a specific contextual need within a specific activity
system
.
The highest quality learning and teaching involves the development of appropriate context-specific approaches (Mishra & Koehler, 2006). A CASA should not be implemented at an institutional level. Such top-down projects are unable to pay enough attention to contextually specific needs as they aim for a solution that works in all contexts. Instead, a CASA should be designed in response to a specific need arising in a course or a small group of related courses. Following Ellis & Goodyear (2019) the focus in designing a CASA should not be the needs of individual students, but instead on the whole activity system. That is, consideration should be given to the complex assemblage of learners, teachers, content, pedagogy, technology, organisational structures and the physical environment with an emphasis on encouraging students to successfully engage in intended learning activities. For example, both the Card and Content Interfaces arose from working with a group of seven courses in the BCI program as the result of two separate, but related, needs. While the issues addressed by these CASA apply to many courses, the ability to develop and test solutions at a small scale was beneficial. Rather than a focus primarily on individual learners, the solutions were heavily influenced by an analysis of the available tools (e.g. Blackboard Tweaks, Office365), practices (e.g. modularisation and learning activities described in course profiles), and other components of the activity systems.

2. CASA should be built using and result in generative technologies. To maximise and maintain contextual appropriateness, a CASA must be able to be designed and redesigned as easily as possible. Zittrain (2008) labels technologies as generative or sterile. Generative technologies have a “capacity to produce unanticipated change through unfiltered contributions from broad and varied audiences” (Zittrain, 2008, p. 70). Sterile technologies prevent this. Generative technologies enable convivial systems where people can be “actively engaged in generating creative extensions to the artefacts given to them” (Fischer & Girgensohn, 1990, p. 183). It is the end-user modifiability of generative technology that is crucial to knowledge-based design environments and enables response to unanticipated, contextual requirements (Fischer & Girgensohn, 1990). Implementing CASA using generative technologies allows easy design for specific contexts. Ensuring that CASA are implemented as generative technologies enables easy redesign for other contexts. Generativity, like other technological affordances, arises from the relationship between the technology and the people using the technology. Not only is it necessary to use technology that is easier to modify, it is necessary to be able to draw upon appropriate technological skills. This could mean having people with those technological skills available to educational design teams. It could also mean having a network of intra- and inter-institutional CASA users and developers collaboratively sharing CASA and the knowledge required for use and development; like that available in the H5P community (Singh & Scholz, 2017).

For example, development of the Card and Content Interfaces was only possible due to Blackboard Learn supporting the embedding of Javascript. The value of this generative capability is evident through the numerous projects (Abhrahamson & Hillman, 2016; Plaisted & Tkachov, 2011) from the Blackboard community that leverage this capability; a capability that has been removed in Blackboard’s next version LMS, Ultra. The use of Office365 by the Content Interface illustrates the rise of digital platforms that are generative and raise questions that challenge how innovation through digital technologies are enabled and managed (Yoo, Boland, Lyytinen, & Majchrzak, 2012). Using the generative jQuery library to implement the Content Interface’s accordion enables modification of the accordion look and feel through use of jQuery’s theme roller and library of existing themes. The separation of content from presentation in the Card Interface has enabled at least six redesigns for different purposes. This work was possible because the BCI development team had ready access to the necessary technological skills and was able to draw upon a wide collection of open source software and online support.

3. CASA development should be strategically aligned and supported. Services to support design for learning within Australian universities are limited and insufficient for the demand (Bennett et al., 2017). Services capable of supporting the development of CASA are likely to be more limited. Hence appropriate decisions need to be made about how and what CASA are designed, re-designed and supported. Resources used to develop CASA are best allocated in line with institutional strategic projects. CASA development should proceed with consideration to the “manageably small set of particularly valued activity systems” (Ellis & Goodyear, 2019, p. 188) within the institution and be undertaken with institutionally approved and supported generative technologies. For example, the Card and Content Interfaces arose from an AEL strategic project. Both interfaces were focused on providing contextually-appropriate customization and support for the institutionally important activity system of creating modular learning activities and resources. Where possible these example CASA have used institutionally approved digital technologies (e.g. OneDrive and Blackboard). The sterile nature of existing institutional infrastructure has made it necessary to use more generative technologies (e.g. Amazon Web Services) that are neither officially approved or supported. However, the approach used does build upon an approach from an existing institutional approved technology – Blackboard Tweaks (Plaisted & Tkachov, 2011).

Scaffolding

4. CASA should package appropriate design knowledge to enable (re-)use by teachers and students. Drawing on ideas from constructive templates (Nanard et al., 1998), CASA should package the diverse design knowledge required to respond to a contextually-appropriate need in a way that this design knowledge can be easily reused in different instances. CASA enable the sustainable reuse of contextually applied design knowledge in learning activity systems and subsequently reduce cost and improve quality and consistency. For example, the Card Interface combines the knowledge from web design and multimedia learning research (Leutner, 2014; Mayer, 2017) in a way that has allowed teaching staff to generate a visual overview of the modules in numerous course sites. The Content Interface combines existing knowledge of the Microsoft

Word ecosystem with web design knowledge to improve the design, use and revision of modular content.

5. CASA should actively support a forward-oriented approach to design for learning.

To “thrive outside of the protective niches of project-based innovation” (Dimitriadis & Goodyear, 2013, p. 1) the design of a CASA must not focus only on initial implementation. Instead, CASA design must explicitly consider and include functionality to support the configuration, orchestration, and reflection and re-design of the CASA. For example, the Card Interface leverages contextual knowledge to enable dates to be specified independent of the calendar to automate re-design for subsequent course offerings. As CASA tend to embody a learning design, it should be possible to improve each CASA’s support for orchestration by implementing checkpoint and process analytics (Lockyer, Heathcote, & Dawson, 2013) specific to the CASA’s embedded learning design.

Assemblages

6. CASA are conceptualised and treated as contextual assemblages. Like all technologies, CASA are assemblies of other technologies (Arthur, 2009) where technologies are understood to include techniques such as organisational processes and pedagogies, as well as hardware and software. But a contextual assemblage is more than just technology. It includes consideration of and connections with the policies, practices, funding, literacies and discourse across levels from societal and down through sector, organisational, personal, individual, formal and informal. These are elements that make up the mess and nuance of the context, where the practice of educational technology gets complex (Cottom, 2019). A CASA must be generative in order to be designed and re-designed to respond to this contextual complexity. A CASA needs to be inherently heterogeneous, ephemeral, local, and emergent. A need that is opposed and ill-suited to the dominant rational system view underpinning common digital learning practice which sees technologies as planned, structured, consistent, deterministic, and systematic. Instead, connecting back to design principle one, CASA should be designed in recognition of and as the importance and complex intertwining of the human, social and organisational elements in any attempt to use digital technologies. It should play down the usefulness of distinctions between developer and user, or pedagogy and technology. For example, the Card Interface does not use the Lego approach to assembly that informs the Next Generation Digital Learning Environment (NGDLE) (Brown, Dehoney, & Millichap, 2015) and underpins technologies such as the Learning Tools Interoperability (LTI) standard. Instead of combining clearly distinct blocks with clearly defined connectors the Card and Content Interface is intertwined with and modifies the Blackboard user interface to connect with the specifics of context. Suggesting that the Lego approach is useful, perhaps even necessary, but not sufficient.

Conclusions, Implications, and Further Work

Universities are faced with the strategically important question of how to sustainably and at scale leverage the knowledge required for effective design for digital learning. The early stages of an Action Design Research (ADR) process has been used to formulate one potential answer in the form of six design principles encapsulated in the idea of Context-Appropriate Scaffolding Assemblages (CASA). To date, the ADR process has resulted in the development and use of two prototype CASA within a suite of 7 courses and within 6 months their subsequent adoption in another 24 courses. CASA draw on the idea of constructive templates to capture diverse design knowledge in a form that enables use of that knowledge by teachers and students to effectively address contextually specific needs. By adopting a forward-oriented view of design for learning CASA offer functionality to support configuration, orchestration, and reflection and re-design in order to encourage on-going use beyond the protected project niche of initial implementation. The use of generative technologies and an assemblage perspective enables CASA development to be driven by and re-designed to fit the specific needs of different activity systems and contexts. Such work will be most effective when it is strategically aligned and supported with the aim of supporting and refining institutionally valued activity systems.

Use of the Card and Content Interfaces within and beyond the original project suggest that these CASA have successfully encapsulated the necessary design knowledge to address shortcomings with current practice and had a positive impact on the quality of the digital learning environment. But it’s early days. These CASA can be improved by more completely following the CASA design principles. For example, the Content Interface currently offers only generic support for module design. Significantly greater benefits would arise from customising the Content Interface to support specific learning designs and provide contextually appropriate forward-oriented functionality. More experience is needed to provide insight into how this can be done effectively. Further work is required to establish if, how and what impact the use of CASA has on the quality of the learning environment and the experience and outcomes of both learning and teaching. Further work could also explore the questions raised by the CASA design principles about existing digital learning practice. The generative principle raises questions about whether moves away from leveraging the generativity of web technology – such the design of Blackboard Ultra and the increasing focus on mobile apps – will make it more difficult to integrate contextually specific design knowledge? Do reported difficulties accessing student engagement data with H5P activities (Singh & Scholz, 2017) suggest that the H5P community could fruitfully pay more attention to supporting a forward-oriented design approach? Does the assemblage principal point to potential limitations with some conceptualisations and implementation of next generation of digital learning environments?

References

Abhrahamson, A., & Hillman, D. (2016). Cutomize Learn with CSS and Javascript injection. Presented at the BBWorld 16, Las Vegas, NV. Retrieved from https://community.blackboard.com/docs/DOC-2103

Alhadad, S. S. J., Thompson, K., Knight, S., Lewis, M., & Lodge, J. M. (2018). Analytics-enabled Teaching As Design: Reconceptualisation and Call for Research. Proceedings of the 8th International Conference on Learning Analytics and Knowledge, 427–435.

Arthur, W. B. (2009). The Nature of Technology: what it is and how it evolves. New York, USA: Free Press.

Bartuskova, A., Krejcar, O., & Soukal, I. (2015). Framework of Design Requirements for E-learning Applied on Blackboard Learning System. In M. Núñez, N. T. Nguyen, D. Camacho, & B. Trawiński (Eds.), Computational Collective Intelligence (pp. 471–480). Springer International Publishing.

Bennett, S., Agostinho, S., & Lockyer, L. (2017). The process of designing for learning: understanding university teachers’ design work. Educational Technology Research & Development, 65(1), 125–145.

Bennett, S., Lockyer, L., & Agostinho, S. (2018). Towards sustainable technology-enhanced innovation in higher education: Advancing learning design by understanding and supporting teacher design practice. British Journal of Educational Technology, 49(6), 1014–1026.

Bezuidenhout, A. (2018). Analysing the Importance-Competence Gap of Distance Educators with the Increased Utilisation of Online Learning Strategies in a Developing World Context. International Review of Research in Open and Distributed Learning, 19(3), 263–281.

Brown, M., Dehoney, J., & Millichap, N. (2015). The Next Generation Digital Learning Environment: A

Report on Research (p. 11). Louisville, CO: EDUCAUSE.

Budiu, R. (2015). Accordions on Mobile. Retrieved July 18, 2019, from Nielsen Norman Group website: https://www.nngroup.com/articles/mobile-accordions/

Cottom, T. M. (2019). Rethinking the Context of Edtech. EDUCAUSE Review, 54(3). Retrieved from
https://er.educause.edu/articles/2019/8/rethinking-the-context-of-edtech

Dahlstrom, E. (2015). Educational Technology and Faculty Development in Higher Education. Retrieved from ECAR website: https://library.educause.edu/resources/2015/6/educational-technology-and-faculty-development-in-higher-education

Dekkers, J., & Andrews, T. (2000). A meta-analysis of flexible delivery in selected Australian tertiary institutions: How flexible is flexible delivery? In L. Richardson & J. Lidstone, (Eds.), Proceedings of

ASET-HERDSA 2000 Conference, (pp. 172-182)

Dimitriadis, Y., & Goodyear, P. (2013). Forward-oriented design for learning: illustrating the approach. Research in Learning Technology, 21, 1–13.

Ellis, R. A., & Goodyear, P. (2019). The Education Ecology of Universities: Integrating Learning,

Strategy and the Academy. Routledge.

Fischer, G., & Girgensohn, A. (1990). End-user Modifiability in Design Environments. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 183–192.

Goodyear, P. (2005). Educational design and networked learning: Patterns, pattern languages and design practice. Australasian Journal of Educational Technology, 21(1). https://doi.org/10.14742/ajet.1344

Goodyear, P. (2015). Teaching As Design. HERDSA Review of Higher Education, 2, 27–59.

Goodyear, P., & Dimitriadis, Y. (2013). In medias res: reframing design for learning. Research in

Learning Technology, 21, 1–13.

Gregory, M. S. J., & Lodge, J. M. (2015). Academic workload: the silent barrier to the implementation of technology-enhanced learning strategies in higher education. Distance Education, 36(2), 210–230.

Jones, D. (2011). An Information Systems Design Theory for E-learning (PhD, Australian National University). Retrieved from https://openresearch-repository.anu.edu.au/handle/1885/8370

Kunene, K. N., & Petrides, L. (2017). Mind the LMS Content Producer: Blackboard usability for improved productivity and user satisfaction. Information Systems, 14.

Leutner, D. (2014). Motivation and emotion as mediators in multimedia learning. Learning and

Instruction, 29, 174–175.

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing Pedagogical Action: Aligning Learning Analytics With Learning Design. American Behavioral Scientist, 57(10), 1439–1459.

Loranger, H. (2014). Accordions for Complex Website Content on Desktops. Retrieved July 18, 2019, from Nielsen Norman Group website: https://www.nngroup.com/articles/accordions-complex-content/

Mathes, J. (2019). Global quality in online, open, flexible and technology enhanced education: An analysis of strengths, weaknesses, opportunities and threats. Retrieved from International Council for Open and Distance Education website:
https://www.icde.org/knowledge-hub/report-global-quality-in-online-education

Mayer, R. E. (2017). Using multimedia for e-learning. Journal of Computer Assisted Learning,
33(5), 403–423.

Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054.

Mor, Y., Craft, B., & Maina, M. (2015). Introduction – Learning Design: Definitions, Current Issues and Grand Challenges. In M. Maina, B. Craft, & Y. Mor (Eds.), The Art & Science of Learning Design (pp. ix–xxvi). Rotterdam: Sense Publishers.

Nanard, M., Nanard, J., & Kahn, P. (1998). Pushing Reuse in Hypermedia Design: Golden Rules, Design

Patterns and Constructive Templates. 11–20. ACM.

Peltier, J. W., Schibrowsky, J. A., & Drago, W. (2007). The Interdependence of the Factors Influencing the Perceived Quality of the Online Learning Experience: A Causal Model. Journal of Marketing Education; Boulder, 29(2), 140–153.

Plaisted, T., & Tkachov, N. (2011). Blackboard Tweaks: Tools for Academics, Designers and Programmers. Retrieved July 2, 2019, from http://tweaks.github.io/Tweaks/index.html

Roberts, J. (2018). Future and changing roles of staff in distance education: A study to identify training and professional development needs. Distance Education, 39(1), 37–53.

Sein, M. K., Henfridsson, O., Purao, S., & Rossi, M. (2011). Action Design Research. MIS Quarterly,
35(1), 37–56.

Singh, S., & Scholz, K. (2017). Using an e-authoring tool (H5P) to support blended learning: Librarians’ experience. In H. Partridge, K. Davis, & J. Thomas (Eds.), Me, Us, IT! Proceedings ASCILITE2017: 34th International Conference on Innovation, Practice and Research in the Use of Educational Technologies in Tertiary Education (pp. 158–162).

Stone, C., & O’Shea, S. (2019). Older, online and first: Recommendations for retention and success.

Australasian Journal of Educational Technology, 35(1). https://doi.org/10.14742/ajet.3913

Yoo, Y., Boland, R. J., Lyytinen, K., & Majchrzak, A. (2012). Organizing for Innovation in the Digitized World. Organization Science, 23(5), 1398–1408.

Zittrain, J. (2008). The Future of the Internet–And How to Stop It. Yale University Press.

Testing out h5p interactives and tracking

Feel free to play around with the following h5p interactive (one of the examples provided on the h5p site). I’m testing out if and how it tracks people’s engagement with the interactive. What does it do when someone who doesn’t have an account on this blog (and/or doesn’t login) interacts? What data can I see?

By default it appears that h5p/wordpress doesn’t really provide anything useful and a quick search didn’t reveal any possibilities.

This post provides a bit more detail on broader possibilites with xAPI and h5p, including this WordPress plugin (h5pxAPIkatchu). This plugin catches all the xAPI statements emitted by h5p interactives and stores them in the WordPress database for downloading for analysis.

[h5p id=”10″]

Exploring knowledge reuse in design for digital learning

This post continues an on-going exploration of knowledge reuse in design for digital learning. Previous posts (one and two) started the exploration in the context of developing an assemblage to help designers of web-based learning environments create a card interface (see Figure 1). Implementing such a design from scratch requires a diverse collection of knowledge that is beyond most individuals. It is hoped that packaging that knowledge into an assemblage of technologies will allow for that knowledge to be used and reused (within Blackboard 9.1) by more people and subsequently have a positive impact on the learning environment and experience.

The card inteface is a simple example of this work. The requirements of the card interface are fairly contained and pre-defined. The next challenge is to explore if and how this can be expanded to something more difficult and open-ended.