An integrated OLE

The following is the next part of the evaluation section of chapter 5 of my thesis. It seeks to evaluate how well the Webfuse system fulfilled the guideline of “an integrated online learning environment” for the period 2000-2004. This is only the first section which talks about the provision of course sites. Subsequent sections will talk about feature adoption, staff and student usage of course sites, and staff and student usage of Wf applications.

An integrated OLE

As described in Chapter 4 (see Table 4.3 and Section 4.3.2) a primary aim for Webfuse was to provide a single, integrated interface for all activities and resources required for learning and teaching. Work from 2000 through 2004, especially from 2001 with the expanded Infocom web team, was aimed at expanding on earlier work and addressing known limitations. Of the lessons learned from the 1997 through 1999 (Section 4.7) work on Webfuse, two are related to this design guideline:

  1. the need for better support for the concept of a course; and,
    This need was address through the design and implementation of the default course site approach in 2001 (Section 5.3.5). This included the development of a number of new page types that offered specific course related services and the ability for the Infocom web team to automatically create course sites for all courses offered by Infocom.
  2. the need for better integration with the requirements & practice of staff and students.
    The main mechanism for addressing this lesson was the adoption of an adopter-focused (Section 5.3.1) and emergent (Section 5.3.2) development process. A process aimed at rapidly addressing the needs of both staff and students within the one integrated system (Webfuse). This included both the idea of the default course sites but also the additional interactive web applications developed during this period.

The overall success of this work can be illustrated by a quote which Danaher, Luck and McConachie (2005) cite as coming from the Infocom annual report for 2003

[t]he best thing about teaching and learning in this faculty in 2003 would be the development of technologically progressive academic information systems that provide better service to our students and staff and make our teaching more effective. Webfuse and MyInfocom development has greatly assisted staff to cope with the complexities of delivering courses across a large multi-site operation.

The following sub-sections provide additional support for the success of work from 2000 through 2004 in implementing an integrated, online learning environment. The first three sub-sections match those used in Section 4.5.2 and enable a comparison between what happened in 1997 through 1999 and 2000 through 2004. These three sections examine: the number of course sites created; the spread of feature adoption within those course sites; and, how much the course sites were used by staff and students. The final sub-section covers usage of the interactive web applications that were first implemented in 2000.

Course sites

The aim of this sub-section is to briefly comment on the number of course sites that were created. From the start of Webfuse, all courses offered by the faculty had course websites created. This practice continued with the default course sites, the major difference is that the actual creation of the course sites was automated. While the creation of default course sites was entirely automated, the gathering of all the necessary data from various institutional sources in order to create the course sites was not entirely automated. Consequently, the resources required were not minimised as much as possible. This problem of limited integration is picked up again in section 5.4.3.

Before the default course site approach was introduced there were always a small number of course sites created without using Webfuse, this practice also continued afterwards in terms of the real course sites. Table 5.8 shows the details of the number of course sites created in Webfuse from 1997 through 2009. It also shows the number and percentage of courses that were created outside of Webfuse.

Table 5.8 – Comparison between default course sites and real course sites (2002-2009)
* – archives of course sites for 1 term in 1998 and 2000 have been lost
Year All courses Real course
sites
% Real course sites
1997 110 3 2.7%
1998* 142 7 4.9%
1999 192 8 4.2%
2000 * 176 13 7.4%
2001 244 16 6.6%
2002 312 27 8.7%
2003 302 29 9.6%
2004 328 16 4.9%
2005 300 15 5.0%
2006 297 15 5.1%
2007 251 4 1.6%
2008 225 2 0.9%
2009 211 0 0%

During 2000 through 2003 all of the non-Infocom faculties at CQU used the institutionally supported LMS, WebCT. At the end of 2003, when CQU was moving to Blackboard as its institutionally support LMS, there were 141 WebCT courses to be migrated to Blackboard. WebCT had been the official institutional LMS at CQU since 1999 and was primarily used for courses other than those offered by Infocom.

Table 5.9 offers a comparison between Infocom and the non-Infocom parts of CQU in terms of course sites, courses and student/courses during 2003. The “# Course Sites” and “# Courses” columns in Table 5.9 shows the number of unique courses or course websites. A number of CQU courses are offered more than once a year, each offering is not shown in Table 5.9. For example, for the 940 non-Infocom courses offered in 2003, there were a total of 1322 course offerings. Lastly, the “# Courses” column in Table 5.9 only shows those courses for which students enrolled.

Table 5.9 – Infocom and non-Infocom comparison for 2003
Faculty # Course sites # Courses # Student/Courses
Infocom 173 150 35,291
Non-Infocom 141 940 76,290

Table 5.9 shows that only about 15% of the non-Infocom courses had course websites, while all Infocom courses had course websites on WebCT. In the same year, Table 5.9 shows that there were more Webfuse course sites than there were Infocom courses. The additional courses come from a number of sources including courses: with no enrolments in 2003; classed as non-Infocom, but taught primarily to Infocom students; and a single test course used during the evaluation process that selected Blackboard as the institutional LMS. So, by 2003 all Infocom courses would have a default course site, while little more than 15% of non-Infocom courses had a course website.

Due to the earlier Webfuse practice of manually creating course websites, the introduction of the default course site approach did not increase the percentage of Infocom courses with course websites. However, it did expand the default level of information and services available through those course sites. In the last term before implementation of the default course site approach, a standard Webfuse course site provided a course synopsis, a link to the course profile, details about course coordinator, and if used, a web-based archive of a mailing list. The default course site design described in Section 5.3.5 added a range of additional information (e.g. assessment item details) and services (e.g. each course had a course barometer). A manually created default site had a minimum of three pages, a default course site had a minimum of 10. In both types of course sites, academics could then choose to add further to the initial course website.

Table 5.10 shows for all Webfuse course sites from 1997 through 2009, the number of Webfuse courses, the number of Webfuse pages for those sites, the number of Webfuse page types used, and an average number of pages per course. From the introduction of the default course sites in the second half of 2001 there is a steady increase in the average number of pages per course from 22.7 to 30.8. However, there are contextual factors to consider before reading too much into the figures from Table 5.10. These factors are discussed below and include:

  • the impact of two Webfuse courses created by the Webfuse designer;
  • changes in the make up of the default course sites in 2004; and
  • related organisational changes in 2004.
Table 5.10 – Number of pages and page types in Webfuse course sites (1997-2004)
* – courses for missing term not included
Year Webfuse courses Number of Webfuse pages Pages/Course
1997 110 2767 25
1998 * 142 2926 21
1999 192 4375 23
2000 * 176 3046 17
2001 244 5532 23
2002 312 9094 29
2003 302 9295 31
2004 328 6023 18
2005 300 6535 22
2006 297 5988 20
2007 251 6038 24
2008 225 5347 24
2009 211 5610 27

As described in Section 4.5.5 two courses designed by the Webfuse creator were significantly different than the standard Webfuse course. For example, in 1999 these two courses accounted for 47.2% of hits on course site pages and 75.4% of page updates. In 1997, just one of these courses accounted for 63.8% of the total pages for Webfuse courses. From 2000 through 2004, the creation of web-based archives of course mailing lists was done using Webfuse page types. In the last term of 2003, 44% of Webfuse pages were web-based archives. In 2004, CQU’s central IT division introduced a centralised service for Web-based archives of mailing lists. Consequently, the default course site was modifed to use this central service. Similarly, a small number of courses used the Lecture page type to create online lectures. The Lecture page type would create one LectureSlide page per slide in a lecture. In the first term of 2001, the LectureSlide page type accounted for 44% of page types. Table 5.12 is a “normalised” version of Table 5.10. It shows the same data as Table 5.10 modified to reduce the impact of these unusual factors.

Table 5.12 – Normalised pages and page types in Webfuse course sites (1997-2009)
Year Webfuse courses Number of Webfuse pages Pages/Course
1997 109 1139 10
1998 * 140 982 7
1999 188 1654 9
2000 * 174 1135 7
2001 241 3192 13
2002 310 5395 17
2003 300 4898 16
2004 326 5324 16
2005 298 5296 18
2006 295 5422 18
2007 249 4782 19
2008 223 4097 18
2009 208 4012 19

The larger page per course ratio for 1997 shown in Table 5.12 was caused by a larger, manually created default site being used for the first year of Webfuse’s operation. To reduce workload, the default course site that was manually created from 1998 through the first term in 2001, was somewhat reduced in the number of pages. As shown in Table 5.12 the introduction of the default course site approach in 2001 saw almost a doubling of the pages per course ratio. In 2002, the first full year’s operation of the default course site there is another 25% increase in the pages per course ratio. This level is essentially maintained for the following seven years with some minor fluctuations. Even though support and development of the default course site idea was significantly reduced after 2004. The fluctuations in the pages per course ratio after 2002 generally arise from increased or decreased modifications on top of the default course site by academics. This behaviour is examined in more detail in a subsequent section.

In summary, the default course site approach provided an expanded range of services and functionality over the previously manually created default course site approach without the need for significant amounts of manual editing of web pages and the subsequent increased chance of human error. As a result, by 2003 all of Infocom’s courses had course websites compared to only about 15% of courses offered by the rest of CQU. The next sub-sections examine the features available within those websites and how much they were used by students and staff.

References

Danaher, P. A., J. Luck, et al. (2005). “The stories that documents tell: Changing technology options from Blackboard, Webfuse and the Content Management System at Central Queensland University.” Studies in Learning, Evaluation, Innovation and Development 2(1): 34-43.

Default course sites and wizards – version 2.0

This is the second version of an earlier post. As I wrote more of the chapter I felt the need to revisit and expand the idea (though the following is still at a rough draft stage). The following describes the rationale and implementation behind the implementation of the default course sites approach within Webfuse. This approach was used at CQU from 2001 through 2009. It has similarities with what Mark Smithers calls a minimium online presence policy (MOPP), though (I think) my take on the value of MOPP differs a little from Mark’s. Not surprising given the following.

As part of writing the thesis I am having to abstract out what I felt was good/important/interesting about the Webfuse work. The “default course site” approach, at least in terms of the principles behind it rather than the specific implementation, is probably going to be part of that. I still have to work on this some more, hopefully in coming weeks, but I’ve put some early thoughts out there.

Hopefully I can work on this argument soon.

Default course sites and wizards

As described in chapter 4, the initial assumptions built into Webfuse was that a course website would simply be an empty page. From this single, empty page it was assumed that each individual teaching staff member would then draw on the variety of page types (hypermedia templates) provided as course website building blocks by Webfuse to construct their course website. The majority, if not all, Learning Management Systems (LMS), are built upon the same assumption. This assumption matches the purpose of CQU’s initial adoption of an official LMS, described by Sturgess and Nouwens (2004) as

to enable teaching staff to develop and manage online courses with little professional support.

The assumption is that the quality of the LMS application is such that teaching staff can create course websites on their own, with a minimum of support. The early Webfuse experience described in Chapter 4 illustrated that only a very small percentage of academic staff engaged significantly in this task.

The majority of academic staff did not have qualifications in teaching and limited experience in the use of technology to improve learning and teaching. In addition, few had large amounts of time to invest in learning these skills due to other work demands, particularly research. In addition, it became obvious that a significant percentage of the tasks associated with creating a course website were fairly low-level tasks, often involving reuse of information and resources already provided. Some of these tasks (e.g. uploading 12 weeks of lectures or study guide chapters) require repetitive manual work increasing the chance of human error. Lastly, there was little or no integration of institutional practice with Webfuse. For example, CQU has a fixed semester schedule where specific weeks occur on specific dates. Initially, academic staff would have to manually enter these dates into course websites when they used them. Another example would be the institutional production of print-based study guides not providing a good set of PDF files, let alone automatically integrating it into the course website.

These observations led to the Webfuse practice where support staff created initial default course sites by manually editing the initial empty course sites and uploading standard information (e.g. course synopsis, staff contact details etc). However, these course sites were of limited quality, failed to encourage further enhancement from academic staff, and required significant workload from faculty administrative staff. It is within this context that it became necessary to:

  • better support the concept of a course; and
    At this stage, a Webfuse course site was created through the combination of page types that were also useful for general web publishing, the only exceptions being the assignment management and quiz page types. As a result, the page types did not assume or use any knowledge of CQU courses and associated data. For example, the home page for a course site was initially implemented by a general purpose information distribution page. If it was felt good practice to include the course synopsis on the course home page it had to be manually entered by a person.
  • encourage greater engagement with course sites from academic staff.
    As described in Section 5.3.1, the assumptions behind the initial Webfuse course site creation process were more developer focused. It was assumed that provision of a collection of tools would result in a majority of academics learning the new skills and effectively engaging in the additional task of the design and creation of a course site.

During 1999 an initial attempt at addressing these problems was commenced as the “Wizard” project. Briefly described in Jones and Lynch (1999) the Wizard project planned to provide an interface based on the Wizards common to Widows programs of the late 1980s. Such an interface would walk the academic through a series of questions about their course, the provided answers would be combined with the Webfuse page types to create a course site. A particular focus of this plan was an adopter-focused development approach, however, due to organisational uncertainty and limited development resources, this project did not move beyond the prototype and planning stage.

The next attempt to address this problem was the creation of an automated and expanded default course site approach for the second term of 2001. This coincided with the broader roll out of CQU’s new student records system. This new default course site approach was made possible by the expanded Infocom web team, the improvements in the Webfuse process and code-base described in previous sections, and was the primary task of the author during the first half of 2001. The automated and expanded default course site approach evolved to consist of a number of components or capabilities that are described in more detail in the following sub-sections. The components or capabilities included:

  • An expanded default course site;
    As described in more detail below, the single empty page default course site was replaced with a much expanded site consisting of five separate sections, containing a range of course related data and services within a re-designed interface. Each offering of every course offered by Infocom would have a new default course site.
  • Course specific page types;
    Once the default course site structure was designed, specific page types were developed to implement the functionality required for each individual page. These page types – described in more detail below – were told which course offering the course site were for and from there drew on existing data sources to provide the necessary services.
  • Automatic creation;
    The creation and population of all default course sites was performed by running a script that, given the details of a specific term and year, would existing information, Webfuse page types, and related scripts and create default course sites for all courses offered by Infocom.
  • A copying process;
    Teaching staff could further modify the default course site by adding additional course information and services. Rather than re-add these additional features for each subsequent term a copying process was developed by which staff could specify what they would like to copy to the new course site.
  • Support for a real course site; and
    There continued to be a number of teaching staff who wished to create their own course website with other tools. Each default course site had support for an optional “real” course site This added another area of the default course site in which the staff member could place their own course site.
  • Support for on-going changes.
    While initially implemented with a single default course site for all Infocom courses, the default course site approach provided a platform for on-going change.

Expanded default course site

The starting point was the design of a common default course site that would be used for all courses. The default course site would use the same structure and look, however, the course site page types would insert information and services relevant to the specific course. The initial default site used a simple hierarchical structure represented in Figure 5.1. A course home page formed the top of the hierarchy with five sub-sections and the hierarchical structure could continue under each of these sub-sections.

Webfuse default course site structure

The five sub-sections were:

  1. Updates;
    The updates section provided a function that allowed teaching staff to provide and store course wide updates or announcements. The titles and post dates of the most recent updates were also visible from the home page.
  2. Study schedule;
    This section provided a week by week breakdown of the course, its topics, content and assessment.
  3. Assessment;
    Provided access to details about the course assessment. By default this would summarise for each assessment item, the title, due date and percentage of the overall assessment.
  4. Resources; and
    All remaining course resources and services were made available via this section. By default this included a link to the course profile (syllabus) document, details of the course textbook(s) (including a link to the university bookshop, and, if used, a discussion forum or mailing list.
  5. Staff.
    This provided both the personal details of the staff teaching the course as well as an area restricted to the teaching staff used for discussion or sharing resources. The personal details of teaching staff included name, contact details and where available a photo.

The initial look and feel for the default course sites is shown in Figure 5.2, which is the home page for one of the course sites from July 2001. The course home page, like each page in the default course site, had three main sections:

  1. header;
    A common header for an entire course site that would show a range of navigation links, branding and administrative information. Navigation links included breadcrumbs to indicate current location and links to other common institutional services such as the student portal, the faculty home page and services such as search, help and feedback. Branding information included common colours and the name of the institution and faculty. Administrative information included the months during which the period ran and the name of the course.
  2. body; and
    The main area of a page and intended to include information specific to the page type. The CourseHome page included a course synopsis and the most recent course updates.

  3. footer.
    A small area for various administrative information such as the Webfuse page update link, details of when and how last updated the page, various disclaimers, and generic contact details.

Webfuse default course site home page

Course page types

For each of the pages identified in the initial design of the default course site and shown in Figure 5.1 a new Webfuse page type was created. These page types were designed to use and provide greater support for the concept of the course and provide services of value to staff and students. When used, each page type would be provided with the course, period and year and draw on existing institutional data sources to provide the necessary content for the body of the page. The following describes the details of what each course page type was designed to produce.

The CourseHome page type was not only responsible for producing the home page for a course as shown in Figure 5.2. It also provided the teaching staff with three additional services. These services were:

  1. some control over the course site;
    When initially created a course site would consist only of the home and updates page. This was necessary as not all course information was initially available due to existing institutional timelines. Once the other required information was ready, the complete course site could be built. This could be done automatically by the Infocom web team or by the teaching staff via the CourseHome page type.
  2. access to information about course enrolments; and
    The staff portal described in section 5.3.6 was not developed until 2002. Prior to this, academic staff could use the CourseHome page type to find out how many students were enrolled in a course and at which campuses.
  3. the ability to add students.
    Students and staff were allocated to a course site based on institutional databases such as the student records system and the Webfuse teaching responsibilities database. Occasionally, these systems – especially the Peoplesoft system during 2001 – were not up to date or staff may wish to provide access to the course site for additional students. The CourseHome page type allowed staff to temporarily enrol students, at least in the Webfuse database.

The RSSUpdates page type provided the ability for the teaching staff to create and maintain a list of course wide updates or announcements. As well as generating HTML representations of the announcements on the Course Updates and Course Home pages, the page type also generated an RSS file that students could access via a newsreader. The student portal also used the RSS file to show students all of the most recent updates in all of their courses.

The CourseSchedule page type provided support for managing a simple study schedule. For each week, the academic could specify the study material, tasks and content the students would be using. Such schedules were a common part of CQU practice and all course profiles (aka course syllabi) contained such a schedule. It was planned for the CourseSchedule page type to use this existing data source to generate the course schedule. However, access to this data was not possible – initially because profiles were produced manually as Word documents and then because Webfuse was not allowed to access the profile system – and consequently course schedules had to be re-created manually.

The CourseAssessment page type would produce a table containing information about all approved assessment items for the course. This information was pulled from a database and included the name, due date and percentage contribution to final grade of each item. Initially, this information was manually transcribed from the course profiles (in Word documents) into a Webfuse database. Eventually, this information was available from a central database. In addition, academics could choose to add a “sub-page” for each assessment item. This item page was used to provide additional details about the assessment item.

The CourseResources page type was designed to provide students with access to all course related resources and services. Some of these resources, such as a link to the course profile and details about any set course texts, were automatically generated. The remaining resources were manually added, deleted and moved by the teaching staff using the page update process.

The CourseStaff page type produced a page that provided a list that included personal details of each teaching staff member associated with the course. This list would include, where available, a photo of the staff member, a link to their home page, and their contact details. The page would also automatically create a “staff only” section underneath the staff page. This “staff only” section was automatically restricted to teaching staff and was used to share information and services restricted to staff.

Automatic creation

Prior to the default course sites, Webfuse courses sites within Infocom were created through manual editing of pages by faculty administrative staff. To remove this workload, and prevent it simply being transferred to academic staff, it was envisaged that the default course sites would be automatically created. The implementation for this automatic creation depended on two artefacts: a collection of identified data sources; and the CourseList page type to create the course sites.

The identified data sources were a collection of databases and files that provided all of the necessary information required to construct the default course sites. This include a range of information including: which staff were teaching which course, the name of the course and where it was being offered, the weeks that made up the term, where the course profile PDFs were located, information about individual assignment items, information about selected textbooks, staff websites and photos, and a range of other information. This information was spread across a variety of institutional systems (e.g. Webfuse itself, student records system etc) and some of the information was not in a machine-readable format. A ever-changing range of workarounds were necessary to convert this information into a format that could be accessed by Webfuse. The inability to access some of this information remained the largest limit on the ability to extend the default course sites.

The CourseList page type was used to automatically create the default course sites and produce a collection of web pages pointing to the course sites. Once the necessary data for a new term was available through the identified data sources, creating the default course sites involved creating a new CourseList page, providing the period and year, and submitting the page. This would display a list of the available courses and provide options about how the sites would be created. After the options were chosen, another submission of the page would result in the creation of the matching default course sites.

A copying process

After a number of terms of using the default course site, it became evident that many courses were made up of the default course site, plus some additional material and services added by the teaching staff. This additional material often remained the same from term to term with only minor changes. Rather than expect academics to use Webfuse to recreate this additional material, it became more efficient to provide a copy process. A process by which academics could specify which parts of the additional material from an old course site they wished to copy to the new course site, and the actual copying of that material.

Support for real course site

The default course site process did create some initial and on-going disquiet around questions of academic independence, consistency and institutional identity. This was particularly a problem for academic staff wanting to create their own course sites, typically using a variety of web or HTML editors or applications commonly used to create websites. This was a particularly common practice amongst academic staff from the multimedia discipline. The ability for an academic to create their own course site was often important in terms of the academics professional identity, but also because of the need for the site design to demonstrate principles associated with the course content.

This raised an interesting and difficult question of how to balance the faculty’s needs to ensure a minimum standard of online presence, with the individual academics’ disciplinary and identity needs. The solution adopted by Webfuse was to add the notion of a real course site to the default course site. The CourseHome page type was modified to include a check box, which when selected would create a real course site. The real course site was essentially an empty directory under the home page of the default course site. The academic could then upload whatever they wished into this directory as the real course site. The default course site would then provide an additional link to the real course site in its header. Staff using the real course site facility would often supplement this link with additional pointers. Figure 5.3 shows the home pages for both the default course site and the real course site for a single course from 2002.

Default course site home page Real course site home page

Support for on-going change

The default course site approach provides an abstraction that allows significant flexibility in responding to on-going change. Change was enabled by a range of features, including:

  • automatic creation and updating of the course sites;
    The CourseList page type and other support services can be used to both create new and update existing course sites. For example, it was Infocom practice to retain old course sites for past students and administrative purposes. At the end of term, to avoid students confusing old course sites with new course sites, all the pages of old course sites were modified to include a clear warning that this is an old course site.
  • the addition of page types and styles; and
    Default courses sites were produced by combining available Webfuse page types with existing Webfuse styles. The creation of new Webfuse page types or styles could be used to implement new and different default course sites.
  • support for different groups of default course sites;
    It was possible for different groups of courses to use different default course sites.

These features were used by Webfuse to implement changes to the default course sites in response to both top-down and bottom-up changes. In terms of top-down or management driven changes, the specification of the default course site provided a guaranteed minimum level of service to students. It was a standard that could be changed at the direction of management. For example, the inclusion of a course barometer (described in Section 5.3.6) in the initial default course site was an attempt to increase formative feedback from students and subsequently improve the quality of learning and teaching (Jones 2002). Post 2004 the barometer was removed from the faculty default course site, again due to an institutional decision.

Bottom-up changes to the default course sites also arose from the adopter-focused, emergent development process used by Webfuse. Such changes arose first from simple changes in the context. For example, when the CQU bookshop started providing a website with details of set texts for courses, the CourseResources page was modified to include a link to the appropriate page on the bookshop website. Such changes also arose from supporting and observing the use of the default course sites. For example, in 2004 a LectureRepository page was added to make it simple for staff to upload and distribute lecture slides. The LectureRepository example also illustrates the flexibility provided by the addition of new Webfuse page types.

From the initial development of the default course site approach in 2001, through to the the final use of Webfuse for course sites in 2009, there was only ever one standard default course site. The site did change slightly over time in terms of appearance and structure, however, it was used by essentially all default course sites. The two main different default course sites were implemented in 2007 when the author was no longer with the faculty or the web team. The home pages for these two different default course sites are shown in Figure 5.4 and Figure 5.12. The default course site in Figure 5.12 has a different look, but a fairly common structure. The default course site shown in Figure 5.4 was radically different and was refered to as the “Web 2.0” course site.

While the “Web 2.0” course site was implemented as a default site using Webfuse page types, none of the functionality – i.e. discussion, wiki, blog, portfolio and resource sharing – were implemented by Webfuse. Instead, freely available and externally hosted Web 2.0 tools and services provided all of the functionality. For example, each student had a portfolio and a weblog provided by the site http://redbubble.com. The content of the default course site was populated by using BAM (discussed in section 5.3.6) to aggregate RSS feeds (generated by the external tools) which were then parsed and displayed by Javascript functions within the course site pages. Typically students and staff did not visit the default course site, as they could access all content by using a personal news reader to view the RSS feeds.

Home page for Web 2.0 course site

References

Jones, D. (2002). Student Feedback, Anonymity, Observable Change and Course Barometers. World Conference on Educational Multimedia, Hypermedia and Telecommunications, Denver, Colorado, AACE.

Sturgess, P. and F. Nouwens (2004). “Evaluation of online learning management systems.” Turkish Online Journal of Distance Education 5(3).

Webfuse as a web publishing tool – 2000 through 2004

This is the second post containing a part of the evaluation section of chapter 5 of my PhD thesis. It looks at how much/well Webfuse acted as a web publishing tool during 2000-2004. There’s not much here, as this is not the focus.

A web publishing tool

As described in Chapter 4, the original design of Webfuse included a specific aim that Webfuse be able to act as a general web-publishing tool. This ability of Webfuse was then used from 1997 through 1999 to manage and publish a range of large, often multi-author websites (see Table 4.13). Experience with Webfuse, the shortcomings of this focus on publishing static web documents, and the adoption of an adopter-focused development process (Section 5.3.1) meant that increasingly the focus of Webfuse development was on services and tasks that were specific to the Infocom and CQU context. In particular, there was a focus on the provision of interactive, web-based applications (Sections 5.3.4 and 5.3.6 ) that provided learning and teaching related services specific to Infocom and CQU. While the development of the default course sites approach (Section 5.3.5) built on Webfuse’s ability as a web publishing platform, use of Webfuse for website management was reduced during 2000-2004 and there were only small and incomplete attempts to address Webfuse’s limitations as a web publishing platform.

During the period 2000 through 2004 Webfuse was only used to manage the personal website of the author and the organisational website for Infocom. This was mostly due to the fact that at this point in time, CQU did not have any institutional system or process for publishing faculty websites. The Infocom website was by far the largest site managed using Webfuse. Table 5.8 breaks the Infocom website into a number of categories. It lists the number of documents accessed during 2004 for each category and the total number of requests for those documents. The definition of document used in Table 5.8 excludes images and CSS files.

Table 5.8 – Infocom web site categories, documents and requests (2004)
Category Requests Documents Description
Teaching 1,062,909 (49%) 68,659 (60.7%) Course websites and other pages used in teaching
Home 262,005 (12.1%) 1 The website home page
Mailing lists 191,563 (8.8%) 11,966 (10.6%) Archives of misc. non-teaching mailing lists
Staff 163,449 (7.5%) 2218 (2%) Home pages for faculty staff
Student support 55,090 (2.5%) 762 (0.7%) Information to help students studying within the faculty (not directly teaching related)
Research 37,819 (1.7%) 570 (0.5%) Faculty research pages
Publicity 37,816 (1.7%) 338 (0.3%) Information about studying
Tech support 18,774 (0.9%) 508 (0.4%) Support for using all technology in faculty
Administration 6421 (0.3%) 608 (0.5%) Intranet, meeting minutes and other administrative uses
Community 1707 (0.1%) 59 (0.1%) Various community related projects

As shown in Table 5.8, teaching was by far the largest category in terms of number of documents (49% of all requests) and requests (almost 61% of all documents). Staff home pages were amongst the most visited per document.

In terms of acting as a general purpose web publishing tool, there were two lessons or limitations identified from the use of Webfuse from 1997 through 1999:

  • An inability to use multiple Webfuse page types to contribute different information to a single web page.
  • A lack of integration between pages managed by Webfuse and more widespread web publishing or editing tools and practices.

As described above, early and uncompleted work was commenced to address both these problems. This work commenced with the move of the Webfuse page type mechanism to a design patterns-based, object-oriented architecture. An initial phase of this work was completed and was used to create page types for the default course sites. The next planned phase would have enabled the output of multiple Webfuse page types to be mixed and matched within a single Web page, including web pages produced using HTML editors. If complete, this work would have addressed both these limitations.

Another set of incomplete changes to Webfuse was aimed at addressing the second limitation – improving integration with more widespread publishing tools. At this time most Web pages were being produced using GUI_based HTML editors that did not require users to know HTML. While normal practice did not require knowledge of HTML, taking full advantage of the Webfuse publishing process did. Since Webfuse’s page types used a standard HTML form interface, the entry of text was initially limited to the use of a textarea HTML form component. This required users to understand HTML if they wished to move beyond simple text. As described above the use of available Javascript-based applications that turned a textarea HTML form component into a mini-GUI HTML editor was commenced but never entered production.

Flexible and support diversity

This is one part of the Evaluation section of Chapter 5 of my PhD thesis. It’s actually the fourth section, but it’s the first to have reached a rough first draft stage. The basic aim is to show how the Webfuse interventions from 2000 through 2004 (and beyond) increased the flexibility and diversity of the system.

By itself this may not make much sense without some knowledge of prior sections/chapters of the thesis and/or CQU experience.

Flexible and support diversity

As described in Section 4.3.4, this original design guideline for Webfuse was based on the recognition that diversity and continual change were inherent in web-based learning. As described in Chapter 4, from 1997 through 1999 Webfuse aimed to achieve this through a range of approaches including “provide the tools, not the rules”. While these were somewhat successful, it was recognised that the flexibility of Webfuse, while allowing for more diversity, also required more skill and experience from academic staff and that this, along with other factors, limited widespread use of Webfuse. For example, Table 4.20 showing that during 1999, just two course sites accounted for 47.2% of requests and 75.4% of page updates.

In the lessons learned section (Section 4.7) of Chapter 4, it was identified that limitations associated with both the product – in terms of design, architecture, programming etc – and the design and support process – in terms of ability to respond to changing needs – were effecting the ability for Webfuse to be flexible and support diversity. The foundation to addressing these limitations was provided by the adoption of an emergent (Section 5.3.2), adopter focused (Section 5.3.1) process. Arising out of that process were two main interventions – interactive, web applications and the default course sites – which illustrate an increase in the flexibility and diversity supported by Webfuse.

Wf applications

Development of Wf applications commenced in 2000 with a single application providing a simple student “portal”, primarily providing students with access to online assignments submission and return, and quiz results. Table 5.18 shows the growth from 2001 through 2004 in the number of Wf applications, the number of people using them and how much they were using them. Figure 5.8 is a graph showing the amount of usage for individual Wf applications. Both the table and the graph show rapid increase in the amount of usage by both staff and students, especially in 2003 and 2004. The doubling in staff users in 2003, represents the spread of Wf application usage beyond Infocom into the broader CQU community.

Table 5.18 – Wf applications: number, requests and users (2000-2004)
Year # Wf applications # Requests # student users # staff users
2000 1 13,908 519 8
2001 14 52,210 4194 178
2002 22 99,688 4738 297
2003 26 384,445 6629 577
2004 24 753,795 9116 570

In terms of the application usage shown in Figure 5.8 and the total number of wf applications shown in Table 5.18 it is noticeable that the majority of Wf application usage is dominated by a few applications. For example, in 2004, 94.9% of Wf application requests arose from just 4 applications: student “portal” (51%); staff “portal” (24.9%); the take quiz application (13.5%); and, assignment management (5.5%). Of the remaining 20 Wf applications used in 2004, all but one contributed less than 1% of overall Wf application usage.

Usage of Wf applications 2000-2004

The significant disparities in the amount of usage between the Wf applications, is indicative of the diversity of the applications and how they are used. The Results Upload application mentioned in Section 5.3.6 handled only 4956 requests during 2004, or just 0.7% of all Wf application requests. Unlike a general purpose, regular use application like a “portal”, the results upload application is only used at certain times by a small sub-set of users – teaching staff in charge of a course offering. In addition, the upload application’s comparatively small overall use hides a significant saving in time on the part of academic staff.

Figure 5.8 offers a comparison between three different Wf applications: student portal, take quiz, and results upload. It shows how much each application was used per month during 2004 (Note: usage data for December 2004 is not included). All three applications have two main peaks, representing CQU having two main terms in which the majority of students are enrolled. However, the peaks appear at different times of the year for the different applications. Student portal usage peaks towards the final weeks of the two main terms, as students submit and complete final assessment. Usage of the quiz application peaks earlier in term, especially in the first term, as quizzes were being used mostly as formative assessment items (though usage is much flatter in Term 2). The upload application peaks at the very end of the two major terms (actually in the first few weeks of the next term).

Figure 5.8 – Comparison of annual usage peaks for student portal, take quiz, and upload Wf applications in 2004 *
* – missing usage data from December
Usage by month of webfuse student "portal" (2004) Usage results by month of take quiz Usage by month of results upload

The Blog Aggregation Management (BAM) Wf application briefly mentioned in Section 5.3.6 represents a different type of diversity. Before BAM, all software services used by Webfuse were hosted on institutional computers. BAM assumed students would use external hosted blog engines of their own choice and register them with BAM. BAM would use the RSS files generated by the blogs to mirror and track student blog posts. In reviewing BAM in the ELI Guide to Blogging, Coghlan et al (2007) suggest that

One of the most compelling aspects of the project was the simple way it married Web 2.0 applications with institutional systems. This approach has the potential to give institutional teaching and learning systems greater efficacy and agility by making use of the many free or inexpensive—but useful—tools like blogs proliferating on the Internet and to liberate institutional computing staff and resources for other efforts.

Default course sites

As described in Section 5.3.5, Webfuse – like most other LMS – essentially supported only the one method for creating course websites. That method could be called the minimum support method, where academics are given a set of tools and left to construct course sites with a minimum of professional support. Initially, Webfuse supplemented the minimum support method by using administrative staff to manually construct sites for academics. The default course site alternative was introduced as part of the adopter-focused development approach with the intent of increasing use of course sites. The results of this in terms of statistics are described in the next section (Section 5.4.5). This section seeks to argue that the default course site approach also evolved to increase flexibility and support for diversity.

The default course site approach ensured that an organisationally approved default course site was automatically available for every course offered by the Faculty. The make up of this default course site – in terms of the appearance, structure and tools used – could be modified for different groups of courses. For example, three very different Webfuse course sites can be seen in: Figure 5.1, the original 2001 default course site home page; Figure 5.2, the home page for the 2007 Web 2.0 course site; and Figure 5.9, the home page for a commercial course offered by CQU at the end of 2007.

EPG02 Course home page

In addition, make up of a default course site could evolve over time in response to organisational priorities or other changes. Examples of the changes to default course sites that could be performed automatically based on organisational requirements, included:

  • Changes in colour based on term;
    At various times CQU had 3, 4 or 5 terms in which courses could be offered. Given that default course sites always remained open, it was necessary to distinguish between different terms. This was achieved by using different colour sets for different terms.
  • Changes based on the current term;
    Once a new term commenced, old course sites were retained for historical and administrative reasons. To ensure students were not confused by old course sites, all old default course sites were modified at the end of their term through the automated addition of an “this is an old term course” message.
  • Changes due to organisational restructures and other institutional changes; and
    In 2005, Infocom was disbanded and became part of the Faculty of Business and Informatics (FBI). In 2009, Central Queensland University re-branded itself as CQUniversity. The default course sites were modified in response to these changes.
  • Improvements in the look and feel.
    In 2005, the default course sites used by the majority of courses were re-designed.

Beyond the flexibility and diversity supported by creation of different default course sites, there was additional diversity for academics in terms of the authoring approach they could adopt. Rather than the single approach assumed in the “minimum support” method, the default course site approach provided academics with three different approaches:

  1. do nothing;
    The default course site automatically create the minimum standard as expected by the organisation. Academics didn’t need to modify the default course site beyond participating in discussions.
  2. make changes with Webfuse; or
    To move beyond the default, academics could modify and add to the default course site by using the Webfuse page types.
  3. create a real course site.
    Lastly, academics could choose to use their own web publishing tools to create a real course site that was separate from the default course site.

A further example of the flexibility of the default course site approach is provided by special topic courses. At CQU, a special topic course has a single course code in which students can enrol. However, the topic of the course would depend on the staff member they had reached an agreement with. Students enrolled in the same special topic course could be studying with different staff on different topics. The course centric design of most e-learning, which assumes that each course has one course web site isn’t a good match of a special topic course. The Webfuse solution to this problem was to have a single default course site for a special topic and then use the real course site area to host a default course site for each topic. Each topic would be have a special course code – created only in Webfuse – and the Webfuse “temporary enrolment” feature would be used to enrol students in that special course code.

Another example of the flexibility and support for diversity offered by the default course site approach is the Web 2.0 course site mentioned in Section 5.3.5. The Web 2.0 course site embraces and extends the BAM idea that increasingly universities do not need to provide all of the e-learning tools to be used by staff and students. Increasingly, staff and students are using a range of readily available online tools for a variety of tasks and wanting to use some of them in their learning and teaching. Created using the same technology as other Webfuse default course sites, the Web 2.0 course site displays content created by staff and students using external applications such as WordPress.com RedBubble.com. Instead of visiting the course website, students and staff can receive notifications of new course activity and contents through a news-reader application of their choice. The news-reader draws on the same feeds used by the course site to display the content. The Web 2.0 content site provided an institutionally branded home for the course, but staff and students never need to use it.

References

Coghlan, E., J. Crawford, et al. (2007). ELI Discovery Tool: Guide to Blogging, EDUCAUSE.

Further analysis of wf application usage

In the last post I outlined some early rough usage figures for Webfuse from 2000-2009. The following goes into a bit more detail with the usage of the wf applications.

Breakdown by application

The following graph (click on it to see bigger version) breaks the usage of wf applications down into the separate applications. In the period from 2000 through 2009, there were 54 or so wf applications that were used. However, as the graph shows, two or three applications were dominant in terms of usage.

Wf application usage

The top three applications by usage as shown in the above graph (in descending order) are:

  1. the student “portal” (the large blue area);
  2. the staff “portal” (the red area); and
  3. student photo (the green area at the end).
    Student photo is a kludge to enable the staff “portal” to still show staff student photos from the institutional student records system. So strictly speaking, its part of the portal.

Breakdown of student “portal” usage

In 2009, usage of the student “portal” could be broken down into the following:

  • 54% of requests for ViewAssessment;
    The page that shows a student the assessment detail for a given course.
  • 23.5% for Home;
    The “home” page for the student “portal”.
  • 14.5% for StartSubmit and ShowReturnedAssignment;
    Students starting to submit an assignment or checking the comments/results on a returned assignment.
  • 7% for checking on an assignment or viewing assessment results.

i.e. By 2009, this was no longer a student “portal”, but almost entirely associated with online assignment submission and management. 0.7% of requests for 2009 were associated with viewing quiz results.

In 2009, 11,482 students logged into the student “portal”.

Come back of the power law

I recently talked about the similarity between the 1999 and 2005 graphs representing how often academics had modified Webfuse course sites. Even though the 2005 graph showed more staff editing course websites more, both years had the same basic pattern. A power law.

The following graph represents the number of times each student used the “student portal” in 2009. See the similarity?

StudentMyCQU Users for 2009

Yet another power law. There’s a trend developing here. I’m wondering if this can be thought of as evidence for the chasm.

Breakdown of staff “portal” usage

A quick breakdown of the 2009 usage of the staff “portal” reveals the following most used services:

  • ~36% of requests to view details about a particular student;
  • ~25% to find a student
  • ~17% to get a class list
  • ~8% finding a course
  • ~5% the “portal” home page.

There are another 17 services provided by the “portal”. 15 of which have less than 1% use.

Staff “portal” usage by staff

So, does the power law hold for staff users of the “portal”?

There are over 1100+ users during 2009. The following graph shows how many requests of the staff “portal” were made by each user. The graph is sorted in descending order on the number of requests.

Staff MyCQU Users 2009

Thar she blows! Yet another power law distribution.

Implications

I am increasingly intrigued by this prevalence of the power law distribution in Webfuse usage, regardless of the application, the time or the users. I’m particularly interested in its implications in terms of how e-learning should be supported and designed. Not to mention why it happens this way.

I’m increasingly interested in whether or not this distribution exists with other systems. Both within the “Webfuse” institution and also at other institutions.

Some rough Webfuse usage statistics – 2001 through 2009

In terms of the thesis, my current focus is on chapter 5. This chapter seeks to describe what we did from 2000 through 2004 and beyond. I’m currently working on the evaluation section, a major component of the evaluation is looking at the usage statistics of Webfuse during this period. It’s slow going and there is still a fair bit of analysis to go, but some initial figures are available.

What analysis has been done?

The statistics shown below show raw hit counts on various “parts” of Webfuse. That is, I’ve taken the raw Apache access logs and extracted relevant hits into the following Webfuse “parts”:

  • Course site access;
    A request for a file from a course website. This could include images, documents and web pages. This one needs to be refined the most.
  • Course updates;
    To modify a course website a member of the teaching staff did a course update. This represents the number of times that happened.
  • FM;
    To upload Word documents etc staff used FM – a file manager.
  • Wf applications;
    From about 2000, a range of interactive web applications including a staff and student “portal”, online assignment submission, quiz management, etc were developed. This represents the number of hits on those applications. Each hit represents a single page.
  • Course profiles.
    For a year or so, CQU course profiles were available from Webfuse as PDF documents. These numbers represent the number of times they were downloaded.

Course sites and updates

First, let’s look at course sites, number of times data was requested, number of times updated and number of times the file manager was used.

These figures are somewhat dependent on the number of courses and students. The following table shows the raw figures and then a ratio with the number of course sites. i.e. in 2001, there were 244 course sites in Webfuse, there were almost 6.5M requests on those course sites for an average of about 26,000 per course site.

Webfuse course statistics 2001-2009
* – 2005 logs for Jan through Apr are missing
Year # of courses Site requests Site updates Fm
2001 244 6,491,238
26,603
4523
19
5679
23
2002 313 5,346,867
17,082
19,483
62
37,874
121
2003 304 4,686,393
15415
21,869
72
14,395
47
2004 328 4,469,301
13,625
19,230
59
12,593
38
2005 * 299 2,422,395
8101*
11,217
38
7010
23
2006 297 3,278,221
11,038
17,752
60
10,625
36
2007 249 1,891,192
7595
9646
39
8296
33
2008 225 1,848,491
8216
6448
29
8547
38
2009 211 1,958,401
9281
9286
44
7963
38

There needs to be more analysis on this data:

  • Why does 2001 have such a large number of requests per website?
    My first guess is that there a small number of courses in that year were generating requests, or a drop in student numbers.
  • What is the spread of these requests per course site?
    e.g. are one or two course sites dominating? Do they go away after 2001?
  • Generate a ratio with student numbers
    There was a drop in student numbers between 2001/2002. Were there simply more students in 2001?
  • What’s the source of the significant increase in site updates per site in 2002 and 2003, why?
    The default course sites were introduced in the 2nd half of 2001, was this the reason?
  • What is the spread between staff in site updates and use of fm?
  • There’s another spike in updates in 2006, why?
  • The drop in site request ratio drops after 2006, why?
    Is this because of the movement of the “real course site” courses. Sites liable to be used a great deal.

Wf applications

One of the major developments in Webfuse from 1999/2000 was the development of the Wf framework which provided the basis for “Webfuse” interactive web applications. There were a broad array of very different applications – ranging from a course barometer and BAM, through to an institutional academic misconduct system – that were implemented with this framework.

The following graph (click on it to make it bigger) summarises requests for these application. The 2009 figure of 6M+ requests, is a bit suspect and requires greater analysis (Further analysis reveals a lot of 401s – the real figure is just over 2M).

Wf application requests

The increase in usage (doubling) in 2003 shows the spread of these applications into institutional systems. More analysis needs to be done on these figures to:

  • Identify the reason for the drastic increase in 2009;
  • Identify how much each application is being used;
  • Examine the number of users of these applications and how heavy their use is.

Academics, course websites and power laws

Last week I was thinking that academics shouldn’t manually create course sites. That arose out of the process of writing up the why/what behind what we did with Webfuse from 1999 through 2004. Today, I’ve been continuing that and looking at the usage statistics from that period.

The following focuses on statistics about how often an academic modified a course site. The Webfuse model was to automatically create a default course site for every course offered by the faculty. Academics could then modify that site as much, or as little as they wanted. The following two graphs (first in 1999 and then in 2005) shows how many staff were editing Webfuse course sites and how many times they made page updates. Staff are ordered on increasing number of updates. What is striking to me is the similarity of the curves – both look like a “power law”. Some rambling on implications below.

1999

Updates 1999

2005

Webfuse page updates  2005

In terms of the 2005, the top 22 academic staff performed 21,298 updates on course sites. That’s over 8,000 more updates than the remaining 127 academic staff (13,472 updates in total). 17% of the academic staff performed 61% of the updates of course websites.

Implications

With the Indicators Project we’ve been using three questions to frame investigations of LMS usage:

  1. What?
    What is actually going on within LMS usage? What are the patterns that can be identified?
  2. Why?
    Why do these patterns exist? Can we identify why this pattern has arisen?
  3. How?
    How can this insight be used to improve practice?

What?

The Wikipedia page on “power law” states “Power-law relations characterize a staggering number of naturally occurring phenomenon”. With the Webfuse data shown above, which is spread over a 6 year time period during which there were significant changes, there is a fair indication that this might be leaning towards a “natural phenomenon”. It would be interesting to perform a similar analysis on more recent data and more “traditional” LMS to see how “natural” this if this might represent a more widespread, “natural” phenomenon.

Based on my experience, and without looking at the data, I suspect that this type of pattern is likely to exist in most universities around use of their LMS.

Why?

So, why do you think this pattern exists? Suggestions?

My suspicion is based on Geoghegan’s (1994) identification of a chasm – a la Moore (2002) – in the adoption of instructional technology. i.e. that there is a chasm/difference between two groups of academics – innovators/early adopters and the pragmatists. It’s the innovators/early adopters that are the big users of instructional technology.

So, one interpretation of the above figures is that the majority of academics are pragmatists. This is not necessarily a negative. They want to do a reasonable job of teaching (as measured by the institution, themselves and their students) but aren’t going to allow other work (mostly research) to suffer. My suspicion is this “pragmatic” perspective is the dominant perspective amongst academics. It’s the type of perspective that environment within universities encourages.

How?

So, if you found support for this perspective, how might it be used to improve learning and teaching?

If it is the university teaching environment that creates this “pragmatic” approach, perhaps it needs to be changed.

If a majority of academics aren’t editing course sites, this suggests that these course sites aren’t that great. Perhaps it also suggests that the quality of the student learning experience isn’t all that great. If this is the case, then continuing the practice of academics having to create course sites within an LMS may not be the way to go. Perhaps, it is time to investigate alternatives ranging from the evolutionary – provided a default course site for academics to build upon – to the revolutionary – such as PLEs etc.

Postscript – Implications for LAMS?

LAMS – Learning Activity Management System has for quite some time been positioned as a better, alternative to the LMS model. From the “About” section on the LAMS site

LAMS is a revolutionary new tool for designing, managing and delivering online collaborative learning activities. It provides teachers with a highly intuitive visual authoring environment for creating sequences of learning activities. These activities can include a range of individual tasks, small group work and whole class activities based on both content and collaboration.

LAMS must be good, it has won a gold medal

Does LAMS usage – within institutions that have adopted it – follow the same “power law”?

The question of how to do an apples versus apples comparison between LAMS and a LMS would be interesting as they follow very different models.

If this could be done appropriately, then my prediction is that yes, in a university environment LAMS would follow this pattern, possibly even more pronounced because LAMS is that much more different to past practice for academics than the LMS.

Also, from the perspective of a typical teaching academic (and perhaps even students?), there’s a lot more to an “online course” than learning activity design. Most of which LAMS doesn’t support directly, hence the need to integrate it with LMSs.

References

Geoghegan, W. (1994). Whatever happened to instructional technology? 22nd Annual Conferences of the International Business Schools Computing Association, Baltimore, MD, IBM.

Moore, G. A. (2002). Crossing the Chasm. New York, Harper Collins.

Workarounds

The following is a first draft of the next section of chapter 5 of my thesis. The aim here is to try and illustrate how the combination of the agile and adopted-focused development process of Webfuse, when combined with the design of Webfuse enabled the rapid development of “workarounds” that enabled the system to adapt to changes in the environment and requirements of the systems users.

Workarounds

The interventions described in the previous sections provided the foundation through which a large number of contextually specific interventions arose during the period from 2000 through 2004 and beyond. The majority of these interventions were not strategic projects identified by management. Instead, the majority of the interventions were workarounds that arose in response to factors or changes that arose during the emergent and adopter-focused design and support of Webfuse. These interventions range from low-level technical changes, through ad hoc combination and integration of existing workarounds, to the implementation of large and complex applications that were later adopted as official, institutional information systems. The breadth and diversity of these interventions illustrate how Webfuse was able to respond quickly to local problems and opportunities.

Student numbers and usernames

CQU, like many other universities, has adopted the practice of assigning students usernames for accessing institutional information systems based on the unique student number assigned to each student as they enrol at the university. Due to CQU specific reasons this has created problems. As of 2001, CQU has two different types of student number: (1) pre-PeopleSoft student numbers that are 9 characters long; and (2) post-PeopleSoft student numbers that are 8 characters long. Circa 2000 authentication systems provided by Windows operating systems, the basis for most institutional systems, could only recognise usernames 8 characters. Consequently, students with 9 character student numbers were told to leave off the last digit. With the advent of PeopleSoft in 2001 and its 8 character student numbers, this advice had to be modified to distinguish between the different types of username. Subsequently, it was common for this to cause problems for new students until they became familiar with the limitations. The design pattern-based, OO design of the Webfuse authentication system enabled Webfuse to work effectively with whatever version of the student number students used.

Evolution of Webfuse user authentication and access control

Jones, Lynch and Jamieson (2003) use the evolution of the Webfuse user authentication sub-system to illustrate the design for repair, not replacement, ethos adopted by Webfuse. Table 5.5 is a summary of the changes made to the user authentication system. All these changes occurred without any apparent change from the users’ perspective and limited change in the Webfuse programming interface (Jones, Lynch et al. 2003).

Table 5.5 – Changes in the Webfuse authentication system (adapted from Jones, Lynch et al. 2003)
Stage Description
Webfuse specific accounts With no access to institutional systems, students had to have special Webfuse accounts.
Student records passwords With access granted to the CQU student records system, students could use the same username/password combination to access student records and Webfuse.
CQU domain accounts The advent of and access to a central Windows NT domain infrastructure allowed CQU staff to access Webfuse via their domain accounts.
PeopleSoft The adoption of PeopleSoft in 2001 meant a change in how student account details were checked. Including working with both 8 and 9 character usernames.
Cached student information In the initial months, the PeopleSoft database was barely available from 9-5 on week days. To enable 24×7 access to Webfuse, student account details were cached on the Webfuse server.

GUI editors

From the start Webfuse used HTML forms as the main authoring environment. This meant that if potential authors (staff or students) wished to use anything beyond simple text, they had to know HTML. This was a significant barrier for many staff and was a mismatch with their growing experience and expectation of GUI word processors and editors. By 2004 a solution to this problem was identified through one of the increasingly available Javascript components that would convert a HTML form text area into a GUI HTML editor. Initial testing of this workaround commenced around the time the author moved back to teaching and was never implemented.

Staff MyCQU, student records and results upload

Prior to the implementation of Peoplesoft, CQU provided academic staff with InfoWeb, a simple web-based application that provided access to the student records system. InfoWeb allowed staff to search for information on particular students such as contact details, to download course lists and upload results at the end of term. For various reasons, the PeopleSoft implementation project did not provide an InfoWeb replacement and when PeopleSoft went live, Infoweb was replaced with use of the PeopleSoft desktop interface (Jones, Behrens et al. 2004). This interface was time consuming, confusing and difficult, restricted to use on the CQU network, only usable via a computer running Microsoft Windows, and was seen by academic staff as a significant step backwards (Jones 2003).

By mid-October 2001, near the end of the first term after the PeopleSoft go live, there were significant concerns within Infocom about the significant workload implications and potential for human-error of the results upload process proposed by PeopleSoft. This concern resulted in the development of a Webfuse results uploading system designed to be significantly simpler to use than the equivalent PeopleSoft process, provide support for faculty specific requirements, and integrate with the PeopleSoft Higher Education system (Jones, Behrens et al. 2004). The system was developed in just over a month and was used for the first time in November 2001. By the end of 2009, over 340 different staff have uploaded final results for over 58,000 distinct students in over 3800 course offerings.

By the end of 2001 it became obvious that the PeopleSoft access to student records was causing significant problems. For example, to generate a list of student details for a course required a 26-step process, two separate applications on a Windows computer, knowledge of internal data representations not in common use by academics, and a time investment of over 20 minutes (Jones 2003) In early 2002, a new member of the Infocom web team was given the task of developing a Webfuse interface to the student records system as a training task. Within a month the interface was available and being used by staff (Jones, Behrens et al. 2004). Initially known as MyInfocom, the system evolved to act as the staff portal and provide access to a range of faculty features and services.

By August 2002, another faculty requested access to MyInfocom for their staff. The integration of MyInfocom with specific Infocom services, meant that access to MyInfocom would be confusing for non-Infocom staff. Instead, the design patterns-based, OO design of the Wf framework made it possible to provide MyCQU, a version of MyInfocom with the Infocom specific features removed. By 2005 the distinction between MyInfocom and MyCQU was removed and MyCQU increasingly became a key centrally supported, university system. As of 2010 MyCQU is still functioning as the staff portal for CQU academic and general staff.

Other Wf applications

A key reason for the continued support of MyCQU was the use of the Wf framework to develop a number of institutional systems from 2005 onwards. These institutional systems were accessed via MyCQU and included the following systems:

  • Academic Misconduct Database;
    Used to store, track and manage incidents of student misconduct.
  • Academic Staff Allocations; and
    Used to store, track and manage details of which staff were teaching which courses and which campuses.
  • Assignment Extensions System.
    Used to request, manage and track student requests for extensions to assignments.

Other previous, similar Wf framework applications included, but not limited to:

  • Timetable generator;
    Automatically produced a personalised timetable based on the courses a student or staff member was involved with for a given term. Also allowed the selection of specific courses. The generator worked for staff and students on all CQ campuses. Only available for a few years, the generator replaced an institutional timetable system that was different depending on which campus you attended, was not integrated with the student records or teaching responsibilities database, required additional knowledge and was time consuming (Jones 2003).
  • Email merge; and
    Given a list of student numbers, would allow staff to create and send an email message to all selected students. The content of the message could be modified based on information specific to each student.
  • Informal Review of Grade (IROG).
    A web-based system developed within 3 weeks to replace an inefficient and error-prone paper-based process by which students across all of CQU’s campuses could request an informal review of their grade.

OASIS

The provision of timely, meaningful feedback on student assessment is a key component of teaching and can influence student results (Jones and Jamieson 1997). It is typical for the submission, marking and management of student assignments to be performed using physical submission. As a multi-campus institution with significant numbers of distance education students this typical approach creates a range of problems including (Jones and Jamieson 1997): slow delivery time; human error; difficulties in timely moderation of marking (especially for overseas campuses); process duplication; and a number of others. For this reason, the development and use of online assignment submission and management systems has been a significant component of Webfuse under the title OASIS (Online Assignment Submission, Infocom System). Arising out of early attempts using email to reduce turnaround times on the assignments for distance education students (Jones and Jamieson 1997), OASIS went through five different generations of development as described by Jones and Behrens (2003) with the last generation described as evolutionary development (aka emergent development as described above). The evolutionary/emergent development of OASIS was through “an on-going process of discussion with the users allowing the system to grow and meet their needs as they arise” (Jones, Cranston et al. 2005).

Combinations and integration

A major benefit of the Wf framework and its design pattern-based, object-oriented design was that it enabled rapid development of new applications. A second benefit of this design was that it also enabled the mix and match of different applications, technologies and views. The Web 2.0 course site described above is one example of this. The following provides two additional examples: grouping students by date of enrolment; and, supporting multiple discussion forums in Blackboard.

Figure 5.3 shows a part of the standard class list page provided by Webfuse. This page shows a range of details of students enrolled in a particular course and was commonly used by teaching academics. As it stands, Figure 5.3 contains a number of examples of the Webfuse ability to combine and integrate different services. For example, at the top of Figure 5.3 there is a button “Mail Merge”. The email merge facility described above was designed to be able to take a list of student numbers from any application. Once the email merge facility was completed the “Mail Merge” button was added to the class list page allowing staff to email all students in the course. In addition, if staff were to narrow the class list to a particular campus, the “Mail Merge” button works as expected.

Class list by name

By 2006, some academic staff were using this facility to send a “welcome to the course” email to students in the course. The purpose of such an email was to create an initial social connection with the students and provide some initial guidance on what they should be doing. This “welcome” email approach was somewhat complicated by the likelihood that students could enrol in a course at times ranging from months before the start of term until two weeks (and sometimes more) after the term had started. Sending the email too early would miss some students, too late and students may already have started feeling lost and sending multiple messages may overwhelm.

In Figure 5.3, the Class list is sorted by name. Prior to 2006, there was no choice, sorted by name was the only option. In 2006, the Model-View-Controller architecture of the Wf framework allowed the creation of a different view for the class list page that displayed student details sorted by date of enrolment in the course (Figure 5.4). The Wf framework’s pattern-based, object-oriented design enabled the link to “Mail Merge” to work on this new view with no code modification.

Class list by enrol date

In 2007, a CQU curriculum design created an instructional design for course using the Blackboard LMS that required the division of hundreds of students into small groups with each group having its own portion of the Blackboard site containing a number of different discussion forums serving very different instructional purposes. This instructional design could not be implemented with the Blackboard LMS due to an in-built limitation around the number and location of discussion forums per group. The solution adopted was to create the discussion forums for these groups within Webfuse and integrate them into Blackboard. Figure 5.5 shows the Blackboard course site and one of the Webfuse discussion forums used by one of the groups.

Integration of Webfuse and Blackboard

References

Jones, D. (2003). How to live with ERP systems and thrive. 2003 Tertiary Education Management Conference, Adelaide.

Jones, D. and S. Behrens (2003). Online Assignment Management: An Evolutionary Tale. 36th Annual Hawaii International Conference on System Sciences, Hawaii, IEEE.

Jones, D., S. Behrens, et al. (2004). The rise and fall of a shadow system: Lessons for enterprise system implementation. Managing New Wave Information Systems: Enterprise, Government and Society, Proceedings of the 15th Australasian Conference on Information Systems, Hobart, Tasmania.

Jones, D., M. Cranston, et al. (2005). What makes ICT implementation successful: A case study of online assignment submission. ODLAA’2005, Adelaide.

Jones, D. and B. Jamieson (1997). Three Generations of Online Assignment Management. ASCILITE’97, Perth, Australia.

Jones, D., T. Lynch, et al. (2003). Emergent Development of Web-based Education. Proceedings of Informing Science + IT Education, Pori, Finland.

Functional fixedness, analytics, and the LMS

A blog post on the website of Gilfes Education Group (apparently a “network of independent education experts”) picks up on the Indicators project and its take on academic analytics. The post seems to largely in agreement with what we’re doing and the reasons behind it.

The following seeks to pick up on a point made in the Gilfus post about the problem arising from ownership of the data and some of the other barriers that have been proposed. The argument I develop in the following that functional fixedness is a major barrier to the effective appropriation of academic analytics to help improve learning and teaching.

But first, an experiment

Imagine if you will that we’re in a room together. I’m going to set you a task. Here’s some matches, a box of tacks and a candle (see the image below). Your task is to attach the candle to a cork board on the wall in way that means that wax from the candle does not drip onto the table that is underneath the cork board.

Candle problem set up

How do you do it?

The solution is given an image at the end of this post.

Apparently, if I rephrase the problem solution a little to the following, it might improve your chances of success.

Here’s some matches, a box of tacks and a candle (see the image below). Your task is to attach the candle to a cork board on the wall in way that means that wax from the candle does not drip onto the table that is underneath the cork board.

Functional fixedness

If you’re anything like my brother-in-law on whom I tested this out in person, you did not arrive at the solution quickly, if at all. This experiment is called the candle problem and has been used to demonstrate the problem of functional fixedness.

Functional fixedness suggests that you have fixated on the design function of the object – i.e. the box of tacks is designed to hold the tacks – so much that you cannot see how it might be put to a different use to solve this problem. To put it in the words of German and Barrett (2005)

Problem solving can be inefficient when the solution requires subjects to generate an atypical function for an object and the object’s typical function has been primed

In other words, the problem description above had the box’s typical function primed as holding the tacks, hindering your ability to see another use for the box.

Academic analytics, the LMS and functional fixedness

For most universities there is an existing set of information systems. There’s the learning management system (LMS) in which learning takes place, and there is the data warehouse and associated business intelligence tools for providing reports and information. The people within these organisations, especially those already supporting (the IT folk) and using (management) the data warehouse, have been primed to see a typical use for these systems. They are fixated on using the LMS and data warehouse in a particular way.

Add into this mix the typical under resourcing/inefficient management of IT, and the typical top-down, techno-rational approach to management and it is simply too difficult for organisational members to see the case for moving aspects of academic analytics into the LMS.

It doesn’t help that it’s messy

The matter isn’t helped much by the benefits of moving aspects of academic analytics into the LMS are somewhat uncertain and messy. Being uncertain and messy aren’t characteristics of an approach likely to overcome functional fixedness. Especially in organisational environments where being efficient (defined as doing what we already do or have strategically planned to do) is the main intermediate goal. But then this is why innovation is hard in organisations, innovation is messy.

References

German, T. and H. C. Barrett (2005). “Functional fixedness in a technologically sparse culture.” Psychological Science 16(1): 1-5.

Solution

The solution to the Candle problem is represented in the following image.

Candle problem solution

Course websites and "libertarian paternalism"

Stephen Downes makes a valid point about my recent question about whether or not academics should manually create websites. I agree with his underlying point that academics should not be forced to use the institutional approach. Given any option I would not suggest such an approach.

Incompetent paternalism

However, at least within some Australian institutions academics are being forced to accept an institutional approach. That approach is typically expressed as “minimum service standards” which are specified by management. The academics are than expected to manually fulfill those standards. I have a problem with this approach, but if it is being adopted, then at the very least implement it in an efficient and effective way.

What is happening in these situations could be described as “incompetent” paternalism. Academics are being treated as children (a theory X perspective of academics underpins this approach). Management as the parents have to specify codes of behaviour. But when it comes to implementing this code of behaviour, management are actually making using an inefficient and ineffective approach.

I disagree strongly with this approach, but if management believe it, is it too much to ask them to do it efficiently and effectively? That’s one perspective, my real interest is in a third way that tries to effectively merge features of both incompetent paternalism and the academic free for all.

The “libertarian” paternalist alternative

The model that evolved in the early part of this decade could be described as a “libertarian” paternalist approach. It’s a bit of a stretch but I think the metaphor works.

The theory was/is that an appropriately skilled group, taking an adopter-focused and emergent development approach could develop a default course site that could effectively be used by a group of courses. That default course site could be automatically created. But since the group was using an emergent development approach, the default course site would continue evolving.

The default course site did not remove the academics freedom of choice. As implemented, academics could modify the default course sites in two ways:

  • Use the “LMS” to modify or add to the default course site; or
    Here’s an example default course site from 2006.
  • Create a real course site.
    Here’s a default course site where the academic has created a real course site.

This wasn’t a perfect solution, it still wasn’t flexible enough. We had plans to enable better merging of the default and real course sites. i.e. if a real course site was created, it could replace the default course site and make better use of the services offered by the “LMS”. But they never got off the drawing board.

In the almost 8 years that this approach was used, the “LMS” averaged around 300 course sites a year. The number of real course sites (i.e. academics doing their own thing) was never reach 10% in any given year. It averaged around 4% of course sites a year.

Lack of appropriation

Importantly, where possible, the aim was to observe what everyone was doing, especially the <10% creating the real courses sites, and use those insights to modify the default course sites.

The current management approach of specifying minimum standards is being driven by external desires, not by the experience of academics and students using the minimum standards.

References

Jones, D. and T. Lynch (1999). A Model for the Design of Web-based Systems that supports Adoption, Appropriation and Evolution. First ICSE Workshop on Web Engineering, Los Angeles.