Assembling the heterogeneous elements for (digital) learning

Month: July 2015

What do "scale" and "mainstreaming" mean in higher education?

@marksmithers has just written a blog post that makes the following point

that talks about a new fund to promote innovation in highered. I know $5M isn’t a huge amount but the principle just seems so misguided. There is no problem with innovation in higher education. The problem is adopting and mainstreaming innovations across a higher ed institutions.

@shaned07 raised a similar question in a recent presentation when he talked about the challenge of scaling learning analytics within an institution.

But the question that troubles me is what do you mean by “scaling” or “mainstreaming” innovations in higher education?

What do you mean by “scaling” and “mainstreaming”?

The stupid definition

This may sound like a typical academic question, but it is important because an naive understanding of what these terms may mean quickly leads to stupidity.

For example, if what I’ve experienced at two different institutions and overhead numerous times at a recent conference is anything to go by, then “scaling/mainstreaming” is seen to be the same as: mandated, consistent, or institutional standard. As in, “We’ll mainstream quality e-learning by creating an institutional standard interface for all course websites”, or, “We’ll ensure quality learning at our institution through the development of institutional graduate attributes”. Some group (often of very smart people) get together and decide that there should be an institutionally approved standard (for just about anything) and every one, process, policy and tool within the institution will then work toward achieving that standard.

Mainstreaming through standardisation is such a strong underpinning assumption that I know of one university where feedback to senior management is provided through an email address something like 1someuni@someuni.edu and another university where achieving the goal of “one university” received explicit mentions in annual reports and other strategic documents.

The problem with the stupid definition

talk to the experts by Mai Le, on Flickr
Creative Commons Creative Commons Attribution 2.0 Generic License   by  Mai Le 

The problem with this approach is that it assumes Universities and their learning and teaching practice is a complicated system, not a complex system. This way of viewing universities is reinforced because the people charged with making these decisions (senior leaders, consultants, internal leaders on information technology, learning etc) are all paid to be experts. They are paid to successfully solve complicated problems. That success and expectation means that they expect/believe the same methods they’ve used to solve complicated problems, will help them solve a complex problem.

As Larry Cuban writes

Blueprints, technical experts, strategic plans and savvy managers simply are inadequate to get complex systems with thousands of reciprocal ties between people to operate effectively in such constantly changing and unpredictable environments……..Know further that reform designs borrowed from complicated systems and imposed from the top in complex systems will hardly make a dent in the daily work of those whose job is convert policy into action.

Much of the content of the talk titled “Why is e-learning ‘a bit like teenage sex’ and what can be done about it?” that @palbion and I gave focuses on identifying the problems that arise from this naive understanding of “mainstreaming/scaling”.

What’s the solution?

Cuban suggests

At the minimum, know that working in a complex system means adapting to changes, dealing with conflicts, and constant learning. These are natural, not aberrations.

The talk I mentioned builds on two papers (Jones & Clark, 2014; Jones, Heffernan & Albion, 2015) that are starting to explore what might be done. I’m hoping to explore some more specifics soon.

Whatever shape that takes, it certainly will reject the idea of mainstreaming through institutional consistency. But in summary, it will probably involve in creating an environment that is better able to adapt to change, deal with conflicts, and constant learning.

References

Jones, D., Heffernan, A., & Albion, P. R. (2015). TPACK as shared practice: Toward a research agenda. In D. Slykhuis & G. Marks (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2015 (pp. 3287-3294). Las Vegas, NV: AACE. Retrieved from http://www.editlib.org/p/150454/

Jones, D., & Clark, D. (2014). Breaking BAD to bridge the reality/rhetoric chasm. In Rhetoric and Reality: Critical perspectives on educational technology. Proceedings ascilite Dunedin 2014 (pp. 262-272). Dunedin. Retrieved from http://ascilite2014.otago.ac.nz/files/fullpapers/221-Jones.pdf

"Me as teacher" – the 2015 focus for NGL

NGL is a course I teach. Participants are asked to spend the semester engaging with networked and global learning as: student, learner, and teacher. They are asked to blog and reflect on this mishmash of experiences throughout the semester. I’m trying to do this as well. Not only to model one version of expectations, but also because I find it a valuable learning process myself. I did “me as learner” last week, this week it’s time for “me as teacher”.

as “meta-“teacher

Last year I wrote this “as teacher” and it had a more traditional focus. I was thinking about me as the teacher of a couple of formal courses. The directions in NGL tries to expand the understanding of “teaching” beyond formal learning to include “as you doing something that helps others learn”. For the rest of this year I’d like to push the boundaries a bit. I’d like to go meta in terms of teaching.

One of my key positions at the moment is that university e-learning is a “bit like teenage sex”. It’s quite horrendous and isn’t dealing well with some difficult problems. Yet another restructure, or a focus on quality standards is not going to help! There’s something more fundamental here. I want to explore what that might be.

In particular, I want to help universities learn how to do e-learning better.

I use that “learn how to” for two purposes. First, to indicate that they need to get better. Second, and much more importantly, is that the way to get better is to focus on the problem as a learning problem. The solution isn’t to analyse the situation and identify the solution (and then implement it). The solution is to recognise that they only way to improve the quality of e-learning is to approach it as a never-ending learning problem. The organisation as a whole always needs to be learning. Trying new things, failing, getting better, finding what works, changing it etc.

Me “as teacher” is actually me as meta-teacher.

Back to the questions asked of NGL participants.

What is your role as a teacher? Who are your students? What is the context?

Let’s keep the context narrow and say within my current institution. The “students” are essentially anyone involved with e-learning at the institution. I’m not formally teaching any of them, but perhaps as I engage within the network of the institution there will be some learning.

Taking a very network centered perspective on learning, my “role as teacher” is to help make connections. Borrowing from the “distributed view” the idea is that

the world is complex, dynamic, and consists of interdependent assemblages of diverse actors (human and not) connected via complex networks.

The mindset underpinning university e-learning is to SET in it’s ways. The question is how to change that?

What role does NGL currently play? How can it help?

My argument is that due to the SET mindset, NGL doesn’t play much of a role at all in university e-learning. While network technology is used increasingly within university e-learning, the practices, conceptions, and processes around it are still largely industrial and completely inappropriate. All the attempts to improve e-learning are trying to do so within the confines of this inappropriate mindset.

I think that really grokking a NGL mindset promises to improve the ability of universities to learn how to do e-learning. Largely because it’s a more appropriate model of the learning that needs to take place.

I’m particularly interested in how the idea of Context-Appropriate Scaffolding Assemblages (CASA) might be implemented and subsequently make it easier for universities to learn how to do e-learning in much more interesting and effective ways.

What difficulties might arise?

The largest barrier is that this requires a mind-shift. A major mind shift. Modern organisations (at least those that still haven’t figured it out) like universities are built on a different mindshift. A NGL mindset is radically different. Different mindsets have been a barrier to reform – especially around computing – before. Pedaling a different mindset is a good way to set yourself up as “strange”.

Then there’s the question of workload. The selfish reason I’m interested in this problem is because I suffer from it. Too much make work, not enough capability to engage in meaningful learning.

Updating "more student details"

“More student details” (see image below) is the most visible part of the additional systems I’ve put in place to make teaching bearable (this presentation talks more about it). The problem I’m dealing with here is that a new semester has started, which means I need to re-configure these feral systems to work with the new semester.

And while I’m at it, I’m hoping I might be able to add a bit of sentiment analysis to it.

MoreStudentDetails

What needs to be done

Tentative process appears to be

  1. Create new course ID (5), group id (53), contextid (575) and course label (EDC3100_2015_S2) for new offering – creating the course in local Moodle.
  2. Create a BIM activity in the new course. – 3 – DONE
  3. run participants/parse.pl – DONE
  4. run the user extras. – DONE
  5. Load the initial activity information.- DONE
  6. Load the current activity completion data. – DONE
  7. Load the BIM data. – DONE
  8. Update the greasemonkey script

The basic aim is to populate a database running on my laptop with the following data

  • Student enrolment data.
    • Source – spreadsheet provided by manual dump from student records
    • Destination – Local moodle tables: mdl_user_extras and mdl_user
    • Method
      • Initialisation – set up the Moodle users data
        • update the course label in extras.pl
        • Play around with the columns in the spreadsheet.
      • Maintenance – participants/extras/extra.pl – DONE
        • Can the CSV file be read appropriately – DONE – but missing some of the data I would have liked to have had.
          Keep on eye out for strange data – e.g. UK post codes.
        • Get the users data for this course.
        • See if a match can be made with local Moodle and Peoplesoft data
        • Any missing students (can’t match)
        • insert into mdl_user_extras table
        • update mdl_user table – to update the phone1 and phone2 values
    • Status
  • Student Moodle user data.
    • Source – CSV file produced
    • Destination – Local moodle tables: mdl_user, mdl_groups_members, mdl_role_assignments
    • Method
      BAM/3100/3100_support/participants/parse.pl

      • Initialisation
        1. Update the library files
        2. Create a new local Moodle course – get id.
        3. ??? do I need to create a group ID???? What else is needed
      • Maintenance – script will do most of this — students aren’t enrolled in the course, but are appearing in BIM.
        1. Parse the HTML file. – DONE
        2. Find any existing users – DONE
        3. Add the new students – DONE
        4. Populate the group data – DONE
        5. Populate role assignments – DONE
    • Status
  • Activity completion data – DONE
    • Source
      • HTML file saved of main Moodle site – DONE
      • CSV file from activity completion
    • Destination – local Moodle tables
    • Method
      • Initialisation – activities/parseActivity.pl – DONE
        1. Have the local Moodle course set up.
        2. Ensure that the script has the configuration for the new course.
        3. Dump the main remote Moodle page to HTML.
        4. Run the parseActivity.pl script
          • Parse the html page – DONE
          • update the activity mapping table – DONE
        5. perhaps have to play with the activity translation stuff especially since Moodle 2.8 seems to have changed their method
      • Maintenance
        • List of activities – Whenever a the activity list to be completed changes on the remote site, save a new HTML file and re-run the script.
        • Activity completion – run the parse completion script
          • get progress data – DONE
            And Moodle has changed the format of their spreadsheet. It appears that the … has been removed. Let’s try that.
          • update the local/remote mapping – DONE
          • check the students – DONE
          • update the database – DONE
    • Status
  • Student blog data.
    • Source – HTML file from BIM on remote site
    • Destination – local Moodle BIM tables
    • Method
      • Initialisation – DONE
        1. Create the course
        2. Create the BIM activity
        3. Run the user scripts
        4. Modify the bim/parseBIM.pl script
      • Maintenance
        1. parse the file
        2. Get the users and check data
        3. insert the data – DONE
    • Status – simply save the updated HTML page and re-run the script when updates required

Greasemonkey script

Well the data is in and the update process should be fairly straight forward – though still somewhat manual. Time to move onto the Greasemonkey script. This is what recognises and updates the Moodle page, communicates with the details server and displays a pop up. Changes will likely need to be made to both the client and server to serve the new course.

Greasemonkey first

  • Recognise the course – 7023
    Add that Moodle course id to an if statement and it should start working? As long as I remember to reload the modified script. Yes. It is recognising it.

Server

  • Map course id to BIM
  • Map remote course id to local id
  • need to abstract out the WEEKS data – at the moment this is hard coded to the weeks, but it will do
  • And the module mapping – DONE
  • Fix up the dates

Still some minor bugs, to do list includes

  • Get the accordion working.
  • Timeago isn’t calculating the time since/till end of weeks.
  • Missing data- GPA, # courses complete
  • Add in the sentiment analysis work.

How might a Book module search function work?

What follows is some explorations of the technology behind Moodle to see how the mockups of a Book module search function might be implemented. Building my own knowledge of how it works and testing whether it might be possible. It’s largely an exploration of how the Search forum block works on the assumption that mirroring that will be a reasonable way to implement a search facility for the Book module.

Two parts to this. First, a look at how the Search forum block is implemented. Second, some rough exploration of how what might need to be done for the Search book block.

Update: It does appear that there was some work done on this. Some working code as a block, but doesn’t appear to have made it into the Moodle plugins database. Will need to chase this up.

Search forum

The Search block

~/moodle/blocks/search_forums implements the search block. The main purpose of this seems to be to generate the HTML necessary to display the block.

All the actual work appears to be passed over to the actual Forum activity module (the file search.php).

Forum activity module – search

~/moodle/mod/forum/search.php is one of those ugly PHP scripts. Apparently hasn’t been changed in almost a year (as you’d kind of expect).

  1. Handle the parameters.
  2. Look at the parameters and attempt to figure out what the search terms will be.
  3. Clear the search terms
  4. check access.
  5. generate an appropriate event
  6. Show the form if we’re not doing a search.
  7. Use forum_search_form to populate the form
  8. Use forum_search_posts to do the search
    • If there aren’t any posts, show an empty form and exit
  9. Display the results
    • Show the start of the response with all search terms entered appropriately
    • Loop through each post and display it (using a long bit of code)
    • Show the footer

Support functions in search.php

  • forum_print_big_search_form
  • forum_clean_search_terms – removes any search strings less than 2 characters long and returns a space-separated search terms
  • forum_menu_list – what is the list of forums the user can view?

forum/lib.php

Contains some other functions used.

  • forum_search_form – generates the much smaller search form.
  • forum_search_posts – returns the list of posts matching the search terms
    Involves the following steps

    1. get the readable forums for the user.
    2. for each of the forums
      • generate different SQL statements based on different types of forums.
      • users search_parser and search_lexer to do some checks on the search strings.
      • further updates the SQL
      • query the database to get the number of matching records.
      • query the database to get a subset (if large) of the records

Search book

Given that the Book module doesn’t have the same variety and complexity in terms of different types of forums etc, implementing the search would seem to be quite easy. Hidden chapters perhaps the main added “complexity”.

Block

Would seem to be fairly straight forward and easy to implement.

001_SearchBooks

Advanced search form

003_searchShowingBooks.tiff

The main additional complexity here would be to generate the list of course topics and the books that belong to each topic.

What I’m calling “topics” are stored in the table course_sections.

  • Can be visible or not.
  • Can be available or not.
    Important: Only search sections the user can see.
  • Name is included.

The modules (activities) that are included in each course section is stored in course_modules

  • module = 3 suggests a book
  • course is the moodle id for the course.
  • section is the section id
  • will be a question of whether or not the user can see the book
    Important: ensuring that only the books the user is able to see are searched will be important.

Searching and displaying the book and its chapters

004_results

book table includes the name and introduction to the book. No indication of who created it.

Important: Is it possible (somewhat easily) to identify someone as the author of a book?

book_chapters table contains information about each of the chapters, including

  • Is it hidden or not
  • Where the chapter was imported from.
    Important: This would be a way of identifying people who have used the import facility
  • The actual content, chapter title, whether it’s a sub chapter or not.
  • pagenum – indicates the order of the chapters/pages
    Important: Use this to order the search results within a book in order of appearance in the book.

Mocking up a search facility for the Moodle book module

Time to start generating a mock up of one of the itches identified in this presentation and from the Moodle “open” book project. Perhaps best summed up from this comment from a past EDC3100 student

It is impossible to go back and remember where to find information…At least on a PDF I can use the Find function

There is no search facility for the Moodle book module.

Update: It does appear that there was some work done on this. Some working code as a block, but doesn’t appear to have made it into the Moodle plugins database. Will need to chase this up.

The question here is whether one can one be added using the existing search forums block as the model?

This post is an attempt to mock up and explore how this might work. The following is intended to provide sufficient ideas to generate feedback and suggestions.

Search book block

The start would be the option to include a “Search Book Block” on a Moodle course page. Current plan would that it would function exactly like the Search Forum block.

001_SearchBooks

Just type in the words you wanted to search for and by default it would search all books in the current course site (results of the search shown below). If you wanted to be more advanced, use the Advanced Search link.

Advanced Search

The advanced search functionality would also mirror the Search forum block. Click on the “Advanced Search” link and you get a page like the following.

002_Search

The extra options being

  1. Search for an exact phrase.
  2. Search for books that don’t include a specific search phrase.
  3. Search for whole words.
  4. Refine the search to only specific books (more on this below).
  5. Search for words within the titles of chapters.
    Addition: should also perhaps have the option to only search for words in the title and the description of the whole book, rather than within chapters.
  6. Search for books written by an author.
    This is a little strange and somewhat borrowed from the Search forum block. Originally the intent was that the Book module might be used by students to create content on a Moodle site. I’m not sure many people use this, but the ability to search only in books written by a particular author might be useful.

Given that a course may have quite a few books (e.g. one of mine has 70+) the ability to select which books to search might be useful. You might wish to search all the books, a few books, or all books within a given topic/module. Hence the change in how to select which books to search in the following.

003_searchShowingBooks.tiff

The idea is that you would be able to select multiple books. All the books in the course would be displayed and would be grouped based on the topics/modules that they belong to.

The search results

The search forum block returns a list of matching forum posts grouped on separate pages (if needed). For each post it displays both the “header” – the author details, the forum, date posted, and the subject line – and the content. The search Book block could mirror that, but instead showing each chapter that matched the search and the book from which those chapters belonged. Something like the following.

004_results

Some of the minor suggested changes are

  • Have the results summary count the number of chapters and books that were found in the search.
  • The header of each search result will provide the full path to the specific chapter. i.e. Topic/module -> Book -> chapter.
    This might be better displayed using the standard display approach used within the Book module, but with the addition of links so that the searcher can go directly to the topic, book, or chapter.

That will do for now. Time to explore the technical feasibility of it all.

Ad hoc notes

  • Probably want to treat titles of books as a special search and result.
    e.g. be clear is a search string appears in the book title and allow to only search (or not include) book titles.
  • Should the results show the entire contents of a chapter, or just a subset?

Predicting System Success using the Technology Acceptance Model: A Case Study

Behrens, S., Jamieson, K., Jones, D., & Cranston, M. (2005). Predicting system success using the Technology Acceptance Model: A case study. In 16th Australasian Conference on Information Systems (Paper 70). Sydney. Retrieved from http://aisel.aisnet.org/acis2005/70/

Abstract

Determining what makes an Information System (IS) successful is an ongoing concern for both researchers and practitioners alike. Arriving at an answer to this problem is compounded by the subjective nature of success and therefore trying to make judgements of what is and is not a success is problematic. Despite these difficulties system use has become more accepted as a measure of system success. Following this logic if a system is accepted it will have a higher likelihood of being used and therefore impact positively on success. The Technology Acceptance Model (TAM) is one of the more widely accepted theoretical frameworks that has been used to measure system acceptance. This paper combines the TAM, as the theoretical framework, with case study research to provide a more holistic account of why a specific IS, an online assignment submission system, has become successful. Initial findings suggest that the TAM measures of perceived usefulness and perceived ease of use are effective predictors of systems success.

Introduction

Measuring success within IS has been a concern for those within the discipline since its inception. Although success is complex and therefore difficult to measure researchers have made efforts in doing so. Traditionally these measurements focus on delivering a functional IS product within certain economic and temporal constraints. Despite this bias there is evidence to suggest that a more accurate measure of success may lie within the realms of system use. Based on the logic that a system must first be accepted to be used ensuring acceptance should increase the probability of system success. One of the more popular theoretical frameworks that predicts system acceptance of technology is the Technology Acceptance Model (TAM). We use this model to try and investigate why a specific IS innovation in use at Central Queensland University (CQU) has become so popular.

Davis et al.’s (1989) work on the TAM Information Systems theory, is a user centred approach which has gained popularity as a measure of technology acceptance. TAM suggests that when users encounter a new IS innovation there are two main factors which will influence how and when they will use it. These are perceived usefulness and perceived ease-of-use. Perceived usefulness is “the degree to which a person believes that using a particular system would enhance his or her job performance” (Davis 1989). Perceived ease-of-use is “the degree to which a person believes that using a particular system would be free from as the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh and Davis 2000) there currently exists only one study confirming its validity and robustness. TAM on the other hand has been tested by many more researchers (Adams et al. 1992, Hendrickson et al. 1989, Segars and Grover 1993, Subramanian 1994, Szajna 1994) with different populations of users and IS innovations.

Due to the testing and support of the model by others we rely on the original TAM proposed by Davis (1989) rather than the extended model of the UTAUT (also known as TAM2) to measure technology acceptance. In this study we apply the model to a twelve year old IS, OASIS (Online Assignment Submission, Infocom System) in use at CQU. We apply the TAM measures of perceived usefulness and perceived ease of use to two different groups; staff non-users and staff users. The non-user group is examined due to their potential to become users and therefore impact on the continued growth of the system’s popularity. The user group is examined due to their importance in maintaining the current level of system use. Preliminary analysis into these two groups through the use of a case study approach reveals that the TAM measures appear to be useful predictors of a successful system. Our study also suggests that the evolutionary development model adopted by the system support team may have impacted positively on user perceptions and beliefs of system usefulness and ease of use.

Research approach

Rationale for using a case study approach

This research focused on determining the significant system acceptance factors that have contributed to the success of OASIS. This level of detail allows us to provide a more complete description and explanation of this particular innovation’s success. We believe that such a holistic focus of success using a proven IS is beneficial for researchers and practitioners alike. Although TAM has been the subject of investigation for much research, many of these studies are limited in several respects. These deficiencies include issues such as the strictly quantitative nature of the research with a focus on the adoption of simplistic technologies such as voicemail, email (Adams et al. 1992, Davis 1989, Segars and Grover 1993), word processing, spreadsheet and graphics software (Adams et al. 1992, Bagozzi et al. 1992). Our study helps to address these limitations in the literature by providing an in-depth qualitative study of a more complex non-mandatory system. For practitioners it provides a unique insight into an authentic and successful IS implementation. Given our main requirement of an in-depth investigation we used the case study approach. This decision is in accord with recommendations from proponents of the case study approach, for example Hamel (1993), Yin (1994), and Stake (1995).

Site selection – unit of analysis

Selection of the research site is the most critical decision in the analytic process of case study research (Hamel 1993). For our investigation, the choice of CQU as our site was a relatively simple matter. This was because CQU met all three of the main selection criteria; ‘no real choice’, suitability and pragmatism (Denscombe 1998). Firstly the site represented a unique opportunity for study. All researchers work at CQU and were able to observe the successful adoption of OASIS, a ‘home grown’ IS used in the support of teaching and learning, as well as various other information system adoption failures. The success of OASIS at CQU was an event that could not be ‘planned or created’ (Denscombe 1998). It therefore represented a ‘one-off chance’ for us as researchers to gain insight into why OASIS has become so successful where other ISs had failed. Consequently there was no real element of choice in ‘deciding’ on CQU as an appropriate site for study. Secondly the site was suitable due to the relevance of the case for testing previous theory. TAM is the theory we believe to be relevant in predicting the success of this system due to its ability to measure user acceptance. Through our own observations of OASIS and a review of the literature we believe that the success of OASIS at CQU contains ‘crucial elements’ (Denscombe 1998) of being a successful IS innovation which can test the theory of TAM. Finally the case is both intrinsically interesting and convenient for investigation. Although these final pragmatic considerations are not enough to choose the site on their own they added to the research experience allowing for a more in-depth study, hopefully appealing to a wider audience (Denscombe 1998).

Case study design type

Our case study is an investigation into why a specific information system, OASIS, is a success. Due to the formation and identification of TAM as the theoretical framework that guided our research, we utilised a descriptive case study design (Berg 2004). Our design follows the advice given by Yin (1994) and includes the five components he deemed necessary; study questions, theoretical framework, identification of the unit of analysis, logical linking of the data to the theory and the criteria for interpreting the findings. The first three (already previously stated) indicated the data we needed to collect and the last two how we would use the data. Although, according to Yin (1994) the last two are the least well developed in case studies we lay the foundations for this analysis early in our research design. Linking the data to the theory was to be done through a “pattern matching” technique (see Donald Campbell 1975 via Yin 1994). This is where several pieces of information from the same case may be related to some theoretical proposition. In our research we would determine whether the data matched or did not match the propositions put forth by the TAM theory through visual examination. The last component of determining the criteria for interpreting the findings is the most difficult of all (Yin 1994). Unfortunately there are very few guidelines on how to arrive at such a
criteria and all that exists is Yin’s very brief advice on matching rival theories. Given these difficulties we decided that we would rely on an “all or nothing” approach similar to Yin’s advice. We had to decide on whether the pattern matched with the theoretical propositions or it didn’t. Where the pattern was deemed to hold true then the theory would be tested in the positive, where it didn’t the theory would have failed to explain our case.

Methodological tactics – Data collection

A case study is an in-depth investigation that seeks to uncover the various nuances, patterns and latent elements that other research approaches may overlook (Berg 2004). It accordingly makes use of different methods to collect various kinds of empirical data (Hamel 1993). Our case study made use of the more traditional methods of collecting data. These were questionnaires, participant observation and the review of documents. Data collection through the participant observation and document review techniques is representative of the 12 year period. Participant observation was a crucial data gathering technique as all researchers have been involved with OASIS at some point over its lifetime. Documentation used in this research included support logs and records from the OASIS system. To capture current perceptions, questionnaires were sent to all current user and non-user groups of OASIS identified as academic staff.

At the time of the writing of this paper 94 responses (34.9%) had been received from users of OASIS and 18 responses (15.3%) from non-users. The analysis in this paper concentrates on the free text responses to two open questions asking staff about the factors influencing their perceptions of the usefulness and ease of use of OASIS. Academic staff members in charge of a course are responsible for making the OASIS adoption decision. Students are only able to use OASIS after this adoption decision is made. For this reason we have initially focused on academic staff as the users and non-users of OASIS. Subsequent research will investigate student perceptions.

Scientific value and study limitations

The case study is a popular research approach across many disciplines both basic and applied (Hamel 1993). Despite their popularity they have many strong critics due to the belief that the approach lacks insufficient objectivity and concern over the ability to generalize research results (Berg 2004). We have been careful in our research to maintain objectivity through deliberate construction of a research design. Construction of such a design does much to increase the rigor of a study and counter the claims of “weak research” (Yin 1994). As part of our research design we maintained as much objectivity as possible by having each researcher separately review the evidence as part of the data analysis phase. Our findings have been examined within the context of our chosen theoretical framework; TAM. As far as generalizing the results of this research is concerned our view is similar to that taken by Berg (2004). He states that “When case studies are properly undertaken, they should not only fit the specific event studied but also generally provide understanding about similar … events. The logic behind this has to do with the fact that few human behaviours are unique, idiosyncratic and spontaneous.” (Berg 2004). Likewise it is our belief, due to the socially constructed nature of IS in general, as well as their reliance on social aspects as determinants of success in particular, that the success of OASIS is not a unique event. Our study should therefore provide an understanding to the wider community of success in similar IS implementations.

Theoretical background

Success of an IS innovation can be determined in a number of ways. However, general organisational measures of success include “on time and on budget” (Standish Group 1995, IT Cortex 2002) with the desired functionality (Mahaney and Lederer 1999). Following the logic inherent within the literature focussing on IS characteristics predictive of success, failure is generally discussed in terms of having the opposite characteristics. For example, Whittaker (1999) described the 1997 KPMG survey on what constituted an IS project failure. The study deemed a project as having failed if it overran its budget by 30% or overran its schedule by 30%, or the project was cancelled or deferred due to non-delivery of planned benefits. However, Mahaney and Lederer (1999) argue that there are degrees of failure and that a project that overruns budget by 5% is less of a failure than one that overruns by 50%. Some determinants of why IS innovations might be considered failures include whether they; have the ability to evolve and grow with the organization, integrate well with the business environment, possess consistency between the initial requirements and the final solution, simply make business sense (IT Cortex 2002).

DeLone and McLean (1992) were leaders in moving to a more user centred approach when trying to judge overall IS success. Their model suggests six interdependent measurements of success; system quality, information quality, use, user satisfaction, individual impact and organisational impact. It is important to note that all of these factors should be considered when trying to measure success under the model and that no single measure is intrinsically better than any other. Further attempts have been made to refine and expand on their model by others (e.g. Seddon et al. 1999) as well as minor refinements suggested by themselves (DeLone and McLean 2003). However as DeLone and McLean (1992 p. 61) themselves point out, “there are nearly as many measures of success as there are studies”.

With the more recent study conducted by Iivari (2003) there is more evidence to suggest the applicability of the DeLone and McLean (1992) model in measuring a system’s success. This work helps to contribute to the shift from organisational measures of success to more user focused measures. Davis et al.’s (1989) work on the TAM IS theory, is a user centred approach which has gained popularity as a measure of a users acceptance of technology. We draw the conclusion that if a system enjoys high user acceptance this will impact positively on system use. Use of the system is a contributing factor to system success especially when that system is not mandatory (DeLone and McLean 2003, Iivari 2005). Based on this assumption we use TAM as a theoretical framework to guide our research. Specifically, do the constructs in the model offer a reasonable explanation for why OASIS has enjoyed such an exponential growth in its adoption and use?

System background

Online Assignment Submission, Infocom System (OASIS) arose out of early experiments in 1994 by a single academic implementing a system to reduce assignment turnaround for distance students (Jones and Jamieson, 1997). Adoption of OASIS by other academics was limited at this time. Only 13 course offerings made use of the system with just over 1900 assignments being submitted in the six years up until 2000. Since 2000 use of OASIS has increased significantly. From the years 2000 to 2005 over 77,000 assignments have been submitted via OASIS by 6892 (72+%)of Infocom students.

Students enrolled in Infocom courses are distributed across a number of campuses as well as being enrolled via distance education. There are five regional Central Queensland (CQ) campuses in Bundaberg, Emerald, Gladstone, Mackay and Rockhampton. Four other Australian International campuses (AICs) in Brisbane, Gold Coast, Sydney and Melbourne managed by a commercial partner. Campuses are also located overseas in Fiji, Singapore, Malaysia, Hong Kong and China. Students may also study from any location in the world via distance education (FLEX). Figure 1 provides a summary of Infocom student numbers from 1996 to 2005.

OASIS usage in percentage

Figure 1. Number and type of students enrolled in Infocom Courses (1996-2005)

Since its inception Infocom has had a small development team responsible for its online presence. In 2001, partly in response to increasing numbers, this team was expanded and additional effort placed on providing services that would help support Infocom’s teaching operations. Using an agile development methodology (Jones and Gregor, 2004) this group, in response to direct user feedback, made a range of additions to OASIS to improve its functionality. The combination of increasing complexity and this on-going development of OASIS appears to have had an impact on usage of OASIS. Figure 2 shows percentage of Infocom students, staff and courses using OASIS from 2000-2005. Specific staff figures are only available from 2002 onwards when a markers’ database was added to the system.

Number and type of students enrolled in Infocom Courses

Figure 2: OASIS usage in percentages of Infocom students, courses, and staff (* figures as of September 30, 2005)

Findings

In this study we applied the TAM measures of perceived usefulness and perceived ease of use to two different staff groups; non-users and users. Non-user perceptions concerning usefulness and ease of use of the system are used to compare against actual beliefs of users. The non-user group is examined due to their potential to become users and therefore impact on the continued growth of the system’s popularity. The user group is examined due to their importance in maintaining the current level of system use. This section reveals the complex underlying belief structures concerning the two constructs of perceived usefulness and ease of use as they pertain to OASIS. This is information that has been missed in other investigations concerning TAM (Segars and Grover 1993). Of particular interest it reveals that usefulness and ease of use, at least in our study, seem to be influenced positively by the evolutionary development model adopted by the system support team. This is in line with other research concerning what makes for a successful system, namely the ability of the system to evolve with the business (IT Cortex 2002). The findings in this study are of a preliminary nature only.

How do non-users perceive OASIS?

Overall non-users of OASIS had mainly positive perceptions of the system. These perceptions centred on the belief that the system would benefit the students just as much as it would enhance course management.

Perceived usefulness factors

The main student benefit perceived by non-users was timely turn-around of assignments. One respondent noted that ‘OASIS will eliminate unnecessary delays’ while another believed that ‘It may also help with prompt and efficient grade information requests in/out’. Other respondents saw OASIS as a method for improving courses by being able to more easily analyse the results from assessment. One respondent noted that ‘each question can be analysed for effectiveness at distinguishing between students passing and failing. With individual questions assessed for how well they are answered teaching can be modified to prepare students better in the identified weak area’. Another was the benefits in being able to track how well students were progressing, stating that ‘I envisage OASIS would be useful to gauge student progress/understanding/level of expertise throughout a particular course/subject’.

Non-users also perceived a number of administrative benefits from OASIS. Of particular note was the ability to track assignments and marking, with one respondent stating their belief that OASIS ‘will encompass safe guards for assignment delivery and return, as well as acknowledgement of assignment receipt for students’. This is an important aspect of course administration, especially with the difficulties in distributing, managing and moderating marking over multiple campuses and markers. Another respondent support this belief, stating that ‘I have used a similar system before and it was quite helpful to my consolidating marks, and not being on campus would probably simplify the marking system’. Respondents believed that OASIS would make this task both possible and easier. Others believed that OASIS would also provide additional benefit by enabling the use of automated plagiarism detection.

While there were few negative perceptions of the usefulness of OASIS, some respondents had beliefs about what types of assessment OASIS was suitable for. One noted that ‘I believe OASIS is suitable for multiple choice questions. But my assignments are essay type with computer program printouts. As of now, I don’t know how I can make OASIS useful for my course’. Another indicated that OASIS didn’t fit in with the way that they currently assessed, stating that ‘I mark all my student’s assignments manually, it is easier for me to sub-edit stories that way.’.

However, respondents generally had positive perceptions of OASIS, with one respondent stating that ‘certainly, any online submission technology would be useful to me. And the precedent of other IT systems made available in Infocom suggests that it would be extremely user friendly for people with very limited computer competence/confidence. The nifty acronym is also appealing’. The successful evolutionary development process adopted by the support team had produced a number of successful systems and helped in developing positive perceptions amongst users of new systems

Perceived ease of use factors

Non-user respondents generally believed that OASIS would be easy to use. They justified this with two belief factor groups. The first group was the technology centric belief that, as non-users had used similar systems, they would be able to easily use OASIS. One respondent noted that ‘I have been using computers for many years, including online application/enrolment. These may not be identical with OASIS, but I believe there will be similarities’. Another respondent stated that ‘It should not be difficult for me to learn since I’m computer literate’. Another respondent believed that the system would be just as easy to use as other systems developed by the faculty, stating that ‘my positive experience with other Infocom systems gives me confidence that OASIS would be no different. The systems team have a very good track record that inspires confidence’.

The second group of factors was based on having not heard negative things about ease of use of the system. One respondent noted that ‘nobody seem to complain too much about OASIS being hard to use, or hindering them in their job’. However, this was contradicted by another respondent who stated that ‘I have heard from another tutor that OASIS is a bit time consuming and a little confusing… but [I] have not used it myself’.

How do users perceive OASIS?

Users of OASIS had generally positive perceptions of the system. Again, these perceptions centred on the belief that the system would benefit the students just as much as it would enhance course management. Yet users were pragmatic in their beliefs and many discussed the “trade-offs” associated with using OASIS. However, a new category of perceived usefulness was uncovered concerning the personal benefits of using the system.

Perceived usefulness factors

Users believed that OASIS gave them a greater ability to monitor student progression while also allowing students to track their assessment through the marking process. One respondent noted that ‘submission records for students are useful in monitoring my students’ progress, hence adjust tutorials/support as needed’. Another respondent supported this by stating ‘it is easy (and quicker) to know if a student has submitted work for assessment by checking the relevant section of the web site’. Others saw the advantage in being able to compare and contrast assessment results, highly rating OASIS’s functionality to give teaching staff the ‘ability to compare your student’s results with overall performance’. Many staff also saw the non-repudiation aspects of the assignment management as being advantageous, with one user stating ‘students cannot say that they were NOT late or did submit the assignment (when in fact they did not)’.

Administratively, users discussed a number of factors that they perceived made OASIS useful. Most of these concerned assignment management issues. One issue identified was the ability for the user to track where an assignment was and what actions had been performed on it, with one respondent noting that ‘[OASIS] makes assignment collection simple and easy also [as you] do not have assignments go missing. [OASIS is a] quick and easy way of returning assignments and collect the assignment marks’. This tracking also facilitates moderation processes, and as noted by another respondent, ‘OASIS allows for the moderation process to be carried out in a timely fashion”. OASIS was also seen to support core academic requirements as exemplified by one user who stated that ‘OASIS is useful in the case of essay and report type assignments as it helps in detecting plagiarism’. This issue is especially important over a multi-campus operation, with another respondent adding that one of the key usefulness factors was the ability of OASIS to ‘perform copy detection, not only within a campus but between campuses’

The users of OASIS also found a personal usefulness factor in the ability to remotely download assessment to mark and moderate. This gave them the ability to mark, moderate and manage assessment from anywhere in the world. One respondent stated that having ‘assignments on soft-copy [was] a tremendous help [because there was] no need to carry them home’. It was sessional staff who seemed to gain the most benefit from electronic access to assessment. One user explained: ‘I am sessional lecturer and OASIS makes it possible for me to download the assignments for marking. I don’t have to go to the campus to get the submitted assignments’. Another supported this by saying that ‘OASIS is useful because it has enabled me to work from home and pick up students’ assignments outside office hours’.

The most contentious usefulness factor was the benefit OASIS was able to provide in the time taken to mark assessment. While many users found the systems fast and efficient to use, others disagreed, but continued to use the system because of other usefulness factors. One user who found that OASIS saved time stated that:

Having also been a marker (both paper-based, and using OASIS), I was stunned by just how much time was saved by no longer needing to handle piles of paper. Virus scanning 100 floppy disks, for example, takes a long time. OASIS provides a neatly formatted, scanned, and correctly-named set of files

However, several users found the process of dealing with electronic assignments time consuming and cumbersome. One respondent stated that ‘practical experience with many assignments that were to be submitted through the OASIS system indicates that … it takes much more time and effort to mark assignments on line’ while another noted that ‘for assignments that where marking can not be automated it is very time consuming to mark electronic copy, especially when there is significant reading to be done. it is also time consuming to provide feedback’. Many staff made comparisons with OASIS and “hard-copy” marking, with one user stating that:

It is a very good way of submission and collection of assignment. But, the hardest part of it, is adding comments electronically during the marking process. It kills time. I did same types of marking to some other university, and later they decided to take hardcopies and to write comments. We found that saving 50% of the overall marking time.

The general experience of users of OASIS was that it was initially slow to mark assessment with, but as one user who stated that ‘[marking with OASIS] takes a little longer to generate a rhythm to freely mark assignments in an efficient timeframe’. Many noted that the time to mark assessment was dependent of the type and complexity of assessment, with one user stating ‘the experience is very dependent on the assessment design’. Even with these negative aspects, users still perceived it as useful with one user summarising by stating that ‘it’s a great system but online marking and commenting takes significantly longer than on hard copy – other than that I like its functionality’.

The only other negative factors that affected perceived usefulness were those concerning support. Some users felt that OASIS was complicated, difficult to understand and lack support mechanisms. One user’s frustration was evident from the comment ‘just trying to understand how to use [it] is a pain’. Yet other users made particular mention of the support services offered by the web team. This perception of the support services will be discussed later as a factor concerning ease of use.

Perceived ease of use factors

Users generally perceived OASIS as easy to use, however two factor groups, technology and support, affected these beliefs. Technology affected users in both positive and negative ways. Users who were comfortable with technology believed OASIS was easy to use and made comments such as ‘being an IT professional, I find it very very easy to interact with’. However, users who found technology intimidating focused on this as impeding their use of OASIS. One user remarked that ‘anything computer-mediated the comments relating to ease of use and technology were focused on difficulties with understanding technology external to OASIS rather than the system itself

A related group of factors concerned support mechanisms. As previously discussed, the perception of a lack of support mechanisms impacted negatively on the perceived usefulness. However, users of the system were divided on the issue. In particular, those users who had used OASIS over a long period of time and had watched its support mechanisms evolve saw them as a positive influence on ease of use. One respondent supported this by stating that ‘It used to be a problem, but I’ve seen the system and supporting documentation improve to the point that I would consider the system fairly easy to use for new users. Support requests for the system have dropped significantly as it has matured’. Another remarked that ‘OASIS is self explaining, there is not much to learn about it in order to use it’. However, others still regarded the system as difficult to initially learn, with one noting that ‘learning OASIS for the first time is difficult because the instructions are not very clear. However, it is easy once you get the hang of it’. Others noted that the lack of documentation and online help procedures was overcome with support from the web team, with one respondent commenting that ‘learning how to do things in the system is not easy but the tech team offer an excellent support and are to be commended for their efforts’. Most users shared these views believing that once users started using OASIS, the perception of ease of use changed. This was supported by one user who stated that ‘OASIS is no more difficult or easier to use than any other web-based system with online help and hyperlinks to the various relevant parts. I think initially I asked colleagues about its general use as the concept seemed daunting at the time (before I’d actually used it)’.

Discussion and conclusion

In general users had very positive perceptions surrounding the usefulness of OASIS. As one respondent stated, ‘I find the system professional and bug free. It’s an excellent assignment management tool. It provides a rigid framework for student submissions. Students appear to have little or no problem with the general concept of online submission and its use. In all I find the system very useful’. If system success can of this system by the majority of students in the Faculty of Informatics and Communication at CQU and its use by non-mandatory nature of the system giving even more strength to the motivations behind its adoption. In the search for an explanation of why it has been so successful we applied the TAM to investigate both staff users and nonusers of the system. Both of these groups revealed very positive perceptions and beliefs surrounding the usefulness and ease of use constructs in TAM. On further investigation these constructs were complex in nature but seemed to centre more on the administrative benefits that the system could provide rather than the pedagogical benefits originally intended by use of the system.

In examining non-users perceptions and users’ beliefs, we have presented evidence that provides an explanation for the continued and growing success of OASIS. Non-users perceive that the system will be useful and easy to use and will not hesitate in using it when the chance arises. This indicates why the believe that the system is useful and easy to use and this explains stability in growth and continued use. If success can be measured in terms of use then we believe that the usefulness and ease of use factors within TAM are reasonable predictors of system success. We also believe that the usefulness and ease of use constructs are positively influenced by the successful application of agile development methods employed by the support staff of the system. As shown above this process has generated a perception amongst staff that the systems produced by this team will be useful and easy to use, or if the systems are not useful or easy to use that the predict that the success of OASIS will continue as long as the beliefs and perceptions concerning the system’s usefulness and easy to use characteristics are maintained through activities such as evolutionary development.

As stated in the previous section, methodological tactics, our findings are limited to initial analysis of two free text questions. Further detailed analysis has to be carried out on all questions. It is envisaged that in the information of respondents. This may impact on their perceptions of perceived usefulness and perceived ease of use of the system. Although the preliminary results of this study offers a more detailed account of a specific systems success it would benefit from expansion in several areas. Firstly, expanding the study to include students. Secondly, using UTAUT instead of TAM as the theoretical framework. Thirdly, including other cases to see whether the results still hold. Finally it may be useful to investigate further the complex structure of the perceived ease of use and perceived usefulness constructs of TAM to other information systems supporting teaching and learning as well as other more complex information system innovations.

References

Adams, D. A., Nelson, R. R. & Todd, P. A. (1992) Perceived usefulness, ease of use, and usage of information technology: A replication. MIS Quarterly, 16, 227-247.

Bagozzi, R. P., Davis, F. D. & Warshaw, P. R. (1992) Development and Test of a Theory of Technological Learning and Usage. Human Relations, 45.

Berg, B. L. (2004) Qualitative Research Methods for the Social Sciences, Boston, Pearson Education, Inc.

Davis, F. D. (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319-340.

Davis, F. D., Bagozzi, R. P. & Warshaw, P. R. (1989) User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35, 982-1003.

Delone, W. H. & Mclean, E. R. (1992) Information systems success: The quest for the dependent variable. Information Systems Research, 3, 60-95.

Delone, W. H. & Mclean, E. R. (2003) The DeLone and McClean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems, 19, 9-30.

Denscombe, M. (1998) The Good Research Guide for small-scale social research projects,
Philadelphia, Open University Press.

Hamel, J. (1993) Case Study Methods, Newbury Park, SAGE Publications.

Hendrickson, A. R., Massey, P. D. & Warshaw, P. R. (1989) On the the test-retest reliability of perceived usefulness and perceived ease of use scales. MIS Quarterly, 17, 227-230.

Iivari, J. (2005) An Empirical Test of the DeLone-McLean Model of Information System Success. The DATA BASE for Advances in Information Systems, 36, 8-18.

IT Cortex (2002) Success assessment: Where is the limit between success and failure? Chapelier, Jean-Pol.

Jones, D. & Behrens, S. (2003) Online Assignment Management: An Evolutionary Tale. 36th Annual Hawaii International Conference on System Sciences. Hawaii, IEEE.

Jones, D. & Buchanan, R. (1996) The design of an integrated online learning environment. IN Allan Christie, P. J., Beverley Vaughan (Ed.) Proceedings of ASCILITE’96. Adelaide.

Jones, D. & Jamieson, B. (1997) Three Generations of Online Assignment Management. IN Kevill, R., Oliver, R. & Phillips, R. (Eds.) ASCILITE’97. Perth, Australia.

Jones, D., Lynch, T. & Jamieson, K. (2003) Emergent Development of Web-based Education. Proceedings of Informing Science + IT Education. Pori, Finland.

Mahaney, R. C. & Lederer, A. L. (1999) Runaway information systems projects and escalating commitment. Special Interest Group on Computer Personnel Research Annual Conference. New Orleans, Lousiana, USA, ACM Press.

Seddon, P. B., Staples, S., Patnayakuni, R. & Bowtell, M. (1999) Dimensions of information systems success. Communications of the AIS, 2.

Segars, A. H. & Grover, V. (1993) Re-examining perceived ease of use and usefulness: A confirmatory factor analysis. MIS Quarterly, 17, 517-525.

Stake, R. C. (1995) The art of case study research, Thousand Oaks, CA, Sage.

Standish Group (1995) The CHAOS report. The Standish Group International.

Subramanian, G. H. (1994) A replication of perceived usefulness and perceived ease of use measurement. Decision Sciences, 25, 863-873.

Szajna, B. (1994) Software Evaluation and choice: predictive evaluation of the Technology Acceptance Instrument. MIS Quarterly, 18, 319-324.

Whittaker, B. (1999) What went wrong? Unsuccessful information technology projects. Information Management & Computer Security, 7, 23-30.

Yin, R. K. (1994) Case Study Research Design and Methods, Thousand Oaks, SAGE Publications.

Import and the Book module: a case of knowledge loss?

The ability to modify open source software (like Moodle) is often identified as one of its strengths. The ability to scratch your own itch, to modify the software to fit your needs is seen as a major plus. Especially when – as with Moodle – that software has an inherently modular (the M in Moodle stands for modulear) architecture that makes it easier for people to scratch their own itch. But making modifications to large bits of software like Moodle requires a fair bit of specific knowledge.

Costello (2014) points out that the “perception that Moodle is easier for institutions to adopt to their own needs is widespread” (p. 193) and that this ability to modify Moodle can form part of the rationale for adoption. Especially for institutions that see themselves being “a co-creator and developer of innovations rather than simply buy(ing) them already packaged up” (Costello, 2014, p. 193). The reality is often messier. The following is capturing one example of that messiness.

The import problem

As outlined in this post I had a problem with the Moodle Book module and its import functionality. A problem illustrated by the following image. In short, the import functionality allows you to provide a zip file containing multiple HTML files. The import should unpack that zip file and place each of the HTML files into the book as separate chapters.

Problem with Moodle book import

But as the image above shows I found two problems

  1. The order of the files were changed from what I wanted into a form of alphabetical order.
  2. The names of the chapters would retain file system components (e.g. _sub in the filename indicating a sub-chapter and the .html extension)

I checked the documentation on importing at the time and seemed to be doing all that is required. This was somewhat frustrating because it required a lot of manual intervention on my part to set right.

Disbelief by kmakice, on Flickr
Creative Commons Creative Commons Attribution-Noncommercial-Share Alike 2.0 Generic License   by  kmakice 

My immediate response was something like that to the right. The Book module has been around for a while. It’s part of the Moodle core. Importing HTML files into a Book is likely to be a fairly common practice. Surely this couldn’t be how it was left to work. After all, “given enough eyeballs, all bugs are shallow” is a mantra of the open source community.

Oh well, I like to code and perhaps this problem is an example of where I can make a difference. I look at the code for the Book module and fairly quickly see where the problem is, can conceive of and implement a solution, and contribute it back to the broader Moodle community. The patch is sitting there waiting to be checked and perhaps might eventually become part of the Moodle core.

A happy story of the open source model working (if a little slowly).

But it’s not as simple as that

I talked about this and other itches around the Book module in this presentation at Moodlemoot’AU 15. Quite a few people attended, quite a few commented positively, and the presentation has been viewed 670+ times on Slideshare. Lots of eyes.

But it’s only yesterday when I get an email from Jonathon Fowler a Moodle developer (amongst other things) where I work. On hearing about the import problem his reaction was much the same as mine. A bit of disbelief. Johnathon knows the original developer and didn’t believe he’d leave the import function sitting in that state. He looked at the code and he was right. It hadn’t been left in that state.

Apparently, the import function will correctly order and name the chapters if you

  1. Put the title of each chapter (HTML file) in the title tags of the HTML header.
    The following chapter will have the chapter name “Overview”
    [code lang=”html”]
    <html>
    <head>
    <title>Overview</title>
    </head>
    <body>
    [/code]
    Why didn’t I do this? Because my authoring process was to edit a single HTML file where the chapters were separated by div tags. A Perl script separated the single HTML file into multiple and stuck them in a zip file. I need to modify the Perl script to add in the title to the single HTML files.
  2. Add numbers to the start of the file names.
    So with the example from the image above, the required file names become
    [code lang=”bash”][/code]
    1. Overview.html
    1.1 Create a new book_sub.html
    1.2 Rewrite an existing book_sub.html
    2 The import itch.html
    [code][/code]
    And the script will need to be modified to add numbers to the file names.

Tested and that all works.

Doh!

/doh by striatic, on Flickr
Creative Commons Creative Commons Attribution 2.0 Generic License   by  striatic 

Appears I’m a bit too quick to engage in a bit of minor “solutionism” rather than try to understand the features of the tool. There was a problem, there seemed a simple (fairly) clean solution and so I solved it.

I’ll reflect a bit on any potential broader implications later, but first.

Improve the documentation

Before I go too much further I need to improve the documentation on importing into the Book module so that no-one else makes a similar mistake.

Uggh, the docs are in some hideous markup language. Moodle wiki? Perhaps best to look at the guidelines for contributors first. It does appear to be using Mediawiki, better go find some formatting help.

Not happy with it, but it’s a slight step forward.

Bugger, just realised that this fixes the documentation problem on the Moodle docs website (for 2.8 anyway), but it doesn’t fix the inline help that appears in the Book module. It still has much the same

I should also update the tracker item I added to make sure no-one else wastes their time on this. Will do that after I post this.

Broader implications?

I wonder if this points to anything broader beyond lack of attention to detail and a tendency to code up a solution.

It does add a small anecdote supporting the idea that (one of) Linus’ laws (i.e. more eyes == bugs are shallow) is a fallacy. Quite a few people saw me mention this “bug”, but no-one (until Jonathon) picked up my inability to identify the existing functionality.

It appears no-one currently involved knows the Book module well enough to have been aware of this functionality. The documentation also shows no explicit mention of this “hidden” but needed functionality.

I wonder how many people have had this problem? Or is it just me? Update: able to do a quick check of an institutional Moodle, 17 (2.2%) of 759 books in that sample were imported. And those were from a single course.

It appears that the “lack of love” given the Book module over recent years is showing in terms of lost knowledge. How does a large, modular open source project detect, avoid, or prevent this type of “knowledge loss”?

All up I’ve invested a fair bit of time in this experience (just this blog post alone consumes a bit of time) for fairly minor improvements. This work is not exactly going to directly transform the learning of 1000s of learners. What would be required if you really wanted to do something interesting, impactful, and complex?

A lot of that time has been spent learning or re-learning knowledge about how to engage with the Moodle ecosystem. You need to know something about software development, PHP, github, Mediawiki, Moodle documentation and associated processes etc and then you get to the really hard stuff, knowing what the people using the system want to do and how that might translate. From all that I’ve seen the Book module once had someone that had that knowledge, but it appears to have been lost.

For an individual (or organisation) to get their head around all that requires quite a large investment of resources/knowledge. Moodle core have the technical knowledge down pat and are adding more developers, but what about the knowledge of the user experience? They are also taking steps in that direction, but that in itself is going to generate a need for more resources.

Leading to questions of prioritisation, which inevitably leads to the problem of stavation.

Has me wondering if it’s time (and possible) to add another layer of abstraction here. Is something like Moodle at too low a level of technical abstraction?

Testing my patch

The next question is what impact my patch has on this behaviour.

Will the normal Book functionality work?

The need to include the numbers within the filename of each HTML file, essentially re-creates what my patch does. Hence my patch should work okay. The order with numbers should be maintained.

Yes, that appears to work. The numbers are removed as expected.

What if I remove the numbers and rely on my patch for ordering?

Yes. This will and does work.

What if my script includes the chapter title in the title tag?

Modified the script to add the filename plus “XXXXXXX” to the title tag within the HTML file, and that’s what is used for the chapter names.

Implications

It appears that my patch plays nicely with the existing functionality. Begging the question, does my patch add anything? Well,

  • If you leave the numbers out of the filenames, my patch will order the chapters in the same order as the zip file (rather than a natural sort).
  • If you don’t put anything in the title tag of the HTML file, then my patch will nicely remove the _sub and .html parts of the filenames.

Not a complete waste of time, some small benefit.

I’m slightly relieved that my code didn’t break anything.

References

Costello, E. (2014). Opening up to open source: looking at how Moodle was adopted in higher education. Open Learning: The Journal of Open, Distance and E-Learning, 28(3), 187–200. doi:10.1080/02680513.2013.856289

Me as learner: 2015 and learning analytics

It’s July 2015 which means two things. First, Semester 2 is about to get underway and thankfully (though it’s a two-edged sword) EDU8117, Networked and Global Learning (netgl) is running again. Second, it’s summer in the northern hemisphere which seems to coincide with another round of xMOOCs running out of the US. Time to combine them both.

Week 1 of netgl includes a task that asks the participants to write a blog post about “Me as learner”. The idea is that during the first section of the course the students should head out into the digital world and try to learn something using netgl. Doing so is meant to provide a focus for what they are reading and doing around netgl. What follows is “me as learner” this time around.

What would you like to learn? Why?

Many, many things, but in this case I’m going to try and focus on educational data mining. i.e. the specific algorithms, approaches and software that can be harnessed to analyse and understand the mountain of data generated through the online courses I teach. There are perhaps three main reasons why I want to learn more about this

  1. Inform research.
    I’ve written and given presentations about learning analytics for sometime now. However, the focus of these have tended to be at the organisational or implementation level. Not at the level of actually applying learning analytics to data and revealing something. I need to do this for the reasons below and because I think it is useful research method and that there may be some limitations about how it’s currently being applied. A major barrier in doing this is that while I’m fairly competent from a technical perspective, I’m not strong on “data science”.

    Beyond specific learning analytics research, completing an edX MOOC (I’m assuming designed more as a xMOOC, than a cMOOC) while participating in netgl will also provide some interesting insights into the different perspectives that inform different approaches to netgl. Especially if I actually follow through and actively participate in the course both within and outside the MOOC specific space.

  2. Improve practice.
    The majority of my students are online. All of my courses are taught heavily online. Being able to harness educational data mining will help understand what’s going on within these courses and inform future re-designs. But I also think that appropriate application of educational data mining can help with the actual teaching (orchestration). In particular, I think it’s an avenue that can enhance student learning, but also address some of the huge holes that exist in the e-learning infrastructure that I’m currently lumbered with.

    Especially because the learning objectives for the MOOC I’ll attempt includes

    How to use methods to answer practical educational questions

    Observing how and experiencing the design of the MOOC may also reveal some “good practice” tips to adopt. After all, this is a large American organisation drawing on lots of resources and lots of smart people. Surely this poor Australian has something to learn from their experience and use of the e-learning literature?

  3. Actually complete a MOOC (of any breed).
    I’m also hoping that by combining this with EDU8117 that I might actually complete a MOOC. Not something I’ve done previously, and to be honest probably not something I’ll do this time. The week 3 material of the MOOC I intend to undertake has just been released. If this were an Australian university, I wouldn’t be allowed to enrol. It’s already past the week 2 cut-off.

The MOOC I’m going to try and complete is Big Data in Education.

Why have I gone with a formal task?

Early birthday present

I had seriously thought about using this as an opportunity to recapture my high school music experience and learn more about playing my birthday present this year (see the image to the right). This is more the type of “learning” I had in mind when designing the “me as learner” task for netgl. The idea was to move beyond more formal learning topics. Some previous participants have taken this approach and tried to learn about topics as diverse as baking bread or playing a particular bluegrass “fiddle” song/approach.

There are two reasons why I didn’t go down this route:

  1. Embarrassment.
    Learning via netgl means (at least for me) making your learning public. Not the end product, but the struggles, small wins, steps backward, and all the other dirty embarrassing missteps that are part and parcel of learning. It’s by making these public that you increase the chances of connections between you, others, and other perspectives and ideas. It’s by making these public that you open up the possibility of netgl really helping enhance your learning. In terms of learning the saxophone that might mean joining Soundcloud, sharing my “performances” (and hopefully their on-going improvement), engaging with critique and learning.

    Being open like that is hard. Especially if you are a novice, and trust me it’s 30+ years since I regularly played a saxophone and it shows. Not to mention if you are a middle-aged professional in a job where you’re expected to be the expert (which is just a bit of a laugh). Add in being an introvert and you get the picture.

    At the same time, I’m meant to be somewhat knowledgeable about learning analytics. Sharing my learning about educational data mining will clearly indicate just how limited that knowledge is (in one aspect). This knowledge is something that is much more central to my identify as an academic, and yet I don’t feel the same sense of embarrassment about it. Even to the extent that this paragraph is being added after taking some time away from this post.

  2. Workload.
    Chances are that this will sound like a cop out (covering up for the prior and real reason) and complaining about a first-world problem from an entitled, middle-aged white man (especially given experiences in higher education like this), but it’s something that I struggle with and which impacts my life. The teaching workload expected of an academic in my position, combined with the expectation to generate research output, and the horrendous mish-mash of poorly inter-connected information systems provided to complete these jobs means that my time and energy are limited. To meet part this semester I’ll need to teach the two courses I’m directly involved with, keep an eye on another, work on the Moodle “open” book project, most likely take on some other tasks within the School, produce at least one journal paper, and that’s leaving out a range of other tasks (including have a life outside of university).

    Focusing on educational data mining allows me to kill at least two (maybe three) birds with one stone.

    And perhaps there is an aspect here that I’d like to keep my personal pursuits (playing the saxophone) separate from work.

It’s suitability to netgl? Benefits and barriers?

Well, my participation with the MOOC is only made possible through the use of digital technologies/the network. It’s apparently designed to use the affordances of those technologies by very intelligent folk.

The topic of the course – big analytics – is largely made possible by the increasing amount of e-learning like this MOOC. It’s learning within digital environments that is generating the “big data” that is the focus of the course. Consequently, it’s not surprise that the learning analytics and data mining communities have quite a presence online. I even follow a few members of that community on twitter and via their blogs.

I also have colleagues with whom I’m working on learning analytics related work. Most of that work is being done via digital technologies. Hence, I already have some knowledge and experience working in and with netgl.

In terms of barriers, I do wonder whether the nature of the MOOC platform will cause any issues with being more open. I doubt it, but I’m trying to find some disadvantages.

What is learning?

Ahh, this old chestnut. Wonder if there’s any value in including this, especially if the other participants react to it like I do. Perhaps I should get over myself (I’m not big on definitions).

Let’s paraphrase/extend a Google search result

Learning is the acquisition and application of knowledge and skills

I’ve added “application” and removed “through study, experience, or being taught”. Those changes perhaps don’t enhance the definition.

Wikipedia appears to add the additional clarification of “acquiring new, or modifying and reinforcing, existing knowledge, behaviour, skills, values, or preferences”

That’s enough, time to get started on the MOOC.

Revising week 1 of EDU8117 – Networked and Global Learning

The following is a description of how I’ve gone about revising the Week 1 material for EDU8117, Networked and Global Learning. The intent is to demonstrate how a tool can be used for something a little different.

The problem

The content for week 1 is a longish web page. It was created in a hurry last year and could use some re-jigging. I need to read through the material and ponder what changes to make. The problem is how to record those thoughts in a way that allows me to focus on reading through the material initially.

If the material was in a Word (or PDF, or Google) document I would probably end up using Word’s (or Acrobat’s, Google doc’s) annotation facility to mark up the text and add comments. But it’s a web page and I’m not going to venture into the wilds of converting back and forth between HTML and other formats.

The solution

Luckily, I do use Diigo which will allow me to annotate web page, and that’s what I’m using. I’m reading through the web page and highlighting sections and making comments on sections I think will change. Once I’m complete, I’ll open the blog post in the WordPress editor beside another browser with the annotated version and make the changes.

The result

Here’s what part of the web page looks like with the annotations.

Original NGL Week 1

And the corresponding information in Diigo. You’ll be able to see that yourself if you visit my Diigo library. That link is actually to Diigo’s list of all links I’ve tagged with “annotate”. I’ll try to keep this the only one.

Diigo annotations on NGL Week 1

It has the advantage of leaving a visible record of the changes I’ve made. Even after I’ve changed the original web page, Diigo has done a reasonable job of keeping the highlights and annotations in place.

It also provides the potential advantage of making explicit

Digital learning: It's déjà vu all over again

Below you will find resources associated with a talk titled “Digital Learning: It’s deja vu all over again”. The slides below are the near final set to be presented at the #dLRN15 conference (abstract available below).

Due to time constraints a slightly longer version of the slides has replaced.

Abstract

The initial steps of my university teaching career commenced in the early 1990s teaching information technology courses to on-campus and distance students. For distance students the learning experience was largely print-based with little or no student-student or student-teacher interaction. Like many academics at that time the increasing availability of the Internet sparked explorations into a range of digital learning innovations designed to overcome the limitations of existing institutional teaching methods (Jones, 1996a, 1996b).

Twenty years later and three years ago – long after digital learning had become the norm in higher education – my teaching career continued at a new institution and in a new discipline. Now teaching pre-service teachers in a program proudly proclaiming itself as being amongst the only in Australia to be available entirely online. Once again I found myself teaching both on-campus and “distance” students. Further extending the sense of déjà vu the last three years have been spent exploring a range of digital learning innovations designed to overcome many of the same limitations of existing institutional teaching methods. Digital learning, it’s like déjà vu all over again.

Using this experience and the BAD/SET framework (Jones & Clark, 2014) the session will argue that the institutional implementation of learning and teaching – be it distance education or digital learning – is underpinned by the SET mindset. A mindset that places more emphasis on reuse and scale than on contextually appropriate pedagogical value and thus creates this sense of déjà vu. The session will seek to illustrate how the combination of both the BAD and SET mindsets can offer useful insights for both research and practice into how digital learning might be harnessed institutionally to achieve appropriate and practical outcomes.

References

Jones, D. (1996a). Computing by distance education: Problems and solutions. ACM SIGCSE Bulletin, 28(SI), 139–146.

Jones, D. (1996b). Solving Some Problems of University Education: A Case Study. In R. Debreceny & A. Ellis (Eds.), Proceedings of AusWeb’96 (pp. 243–252). Gold Coast, QLD: Southern Cross University Press.

Jones, D., & Clark, D. (2014). Breaking BAD to bridge the reality/rhetoric chasm. In B. Hegarty, J. McDonald, & S. Loke (Eds.), Rhetoric and Reality: Critical perspectives on educational technology. Proceedings ascilite Dunedin 2014 (pp. 262–272). Dunedin. Retrieved from http://ascilite2014.otago.ac.nz/files/fullpapers/221-Jones.pdf

Female/male participation rates in IT: an example of what's easy to log??

There’s an identified problem with learning analytics captured by the quote from Buckingham-Shum in the image below (the image is from a presentation I gave yesterday on learning analytics, so my head is in that space). In the case of learning analytics that data that is easy to capture is generally mouse clicks. The number of times a learner clicks on a website. Computer systems log this information almost by default. It’s the data that exists so it is what learning analytics analyses. It’s the data which defines the way people think about the problem.

Easy to log

This morning there were two different resources come across my PLN about the question of female participation in IT related courses. First, was this research project into female participation in ICT in schools. Second, was this analysis of female participation in IT and Engineering degrees at University.

This is an important issue, both in general and for me. This semester I’m somewhat involved with a course that is helping Secondary Computing pre-service teachers develop their discipline specific knowledge and identity. Interestingly, at least 60% (n is quite small) of the students in the course are female.

I did some initial exploration of enrolments in IPT (a Queensland senior secondary ICT course) a few years ago. The following graph is one artefact from that. It shows the percentage of all OP students (those who will be eligible to apply directly for University enrolment after year 12). It shows that the % of female OP students never got above 10%. It also shows that as of 2010, less than 20% of male OP students enroled in IPT.

Percentage of gender enrolments

Gender is something that appears in most enrolment databases and in most surveys. It’s data that is available. I wonder if there is some “factor Y” (or multiple factors) that isn’t in the data that’s already been gathered?

I imagine the research in this field has already done some thinking about this.

Over the last couple of years (in particular, but not only) parts of the broader computing community/culture haven’t exactly covered themselves in glory when it comes to questions of gender. As a long-term male part of that community/culture I don’t wish this question to be seen as suggesting moving focus away from the question of female participation in computing. There are obviously some important questions to be explored.

References

Buckingham Shum, S. (2012). Learning Analytics. Moscow. Retrieved from http://iite.unesco.org/pics/publications/en/files/3214711.pdf

Does branding the LMS hurt learning

The LMS used by my institution is Moodle, but the institution has “branded” it as “Study Desk”. Meaning students and teachers talk about finding X on the “Study Desk”. They don’t talk about finding X on Moodle. The following suggests that this branding of the LMS may actually hurt learning.

Update: Via twitter @georgekroner mentioned his post that has some stats on what institutions are branding their LMS.

Google the name (information literacy?)

The biggest course I teach is aimed at helping pre-service teachers develop knowledge and skills around using digital technology to enhance and transform their students’ learning. Early on in the course a primary goal is to help the students develop the skill/literacy to solve their own digital technology problems. The idea is that we can’t train them on all the technologies they might come across (give them fish), we can only help them learn new technologies and solve their own problems (teach them how to fish).

A key part of that process is the “Tech support cheat sheet” from XKCD. A cheat sheet that summarises what “computer experts” tend to do. One of the key steps is

Google the name of the program plus a few words related to what you want to do. Follow any instructions.

How do you “Google the name of the program” if the institution has branded the LMS?

Does branding the LMS mean that students and teachers don’t know “the name of the program”?

Does this prevent them from following the tech cheat sheet?

What impact does this have on their learning?

A brief investigation

Early in the year I was noticing that a few students were having problems with “Google the name”, so I set an option activity that asked them to create a “technology record”. i.e. a record the names of all the technology that they are using. The idea is that having a record of the technology names can help solving problems. I included in that “technology record” that they specify the name of the software that provides the “Study Desk”.

There were 40 (out of ~300) responses including

  • 10 that identified uconnect, the institutional portal;
  • 8 that weren’t sure;
  • 8 that didn’t provide an answer for the Study Desk question;
  • 4 that identified their web browser;
  • 4 that firmly identified Moodle;
  • 3 that identified Moodle but weren’t sure;
  • 2 answered with the URL – http://usqstudydesk.usq.edu.au;

20% of the respondents were able to identify Moodle.

These are 3rd year students. Almost all will have completed at least 16 courses using Moodle. These are students completing an optional activity indicating perhaps a slightly greater motivation to do well/learn. A quick reveal that most of the students have a GPA above 5.

The still don’t know the name of the LMS.

I wonder how many teaching staff know the name of the LMS?

Does this hurt learning?

Perhaps if everything works with the LMS then this doesn’t create any problem. But if the students wish to engage with social and information networks beyond the institution, they don’t know the common name for the object they want to talk about. That has to hurt learning.

I imagine that there are librarians and others who can point to research identifying the inability to know the correct search term hurts search.

What do you think? Does branding the LMS hurt learning?

Re-using a #moodle course design

This semester I’m course examiner for a new course on Secondary Computing Curriculum and Pedagogy. As the name suggests, the course is intended to help pre-service teachers who are aiming to teach computing in Secondary schools. While I’m the course examiner, the course is being developed and will be largely taught by a couple of practicing and experienced Secondary computing teachers (how’s that for “recency of practice”?).

Two weeks before semester start my institution opens up the course sites for students to become familiar with what’s on offer. There are some minimum expected standards to meet. My task today is to meet those standards and in doing so set up the skeleton of the course site for the rest of the semester. To do this I’m going to reuse the structure from EDC3100, perhaps with a few tweaks. Besides saving me some time, four of the five students currently enrolled in the course have done EDC3100.

This is also a bit of an exploration of the difference between an empty Moodle course site (even one with a standard look and feel) and one with a structure.

What I’ll need to do

Can I list all I need to do

  • Structure of the site
  • Study Schedule
  • Assessment
  • Teaching team
  • The initial orientation message.

Structure

The basic structure is going to match the EDC3100 template. A collection of topics tied directly to each week of semester with a “jump to” bar at the top of the course site. There will also be a collection of “adminstrative” topics.

The following image shows the top of the 2012 version of the EDC3100 site. In 2015 the institution has adopted a default course structure that does away with the need for the “Course Content” and “Course Background” boxes.

Welcome

The one question about this approach is that EDC3100 has quite a bit of content in each week. Not sure that EDS4406 will have the same quantity. Hence the separate topics for each week may be a bit of overkill.

As it stands each topic does have a formal title meaning it’s probably valuable to make use of the macro facility I’m using in EDC3100.

Process setting this up includes

  1. Copy the “course format” used in EDC3100.
    “Weekly format” with 12 sections. 10 teaching weeks + an orientation and a “resource” week.
  2. Update the names of each section/topic/week of the course.
    Using the macro facility the names entered into a moodle are in this form {{{W0_TITLE}}} ({{{W0_PERIOD}}}). A bit of Javascript will replace the “variables” with appropriate values.
  3. Put in place the Javascript file to do the translation.
    I’ll create a new one for this course. Copy the EDC3100 js file across and update the values. Week titles first. That’s all done. The problem with week numbers changing because of holidays reared it’s head again.
  4. Stick in the “jump to”
    Oh, that was nice. Copy the HTML from EDC3100, replace the course id, paste it into the EDS4406 site, and hey presto it all works. Even the tooltips get updated with the new topic names.

“Administrative” content – Study schedule, assessessment, and teaching team

These three sections are important but don’t form part of the students’ weekly learning activities. The institutional default course structure provides default tools for display this information, but IMHO they aren’t as useful as using a Moodle book to provide course specific information.

At this stage, the detailed information for these sections isn’t yet written, I’ll just be putting in the initial skeleton. That involves

  1. Creating a Moodle book resource for each section.
  2. Updating the js file to point the default course structure links to these books.
  3. Put in some basic information into the books.

Again the macro system is nice. Copy and paste some HTML from the EDC3100 book that is using the macro approach. Link in the EDS4406 js file and the content automagically updates to EDS4406 information.

Orientation message

This will need to be a little more than a message. Will work on that tomorrow and update this.

Can the Moodle book module be made open and other enhancements

Next week I’m off to Melbourne to Moodlemoot’AU 2015. This post will hold the details and resources associated with this talk which is talking about how the Moodle book module might be enhanced, including by making it more open. It’s the first public talk about the USQ funded Moodle book project.

There’s a Moodle site associated with the Moot presentation.

Slides

Synopsis

Since 2012 the course EDC3100, ICT and Pedagogy at the University of Southern Queensland has been taken by over 1500 pre-service teachers learning how they can use digital technologies to enhance their own teaching. In that time, the course has evolved from a fairly traditional “textbook, lecture, and 2 tutes” type of online course into an online course reliant on Open Educational Resources (OERs) and individual student blogs. Core to that evolution has been the Moodle book module. Each week’s “learning path” consists of a range of activities and resources with the Book module providing the scaffolding. In the most recent offering, the course included at least 73 Moodle books with 670 chapters. The success of this approach is evident in student evaluations of the course, with comments such as

Learning paths were great. They were informative and interactive

The most helpful aspect of this course was learning paths.
It was clear what to do and how to do it.

This experience has shown that the Book module can be an integral part of an effective course design. However, it has also revealed a number of areas where the Book module could be usefully enhanced.The aim of this session will be to start an on-going conversation about what enhancements could be made to the Moodle book module and how those might be best made.

As a spark for that conversation the session will outline a range of possible enhancements derived from the experience in EDC3100; discussion with other users of the Book module; and, an analysis of discussions on the Moodle tracker and forums. In particular, the session will outline possible enhancements to the Book module that would provide the option for content managed by the Book module to be transformed into Open Educational Resources (OERs).

Background

Much of this work is being done as part of the Moodle “open” book project funded by a grant from USQ’s Open Textbook Initiative. The project aims to enable Moodle books to be leveraged as OERs and implement more general enhancements to the module. The aim is to ensure that any and all work done is contributed back to the Moodle community.

Initial pre- and post- session discussions will take place in a site associated with the Moot before migrating to a more appropriate location.

Re-building results processing

It’s once again time to process final results for a course and return the final assignment. A process that involves

  1. Checking overall student results for a course, before returning the final assignment.
  2. Identifying all of the students who won’t have final results available by the required date.
  3. Analysing this offering’s performance across the course and comparing it with prior offerings.
  4. Returning the final assignment.
  5. Ensuring the overall results are entered appropriately into the student records system.
  6. Preparing a report for the examiners meeting.

There are three problems driving this

  1. What institutional processes/tools are provided to help with these tasks are far from user friendly.

    e.g. the greasemonkey script I wrote last year to help with task #5 above.

  2. There are no institutional processes/tools for some of these tasks.

    e.g. the only way to get all of the assignment marks into one place is to put them into the Moodle gradebook. But I believe if I do that then the students can see the marks. Given that I’m still moderating the last assignment marks them seeing the marks is not so good.

  3. The management of online assignment submission has changed this year, so some of prior workarounds are no longer viable.

What I used to do

Part of this activity is identifying what I’ve already got. It’s around 7/8 months since I last had to do this, so I don’t remember. I’m not sure the impact of this temporal distance is something that the designers of institutional systems and processes are fully cognisant of.

The old process appears to have been

  1. Extract individual CSV files for each assignment from EASE.
    The old online assignment system generated 2 separate CSV files. I needed both.
  2. Get another CSV file with Professional Experience results.
    Whether a students has passed their Professional Experience is determined by another part of the institution using a different system.
  3. Run a perl script which would
    1. Extract all the data from the CSV files.
    2. calculate the final results (for each student).
    3. Assign a grade to the student based on circumstances
      • FNP if at all assignments not submitted.
      • F regardless of mark if PE failed.
      • IM if PE mark has not yet been received
      • RN if assignment submitted but still being marked.
    4. Generate grade breakdowns for each campus and overall
    5. Output all that as a CSV file, including comments that I’d added
  4. Import that into Excel and do some further checking.

What I need to do

The process will be largely the same. The main difference is the source and format of the CSV files. The changes now are

  1. Assignment 1, Assignment 2 and PE marks are now in Moodle gradebook.
    The will all export nicely. Though it does use the long version of the USQ student number. May need to do a workaround for that.

    what sort of institution is silly enough to have at least 3 versions of the same student number?. Long with leading 0s. Long without leading 0s. Short!

  2. Assignment 3 mark will be in Moodle assignment activity.
    That will work. The grade is there.

    But yet, it uses a different version of the student number (long without leading 0s). But both do have email address, that might be the candidate for unique id. Or the script will just pre-pend leading 0s.

The other major difference is that neither of these CSV files include any mention of the campus or mode the students are using. Which prevents breaking results down. But thankfully I do have database table populated with this information.

Ahh, that’s right. I also have to change the PE mark for some students to -1 to indicate they were exempt. This is to discern students for whom there is no PE result yet, from those students who aren’t going on PE and should still pass.

My other problem is that the CSV file for A3 has a “status” field that includes new line characters. My poor little Perl CSV parsing module doesn’t handle that well.

It’s working well enough to help moderate A3 results. Time to do that.

Still to be done

  • Modify gradebook.csv to have -1 for PE for those students who will never go **** what about when it’s updated?
  • Add the campus calculations back in
  • Figure out how to handle the status field in Moodle assignment csv.
  • Have the script produce an Excel file, not a CSV file

Powered by WordPress & Theme by Anders Norén

css.php