Some MAV tasters

MAV is just about up and running at USQ as one of the Technology Demonstrators. It’s taken longer than I thought, but it’s there.  The following demonstrates MAV running on a course I teach and is intended to illustrate some of what it can do.

Hoping we’ll get an opportunity to use this type of process to support others to use MAV to explore what’s happening in their courses. The aim being to explore what, if any, insights MAV provides teaching staff.

See usage of any Moodle link

Once MAV is installed on Firefox, whenever you view a Moodle course page every Moodle link on that page will be modified to show how it’s been used by students.

You can see the number of students who have accessed the link.  Even the course link that’s include via the Diigo widget get’s highlighted.  You can also see that no student has been able to use the hidden “Where are we going” forum.

EDC3100 2016 S2 - MAV - Students

Or you can the number of times students have clicked on the link.

Interestingly, but perhaps not surprisingly, in the above the link about the importance of student’s completing course evaluations is clicked about once per student (51 students with 54 clicks), but the Assignment 3 specification has been visited at least 7 times per student (93 students with 696 clicks).

EDC3100 2016 S2 - MAV - Clicks

On any course page

This happens on any Moodle page for the course. The following shows the MAV view on a Moodle Book page from the S2 2016 offering of my course. It shows the first page which gives an overview of the rest of the book, including links to those specific pages.

It shows that the recommended learning process page was visited by 90 students, however the rest of the pages in this book were visited by 86 or 84 students.

EDC3100 - Book - MAV

And here’s the same page from the Semester 1 offering. There is also a similar slight increase for that same page.

S1, 2016 Book page

Find out who has or hasn’t viewed the page

MAV adds in an indication of the number of students (or clicks) for each Moodle link. If you click on students link, MAV will show you a list of the students who have (or have not) visited that page.

At CQU there is a link from this view to a system that allows the managing of nudges (communication attempts) with the students.

MAV - students no access

View specific groups

I’ve always been interested in the difference in engagement between on-campus and online students. MAV allows you to focus on specific groups. Here’s the S1 2016 book page showing the Springfield students Springfield student usage

Here’s the same section for the Toowoomba on-campus students.

It appears that the Toowoomba students are using this book less, and there is also NOT the same peak for the recommended learning process page.

Toowoomba student usage

Exploring more frameworks to understand OER/OEP

What follows is a continuation of an earlier exploration into extant “frameworks” to understand OER/OEP.

The OPAL OEP guide

A 2010/2011 project partly funded by the European Commission aimed “at establishing a forum which works to build greater trust in using and promoting open educational resources”.  Has a particular focus to move beyond OER to “focus on innovation and quality through open educational practices (OEP). Ehlers (2011) describes it in more detail and suggests that

Individuals (learners, professionals) likewise can use Matrix 1 to better understand OEP and to self-assess and position themselves to the extent that OEP constitutes part of their own learning/ teaching abilities. They can use the second matrix to analyse the OEP landscape in which they operate, which can be represented in the degrees of freedom to practice open education and the extent to which it is embedded in an open social sharing and collaborative environment. (pp. 6-7)

The diagram (Matrix 1) below show different stages of OEP using a combination of OER usage and learning architecture.  Coughlan and Perryman (2015) label this the OPAL open educational practices maturity model and use it to evaluate the practices of three global health projects. Coughlan and Perryman (2015) suggests that it

has been the dominant OEP analysis framework since its development in 2011 in connection with the Open Education Quality Initiative (p. 177)

So the OEP matrix consists of

  1. OER;
  2. Learning architecture aka pedagogical practice.

Stages of OEPs

OPAL (2011) offers the following example

field “H” could relate to “I am sometimes using OER for normal lectures”, field “B” would represent rather “I am using open educational resources in open seminars and learning scenarios”

OPAL (2011) offers the following explanations of the low, medium and high levels of learning architecture (p. 5)

  • Low” if objectives as well as methods of learning and/ or teaching are rooted in “closed” one way, transmissive and re-productive approaches to teaching and learning. In these contexts, the underlying belief is that teachers know what learn- ers have to learn and mainly focus on knowledge-transfer.
  • Medium” represents a stage in which objectives are still pre-determined and given, but methods of teaching and learning are represented as open pedagogical models. They encourage dialogue oriented forms of learning or problem based learning (PBL) focusing on dealing with developing “Know how”.
  • High” degrees of freedom and openness in pedagogical models are represented, if objectives of learning as well as methods (e.g. learning pathways) are highly determined and governed by learners. Questions or problems around which learning is ensuing are determined by learners (SRL – self regulated learners), and teachers facilitate through open and experience-oriented methods which accommodate different learning pathways, either through scaffolding and tutorial in- teractions (ZPD Vygotskian inspired approaches) or contingency tutoring (Woods & Woods strategies of re-enforcement, domain or temporal contingency)

Ehlers (2011) then offers “Matrix 2” in the following image. It’s intended to be used to “categorise, assess, and position the existing landscape of OEP within a context”.  Based on the freedom to participate and the involvement of others.

Diffusion of OEP

Ehler (2011) positions OEP as the 2nd phase of open. It has a focus on actually using OER to improve learning. A move that requires the combination of OER and open learning architectures.  The following bullet list describing phase 2 is provided by Ehler (2011, pp 3-4)

  • builds on OER and moves on to the development of concepts of how OER can be used, reused, shared, and adapted
  • goes beyond access into open learning architectures, and seeks ways to use OER to transform learning
  • focuses on learning by constructing knowledge assets, sharing them with others, and receiving feedback and reviews
  • follows the notion of improving quality through external validation because sharing resources is in the foreground
  • is about changing the traditional educational paradigm of many unknowledgeable students and a few knowledgeable teachers to a paradigm in which knowledge is co-created and facilitated through mutual interaction and reflection
  • strives to understand that OER has to contribute to institutions‘ value chain.

The last point is potentially questionable.  What is the institution?  If we’re focused on teacher education, is the institution our respective universities or the teaching profession?

Limitations

However, Coughlan and Perryman (2015) found that

the pedagogy and object-focused OPAL framework is not sufficiently comprehensive to cover the open collaboration featuring in our case studies (p. 177)

And supplemented it with four dimensions from the OEP social configuration framework from Vrieling, Van den Beemt and De Laat (2016) and also Schreurs et al (2014). What’s missing is deemed to be more about the behaviour of those involved, they go onto argue that

that current OEP evaluation frameworks are not sufficiently comprehensive nor nuanced to capture all of these practices; indeed, the models reduce the three case studies to appearing very similar (p. 184)

Other weaknesses are identified, but I wonder how much of this is simply weaknesses inherent in all models (they are all wrong at some level/perspective). For example, the following from Coughlan and Perryman (2015)

For example, influential frameworks such as the OPAL matrix, with its language of teachers, courses and educational institutions, are overly narrow and do not map easily outside academia.

Open Educational Practice Maturity Matrix

OPAL (2011) the proceed to provide a maturity matrix to position an organisation re: uptake of OEP.  Three sets of questions based around: positioning the org in an OEP trajectory; creating a vision of openness; and, implementing and promoting OEP

Relationship between OPAL and Stagg

The last exploration of OEP frameworks was largely focused on Stagg (2014) who developed a continuum of open practice.   Stagg and most of the other frameworks covered in that earlier exploration have an institutional focus. They include consideration of what the institution can/should do to support OEP.

But what about moving beyond the institution? Integrating OEP into teacher education would seem to place some value on engaging with the professions, which is beyond any single university.  How does that work? What is required there?

Leaving behind those questions for now, what’s the connection between the Stagg, OPAL and other frameworks?


A continuum of practice - OEP

The Stagg continuum appears to be finer in the granularity with which it divides OER resources/practices, but at the same stage it isn’t as fine grained around pedagogical activities.  Hence the following mapping of Stagg’s continuum against the OER usage dimension of the OPAL matrix doesn’t quite work.  e.g. Student co-creation from Stagg doesn’t map against this dimension.

  • Low – no OER (re-)usage
    No equivalent from Stagg
  • Medium – OER (re-)usage or creation
    Equivalent’s from Stagg might include and other following, as long as the either/or relationship exists between usage and creation.

    • Awareness/access – there is a basic level of (re-)usage of OER
    • Passive remix – some level of (re-)usage
    • Active remix – a higher level of (re-)usage
    • Sharing a newly authored OER
  • High – OER (re-usage) and creation
    The equivalent from Stagg above is in the combination of both (re-)usage and creation.

Judith & Bull and OPAL

Judith and Bull (2016) analyse OER literature and identified a set of categories for strategies that can be used – each having “increasing levels of collaborative support involved in OER implementation”

  • individualised strategies;
    individual or small team.
  • programmatic strategies;
    Organised programs within institutions.
  • institutional strategies;
    Approaches more embedded within normal institutional activities.
  • networked or user-shaped strategies.
    This appears to be touching more on Coughlan’s and Perryman’s (2015) use of the social configuration framework, but perhaps that’s in each of them.

This is then converted into a continuum of openness that maps some aspects of the above.

References

Coughlan, T., & Perryman, L. (2015). Learning from the innovative open practices of three international health projects : IACAPAP , VCPH and Physiopedia. Open Praxis, 7(2), 173–189. Retrieved from http://openpraxis.org/index.php/OpenPraxis/article/view/188

Ehlers, U. (2011). Extending the Territory: From Open Educational Resources to Open Educational Practices. Journal of Open, Flexible and Distance Learning, 15(2), 1–10.

Stagg, A. (2014). OER adoption: a continuum for practice. Universities and Knowledge Society Journal, 11(3), 151–164. http://doi.org/10.7238/rusc.v11i3.2102

Vrieling, E., Beemt, A. Van Den, & Laat, M. De. (2016). What’s in a name: dimensions of social learning in teacher groups. Teachers and Teaching, 22(3), 273–292. http://doi.org/10.1080/13540602.2015.1058588

Some findings from initial exploration of Moodle Book usage

I recently gave a talk at the Moodlemoot’AU 2016 conference in Perth. The talk was titled “How and why do people use the Moodle Book module?”  It reported on analysis of data from one University’s Moodle isntances to explore how courses, learners, and teachers made use of the Moodle Book module (aka the Book) from 2012 through 2015. The slides, abstract, and links to various interactive graphs used in the presentation are available via the presentation webpage.

This talk is part of a broader project that is looking more into post-adoption usage of e-learning systems, with an initial focus on the Book module.

The following provides a summary of some of what was found.  The links are to web pages that contain relevant interactive graphs.

Most books aren’t completely read

The percentage of students who completely read all of the Book resources appears quite low. In 2012, half the books were completely read by less than 47% of students. In 2015 half the books were completely read by less than 2% of enrolled students.

More integrated design appears to increase completion

I teach a course where Moodle book resources are integrated into the course and contribute a small part toward the final result. Not unexpectedly this produces better completion rates. However, questions remain the quality of the “reading” and some ups and downs in the data.

Courses using the Book tend to be bigger

Courses that use the book tend to have more students (median course enrolment of 85 in 2015) than those that don’t. (median of 11 in 2015). Raising questions about what is the difference with larger courses that seems to drive the use of the Book module.

..and have tended not to be for online only students

The majority of courses that use the Book do not include (m)any online students, especially when compared with all courses.

Again raising the question of the motivation/purpose for using the Book.

However in 2015 there was an increase in the number of courses with online students using the Book module.

This appears to be a result of the discontinuation of an institutional system that was used to transform more traditional distance education study guides into web-based resources.

Size of books and number of books

In 2015, the median number of Book resources per course was 3, however, there were two courses that had almost 80 Book resources.

In 2015, the median number of chapters per Book was 4, however, there were 3 that had more than 80 chapters. In 2014, there was a Book resource that contained over 74,000 words.

Most people are not using the Book import functionality

The Book module provides two broad methods to create resources:

  1. create; and
    Use a web-based editor to type (or perhaps copy and paste is the most popular method) text into the Book resource.
  2. import.
    Given a HTML file (or collection of such files) created externally, you can then import that file and it will be broken up into chapters.

It appears that very few people are using the import facility.  The following graph shows that roughly 10% of chapters, books and course offerings at this institution involved the use of the import method.

After the presentation, someone from the institution’s central L&T group came up for a chat. Indications are that between he (supporting another teacher) and I (in my course), we potentially account for all use of the import functionality.  Further discussions reveal issues arising from people copying and pasting from Word into the Book module.

It appears that content authoring might remain an unresolved problem.

Create or import