Assembling the heterogeneous elements for (digital) learning

Month: February 2011 Page 1 of 2

ICTs for learning design – the first week

So, it’s the end of O-Week and a four day residential school for Graduate Diploma of Learning and Teaching Students – a post-graduate, pre-service, teaching qualification. About 100 students got a lot thrown at us for four days, now it’s time to get into some work. The following is an attempt to get some idea of and reflect upon the work I need to do with the “e-learning” course. Others will probably follow.

This course actually requires use to use a blog for reflections, so there will be more of that as well. This is a rough and ready summary of what I was doing.

Overwhelmed and Moodle

It’s the nature of this week and starting a new program that people will feel overwhelmed. Especially when there are some students who aren’t gung-ho technology users. Especially when one of the courses is around the use of Information and Communication Technologies (ICTs) in learning and teaching.

Given my background, I’m not one of those students, but I’m still feeling somewhat overwhelmed. In particular, I have this feeling that there are activities or tasks that I need to complete that I’m not aware of. I feel I’m missing something. That’s part of the reason for this post, to try and develop a list of the tasks I need to complete over the next week or so.

Beyond just the nature of starting a new program, I’m also wondering how much of this is connected with Moodle and, in particular, the design of course sites within Moodle. More on this as I generate the list.

The list

  • Add entry to “My profile” wiki (students placed in groups).
    Had some troubles with the Moodle wiki, got there in the end. Currently only two members of the group with entries. Have already experienced the problem with differentiated messaging. i.e. the other group member asked a question about the Moodle Wiki using the Moodle messaging system. I didn’t see that message until days later, long after seeing the person at the residential school.

    It is interesting that the course isn’t using some sort of aggregator to bring student blogs together, we’re being asked to add blog addresses to a wiki page.

  • Readings
    • Expert jigsaw;
      Contact one of the profile wiki members and choose a reading (about learning theories) each and use a PMI to present findings on the wiki.
    • Read the “study guide” – 3 or 4 pages
      • Do the F&S learning style test and reflect in a blog post.
      • Do a MI test and compare result with other students in a discussion forum.
      • Blog about 21st century students and Prensky.
      • Blog about some of the challenges arising from connectivism.
  • Create blog and add it to the blog list.

Blogs

The “study guide”

Most of the work for this course seems to be described in a “study guide”. A single web page within the Moodle course with some blurb, a reading/activity, and some more blurb. So here’s a summary and some reflection.

Are leaners different?

From the guide

Today’s students are fundamentally different from the learners of 40 years ago.

Mmm, not sure I agree. Sure, there is probably a greater percentage/diversity of learners at schools, especially secondary schools. But are students fundamentally different? That’s a pretty strong statement.

There is no difference

Suggestion that whether f-t-f or e-learning, that knowing the learner and their needs is a key element.

This highlights one of the misassumptions around ICTs/e-learning. People seem to think that it is different. That they haven’t been teaching and learning with technology all along. Most of the face-to-face activities we engaged with at the residential school were mediated by some form of technology. Examples included: paper handouts, sheets of paper cut up and distributed in a bag, whiteboards, rooms and furniture configured in different ways etc. I do wonder when (or perhaps if) we’ll ever get over this tendency to treat ICTs as different?

Learning styles

And so, we’re into learning styles, multiple intelligences etc as the way to know students. I have a vague disquiet about these approaches. My concerns arise from two parts. First, the tendency for these approaches to be used to pigeon hole people into a small number of categories rather than celebrate the true diversity of people. Second, the attention these approaches take away from the more complex and nuanced approaches to understanding diversity. i.e. we apply the test, identify the categories and feel satisfied that we know the students. Perhaps that’s just one source of disquiet intermixed.

Okay, so now I’m doing the survey and finding myself saying “neither a nor b”, “it depends”, “sometimes a bit of a and a bit of b”. Another concern I have about these instruments.

Seems I have to post another entry on this question.

Multiple intelligences

So, we’re meant to take this test based on Gardner’s multiple intelligences. Here are my results.

My "multiple intelligences"

As with most of these tests, it meets my basic expectations, but there continues to be a uncertainty at the back of my mind about this work. I need to read and reflect a bit more. Trying to balance the desire to dig deeper with the desire to complete the study is interesting.

The following quote from Howard Gardner is interesting in this context

I have drafted generic letters that answer the most frequent questions-for example, “Is there a test for multiple intelligences?” (Answer, “Not one that I endorse”)

The associated PDF of FAQs is interesting reading.

The following quote from those FAQs about learning styles and intelligences is interesting (to me)

Educators are prone to collapse the terms intelligence and style. For informal matters, that is no great sin. However, style and intelligence are really fundamentally different psychological constructs. Styles refer to the customary way in which an individual approaches a range of materials—for example, a playful or a planful style. Intelligence refers to the computational power of a mental system: for example, a person whose linguistic intelligence is strong is able readily to compute information that involves language. Speak of styles, speak of intelligences, but don’t conflate the two if you can help it.

And while it’s somewhat off the current topic, I need to save this comment on creativity

Finally, as stressed by my colleague Mihaly Csikszentmihalyi, creativity should not be viewed simply as a characteristic of an individual. Rather, creativity emerges from the interaction of three entities: l) the individual, with his given talents, personality, and motivation; 2) the domain—the discipline or craft in which the individual is working; 3) the field—the set of individuals and social institutions that render judgments about quality and originality

The resonance between this quote and my thoughts about creativity within education and the negative impact of institutional policies and systems – especially LMSs and how they are supported – is quite strong.

And this on how intelligences are not set in stone

The extent to which intelligences develop is a joint product of biological (genetic potential), the emphasis a culture places on an activity, the excellence of the instruction, and the motivation of the individual. Any individual can strengthen an intelligence if she is wellmotivated; if her ambient culture values that intelligence; if there are human and artifactual resources (i.e. texts, computer programs) on which she can draw.

And at the value of the MI tests

As you may know, many other people have devised MI tests. The best known is probably the MIDAS test, developed by Branton Shearer. These tests typically give a rough-and-ready sense of people’s interests and preferences. They suffer from two deficiencies: l) They don’t actually measure strengths—you would need performance tasks to determine how musically intelligent, or spatially intelligent, or interpersonally intelligent a person is; 2) The tests assume that the person has good intrapersonal intelligence—that is, he or she knows himself well. But many of us think that we know ourselves better than we really do. I doubt that anyone would score herself or himself low in the personal intelligences, but some of us must have lesser personal intelligence than others.

I have nothing against a person using the MIDAS or other measurements to learn about the idea of multiple intelligences. Indeed, people—both young and old—often find it interesting and illuminating to think about their own and/or other person’s intellectual profile. And yet, I must stress, we can only learn about a person’s intelligences if we actually measure how well they perform on tasks that presumably draw on specific intelligences.

My take away from this is that there remains significant questions about the value of using the results of a multiple intelligence test as a direct guide to a learning design. Essentially because it’s highly questionable that they actually give a valid result as to what they seek to test. A true measure, perhaps, is only possible through observing participants while performing associated tasks.

There is, however, still value in these sorts of tests as a way of getting learners to think about different intelligences within themselves and their fellow learners.

Another interesting quote is

That is, we reject the focus on a single scholastic intelligence that is measured by a certain kind of short answer test

I find it somewhat ironic that responding to “a single scholastic intelligences that is measured by…short answer test” was one of the reasons for the development of the theory of MIs, and that we are now in a situation where your MIs can be measured by a MCQ.

In summary

Briefly, my theory can reinforce the idea that individuals have many talents that can be of use to society; that a single measure (like a high stake test) is inappropriate for determining graduation, access to college, etc.; and that important materials can be taught in many ways, thereby activating a range of intelligences.

21st century learners

Ahh, Prensky, immigrants and related perspectives.

First up, the Engage me or enrage me article. The suggestion is that today’s technological environment means that all students have technologies of various types that are working hard to engage them. That’s a very broad generalisation. Then there are claims like “Many of today’s third-graders have multiple e-mail addresses” – no evidence to support this and frankly my experience doesn’t support it.

Then there’s an argument about how the top 3 selling games of the era offer vast and engaging experiences that kids aren’t getting at school – this is showing how backward school is. But this 2008 US survey suggests that the average age of gamers is 35. Only 25% of gamers is under 18, 75% are over. So it’s perhaps not the students purchasing those games.

Okay, now there is a point about students being able to handle systems or problems much more complex than what they are being given at school. There’s the example of a 4yo gamer being bored to tears by “learning games”. Mmm, final blog post on this section.

A brief overview of learning theory

So, the standard behavourism, cognitivism and constructivism summary (this seems an useful overview). Interesting that connectivism is included.

Reflections

There was a lot of reading this week, but even that only barely touched the surface of what is out there. I found it interesting, but without my prior experience this would have been a very daunting week.

Thoughts and applications of connectivism

So, post #3 for week 1 of the ICTs for Learning Design, this time on connectivism. There are two parts to the question, the second is somewhat easier and asks fo examples of how connectivism could be used in a classroom/learning context. The first is somewhat more difficult.

Position on connectivism

The question suggests that connectivism is contested by some and that some of this may arise from its challenging nature to some of the broadly accepted, established concepts or practices. The first problem with completing this arises from the fact that the guide we’re working from doesn’t really point to the folk that challenge/disagree with connectivism. This can be addressed by a quick search. That reveals a growing body of literature and challenges that can be quite complex and time-consuming to engage effectively with. Even with a history of a few years of engaging (not always deeply) with connectivism and its proponents, I don’t feel really all that comfortable enough to express a firm opinion. I’m beginning to wonder if I am over thinking this, so lets go quick and dirty for the purposes of assessment.

At the root of these learning theories/paradigms is a description of learning. I guess one approach to expressing a perspective is to examine the validity of the description of learning provided by the theories/paradigms. Based on this approach I think connectivism provides a model of learning that is perhaps closer to reality than the others. The black box of behaviourism doesn’t say anything about learning. The cognitivist approach is based on an information processing perspective. A learner as computer perspective which I don’t think captures how the brain actually works. The constructivist approach seems to wave its hands and say “learners construct knowledge”, each differently. It strikes me that there is some similarity with the network-based perspective of connectivism. i.e. students are constructing their knowledge by building and pruning networks.

That’s really broad-brush. Perhaps I should take the time to read a bit more (e.g. Kop and Hill, 2008)

One of the points in the question is

It is unsettling to be challenged about existing perceptions of “knowing”, in particular, the lack of purpose in asking our students to KNOW and be able to RECALL what they know in assessment

I’m not sure that there connectivism suggests that there would be a lack of purpose in asking students to know and recall. Connectivism, at least according to Siemens, suggests that the capacity to know (learn more) is more critical than what is known. One interpretation of this is that it doesn’t mean that what is known is unimportant, it’s just not as important as the capacity to know. In the end, demonstration of the capacity to know would seem to require an ability to demonstrate what is known.

Perhaps the view I expressed above connects more closely with Kop’s and Hill’s (2008) discussion of epistemological frameworks for learning. The final paragraph of Kop and Hill is interesting in this context

A paradigm shift, indeed, may be occurring in educational theory, and a new epistemology may be emerging, but it does not seem that connectivism’s contributions to the new paradigm warrant it being treated as a separate learning theory in and of its own right. Connectivism, however, continues to play an important role in the development and emergence of new pedagogies, where control is shifting from the tutor to an increasingly more autonomous learner.

Using this theory in the classroom

Currently I have only a theoretical/anecdotal understanding of what a high school classroom context would be like. I don’t fully appreciated the constraints of such a context, so the following will be limited by that lack of understanding.

I’ll draw on Downes comments on teaching and learning within connectivism here

to teach is to model and demonstrate, to learn is to practice and reflect

and use these within the context of teaching Information Technology – particularly programming. This is a good opportunity as it allows me to make concrete some vague ideas I’ve had for a while.

One aspect of IT is learning how to program. Often programming is taught through “pretend” authentic projects such as creating a reservation system for a restaurant. Other limitation of these projects is that the student starts from scratch and does the programming by themselves. The trouble is that increasingly most software development occurs within broader frameworks. e.g. developing a plugin for Moodle, for WordPress etc.

I’m interested in exploring how programming could be taught be encouraging students to do exactly this. Pick a module, open source application – like Moodle or WordPress – and over time develop or modify a plugin. In terms of teaching it would be my task to model and demonstrate the practices and knowledge required to do this (e.g. like my work on BIM) and the students would be required to engage in the existing developer networks around these open source projects. In terms of encouraging reflection and making connections between the students and those external, the students would be expected to maintain blogs on their practice, must like I do with BIM development.

There’s much more to this, but that’s the basics.

References

Kop, R., and A. Hill. 2008. Connectivism Learning theory of the future or vestige of the past. The International Review of Research in Open and Distance Learning 9, no. 3. http://www.irrodl.org/index.php/irrodl/article/viewArticle/523/1103%22.

Prensky, immigrants and old problems in new bottles

Okay, so blog post #2 arising from week 1 of an ICTs for Learning Design course I’m doing. This post is intended to address the question of 21st century learners. In part this connects back to a sentence from the start of this weeks study guide with which I had problems. That sentence was

Today’s students are fundamentally different from the learners of 40 years ago.

That struck me as a big claim, and to some extent it still does.

Today’s learners

In terms of today’s learners, I tend to think that they are not fundamentally different. Some (but not all) have certainly had some very significantly different experiences through the widespread availability of ICTs. I don’t, however, agree with Prensky that this experience has fundamentally changed those learners. Not the least because a significant percentage of students don’t have significant ready access to ICTs, so they can’t have changed fundamentally because of technology. Then there are questions about the type of engagement the remaining students have with technology and how much such engagement can fundamentally change them.

I currently think that today’s learners might, at a fundamental level, still have significant commonality with “yesterday’s” learners. At a less fundamental level they certainly do have some significant differences, but I am not yet convinced that those differences are fundamental. I’ll pick up on this in the next section.

Engage or enrage me

Prensky’s argument in this article goes something like this

  • Students are bombarded in their everyday life with technologies (games, phones, apps etc) that work really hard to engage them.
  • Schools just aren’t working that hard to engage them.
  • Schools need “damned good curricula gameplay” to address this problem.

I think this is an example of where there isn’t a fundamental difference between today’s and yesterday’s learners, it’s just that today’s learners have some different experiences that may well be emphasising a long-term problem. Prensky starts off his paper by dividing students into three groups:

  1. Those that are truly self-motivated.
  2. Those who go through the motions.
  3. Those who tune us out.

The last group are described as

These students are convinced that school is totally devoid of interest and totally irrelevant to their life.

I don’t see anything new in these groupings. If I go back 25+ years to my high school experiences I could quite easily place all my fellow students into those three groups. One of the major differences between then and now is that significantly more of the third group are now expected to complete years 11 and 12 of school. 25+ years ago most of the third group would have left at the end of year 10. If I go back to my parent’s days at school then the third group, such as my father, would have been leaving school much earlier.

Based on this, you might suggest the problem with “engage or enrage me” has more to do with changing societal expectations around schooling than technologies. Society now requires schools to look after the third group (and based on some recent discussion it really is a case of “babysit”) of students for much longer than prior without any fundamental change.

From another perspective the importance of learning that engages students interest and its positive effects on outcome is fairly well known/accepted. I don’t believe that the importance of engaging students has changed simply because a the most recent cohort has significantly expanded experience with technology. i.e. the change in technology is not a fundamental change.

It may be, however, a change that is important. If, and it remains a big if, students are increasingly familiar and comfortable with technologies then it might be that increase student engagement in learning may require greater and more appropriate use of engagement.

Another problem I have with Prensky’s argument is that it assumes that the fundamental change required of schools is “damned good curricula gameplay” and not a change in some of the other fundamental characteristics of schools.

Roger Shank (and many others) believe that the entire system is broken. For example, that the list of subjects which in the US owes more to the thoughts of a Harvard University professor from the 19th Century than the need to engage 21st century students.

Ira Socol goes to some lengths to show that the fundamental design of the education system (again US focused) is intended to help it fail.

So, I think there is some substance to the “engage or enrage” argument, but not in its use as the technology that is available to today’s students as being the substance. It’s the need to engage students and the difficulty of achieving such engagement, especially within the constraints of the existing education system that is important. Technology might help, but then the constraints of the existing system might prevent it from happening.

Learning styles, teaching and digital pedagogy

The following post if for the course EDED20491 and is in response to the following activity

Access the Felder and Solomon website and take the online questionnaire

  1. What is your learning style? What sorts of learning experiences would suit you best with your learning style?
  2. In a traditional classroom of 25 students, how would you support the range of learning styles each lesson?
  3. With your current knowledge of ICT, how could your design and digital pedagogy support your learners better?
  4. What sorts of profiling questions would you be asking about your learners to ensure you cater for everyone’s preferences?
  5. How does ICT support differences in learning styles?
  6. Create an entry in your blog (when it is created), and respond to these questions, and any you wish to pose, in the blog.

What is your learning style

The following image – click on it to see it in a larger form – summarises my results for the questionnaire.

Learning Styles

What that represents is that I am (supposedly)

  • A heavily reflective, rather than active learner.
    Reflective blogging, like this an related posts, is an example of a learning experience likely to suit this learning style. As suggested by Felder and Soloman

    Don’t simply read or memorize the material; stop periodically to review what you have read and to think of possible questions or applications. You might find it helpful to write short summaries of readings or class notes in your own words.

    Along similar lines, I’ve found having to teach something an effective way to learn as the act of teaching has required me to reflect upon what I know, how it is organised, how it can be explained, and what activities best encourage learning in others.

  • A heavily intuitive, rather than sensing learner.
    The explosion in information available via the Internet helps me here. I’m able to use Google and other means to find other interpretations, perspectives or theories on a topic. Establishing practices such as listening to a range of podcasts, following various edubloggers etc also open me up to alternate perspectives that enable and encourage connections. The capability to search digital information also helps in making connections to prior writings. e.g. searching my blog for posts that I believe connect in someway to a current topic. Both of which link with Felder’s and Soloman’s advice

    Ask your instructor for interpretations or theories that link the facts, or try to find the connections yourself

  • Slightly more a visual than verbal learner.
    On the verbal side the suggestion is “Write summaries or outlines of course material in your own words”. This seems to link nicely to this particular exercise. I’m essentially paraphrasing the material by Felder and Soloman and linking it to my individual context. For the visual side, the screen shot (though it’s a very text-based visual representation) of the survey results is a step in the right direction. Taking it one step further, I think the generation of slide presentations that have a highly visual component is one strategy that helps. For example, this presentation I did a couple of years ago.
  • Very slightly more global than sequential.
    Another blog post that I’m developing is essentially an attempt to develop a personal oveview of what I’m meant to be doing this week for this course. Again, a practice that links with some advice from Felder and Soloman

    Before you begin to study the first section of a chapter in a text, skim through the entire chapter to get an overview

    The post started with me skimming over the readings and activities for this week and generating a summary. When I was happy with that overview, that’s when I started working on this more detailed post in response to one of the activities.

How would you support a range of learning styles

I’m big on context. Depending on the context of the classroom of 25 students, the answers to this question would vary hugely, at least in the specifics. If I were to attempt for something abstract, the two-pronged approach would probably be

  1. Where possible adopt an appropriate array of different activities designed to support different learning styles.
    i.e. rather than simply always (or never) use classroom discussions, use them sometimes and supplement them with strategies that better suit more reflective learners. Or adopt modifications of classroom discussions that incorporate some aspect of reflection or at least time and space for reflective learners to engage.
  2. Capacity building.
    Students are not always going to find themselves in situations where their preferences are catered for. It would seem important to develop within them the capacity to deal with situations like this either through developing their skills on the other side and/or identifying strategies that help them deal with those situations.

The last point may well be essential for their school education as the context of schools may well mean that few of their learning experiences will provide exactly what their preferences require.

Design of digital pedagogy

To a large extent, I think my answer to this question is embodied in the previous two responses. First, I don’t believe there is any fundamental difference between non-digital and digital technology. Digital technology provides just another set of tools/technologies to help with learning and teaching. As it happens, I was listening to an interview of John Seely Brown this morning. In that interview Seely Brown (or it might have been the interviewer, Steve Hargadon) said something that summed up the point I’m trying to make here. The quote went something along the lines

With digital technology it’s not about the technology, but about the effects it can produce.

So, I would be using the same approach to design a digital pedagogy as I would a non-digital pedagogy. I would be aiming to fulfill the two-pronged approach from the previous question. To fulfil those two prongs I would draw on the advice and insights briefly discussed on this page and more deeply in other literature.

Which is essentially what I did in the first question where I equated the advice from Felder and Soloman with the practices/effects I was currently using this blog and other digital technologies with. i.e. I was looking for digital technologies that provide the effect suggested by the advice from Felder and Soloman.

Profiling questions

Well, I probably wouldn’t be using the Felder and Soloman learning styles questions as in the FAQ Felder explains how these questions have only been validated for college age students. The questions might retain some usefulness for younger students, however, the validity is somewhat questionable. In addition, there is the whole problem of self-reporting. i.e. I’m certain that a percentage of school students might see this as an opportunity to have some fun by answering exactly the opposite to their preference. This raises the question of whether or not there are any similar instruments that have been validated for use in school children. Not to mention validated for use with the cohort of children that I’m likely to teach.

That said, I would probably lean towards drawing on versions of these questions asked in the flow of classroom activity (rather than in a formal survey) combined with classroom observation. If subsequent research revealed validated instruments for school children, I might rely upon them.

I’m guessing that the answer to this question will have to be refined further prior to the completion of assignment 1 for the course.

How does ICT support differences in learning styles?

Again, I’d make the distinction that it is not the technology that supports the differences, but the capabilities for effects that it makes available. (This has me thinking about whether this is to fine/academic a distinction to make, for now I’ll stick with it.) Different types of technology provide effects that were previously not possible. e.g. the Blackboard provided a way for written or drawn information to be shared with a whole classroom. While a wiki allows multimedia information to be shared and modified with any group of people from anywhere with Internet access.

So, ICTs support differences in learning styles by providing effects/capabilities that fulfil some of the advice provided by Felder and Soloman. Especially when such capabilities were previously unavailable, too expensive etc. e.g. a personal blog provides a place for a reflective learner to work on and share their reflections in a way that is concrete and visible to others. Something that isn’t easily possible for a paper-based diary.

From a different perspective, ICTs support differences in learning styles because of their capability to manipulate digital information. For example, it’s possible that a verbal learner may not get much out of this blog post, but there are services that will automatically convert blog posts into audio. Hypermedia makes it easier to support both global and sequential learners in their approach to texts. Hypermedia and services like youtube make it much easier to support visual learners. e.g. 10 years ago (even 5 years ago) the readings for this course would not have been sprinkled with half a dozen videos.

From yet another perspective, ICTs support differences in learning styles to the extent that the learners, teachers, technologies, education systems and policies involved in learning allow ICTs to support differences in learning styles.

How many pages of a course profile are necessary?

This week brings the first formal tasks of my new phase as a teacher in training. We, the students enrolled in the Graduate Diploma in Learning and Teaching, are required to attend four days of a residential school. These four days coincide with Orientiation Week and we’ll be attending bits of standard O-Week, but mostly focusing on the res school.

Apart from pen and paper, the only items were advised to bring are printed versions of the course profiles. Now there was a time when the university provided print copies of the course profiles to all students. But now, mostly to save costs, these are distributed electronically and it is up to individual students to do the printing. Regardless of the course, the course profile follows a fairly standard template, which has me wondering. How many pages of a course profile do I really need to print?

After all, if it’s okay for the institution to save costs, it must be ok for me. Here’s what I found.

Course 1

3 pages of what appears to be reasonably important information, but information that is common can go. Surely it is covered in one of the myriad of guides and primers that we’ve been encouraged to read. In addition, having worked at the institution for 20 years in learning and teaching, I’m familiar with most of it.

Each assignment includes a rubric for marking. It appears to be a direct copy of what might be used by the marker. Even down to the dotted lines for comments, the lecturer’s name and a list of grades to be circled. Can’t see the need for that, especially when for 2 of the assignments that information is the only think on the page (it’s the last bit of information before the next assignment which obviously they want to start on a new page).

16 pages down to 11.

Course 2

This is the minimal course. What I think of the traditional course taught that arises when by a long-term member of staff is teaching a topic he/she knows well. Shall be interesting to see the differences evolve as the term progresses.

For example, the original course profile was only 9 pages long. The same 3 pages first removed from the profile course #1 were removed.

9 pages down to 6.

Course 3

10 pages down to 7 – same three pages.

Course 4

Again, only the same 3 pages: 13 down to 10.

Reflection

So, overall not that bad, but still 14 pages. With some work on spacing in the layout of the profile more pages could be saved. Of the 3 pages I removed, there’s an argument to remove them, but then there’s also an argument that some students have had problems by not knowing that information.

A proposed link between academic involvement and student evaluation response rate

In the following I ruminate on a possible correlation between academic participation in a course and response rates on end of term student evaluation. There are two points:

  1. It should be an relatively easy correlation to test.
  2. It would offer support for ways in which student evaluation response rates can be increased.

Understanding low response rates for student evaluation is important, at least for some institution, because they are expending significant resources in an attempt to increase response rates. Personally, I think they are generally barking up the wrong tree and that this sort of work might reveal a better tree.

Background – LMS, analytics, correlation and student participation

One aspect of the Indicators project has been using the data that is available from information systems such as an LMS to test correlations.

For example, in an early publication one of the correlations we explored was the link between student participation – measured with clicks on the course site or better yet with posts or reading of the course site discussion forum – within the LMS and student grade. Some work by Dawson (sorry don’t have the reference) had established the pattern that the greater the student LMS participation then the greater the student result. i.e. HD or A students used the LMS more.

We found this pattern with distance education students. As shown in the following graph that shows the connection between participation in a course discussion forum and student grades. (Click on the graph to see it bigger)

Average posts & replies for FLEX students

HD (high distinction – the top grade) distance education students were, on average, posting 13 replies per course discussion forum compared to 11, 9, 6, 5 and 3 for the lower grades.

But we didn’t find that on-campus, international students. As shown in the following graph using the same measure.

Average posts & replies for AIC students

The HD AIC students were actually making, on average, less replies than all other students except those that failed.

It should be remembered that we don’t know why this is happening, we’ve simply established an interesting pattern.

Another interesting pattern we established was that increase staff participation in the LMS course site also influences the correlation between student grade and student LMS participation.

Students in courses with high staff participation (staff clicked on the course site more than 3000 times in a term) gave this pattern.

Average student posts/replies on discussion forums for high staff participation courses

Students in courses with super-low staff participation (staff clicked on the course site less than 100 times in a term) gave this pattern.

Average student posts/replies on discussion forums for super low staff participation courses

Student evaluation response rates

For various reasons it has become important to some university folk to increase response rates on student evaluations. These are the anonymous, generally end-of-term, evaluation forms that ask the student about their course experience.

One institution I’m aware of is reporting figures that suggest an average response rate across all courses of about 16%. 3 of the courses has 100% response rates, but they all had 6 or less students. The highest response rate for a decent size course (n=99) was 68%.

The most visible strategy being used to increase response rates is lots of visible encouragement from the Vice-Chancellor and a 825×305 “banner” image on the LMS home page when students login.

If they want to increase response rates, I think they are barking up the wrong tree.

The right tree?

Based on my experience, observations and the above results, I believe there’s a strong correlation between the sense of responsiveness/participation students get from the academic staff teaching their courses. After all, one (but not the only one) of the reasons why I think it’s easy to get 100% response rates for a course with 6 students is that with 6 students a teacher can be very responsive.

It’s hard to get a computable indicator of that sense of responsiveness from a face-to-face classroom situation, but the above patterns from the indicators project suggest that a proxy figure – though I do admit that it is a somewhat flawed proxy – can be generate from the LMS.

I think it would be interesting to map the response rate on student evaluations with the level of staff participation in the LMS course site. If I’m right, then the best response rates should come from those courses with high staff participation.

There will be confounding situations, but in general I think the pattern would hold. In fact, the confounding situations would be interesting to investigate. What is different about those situations? What lessons might an institution learn? It might be interesting to survey some or all of these staff in order to find out what is the thinking behind their practice, is being responsive high up the list?

Of course, it’s still the wrong tree

In the end, however, there’s no getting away from the problem that such student evaluations are broken and not really worth it.

At the very least, the results of the above might just encourage a few staff to think about how they can improve student perceptions of their level of responsiveness and participation.

bim2: manage marking services – Part II

Time to continue the implementation of support services for manage marking started in the last post. Services left to be implemented are

  • Releasing marked posts.
  • Registering a blog.

Releasing marked posts

Standard order for this stuff

  1. Controller.
  2. Model.
    In this case, the model will be making changes to the database and storing the outcome.
  3. View.

Model

First, parse the parameters. There are two: marker and question. The possible combinations are

  • question and marker empty == release all marked posts for this activity.
  • question set only == release all marked posts for that question.
  • marker set only == release all marked posts for that marker.
  • marker and question = release all marked posts for that question and that marker.

So, have the model processing and updating the database. Also getting list of changes to report in the view.

The view will need to be able to translate student uid and question id into question title and user name. So need to include students and questions in the model.

View

A simple one, simply suggest success and summarise the changes made. Yep, all done. Need to do a bit more checking

Misc bugs

On manage marking there are a couple of minor bugs to fix

  • Empty table and email button showing up when there are no unregistered students.FIXED
  • The “release” link is appearing within a question heading when there are no questions to release.
    This one is probably going to be a bit more difficult to fix. Actually, the header code is correct in what it is doing. Time to check out how the rows are calculating whether there is anything marked. The display of marker details appears to be working, it’s the calculating of the stats that seems to be letting itself down.

    Ahh, all that and it’s not a code problem, but with the configuration of the users and course members. Mostly.

This is the type of bug that is going to have to be looked for in testing.

Registering a blog

This allows the coordinator to register a blog for unregistered students. It is essentially the same as the process for a student to register, just a slightly different interface. Hopefully, there can be some significant code reuse. Let’s look at how I did it for the student.

The student interface itself is a bit too complex. Registration is handled by some overly complex code within a method of one of the view. i.e. it’s not that stand alone. It is complicated because it also includes processing of the form.

Am going to have to put this into a stand along object that can be easily called elsewhere like this
[code lang=”php”]
$form = new process_register_form( $this->factory );
$form->process();
[/code]

There are going to have to be some differences between using it with a student and with a coordinator, including

  • heading – this (including breadcrumbs) will have to change.
  • return URL – successful processing generates a “return to ??”, ?? will change based on where it’s calling from.

In terms of the heading, it’s even more complex as the student and coordinator should see radically different headings when this is called. In the end, the code looks like this
[code lang=”php”]
$form = new process_register_form( $this->controller );
$form->set_header( $this, "view_header", "registeration" );
$form->set_return( "?param=regOK&id=".$this->model->factory->cm->id);
$form->display();
[/code]

The set_header method is used to tell $form which function to call to display the header that is required. set_return provides the URL params which let $form now where to redirect output when successfully registered.

So, time to add this in the appropriate place for manage_marking. There is some difference here in that it’s not being called from a view, but instead a controller. Another wrinkle to consider. Let’s create a view just for register and stick the call in there, this allows the view to be handled.

Ahh, no another problem, the URL for the register your blog has to be updated. Ahh, that’s the first parameter in the form. Done.

Next problem, when the student is registering their blog, the userid is based on their login, but that can’t happen when the coordinator is doing it. Has to be based on the student parameter. So, will need to pass in the userid for the student with the blog to be registered as a parameter. So this means I need to

  • Generate the form with an extra hidden parameter for coordinator. done
  • Same for student – how do I get id of browser user? (this->factory->userid) done
  • Modify processing of the form to use the hidden parameter. done

What’s next?

So, manage marking is essentially done. Time to move onto “Your students”, some new code there, also a fair bit of work to do, but should get quicker.

bim2: manage marking support services

As per the last post time to complete some support services for manage marking.

  • Releasing marked posts.
  • Displaying student details that match certain criteria.
  • Registering a blog.

Getting the controller going

First step is to set up the controller to start processing.

The current plan is that the processing will be done within the manage marking function, so tab is the same. The parameter op specifies which of the three services to execute. The view base class is already set up to allow sub-operations via the methods hash
which originally looks like this for manage marking
[code lang=”php”]
"marking" => array( "default" => "manage_marking" ),
[/code]
which suggests only the one method. If the op URL parameter is set, then it will look for alternatives, so it changes to
[code lang=”php”]
"marking" => array( "default" => "manage_marking",
"view" => "view_details",
"release" => "release_posts",
"register" => "register_blog" ),
[/code]

All that’s left is to define the methods such as view_details. Initially all with simply print statements, but subsequently with models and views.

Done.

View details

So, this should be a standard model/view combination

Model

The process here is

  • Check the additional parameters which specify which group of students to display.
  • Get the data for those students.
  • Display the details for those students.

There’s already a class that gets details for all students, seems to make sense to modify that to handle the ops.
Now have to get list of students for a marker, there is already a markers class, but it does it for all. Can it be made to work for just a single marker? Yep, that works, much the same approach as above.

So, the basic model is working. Needs a bit more work on the missing value, but will do for now.

View

The idea is to display the student details that match the chosen values. Students can be either registered or unregistered, so treat them differently.

While I wish to display it as part of manage marking, this display will also be required elsewhere. So, will implement with two views

  1. bimtwo/coordinator/view_details_view.php – responsible for the header/footer and getting the navigation to fit within manage marking.
  2. bimtwo/lib/students_view.php – to display the actual body of the page, the student data. This can be re-used elsewhere.

In fact, there is already a bimtwo_unreg_students_view class that takes care of the unregistered students.

Add in a bimtwo_reg_students_view and we’re just about there. However, it’s a little more complex as it wants to show information about the student’s feeds (how many marked, how many posted etc) and also progress against each question for the activity. This will take some time, however, it will be re-used as it serves a major component of the next tab to be implemented.

There is a bimtwo_posts class that gets all the posts for an activity, this could be modified to do the same

So, that’s basically working. It’s displaying information for both unregistered and registered students. But there’s a problem. It’s only showing details for one student and he is being listed as both registered and unregistered. Indicates a bug somewhere in the model code.

So, the model is passed to the view. It is a bimtwo_students class. This is passed to two separate views

  • unreg_students_view – which loops through an array unregistered to display details
  • reg_students-view – which loops through an array feed_details to display student details.

This is the correct behaviour. So, these are being populated incorrectly. Yep, the user is appearing in both. They shouldn’t be in the unregistered array. Ahh, unregistered is formed by this bit of code
[code lang=”php”]
$this->unregistered = array_diff_key( $this->user_details, $this->feed_details );
[/code]

The problem is that the keys for the two different arrays (user_details and feed_details) are the ids for different objects. So, construct a hash with the right keys, use it and hey presto it’s working.

Time to put in the criteria blurb at the top, basically tells the user what criteria is being used to search for students. So, that’s working. Here’s an example, the breadcrumb navigation will need looking at.

bim2: View details

What’s next

Will post this, still have to implement

  • Releasing marked posts.
  • Registering a blog.

bim2: Coordinator's manage marking tab

The next step in bim2 development is implementation of the manage marking tab.

What is it?

bim arises from an institutional context where it is common for courses to have at least a couple hundred students and greater than 10, sometimes greater than 20, teaching staff. The model at this institution was to have one or two of the teaching staff to take on the role of “coordinator”. The one in charge. One of the tasks for a coordinator is to manage and moderate the marking performed by other staff. Moderation is intended to ensure some consistency in marking.

To do this, the coordinator needs some way to observe how each other staff member is going with their marking and also to be able to access and view specific comments and marks awarded by the other staff. This is the purpose of the manage marking tab.

Another bim specific task was the releasing of marked posts. Since the coordinator had to moderate all marking before release, bim was designed so that only a coordinator can release marked posts. Students cannot see the mark and comments for their post until it is released.

In bim, the manage marking tab shows:

  • Unregistered students.
    This is a list of students who have not yet registered their feed with bim. bim provides a mechanism by which these unregistered students can be contacted via email using the Moodle bulk mail service.
  • Marking summary.
    A table with a row for every marker and columns for each question. In each question column it would show how many of the marker’s students have posts in the different states (i.e. submitted, marked, suspended, released, missing). Each of the different states is a link that will show the details of the students that have posts in that state. It also provides various links to release sets of posts.

What’s required?

There will four main pages for manage marking

  1. Main page.
    This shows the information described above.
  2. Results of release page.
    Showing the success (or otherwise) of an attempt to release posts.
  3. Details page.
    Shows the details of students with posts that meet certain criteria.
  4. Change blog registration.
    A way in which the coordinator can register a blog for unregistered students. This will be a page used in a number of other spots.

Each of these will have to be implemented separately, will attack them in order.

Main page

This is going to be a fairly straight-forward typical model/view combination. No form processing with this one. The only “difficult” aspect will be on deciding exactly how to create the support classes to get some of this information, rather than build it straight into the Model. Most of this data could/will be used elsewhere.

Model

The data required for this page includes

  • To calculate unregistered students
    • List of all student users for the course.
    • List of all registered feeds.
  • List of all questions for the activity.
  • List of all the markers for the course/activity and also a list of the students that are allocated to those markers.
    This is quite a complex task requiring a combination of a range of information. In bim, this was implemented as a hash where the keys were the userid for the markers and the value was another hash consisting of
    • marker – the markers id.
    • students – a hash of user details for all students allocated to the marker keyed on user id.
    • details – the markers user details.
    • statistics – a hash of stats for markers progress. i.e. how many posts are in the various states. The keys for this hash were either the question ids (which gave stats for that question) and “total” (which gave the overal total stats).

Current planning is to implement the above using the following classes

  • bimtwo_students – get all student details, including unregistered etc.
  • bimtwo_questions – this is an existing class.
  • bimtwo_markers – for the markers details.
    It has the following data members
    • DATA – hash of hashes, key is marker id, each element contains
      • firstname/lastname
      • students – hash of hashes, key is student id, value is entry groups_members
      • statistics – hash of hashes, key is question id, value is status and num in that status.
    • questions point to questions for activity

    View

    As implemented there are two main parts to the view

    1. Manage marking
      • Link to unregistered students and indication of how many there are. DONE
      • Main table for overview of marking stats for each marker. DONE
    2. Unregistered students – implement this with new view class
      • Another message about how many there are.
      • A button to email unregistered students.
      • Table listing the unregistered students and their details.

    There are also some gotchas, cases where special displays are required

    • No questions – the manage marking table doesn’t come with question columns DONE
    • No markers – no manage marking table DONE
    • No students – can’t do anything DONE

    Implementing unregistered students view

    This is a separate class as it will be used again.

    First, is there still a bulk email message service in Moodle 2? Yep, hasn’t changed. Same place. That works.

    So, now for the message and details table. Done. Here’s what it looks like

    Manage marking

    What’s left

    In terms of display, Manage marking is done. What’s left, however, are the various “services” which are linked to from manage marking. These include

    • Releasing marked posts.
    • Displaying student details that match certain criteria.
    • Registering a blog.

    I’ll leave these until the next post.

A glimmer of what I'm thinking

This time next year I should probably be teaching high school IT and maths. Because of this I’ve been thinking a lot about what type of teacher/educator/facilitator I’d like to be. What will be my ethos? I’m sure I’ll be asked that in coming weeks as I study.

This talk from Chris Lehman captures a significant part of what I’ve been thinking about.

Making high school not suck. Engaging with students in making stuff that matters.

How that can be achieved within the constraints of existing organisations should be interesting.

How hard is it to get a personalised class timetable?

Apparently, it is too complex to create a personalised class timetable for students at the institution I’m attending. I have previously described what I (as a student) have to currently do to create my class timetable as well as explaining that I helped implement a personalised class timetable system at the same institution about 10 years ago.

The purpose of this post is to find out just how difficult it would be to do it today.

The context

The institutional context includes:

  1. A collection of static web pages that contain class timetabling information.
  2. A newly introduced Google Apps for Education for students (i.e. students all have institutionally provided individual Google calendars).

The plan

The previous personalised timetable system at this institution was a stand-alone web application. With some input it created a one week summary timetable as a single HTML page. Given the change in context, in particular the availability of Google calendar, the plan is to respond to this change. Rather than having a stand-alone institutional application, the plan is to integrate with what students are using.

That is, the plan is

  1. Web scrape the institutional web page to get the data.
  2. Use the data to create an iCalendar file or similar with the timetable information.

The idea is that the student can then important the iCal file into Google Calendar or any other calendar program that supports that file format (which is most). If this were an institutional system, it might be able to automatically pre-populate all students’ individual Google calendars with their class timetable.

Missing access

The last time I implemented a system like this I was within the institution, this time I’m not. This means I don’t have access to information such as

  • The list of courses a student is enrolled in.
    For the test I’ll hard-code it with the four courses I’m enrolled in. Wouldn’t be hard to taken any list of courses and generate a calendar.
  • The dates for each week in the University calendar (e.g. week 1 is Feb ?? to Mar ??)
    I’ll also have to hard-code this.
  • Assessment due date information which could be added to the file.
    I’ll leave this data out of this little test.
  • Ability to automatically import information into the institutional Google calendar.
    Not sure if this is possible. If it were the institution could automatically insert into a student’s Google calendar their timetable.

Web scaping

I don’t think the web page format for the timetable has changed too much. Am hoping that I still have the code that can easily web scape the appropriate web pages.

Yep, there’s a bit of code from a couple of years ago, let’s see if that will work.

I think the hardest part here will be getting this ancient Perl code to run on my new computer when I haven’t done anything with it for years. Yep, get that working and hey presto it is extracting the information.

One of the limitations of the old Perl install is that the database stuff isn’t working. This is okay as I don’t need it for this little exercise. So, remove the database stuff and create a hashed data structure that allows manipulation.

Done.

Create an iCalendar file

Can this be done in Perl? Yes, a quick Google reveals the Data::ICal Perl module and some code that used Data::ICal to do something similar as I’ve planned (though only for the 2010 FIFA World Cup).

Say what you like about Perl, but CPAN rocks. Simple single statement and Data::ICal and all other necessary files are installed and working.

A couple of hours later (including a stop for lunch) and I have produced an ical file that will get imported into Google calendar. Main problem at the moment is that the timezone isn’t quite correct. Google calendar is showing the events 10 hours after they should have been – I think this is timezone related.

The big question now, “is there an easy way to delete all these entries?”. A google search reveals an option under setting for Google calendar.

Yep, slight change in the timezone setting and the times are okay. What about the dates?

Oops, week 12 dates showing up early (typo in hard-coded start date). Week 8 on a Tuesday, not Monday. Ahh, public holiday that Monday. So official week start date is the Tuesday. My code assumes the Monday. A more detailed version of this would need to figure out the public holidays. Same for week 9.

It’s all working

So, what’s been implemented is a script that

  • Automatically scrapes the institutional web page with timetabling information.
  • Extracts only the courses being taken by a student.
  • Generates an iCal file with the weekly personal class timetable.

Week 1 for me looks like the following. Click on the images to see a larger version.

Week 1

The monthly view on Google calendar, with one of the events highlighted looks like this

Monthly

Concept proven, work to do

It took me just over 3 hours to complete this. There was a break for lunch and most of the time was spent remembering how to interact with Perl and the Webfuse code.

This is not an “enterprise” solution, not yet. But it wouldn’t be that difficult to do. It’s certainly not impossible.

If you have followed some of my previous work or blog posts it will not be a surprise that I believe there are significant barriers in the processes around institutional IT systems that limit the possibility of these types of innovations.

Getting an overview of the term ahead

In the next couple of weeks I re-commence my face-to-face university education. This time in a Graduate Diploma in Learning and Teaching in order to become a high school teacher. As it’s less than two weeks from the start of term, the materials for the courses are now becoming ready. The following is an ad hoc collection of thoughts and experiences as I try to get an overview of the material and the term ahead.

Google being ruined?

Theoretically the institution is moving to Google Apps for Education for students. So the horrendous student email system originally being used is being replaced by gmail. It shall be interesting to see how the Google applications can be ruined by an institutional implementation.

Interesting to note that single-sign on still does not work for all applications. In this case getting the course profiles – the document serves as the “contract” between university and student – system requires another login.

Portals, timetables and ical

So there is a new look portal intended to bring everything into one place, to make life easier for students. However, it is still missing some fundamental elements. For example, a personalised class timetable, as I’ve complained about before.

There is a calendar in this new portal. Ahh, it’s Google calendar. But it’s not pre-populated with anything. I’m currently looking through the course profiles at the assessment. Slowly building up a calendar of due dates, wouldn’t it be nice if this could be done automatically? After all the institution has the due dates in a database, it knows which courses I’m enrolled in. At the very least, the ability to download an ical file or similar to import might be nice.

The absence of this sort of thing, especially if it remains absent in the long term, is another example of the problems facing institutional systems touched on briefly in the post around open source LMS. The IT folk at this institution are focused on package provision, not on the user. They will provide the calendar, but joining together two separate systems is not something they – or the applications – will do automatically.

Personal versus institutional

The other problem I face, is that I already have a gmail account, I already have a google calendar. I don’t want the institution to provide me with another gmail account and another google calendar. I want it to work with what I have. Ahh, there’s the Google docs link, I already use that.

Come on guys, catch up with the world.

Here’s an example. With the old student email systems – a web-based interface – the institution actually modified the system to remove the capability for students to forward email. So, instead of having the email all come nicely to the one place, I had another place to check.

So, can I automatically forward my new institutional gmail account? Yes, I can. It’s baked into gmail and I imagine they couldn’t convince (or pay) Google to remove it. Ahh, it’s also pop enabled, so I can bring it all into the one place (I use Thunderbird as an email client). Ahh, but will the screw around with hostnames, single signon get in the way? Yes, it appears like it does. Should I ask?

Bugger it, forwarding works. Will stick with that.

Of course, my decision here is another example of why institutional systems don’t improve. In the absence of anyone not complaining about the ability to use POP/IMAP the institutional IT folk will claim that there is no need for it, no-one wants it. It never occurs to them that people are smart enough (or cynical enough) to understand the futility of asking.

Of course, it now appears that forwarding may also not work. Actually, I just had to wait. It’s working now.

Bulk email to students

Oh dear, the uni is trying to increase response rates on student evaluation by bulk email to students from the institutional leader. At the very least you could limit this to students actually enrolled in the term. i.e. I was not enrolled in any courses during T3, 2010.

Assessment

By course the assignments are

  1. Course 1
    1. 40% – 10 “investigations” – 2000 words equivalent.
    2. 60% – 3000 word report, a range of supporting activity.
  2. Course 2
    1. 20% – Learning design brief, 1000 words – essentially a reflective blog (on first skim).
    2. 40% – Blog including 1600 word reflective synopsis.
      Oh dear, you have to upload your blog url via a Word document into Moodle. Have these folk not heard of BIM? It’s even installed.
    3. 40% – 2000 words – a learning design?
  3. Course 3
    1. 50% – Lessons plans, implementation and critical de-construction – 23 May
    2. 50% – Unit of work and evaluation – 6 June
  4. Course 4
    1. 50% – 2000 words report – 21 April
    2. 50% – 2000 word report – 26 May

bim2: Some more coordinator tabs

The last work on bim2 ended up with the first 2 of 5 tabs for the coordinator interface completed. This post continues on with that development.

Allocate markers

This is the tab which allows the coordinator to allocate different teaching staff to different groups of students. This structure arises from the context of the first institution to use BIM/BAM. In that context, it was not unusual for a course to have 10+ academic staff teaching the course, each responsible for a different cohort.

The basic algorithm for this tab is/was

  • If there are no teaching staff listed for the course, display a message and stop here.
  • If there are no groups for the course, display a message and stop here.
  • Otherwise, get all of the current bim marker allocations (stored in a bim table).
  • Display a form that contains a select menu for each marker. The menu contains all groups and highlights those that are allocated to the marker.
  • On submission, check the allocation to each marker and update database appropriately.

To a large extent this follows the same structure as the manage questions tab developed in the last post. This implies I’ll need to implement the following classes:

  1. allocate_markers – model.
    The data needed includes
    • List of all teaching staff for the course.
    • List of groups for the course.
    • List of bim2 marker allocations for this activity.
  2. allocate_markers_view – view.
    Which will implement much the same “structure” as the manage markers view in terms of displaying and processing the form.
  3. marker_allocations – object.
    Simple class to retrieve list of current marker allocations.
  4. bimtwo_allocation_form – the form.
    Moodle form class extension that implements the form.
  5. bimtwo_process_allocation_form – object.
    Processes the form on submission.

To do: The intent is that bim2 will recognise groupings, an aspect of Moodle’s operations that I have yet to grok. For now, I plan to ignore this and get a version of bim2 up and going. Given the OO structure, in theory it should not be difficult to come back and add support or groupings at a latter date.

Okay, that looks like it is working. The initial Allocate Markers page is very simple

Allocate markers

Just a list of teaching staff with a select menu full of course groups. The results of any processing are given a simple unordered list.

Allocate markers processing

There is a small problem with these images in that there are some missing images from the theme. They weren’t missing in the Manage questions dumps from earlier in the day. So I’m assuming there is some minor problem with the theme rather than a fundamental mistake in how I’m doing something in Moodle.

What’s next?

Hopefully tomorrow will see the last two coordinator tabs completed. Manage marking is a fairly simple in terms of form processing. “Your Students” is more complex, but it also serves as the interface for markers.

So, I’m hoping that the complete staff interface should be finished by Tuesday. i.e. most of bim2 will be working.

Then I move onto backups and other background tasks. Won’t be able to predict deadlines for those tasks until I take a closer look at the changes in backups for Moodle 2.

On the potential flexibility of open source LMS and its limits

Today a mate posted to his blog about a small project he’s involved with. The context of this project seems to be a good opportunity to comment on the potential flexibility of open source LMS and the limits of that flexibility within an institutional context. It’s also an attempt to link it back to the design theory described in my thesis (if you want more of the theory behind the following, look at the thesis).

The following uses Moodle as an example, but I believe that similar limitations exist regardless of the open source LMS. This is in part because a significant limit on the flexibility is not the LMS, but instead the institutional governance processes and associated factors..

The need

In this case, the need is to send students an email with a link to a survey. The link has been personalised based on the students’ membership of Moodle groups. They survey asks them to answer questions based on their experience of a group task.

My initial thought would be that this sounds like something Moodle should be able to do. Given the increasing emphasis on group related work I doubt that this is a novel requirement. So, there might be something in Moodle that can do this, however, based on my limited knowledge of Moodle I can’t think of anything off the top of my head.

I believe that there might be the functionality within Moodle to do each of the components of this task. There is probably a way to send emails to members of a group. There might even be a way to customise that email to some extent (there is a bulk email facility in Moodle 1.9, but, from memory, it seems somewhat limited). There is also probably a way to do a group-based survey (a MCQ might be the obvious solution).

But I doubt that there is an easy way to combine these separate functions so that the group email can automatically include the link to the group’s MCQ/survey.

Doing it outside of Moodle

There is another interesting and related comment in post describing this project

hile not ideal in that it is a separate system from the LMS, it is hoped that this trial will help inform the development of a Moodle module that will perform the same function albeit in a more integrated and seamless way

Over 6 months ago, I used to work at the institution being described. Based on that out-of-date experience, my initial guess is that “doing it outside of Moodle” is deemed to be easier than engaging with Moodle and the institutional IT department.

Two limits of open source LMS flexibility

Drawing on the above examples, I’d like to propose at least two, somewhat related limits on the flexibility of open source LMS:

  1. Inflexible institutional structures and processes.
  2. The difficulty of producing/the absence of scaffolding conglomerations.

Inflexible institutional structures and processes

Modifying an enterprise implementation of Moodle effectively and efficiently is hard. You don’t want your institution’s Moodle instance to be unavailable to students and staff because a code change has broken something drastically. The Moodle code-base is itself quite difficult to get a handle on. Not overly difficult, but a non-Moodle developer can’t simply front-up and start making changes quickly. They need to be enculturated into the Moodle way, to learn what works and what doesn’t. Such a requirements means that someone who is able to modify Moodle is often a scarce and expensive resources. Especially within most universities who often don’t have someone dedicated as a Moodle developer.

To address this difficulty and also to CYA (some might argue that CYA is the major reason) institution’s spend a lot of time and effort setting up appropriate governance structures. The theory being that these are objective and rational ways to manage the difficult process and the expensive and scarce resource.

The trouble is that the difficulty and expense involved means that it becomes difficult for such processes to effectively engage in “small” problems like this one. i.e. problems that don’t actually require development of any significantly new functionality or large-scale modules. It just needs a few minor changes or wrappers around existing functionality. For example, the requirement above could possibly be solved (the following is an example description given off the top of my head without any investigation as to whether this would work) by

  • Modifying the Moodle quiz function to populate a database table linking groups to URLs for group specific quizzes.
  • Modifying the existing Moodle bulk-email facility (or perhaps adding a wrapper around it like I did with bim) to use this database table to send personalised emails to group members.
  • Perhaps add a new quiz report that allows viewing/comparing within/across groups.

For a variety of reasons traditional institutional LMS policies and processes are too heavy-weight to respond to this sort of need. Instead, in order for something like this to be considered, it has to be blown up into some institutional priority. e.g. a system to support peer and group-based assessment for the entire institution. A project that will require a significant amount of time doing a needs analysis,……..

A big project that requires lots of resources is expensive enough to be efficiently considered by the governance and related processes. Small projects are too cheap to be efficiently considered by the expensive institutional processes.

In the hardware/operating systems field, this is a problem known as starvation or indefinite postponement. The situation where a task is forever ignored because of a flaw in the priority mechanism.

So, I’m proposing that the institutional implementation of open source LMS end up suffer from the “starvation limit” on flexibility.

The need for rapid development of scaffolding conglomerations

The need in this case, at least to me, sounds like an example of what I termed scaffolding, context-sensitive conglomerations. Rather than necessarily requiring a brand new Moodle module or block, this problem sounds like something that actually needs to combine the functionality from a number of existing Moodle services. Something that conglomerates the lower-level functionality provided by Moodle into something that better meets this higher-level need.

A large part of the popularity of Moodle arises from its modularity. A feature that allows for the easy development of lots of new functionality. Something that increases the flexibility of Moodle.

The problem is that this flexibility arises, in part, from keeping these different modules separate. It’s the separation that makes it easy to add a new function without (theoretically) worrying about how it will effect the other modules. They are meant to be independent. The current problems moving to Moodle 2.0 is an example of the problems that arise from dependency. All the third-party modules depend on the Moodle core, so when the Moodle core changes all the third-party modules have to change.

A strict separation between modules makes it more difficult to combine parts of these different modules into a scaffolding conglomeration.

So, I’m proposing that open source LMS have an “over reliance on module independence” that limits their flexibility.

It’s really all about balance

I can already here proponents of traditional institutional IT governance processes or strict software engineers bemoaning the problems of not having institutional governance or of module dependence. And I do agree. There are dangers and problems. I’m not suggesting that they should necessarily be done away with.

I do, however, think that too often the balance has gone too far one way. There needs to be more recognition of a need for balance the other way. A bit less of a focus on the objective, best ways of technical implementation, and a bit more on the subjective, best ways to improve learning and teaching.

bim2: Starting on the coordinator models

The last bit of work on bim2 set up the structure for the coordinator interface and operations. This post documents the work of implementing the models, i.e. starting the process by which the coordinator interface actually starts actually working. Most of this should be a reasonably straight translation/update of bim into the OO structure of bim2.

This post gets the first two tabs done. Three to go.

Configure

The aim here is to display basic configuration information and link to the standard configuration form. This is somewhat simpler because there’s already a class for doing this that is used by the student. So, will have to borrow that and perhaps add a bit more beside.

Yep, all done. Also discovered that the problem with help buttons was due to the theme I was using, not my use of Moodle. Progress!

Time to double check and get the actual configure process working. First, a win with the help buttons, finally figured out what I was doing wrong. Next was a bug in the database table to hold the configuration and now there is a problem with the description being saved. Yep, another transition mistake in moving to Moodle 2. Fixed. (Click on the screenshot below to see it larger).

configure

Manage questions

Most of the remaining tabs for a coordinator (except Manage marking) require the use of forms. Manage questions presents a form that contains space to add a question and form elements for each existing question. It’s used to modify, add and delete questions. Handling forms in this structure is a little different – and more complex – than what has happened so far. So, first step here is to re-acquaint myself with the kludge I used for the student form for registering a new blog.

The process is that the controller method remains essentially the same – create model, create view, pass model to view and use view’s display method – the difference is that the view embodies the Moodle form process. Which is essentially

  • Create a form object;
  • Implement 3 or so different options based on a value from the form object including: display the form, handle a cancellation and process the form contents (i.e. update the database).

Actually, that’s not too different. First step is to create the model. In this case the model has to know all about the questions for this bim2 activity. For which there is already a class, but it will need to be embedded in a model object. So, copy a model from the student over and update it to use the questions class. Done that was quite simple and the view is dumping its contents.

The step now is to use the data in the model to display and handle the form. The rough, initial version of the code looks like this
[code lang=”php”]

function display() {

global $OUTPUT;
global $CFG;

// construct the form
$cm_id = $this->model->factory->cm->id;
$form = new mod_bimtwo_questions_form( "view.php?id=$cm_id" );

if ( $form->is_cancelled() ) {
$this->view_header( "questions" );
echo $OUTPUT->heading( "Manage questions" );
$form->display();
} else if ( $from_form = $form->get_data() ) {
echo $OUTPUT->heading( "Process the form" );
} else {
$this->view_header( "questions" );
echo $OUTPUT->heading( "Manage questions" );
$form->display();
}

//print_object( $this->model->questions );
$this->view_footer();
}
[/code]

What’s left to do is create the bimtwo_questions_form class and put in the real code to process the form.

The form class should be a straight copy and slight update of the form used in bim. Yep, that’s working fairly simply. Most of the work was involved with updating the language strings.

One extra bit of work is that the questions form from bim does assume the existence of an additional bit of information for each question, i.e. a status array showing number of students in each state. The question class in bim2 doesn’t support this, but I should be able to simply add a method, implement status and Robert is a parental male sibling. Done.

Now to process the form and update the database. First, the outline of the process from bim

  • If a title and post for a new question then add a new question.
  • Loop through each of the existing questions and do a range of checks for changes
  • Re-display the manage quesitons form.

In bim this is implemented as quite a long winded procedural function. Probably a candidate for methods or perhaps a new class. I’ll go the new class, don’t want to muddy the “view” waters with stuff that is manipulating
questions.

Forgot the function to add the question data into the form.

Okay, so have a processor object that is doing the form processing. Have implemented the checks for a new question and deleting questions. Also have adopted the approach where the view (which creates the processor) figures out how to display what was done. Time to move onto the handling of modification of existing questions. The process will be for each existing question

  • for each existing question
    • check if there has been any change;
    • if so, make a copy of the changes in “changed_question”
  • update the database with the changes;

Question: what happens if a question has already by deleted when it comes time to update? At the moment the question will have already been deleted from the database as deletion is handled first. So will prevent “deleted” questions from being modified.

Okay, now there is a problem with the update_record method, “update_record_raw() id field must be specified.” Appears for some reason that the structure being passed in is having all the fields cleared before insertion! Ahh, silly user error and unhelpful error message. I was using an old bim v1 table name, not the new version and Moodle wasn’t giving an error that suggested this as the problem.

With that fixed, manage question is working. Based on my naive understanding of good Moodle 2 practice I can no longer do the automatic redirect that bim v1.0 used. Instead, you start Manage Questions with the following page that shows a list of all existing questions and space to add a new one.

Manage Questions

You make your changes on that page and hit one of the submit buttons. bim2 then processes the changes and displays details about what happened.

Manage questions update

There is a link at the bottom of that page to return to the first Manage Questions page. In bim1, this would have been where the automatic re-direct would’ve happened.

What’s next?

Well, that’s two of the coordinator tabs down. Another three to go. Will start another post for those.

Standardised testing is bad, but what is the alternative?

Spent yesterday playing cricket in the local competition. A first step in returning to a “normal” life. As it happens one of the guys I was playing with – and have known for a long time – is also a Deputy Principal at one of the local primary schools. He asked if I was still working at the Uni and I explained no, I was heading into high school teaching. From there the discussion turned to teaching and its issues and benefits. Before too long I asked about NAPLAN testing and its impact. “It’s all we’re teaching at the moment”, was the response, essentially much the same as I I mentioned in a previous post.

Over the last six months or so, I’ve been listening to and reading a lot of K12 education related material. All of it, especially that by folk with first-hand knowledge of actual teaching, has been aware of the negative implications of standardised testing, especially when tied to rankings or funding. It corrupts the teaching process and becomes the almost primary focus of learning. At the core, this seems to be a wonderful example of Goodhart’s law which Wikipedia describes as

states that once a social or economic indicator or other surrogate measure is made a target for the purpose of conducting social or economic policy, then it will lose the information content that would qualify it to play such a role

It is obvious that school performance on the NAPLAN test has become a target. School’s and teachers are being told by regional management to focus on improving performance on that target. Consequently, I believe that NAPLAN test scores have lost any value, the information content in such scores does not represent the quality of education. It represents how well the educators have been able to prime students for the test. More importantly, it fails to measure what has been lost in the chase for good NAPLAN scores.

What is the alternative

The idea of standardised testing, like the NAPLAN tests, arise from the idea of managerialisation which is seen by some as a key component of globalisation. It is the idea that it is important for government to measure and assure the quality of services like education. Measurement that requires objective, quantifiable measures. It is unlikely that this requirement for measurement is going to disappear anytime soon.

Given, as described above, that I believe the current approach to this measurement is having extremely negative effects on what it is trying to measure. Then the question is, what are the alternatives? Is there a way to provide government and society with the “measurement” that shows the quality of the education being provided to children?

Surely there has to be some folk within the educational research community coming up with alternatives? Even some folk trying them out in real education settings.

This is not a problem limited to K12 education. Universities, especially those in Australia, are increasingly having to deal with this problem and they aren’t doing it any better.

This is likely to become one of my underlying questions for the next year or so.

Page 1 of 2

Powered by WordPress & Theme by Anders Norén

css.php