Some assemblage required

Assembling the heterogeneous elements for (digital) learning

Month: April 2012

Questions and insights on (education) hack days/hackathons

By

On April 25, 2012

In thesis

A few weeks ago, inspired by @sthcrft, I speculated around the idea of having a eduhack session at ASCILITE’2012. There was some interest and I had other tasks, but now it’s time to think a bit more about it.

The following is :

  1. an attempt to generate a list of questions to answer/ponder;,
  2. a summary of some material I’ve found so far, and
  3. an attempt to generate suggestions from others.

Questions

  • Can a hackathon idea work with the mix of people at ASCILITE’12?
    The crowd will be most education focused folk – educational developers (aka curriculum/instructional designers, academic staff developers etc.), managers, and academics – with a sprinkling of technical people. Hackathons are traditionally very focused on producing a prototype typically through software development. Will this work with the ASCILITE crowd?
  • What type of “prototypes” would an ASCILITE’12 hackathon produce?
    Some would be software, but what other types? Training resources? Plans? Policies? What prototypes/outcomes that will be useful, that will spark on-going work.
  • Are we talking about a hackathon that isn’t really about software developers (having some around would be nice) and what does that mean for how it will work?
  • How to generate the ideas to put to people?
    The aim should be to work on interesting and so far unsolved (unthought of?) needs.
  • How to organise the event to get the collaboration, cross-fertilization, and community building going?
    These are not easy things to do.
  • How long should it be/when should it run?
    Hackathons are typically marathons lasting much longer than the typical conference workshop. Can you push the ASCILITE crowd into something like that? Can it be resourced? Do you stick with the typical 3/4 hour workshop sessions?
  • Can a session like this be resourced?
    It needs really good net access and a more casual/collaborative environment than typical of conference workshops. Are there spaces that might suit?
  • What about online?
    I know there will be folk who want to participate, but can’t make it to the conference. Can this work as a “dual-mode” offering? How?

The resources

Let’s start with the Wikipedia page for Hackathon. It has an overview of the origins, purpose, and how such events are often run.

Remaining resources divided into categories

Examples

Education Hack Day held in late 2011 at a US high school. 80 or so software developers and designers were given crowdsourced ideas from teachers and had a day to respond and compete to win prizes mostly aimed at enabling a company to work on the idea.

Random Hacks of Kindness – Hacking for humanity a “global community of innovators building practical open technology to make the world a better place”. Lots of support, large, global. Apparently focused on disaster management and crisis response.

Literature

I like to build on what has gone before, so some ad hoc Google Scholar searches reveal.

Wheatley, P, Middleton, B, Double, J, Jackson, A and McGuinness, R People Mashing: Agile digital preservation and the AQuA Project. In: http://ipres2011.sg/conference-procedings. IPRES 2011: 8th International Conference on Preservation of Digital Objects, 1-4 November 2011, Singapore

From the abstract (emphasis added)

Two AQuA Mashup events brought together digital preservation practitioners, collection curators and technical experts to present problematic digital collections, articulate requirements for their assessment, and then apply tools to automate the detection and identification of the content issues. By breaking down the barriers between technical and nontechnical practitioners, the events enabled grass-roots digital preservation collaboration. This paper describes the AQuA Project’s novel approach to agile preservation problem solving and discusses the incidental benefits and community building that this strategy facilitated.

Steven Leckart, The hackaton is on: Pitching and programming the next killer app.

The obligatory Wired story on Hackathons.

Mark Headd, Open Government hackathons matter

Blog post delving into the apparently thriving culture of civic hackathons. Some comments about the results not being long term, but also an argument that it’s more than just the product.

The 2006 NESCent Phyloinformatics Hackathon: A Field Report Evolutionary Bioinformatics

Gives some background rationale and process of organising a hackathon for phyloinformatics.

Lisa Petrides and Cynthia Jimes (2008), Building Open Educational Resources from the Ground Up: South Africa’s Free High School Science Texts, JIME.

Hackathons were one of the successful methods used in getting OER content written as part of a broader project.

A guide to organising a successful hackathon for social good – post giving tips. And another set of tips

Mind the gap – a big problem with institutional e-learning

By

On April 13, 2012

In elearning, thesis

The almost 300 students in the course I’m teaching have their first assignment due on Monday. I’m currently laying the ground work to help the small course team to mark those assignments. Actually I’m busy procrastinating about the task by writing this blog post. Mainly because it is more work than it needs to be and it is this way because of one of the fundamental flaws of how universities are currently implementing e-learning systems.

A flaw which means they can never ever bridge the gap between what people actually need to do with the system and what the system does. For many the gap won’t be large, but when you are marking 100+ assignments within a couple of weeks what was once a small gap grows quite significantly. A situation where forewarning of the gap isn’t sufficient, it should be filled.

mind the gap

I’m going to illustrate my point with the current online assignment submission system I’m working with. There may well be a range of contextual reasons why this system is designed the way it is, and it is a step up from some I’ve seen. But those contextual reasons are part of my point, the context within which e-learning systems – like an LMS – are supported and operate is not conducive to bridging the gap (let alone doing something innovative). The point is broader than online assignment submission.

Manual distribution

For example, there are three people who will mark these assignments. They will be marking different cohorts. The system has been told which cohorts they will be responsible for marking.

I assumed that as students submitted their assignments these markers would be able to access those assignments. i.e. the system would automatically allocate them to the marker. No.

Instead a human being need to be involved to manually allocate. Always fun to do with 300 assignments.

Marking

The students have created a page in Mahara. They are submitting the URL, which is one of the options in the system. The marker sees a page that lists each student they are marking. For each student there is the link they submitted, some space to add a comment, add a mark, and space to upload a file. In our case, we’re going to use an Excel spreadsheet to give feedback.

This means that for each student the marker will have to manually make a copy of the spreadsheet for this student, enter the student’s number and name into the spreadsheet, click on the link, mark the submission, copy the grade created by the spreadsheet into the form, and upload the spreadsheet.

If you’re marking 10, not a big problem. When you’re marking 100+….

Filling the gap

There are many ways to fill this gap. The approach I’m going to use (I hope) is to write a script that will allow me to give each marker a zip file. The zip file will contain a collection of Excel spreadsheets. The file name of each spreadsheet will match the name and number of one of the students they will mark. Each spreadsheet will include the student’s name, number and the URL for their Mahara page. Without entering the submission system, the marker can work through the spreadsheets, click on the URLs and mark the assignments.

It’s just a pity the marker can’t zip up the spreadsheets and upload them into the system in one go.

A new system will solve it

There seems to be interesting work being done around Moodle’s assignment submission/management system. My institution will probably move to that at some stage. The move will be promoted as solving all sorts of problems.

But it won’t solve the problem of filling the next gap. With the new system there will be some gap that is identified. Some new practice that would be really good. The trouble is that how e-learning systems are implemented and supported within most institutions means that this gap/new practice will not be implemented. It will have to wait until the next grand, new system. During that time huge amounts of staff and student time will be wasted jumping over the gap, filling it in, or worse, ignoring it.

Big deal?

Using an approach better suited to filling the gaps we implemented an online assignment management system that had filled many (but not all) these types of gaps almost 10 years ago. A system that was well received by its users.

For almost 10 years there have been institutions of higher education using online assignment submission systems that are wasting time and money. And this is just one of those boring, administrative jobs at the current core of higher education. Imagine all the other unfulfilled gaps and opportunities that have been missed.

A more in-depth extension of this argument and one proposed solution forms the foundation of my thesis.

bim2 – working on coordinator – part 1

By

On April 11, 2012

In bim2

More work on bim2, this one starting work on the previously identified tasks. Find out what BIM is here.

Status: At this stage, the user screens from bim2 can be considered in alpha. i.e. they basically work, but there will be bugs and some of the underlying “infrastructure” (i.e. backups and restores, gradebook integration etc) need to be fixed. The current code can be gotten here.

Configure BIM improvements

  • Help buttons for configuration options. DONE
  • Have the “General Steps” actually link to the appropriate screen.
  • Change “provided below” link to something more human readable. DONE – well kind of.

Problem with changing the configuration

Ahh, this may be related to the gradebook issue. When the configuration calls for adding an entry to the gradebook, we’re getting an error. Confirmed, if I turn “grading” off, it works fine.

This will/should be due to API changes in Moodle 2.x. The question is do I fix it now or leave it until later?

Well, as with most of the Moodle docs, it is incredibly hard to find the canonical answer. Instead, I’ll have to revert to looking at the BIM code and comparing that with code in other modules.

The changing of the grade entry for configuration is done in bim_grade_item_update

[sourcecode lang=”php”]
require_once($CFG->libdir.’/gradelib.php’);

$item = array();
$item[‘itemname’] = clean_param($bim->name, PARAM_NOTAGS);
$item[‘gradetype’] = GRADE_TYPE_VALUE;
$item[‘grademax’] = $bim->grade;
$item[‘grademin’] = 0;

grade_update(‘mod/bim’, $bim->course, ‘mod’, ‘bim’, $bim->id, 0, null, $item);
[/sourcecode]

The error I’m getting is “Column ‘grademax’ cannot be null”. Mm, that will take a while to solve. Save it for later.

Manage questions – add new not working

Okay, this seems to boil down to some changes with how to work with the htmleditor in Moodle forms.

Manage questions – improvements

  • help buttons for elements to add. DONE
  • Work on the introductory text. DONE

Allocate markers

It was crashing, ended up being simple typo. Up and going.

Manage marking

Displaying the heading, but then crashing. Problem is in bim_get_all_marker_stats. The in_or_equals needs to be used properly as it hasn’t made the leap over…complex data structures

Need to look into the problem with html editor within form not showing data in the database. Confirmed that it is getting the data from the database.

Okay, this seems to be connected with the more complicated method for using the editor and passing data into and out of it. Surely this has to be documented somewhere???? Can’t find it but I’ve figured out how to fix it.

Find student

When actually doing the find generates an error in line 101 of find_student.

get_records_select syntax has changed, fixed that up. Had to make some changes to the “where” part of the SQL used here.

Note: wondering if this will make any difference to cross-platform operation

Now the process for showing multiple users isn’t working. Problem with get_records_select..again. Also problem with table display. Need some rewriting to get the table display working well.

Also some problems with the language strings, which look like they would’ve caused problems for bim 1. That’s fixed. Another problem if the search finds one student who hasn’t been registerd. Another hang over from bim 1. That’s fixed.

Why learning management systems will probably go away

By

On April 6, 2012

In design theory, elearning

I’d like to argue – and yes it is perhaps a case of wishful thinking – that there are reasons to suspect that learning management systems (LMSs) will probably go away. Or at the very least, identify some forces that are pushing that way.

The post is mostly in response to Tony Bates post “Why learning management systems are not going away”, but it is also inspired by this post from “Music of Deskchairs”.

To some extent, as I’ll suggest below, this could become an argument about the definition of what an LMS is and just how long it, whatever it is, is going to stay.

But first, a word from Chris Dede (a Harvard Prof of education no less) to lay the foundation for an aspect of my argument. In this 1m39s mp3 Dede argues that the assumptions about learning that underpin “our best learning environments” are severely flawed.

Why the LMS may go

Bates’ lists four reasons he thinks the LMS is here to stay:

  • Instructors and students need a structure for teaching in terms of topics, sequences etc, the LMS provides that structure.
    Yes, based on my experience a vast majority do. But there’s debate to be had around whether the LMS is the tool for this structure to be provided. Whether the limitations of the structure provided by the LMS will be too constraining as more interesting and useful tools develop.

    The broader question is what will happen as different pedagogical models – e.g. MOOCs, open teaching and other more social models currently being explored by a range of folk – become more prevalent. The diversity of learning is going to push against the constraints of the LMS as we currently know it.

  • Instructors and students need a private place to work online.
    I don’t deny the pressure on academics and students to modify their statements when in the open. No matter how much I deplore the need. However, the LMS is not the only private place on the Internet. Any number of spaces can be private. But the assumption that these places are truly private is just a bit mistaken. The ease and lossless nature of copying in an online world means that what is private can become public very quickly.

    Frankly I don’t say anything in an online medium – public or private – that I won’t stand by because I’m aware that there’s always a chance what was private could become public.

  • The choice is not either an LMS or web 2.0 tools
    I agree with this one. I’ve been working on BAM/BIM since 2006 as an example of this trend. We also did some work around a “Web 2.0 course site” in 2007, see the image below. Based on this experience, however, I do believe that when an “LMS” really starts to work effectively with Web 2.0 tools, then the very nature of the system and how it is supported also needs to change. In particular, it moves away from the “you can only use what’s in the integrated system (LMS)” approach currently popular. This will be a challenge, just a few weeks ago I was told “not to use any non-ICT approved cloud services”.

    Home page for Web 2.0 course site

    Rather than be the central provider of all online services as it is know. The LMS will have to become the “glue” that binds web 2.0 tools and institutional requirements together. To work really well the task of choosing and integrating Web 2.0 tools has to start moving into the hands of the users, not the developers. This starts becoming a very different type of beast. See the argument below.

  • Institutions are increasingly reliant on the LMS, especially for accountability.
    The argument here is that administration is increasingly reliant on the LMS as the place to check on student performance, for appeals and accountability. Bates does not suggest this is a good thing, but states it is the reality.

    Of all the arguments this is probably the strongest one in terms of creating inertia around the LMS. In reality, it is management that makes decisions about LMS selection. They may engage in democratic and open decision making, but in the end, if they want something…This perspective does ignore a couple of points, which I’ll pick up below.

    One point I will make now is that this perspective seems to miss the post-LMS ideas of the loosely coupled gradebook that are tightly connected to the last point. With the rise of big data there are just as good, if not better, methods for accountability/analytics available for use with Web 2.0 tools as with the LMS.

VLEs and not LMS?

Which raises the whole question of what an LMS is. From an information systems perspective, the historically dominant product model of an LMS has been the monolithic, integrated product model. Everything you ever need for X is in the integrated product that is sourced from a single vendor. You don’t need anything else and in fact integrating it with anything else is really difficult.

But there is another product model – best of breed. Light, Holland and Wills (2001) perform a comparative analysis of the monolithic versus best of breed approaches. That’s where the following table comes from. If an LMS is going to effectively integrate the rapidly growing number of high quality external applications (including Web 2. 0 tools) then it is going to have to move heavily towards a best of breed model, not an ERP/integrated model.

But it doesn’t stop there. How an organisation supports and uses a best of breed model is radically different from an ERP/integrated model. For example, with a best of breed approach there is a lot more time and effort placed on being sympathetic to the context, rather than making the context suit the tool. The organisational units supporting a best of breed system require radically different skill sets.

Bates’ suggests

we should be thinking more broadly than just the LMS. Instead we should be thinking about virtual learning environments

If we’re to think about more broadly about the VLE, rather than the LMS, then I would argue we’re having to think more about the best of breed approach than the ERP/integrated approach.

Table 2.3 – Comparison of major differences between ERP and BoB (adapted from Light, Holland et al. 2001)
Best of breed Single vendor ERP
Organisation requirements and accommodations determine functionality The vendor of the ERP system determines functionality

A context sympathetic approach to BPR is taken A clean slate approach to BPR is taken
Good flexibility in process re-design due to a variety in component availability Limited flexibility in process re-design, as only one business process map is available as a starting point
Reliance on numerous vendors distributes risk as provision is made to accommodate change Reliance on one vendor may increase risk
The IT department may require multiple skills sets due to the presence of applications, and possibly platforms, from different sources A single skills set is required by the IT department as applications and platforms are common
Detrimental impact of IT on competitiveness can be dealt with, as individualism is possible through the use of unique combinations of packages and custom components Single vendor approaches are common and result in common business process maps throughout industries. Distinctive capabilities may be impacted on
The need for flexibility and competitiveness is acknowledged at the beginning of the implementation. Best in class applications aim to ensure quality Flexibility and competitiveness may be constrained due to the absence or tardiness of upgrades and the quality of these when they arrive
Integration of applications is time consuming and needs to be managed when changes are made to components Integration of applications is pre-coded into the system and is maintained via upgrades

The Blackboard ERP/BoB approach

Reflecting on the Blackboard purchase of some open source service providers, Music for Deckchairs points to this video from Blackboard which starts by asking “Where does the educational experience need to go?” And closes with the very “best of breed”-like quote “And it should build on everyone’s best”.

At the same time it’s showing how the broad range of Blackboard’s tools and systems can invade the entire educational experience, at least in terms of a formal university experience. When I watch this video I am very much reminded of the Kaplan University video showing the brave new world of education and the “TPACK” mashup video pointing out how this wonderfully exciting view of the future is mired in the practices and assumptions of the past.

Diversity, complexity and disruption

Which brings me back to the Dede quote from above. Too much of the past decision-making around education within universities is based on the assumption of sameness and ignorance of the diversity and complexity of learning and the context surrounding learning.

Sadly, I don’t see much evidence of organisations that are grasping the increasing importance of this fact. So I’m guessing to some extent Bates is right, the LMS is probably here to stay.

But I also think that a range of factors are providing those interested in this diversity and complexity with the capabilities to route around the on-going sameness underpinning universities. So the LMS may well go away, eventually.

References

Light, B., Holland, C. P., & Wills, K. (2001). ERP and best of breed: a comparative analysis. Business Process Management Journal, 7(3), 216-224.

bim2 – What's working for coordinator

By

On April 5, 2012

In bim2

More detail on what BIM is can be found here

The following documents initial forays into getting the coordinator view of bim. The aim is to summarise what is or isn’t working in Moodle 2.1

What’s working, what’s not

The main aim in this section is to identify what’s working, what’s not and ideas for minor improvement.

The coordinator (lead teacher role) is the most complex of the roles. It’s major screens and their related activities include

  • Configure BIM – set up basics of a BIM activity
    • Display configuration – Basically working
      Improvements that could be made

      • Help buttons for configuration options.
      • Have the “General Steps” actually link to the appropriate screen.
      • Change “provided below” link to something more human readable.
    • Change configuration – not working
      • Problems on inserting data into the database. Some insertions seem to be putting data in/modifying database, but errors happening
  • Manage questions
    • Display existing – Working
      Improvements

      • help buttons for elements to add.
      • Work on the introductory text.
    • Add new – NOT WORKING
      • All seems to be working, but the text of the question isn’t appearing in the form and may not be placed into the database. The text is being added when changing an existing question.
      • The pause while displaying results of add new/delete etc is not waiting long enough.
    • Delete existing – Working
    • Change existing – Working
  • Allocate markers
    • display – Working
      Improvements

      • Layout of the submit buttons.
      • Rethink intro text
      • Addition of some help buttons
      • Links to show current course groups and change them.
    • Change – NOT WORKING
      Seems to process the change, but an error prevents it from finishing. Perhaps when deleting/removing any group allocations that have changed. When simply adding a group where none is currently allocated, this does work.
  • Manage marking – NOT WORKING
    Doesn’t display, crashes and burn after displaying heading.
  • Find student
    • Display – works
    • Find student – NOT WORKING
      Error in SQL command, will need fixing.
  • Your students – much the same interface as for the markers, so hopefully will work
    • View student details
      • Display – working
      • OPML feed – working
      • change blog – working
        Improvements

        • Once changed, navigation to return to “Your students” is impossible. Need to improve.
    • Mark posts – working
    • Allocate questions – basically working but…
      improvements

      • The progress message after allocating a question occasionally pops up this “[[marker_change_alloc_descriptor]]” indicating a missing language string.
    • Mark post – working

Time to watch a horse race.

bim2 – Marker interface

By

On April 3, 2012

In bim2

In case you’re wondering, BIM is a Moodle activity module that helps teaching staff manage individual student blogs. This post is one of a number of posts documenting the development of a version of BIM that will work with Moodle 2.x. More detail on BIM here

In the following, I’m continuing work on the marker interface from prior work. Work to do includes

  • Register a blog for a student DONE
  • Mark posts – viewing details DONE
  • Allocate questions DONE
  • Mark a particular post – which includes a range of changes DONE

This means that the marker interface for BIM2 is working. Next step will be the coordinator interface.

Register a student blog

That actually seems to be working as stands. Not surprising as it reuses the student functions for registering.

However, the main marker screen isn’t handling the absence of “unregistered” students well. i.e. if there are none, it’s still showing the heading and empty table.

Fixed.

Mark posts

This is the main marking interface and has a number of components. We’ll start with simply displaying the main interface.

Current problems

  • Has the same “showquestions” problem as the student had. DONE
  • Help button not showing in the right place. DONE
  • It’s crashed and burn, not completing. DONE
    Needed to include tablelib.php

Allocation questions

This seems to be displaying first up.

Some things to improve

  • Funny “# posts mirrored” heading appearing – DONE
  • Allocating a post doesn’t seem to be showing actual answers & required answers
    This problem seems to be due to displaying the stats with old data. It is being updated in the database. A fairly major flaw in the original design of the code.

Mark a post

Coding error…seems a database problem in marker/view.php 868. Ahh, the syntax for DB->get_field has changed. Should check that all over. That’s it.

Another problem with implode. That’s working.

Now problems with help buttons again. Done.

Title for showing the student post is being corrupted. That’s fixed.

Now the marker comments title and editor aren’t showing up properly. Ahh, it appears “htmleditor” is deprecated. But the problem is really just the width of things. i.e. use has make the browser window large.

When updating a post having a problem with addslashes still being used. Remove those, no longer needed. Do a quick check where else it is used

To test: Do need to populate the test BIM with more questions and students with answers. Done.

Learning analytics: Starvation and telling us what we already know?

By

On April 1, 2012

In indicators

This article from Inside Higher Ed reports on some initial findings from the Gates Foundation data mining project in the US. The key finding reported in the article is

New students are more likely to drop out of online colleges if they take full courseloads than if they enroll part time

This article gives me pause to reflect on two observations – starvation and tell us what we already know – and talk a bit about patterns.

What we already know

Online students are for all intents and purposes distance education students. Long before online learning many universities across the world had long experience (and lots of research) with distance education. One of the reasons I started in e-learning was that I taught at one of these institutions and print-based distance education just didn’t cut it.

Here’s an excerpt from a paper I wrote in 1996 titled Computing by distance education: Problems and solutions”

Most distance students are either returning to study after a prolonged absence or studying seriously for the first time. The initial period of readjustment to the requirements of study can cause problems for some students.

The circumstances under which distance students study can also generate problems that lead to poor performance. These circumstances include workload, family commitments, illness, economic situation, geographic location and general lack of time.

For these and other reasons, anyone with time in a distance education institution new that a large percentage of distance education students would drop out in the first year because they weren’t all that familiar with the requirements of DE study and over-extended themselves in some way.

I don’t find it at all surprising that students of “online colleges” would suffer the same problem. Especially if they took on more courses. All we’ve done is exchange the platform/medium.

I really don’t think this particular finding is, as suggested in the article,

challenging conventional wisdom about student success.

It does, as suggested further into the article, raise questions about some of the assumptions built into financial aid practices, but doesn’t really challenge wisdom about student success.

Starvation

This sounds like a big project. A $1 million grant, 6 institutions, 640,000 students, and 3 million course-level records. All focused on at-risk students or student success. Given the other rhetoric around learning analytics it isn’t hard to see what management are really interested in learning analytics. A perfect tool for them to see what is going on, be informed and subsequently take action. Leaving aside all of the likely problems arising from management taking action, my biggest worry is that this approach to learning analytics is going to starve the other uses of “learning analytics”.

Taking a higher ed focus, there are (at least) four roles at universities for whom learning analytics might provide benefits, including:

  1. Administration – retention, success, at-risk, efficiencies etc.
  2. Students – seeing their performance in the context of others (somewhat related to success, retention etc)
  3. Teachers – knowing what’s going on in their courses and what happens when they make changes.
  4. Researchers – as a research method that complements other quantitative and qualitative methods for figuring out the why, why, how, who etc with e-learning.

So which roles do you think learning analytics, as implemented at universities, is most likely to serve?

Who holds the purse strings?

Over the last 16 years or so I have observed universities spend tens of millions of dollars on administrative information systems. Enterprise Resource Plannings (ERP) systems like Peoplesoft etc have consumed vasts amounts of resources.

At the same time learning and teaching systems have had comparatively little spent on them. And that’s before you factor in staffing. Compare the number of analysts, programmers and associated support staff an institution employs around its ERP system with the number it employs around its LMS.

For various reasons, administrative systems tend to starve learning and teaching systems (let alone research systems) of funds and resources.

I can see the same thing happening with analytics.

Patterns

These sorts of patterns are interesting. We’ve started documenting some of these patterns over here. The trouble is that all too often the answer to “why” is provided via the schemata of the people involved. Rather than research.

For example, Col published yesterday about another pattern

The later a student first accesses the LMS course site, the lower their final grade will be

Now does this mean that the keener, better organised students are those that access the course site early in term? Or is it because these are the students who don’t have disruptive life situations to deal with?

Powered by WordPress & Theme by Anders Norén

css.php