Month: March 2016

“PEBKAC” (CC BY-NC 2.0) by
Rafa Esteve 

Apparently there’s a problem with digital skills in Australian schools. Only 52% of Year 10  students achieved a minimum standard of digital competence, and the teachers tasked to help develop that competence feel they aren’t competent. Closer to home, I’ve previously pointed out that the pre-service teachers I work with are far from digital natives harnessing digital technologies seamlessly to achieve the learning, teaching, and life goals.

Given the perceived importance of digital competence, then something must be done. Otherwise “we run the real risk of creating a generation of digitally illiterate students”.

But what?

Mcleod and Carabott suggest

explicit teaching of digital competence through professional development for teachers. This is also important in teacher education programs…

digital competence tests should also be required for teacher registration

What do I think of those suggestions?

Well they certainly have the benefit of being familiar to those involved in formal education. Expanding as they do existing ideas of testing teachers.

But I’m not sure that’s a glowing recommendation. There’s an assumption that those familiar practices are working and should be replicated in other areas.

Limited views of knowledge – blame the teacher

Beyond that they seem based on a fairly limited view of knowledge. Di Blas et al (2014) talk about the knowledge required to integrate digital technologies into teaching as having

consistently been conceptualized as being a form of knowledge that is resident in the heads of individual teachers (p. 2457)

The type of view that sees the problem with a perceive lack of digital competence to be fixed only by filling the heads of the teacher with the necessary digital competence, and then testing whether or not it’s been filled appropriately. If it hasn’t been filled properly, then it tends to be seen as the teacher’s fault.

The limitations of this view means that I don’t think that any approach based on it will be successful. (After all, a deficit model is not a great place to start).

A distributive view

In this paper (Jones, Heffernan, & Albion, 2015) some colleagues and I draw on a distributive view of learning and knowledge to explore our use as teacher educators of digital technologies in our learning and teaching. Borrowing and extending work from Putnam and Borko (2000) we see a distributive view of learning and knowledge focused on digital technologies as involving at least four conceptual themes:

1. Learning/knowledge is situated in particular physical and social contexts;
2. It is social in nature;
3. It is distributed across the individual, other people, and tools; and, that
4. Digital technologies are protean.

How does this help with the digital competence of school students, teachers, and teacher educators? It suggests we think about what these themes might reveal about the broader context within which folk are developing and using their digital competence.

Schools and digital technologies

Are schools digitally rich environments? Each year I teach about 400 pre-service teachers who head out into schools on Professional Experience for three weeks. During that time they are expected to use digital technologies to enhance and transform their students’ learning. As they prepare for this scary prospect, the most common question from my students is something like

My school has almost no (working) digital technologies? What am I going to do?

Many schools are not digitally rich environments.

If they do, then digital technologies are often seen in ways that mirror reports from Selwyn and Bulfin (2015)

Schools are highly regulated sites of digital technology use (p. 1)…

…valuing technology as

1. something used when and where permitted;
2. something that is standardized and preconfigured;
3. something that conforms to institutional rather than individual needs;
4. something that is a directed activity. (p. 15)

As teacher educators with large percentages of online students, our digital environment is significantly more rich in terms of the availability of digital technologies. However, in our 2015 paper (Jones, Heffernan, & Albion, 2015) we report that the digital technologies we use for our teaching matches the description from Selwyn and Bulfin. Our experience echoes Rushkoff’s (2010) observation that “instead of optimizing our machines for humanity – or even the benefit of some particular group – we are optimizing humans for machinery” (p. 15). More recently I worked on a paper (Jones and Schneider, in review) with a high school teacher that identified the same problem in schools. Digital technologies that were inefficient, got in the way of effective learning and teaching, and failed to mirror the real world digital technology experience.

How do students and especially teachers learn to value and develop their digital competence in such an environment?

In the recent paper (Jones and Schneider, in review) we wondered what might happen if this environment was modified to actually enable and encourage staff and student agency with digital technologies. Allow people to optimise the technology for what they want to do, rather than optimise what they want to do to suit the technology. If this was done:

• Would it lead to digital environments that were more effective in terms of learning and teaching?
• Would it demonstrate the value of digital technologies and computational thinking to teachers in their practice?
• Would this improve their digital competence?

If you could do it, I think it would positively impact all of these factors. But doing so requires to radically rethink a number of assumptions and practices that underpin most of education and the institutional use of digital technologies.

I’m not holding my breath.

Instead, I wonder how long before there’s standardised test for that.

References

Blas, N. Di, Paolini, P., Sawaya, S., & Mishra, P. (2014). Distributed TPACK: going beyond knowledge in the head. In Society for Information Technology & Teacher Education International Conference (pp. 2457–2465). Retrieved from http://www.editlib.org/p/131154

Jones, D., Heffernan, A., & Albion, P. (2015). TPACK as Shared Practice: Toward a Research Agenda,. In L. Liu & D. Gibson (Eds.), Research Highlights in Technology and Teacher Education 2015 (pp. 13–20). Waynesville, NC: AACE. Retrieved from http://www.editlib.org/d/151871

Putnam, R., & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29(1), 4–15. Retrieved from http://www.jstor.org/stable/1176586

Selwyn, N., & Bulfin, S. (2015). Exploring school regulation of students’ technology use – rules that are made to be broken? Educational Review, 1911(December), 1–17. doi:10.1080/00131911.2015.1090401

The last post outlined the process for extracting data from ~300 student submissions. This one outlines what was done to actually do some analysis on that data.

The analysis has revealed

• Around 10% of the submissions have an issue with the URL entered.
• About 16 lesson plans that have been evaluated by more than 1 student.
• At least 100 students have evaluated a lesson plan found online with the Australian Curriculum Lessons site being the most popular.
• An education-based site set up by the industry group Dairy Australia appears to be the most progressive in terms of applying a CC license to resources (apart from the OER commons site)
• It’s enabled allocating related assignments to a single marker, but the process for doing so with the Moodle assignment management system is less than stellar.

Time to do some marking.

What?

Having extracted the data, the following tests can/should be done

1. Is the lesson plan URL readable?
2. Are there any lesson plans being evaluated by more than one students?
   Might be useful to allocate these to the same marker.
3. What is the distribution of where lesson plans are sourced from? Did most use their own lesson plan?
4. Pre-check whether the lesson can be used?

Is the lesson plan readable?

Code is simple enough, but using LWP::Simple::head is having some problems.

Let’s try LWP::UserAgent.  That’s working better.

Seems that if they successfully enter the URL it’s readable.

3 students have used file:/ as a URL – not online.

Distribution of lesson plans

Aim here is to group all the URLs based on the hostname. This will then allow the generation of some statistics about where the lesson plans are being sourced from. Findings include the following counts for domains

• 1 – domain = UNI (a problem)
• 3 – that don’t have a domain
• 96 that appear to be using their own site as the course, indicating their own lesson plan
• 19 domains indicating a lesson planning site
  Accounting for 107 students
• 32 with some sort of ERROR

That’s still not 300. Ahh, we seem to have some problems with entering URLs correctly, common mistakes include

• Just leaving off the http:// entirely
• Mangling bits of http:// (e.g. ttp:// or http//)
• Using a local file i.e. file:////
• Having the URL as “To complete”
• Having the URL empty

Fix those up as much as possible.  Most of these appear to have put something in the cover sheet – if they have one.

Duplicate URLs

There are 16 lesson plans that are used by more than one student.  Most by 2, 4 by 3, 1 by 4 and 1 by 5 students.

Identifying these mean I can allocate them to the same marker. Would be nice if there was an easier way to do this in Moodle.

Pre-check use

At least 107 of the students are using a lesson plan find online. The question is whether or not they can use that lesson plan as per copyright etc.

I could manually check each site, but perhaps to short cut it I should check the spreadsheet for a couple of students, see what argument they’ve mounted for fair use, and then confirm that.

The sites to check are

• http://www.oercommons.org
  Not surprisingly under a CC-NC-SA license, though the NC is a little bit of a surprise.
• http://www.australiancurriculumlessons.com.au
  Requires permission to upload.
• http://readwritethink.org
  Seems to allow reuse.
• http://www.capthat.com.au
  Copyright applies, but they’ve been good in granting permission for students to use for this assignment.
• http://www.dairy.edu.au
  Dairy Australia showing off their online nouse to have applied a CC license.
• http://cdn.3plearning.com
  Most students appear to have had to request permission.

Interestingly there is some large variation between people using the same site.  Should allocate these to the same marker.  Will cut down on time for them.

A couple of weeks ago I wrote this post outlining the design of an Excel spreadsheet EDC3100 students were asked to use for their first assignment. They’ll be using it to evaluate an ICT-based lesson plan. The assignment is due Tuesday and ~140 have submitted so far. It’s time to develop the code that’s going to help me analyse the student submissions.

Aim

The aim is to have a script that will extract each students responses from the spreadsheet they’ve submitted and place those responses into a database. From there the data can be analysed in a number of ways to help improve the efficiency and effectiveness of the marking process; and, explore some different practices (the earlier post has a few random ideas).

The script I’m working on here will need to

1. Be given a directory path containing unpacked student assignment submissions.
2. Parse the list of submitted files and identity all the spreadsheets
3. Exclude those spreadsheets that have already been placed into the database.
   Eventually this will need to be configurable.
4. For all the new spreadsheets
   1. Extract the data from the spreadsheet

At this stage, I don’t need to stick the data in a database.

Steps

1. Code that when given a directory will extract the spreadsheet names
2. Match the filename to a student #id.
3. Parse an individual Excel sheet
   1. Rubric
   2. About
   3. What
   4. How
   5. Evaluation
   6. RAT
4. Mechanism to show the values associated with question number in the sheet.
   Look at a literal data structure.
5. Implement a test sheet
6. See which student files will give me problems.

Extract spreadsheet names

This is where the “interesting” naming scheme by the institutional system will make things interesting. The format appears to be

SURNAME Firstname_idnumber_assignsubmission_file_whateverTheStudentCalledTheFile.extension

Where

• SURNAME Firstname
  Matches the name of the student with the provided case (e.g. “JONES David”)
• idnumber
  Appears to be the id for this particular assignment submission.
• assignsubmission_file_
  Is a constant, there for all files.
• whateverTheStudent…
  Is the name of the file the student used on their computer. It appears likely that some students will have been “creative” with the naming schemes.  Appears at least one student has a file name something.xlsx.docx

Match the filename to a student id

This is probably going to be the biggest problem area. I need to connect the file to an actual unique student id. The problem is that the filename doesn’t contain a unique id that is associated with the student (e.g. the Moodle user id for the student, or the institutional student number).  All it has is the unique id for the submission.

Hence I need to rely on matching the name.  This is going to cause problems if there are students with the same name, or students who have changed their name while the semester is under way. Thankfully it appears we don’t currently have that problem.

Test with 299 submitted files

Assignment due this morning – let’s test with the 299 submitted files.

Ahh, issues with people’s names: apostrophe

Problem files

Apparently 18 errors out of 297 files.  Where did the other 2 go?

“Bad” submissions include

1. 10 with only 1 file submitted;
   All 10 only submitted the checklist. Not the cover sheet or the lesson plan.
2. 26 with only 2 files submitted (3 total required)
   1. 25 – Didn’t submit the lesson plan
   2. 1 – Didn’t submit the checklist
   3. 0 – Didn’t submit the coversheet
3. 18 files that appear to have the bad xlsx version problem from below.

That implies that some of the people who submitted 3 files, didn’t submit an excel file?

Oh, quite proud in a nerdy, strange way about this

[code lang=”bash”]
for name in `ls | cut -d_ -f2 | sort | uniq -c | sort -r | grep ‘ 3 ‘ | sed -e ‘1,$s/^.*[0-9] //’`
do
files=`ls *$name*`
echo $files | grep -q ".xls"
if [ $? -eq 1 ]
then
echo "found $name"
fi
done
[/code]

I’m assuming there will be files that can’t be read. So what are the problems.

Seem they are all down to Microsoft’s “Composite Document File V2 Format”.  These files will open in Excel, but challenge the Perl module I’m using.

Out of the 297 submitted so far, 18 have this problem.  Going to leave those for another day.

I’m spending this week attending the SITE’2016 conference (SITE = Society of Information Technology and Teacher Education). This is my first SITE and the following outlines some of my expectations and intent.

It’s big

Site is one of a raft of conferences run by AACE. I’ve been to two of them previously: EdMedia and Elearn. These are big conferences. 1000+ delegates. Up to be beyond 10 simultaneous sessions. Lots of in-crowds and cliques. Lots of times when there is nothing you’re really interested in, and lots of times when there are multiple things you are very interested in. A lot of really good stuff lost in the mass.

Observations that have been borne out by my first glance at the program.  Too much to take in and do justice to.

At face value, a fairly traditional large conference. With the same breadth of simple to complex, of repetition to real innovation, from boring to mind-blowing. Probably the same ratio as well.

As I’m far from being an extroverted and expert networker, I instead rely on actively trying to make some connections with what I see and I’m doing/going to do.

Join a clique?

While our paper didn’t get an overall paper award, it was successful in winning a TPACK SIG Paper Award.  Given that our previous paper was also TPACK related and won a paper award. This might suggest a “clique” with which I might have some connection, and there are a couple of related papers that sound interesting.

There also appear to be other “cliques” based around computational thinking, design thinking/technologies, and ICT integration by pre-service teachers (more generally than TPACK).  All of these are interests of mine, they connect directly to teaching.

I’m thinking a particular focus for the next few days will one identifying and sharing ideas for using digital technology in school settings with the current EDC3100 crew.

There was one explicit mention of OER in the titles. Pity I can get access to the content of talks yet (thanks to how I was registered and the closed way the organisers treat the proceedings – online now but only for those registered)

Time to get the presentation finished.

EDC3100 tries to get students engaged with writing their own blogs and following the blogs of others via a feed reader. Yes, just a bit old fashioned.

But then one of the problems with doing something a bit different is that it takes a fair bit of extra work to implement. Once you automate this bit of extra work it creates a bit of inertia that prevents change. Not only because I don’t want to lose the effort that went it into automating the process. But also because I know that if I did something different that was more modern, I’d have to invest more time in automating that process (i.e. working around the limitations of the current institutional learning environments.

So documenting the process.

Students create and register their blog on the LMS

About 250 have done this so far. Another 100 to go.  But can’t wait, need to get the OPML files out to students so they can start making connections.

Get the data that identifies students by specialisation

By default the institutional learning environment doesn’t provide this. That’s why I had to spend Friday doing this.

Though I do now have to update the data

• Get the most recent participants.
• Get the most recent registered blogs.
• Double check “can’t find data” students.
  • Find a buggy example
  • Check the local course enrolment – shows up
  • Check users_extras – not there.  That’s the problem. Don’t have the extra data from student records for students who weren’t enrolled a couple of weeks before the start of semester.

Run the script

Once the data is available, I can configure and run a script that will produce the OPML files.

• Fix the configuration settings
• Do something with the NOPLAN students
• Modify the script to handle change in data format and the dirty data

Test the OPML files

All appears to be good.

Write the instructions for students

That’s the next task.

Semester has commenced. First lecture yesterday. Big plans to use Zoom to “broadcast” the lecture to online students and to make a recording that could be shared with those who didn’t want to/couldn’t listen to my dulcet, droning tones at 8am on a Tuesday morning.

Zoom performed as expected. Easy to set up and get working. As expected a small issue with participants not following the advice to mute their microphones. Hence my dulcet, droning tones and the nodding off of the face-to-face audience were occasionally interrupted by the sounds of the domestic life of the online audience. Thankfully Zoom has the capability for the host of the session to mute mics of participants.

PEBKAC

“PEBKAC” (CC BY-NC 2.0) by  Rafa Esteve 

The tendency for people to forget to turn off their mic appears to be an example of PEBKAC. A reasonably well known term amongst computing people, especially those in technical support. As the image to the right explains PEBKAC is an acronym that expands out to

• Problem
• Exists
• Between
• Keyboard
• And
• Chair

i.e. the object that exists between keyboard and chair is the user. It’s user error.

It’s a term that expresses the bewilderment of technical people when the person with the problem has done something that clearly demonstrates a lack of basic understanding. At least in part, the term arises because technical support people see this type of problem all the time.

Why is it so common?

Poor mental models as a source of PEBKAC

This is a question which the content of this week’s lecture has an answer to provide due to a combination of the nature of digital technologies (ICT) and how people learn.

Ben-Ari and colleague (Ben-Ari, 1999; Ben-Ari & Yeshno, 2006) suggest that the problem is that many people have superficial mental models of how the technology works. In the absence of a reasonable correspondence between their mental model of the technology works, and how the technology actually works people are left to “aimless trial and error” when they attempt to use digital technologies. By definition, the reliance on trial and error means that errors will occur and PEBKAC will become evident.

In the Zoom lecture experience the participants joining the Zoom lecture – perhaps many for the first time – don’t understand (they don’t have a mental model) how Zoom works. They don’t understand that their mic is on by default. When it is on, any noise made where they are is shared with all the other participants in the Zoom session. Including, the 40 odd people in the lecture theatre in Toowoomba.

So blame the user?

The opaque nature of digital technologies

Maybe we can blame the technology.

Koehler and Mishra (2009) have this to say

Digital technologies—such as computers, handheld devices, and software applications—by contrast, are protean (usable in many different ways; Papert, 1980); unstable (rapidly changing); and opaque (the inner workings are hidden from users; Turkle, 1995).On an academic level, it is easy to argue that a pencil and a software simulation are both technologies. The latter, however, is qualitatively different in that its functioning is more opaque to teachers and offers fundamentally less stability than more traditional technologies. By their very nature, newer digital technologies, which are protean, unstable, and opaque, present new challenges to teachers who are struggling to use more technology in their teaching. (p. 61)

Digital technologies are opaque. It’s not easy to get a handle on the models that underpin the design and implementation of digital technologies. It’s difficult for a student sitting at home in front of their computer to hear the sound of their neighbour’s lawn mower echoing around the R113 lecture theatre on the Toowoomba campus and connect that to their mic not being muted in Zoom.

There is a picture of a mic on the Zoom interface. But you have to click on it to see the option to mute the mic. That requirement makes it difficult for a person using Zoom for the first time (especially if they are new to video-conferencing) to be aware of how to mute the mic, let alone the need for it.

People who have a mental model that more closely corresponds to how the technology works are better able to prepare for (avoid) or solve problems.

Before starting the lecture I thought this might be a problem (based on prior experience) so I explored the Zoom interface to see if it had an feature that would allow me (as meeting host) to mute the mics of other people.  It did, and that’s what I used to address this problem.

Developing mental models through conceptual models

Ben-Ari and Yashno (2006) found that if they presented people with a conceptual model of how an ICT works, those people were able to move beyond trial and error and solve problems conceptually.

At the moment, my mental model of Zoom is that I’ll have to be manually muting participants next week. As I’m sure, even with all the recommendations, the opaque nature of the Zoom, and the limited mental models of Zoom held by people will once again create the problem.

I wonder now if Zoom has a feature by which you can specify that participants mics are muted automatically as they join.  This is an example where my mental model of Zoom breaks down. Time to play with Zoom.

Hey presto, it does indeed.  The image below shows the “Mute All” button that includes the ability to mute all participants, including new participants. I’ve just learned something new.

Interestingly, however, for me this reinforces the opaque nature of digital technologies (or at least their user interfaces). The button “Mute All” suggests to me that it will mute all existing participants. I didn’t assume it would include participants yet to join. Minor, but useful example.

Environment/context plays a part as well

But it’s just not the technology that is to blame.

As explained above, due to a combination of technical training and experience with video conferencing I have a fairly good mental model of how video conference works. But even with that I still make mistakes.

About 3/4 of the way through the hour lecture yesterday I realised that I hadn’t hit the record button. This is a problem as I’d planned to share that recording with those who couldn’t attend.

So if I had a good mental model of the technology, why did I make the mistake.

I blame the environment. This was my first time trying to use Zoom in a lecture theatre. Due to the theatre set up I was using a Windows computer for the presentation (I’m normally a Mac user). I also had to set up my Mac as secondary machine so I could observe the chat. I also had to worry about the lapel mic and getting that to work. Lastly, it was the first lecture of new semester. A lecture that I’d only finished preparing 30 minutes before the start of the lecture.

It was a novel environment, I was feeling rushed. So even though I knew the importance of hitting record, I didn’t hit record.

Improving my mental model

That mistake and having to re-record the lecture this morning means that I’m unlikely to make this same mistake again. Human beings learn best from making mistakes (especially public mistakes) and reflecting on them.

Broader implications

Some ad hoc ponderings and hypotheses.

The learning activities this week should be designed to require people to make mistakes and then build their conceptual models from there. The pre-packaged errors won’t be as beneficial.

Will the conceptual models that have been provided of the technology and the course be useful enough to help people develop useful mental models?

Are all the problems with staff using the Moodle Assignment activity down to this problem of opaque technology and limited mental models? Could this be fixed by sharing accessible conceptual models with staff? How do overcome the sheer complexity of the model of the Moodle assignment activity?

What role does the teaching context play in these limited mental models?

Would improving the mental models of teaching staff address the perceived quality issues around University digital learning?

References

Ben-Ari, M. (1999). Bricolage Forever! In Eleventh Workshop on the Psychology of Programming Interest Group (pp. 53–57). Leeds, UK. Retrieved from http://www.ppig.org/papers/11th- benari.pdf

Ben-Ari, M., & Yeshno, T. (2006). Conceptual Models of Software Artifacts. Interacting with Computers, 18(6), 1336–1350. doi:10.1016/j.intcom.2006.03.005

Koehler, M., & Mishra, P. (2009). What is Technological Pedagogical Content Knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70. Retrieved from http://www.editlib.org/p/29544/