Month: December 2015

Tweaking Moodle book search

On December 15, 2015

A couple of weeks ago I gave a presentation showing off some work from the Moodle open Book project. The middle of the presentation was a live demonstration of the Moodle Book and various features. At one point in the presentation members of the audience (including a number of academics who used the Book module in their Moodle sites) gave an audible gasp. This occurred when I showed off the search block for the Book module. A tool that allows the user to search the contents of all the books in a Moodle course. The gasp indicated just how much teachers and students desire this feature. A feature I’ve been calling out for quite some time.

The GitHub repository has been around for 2 years. So why isn’t this block more widely available in Moodle sites? There’s a security flaw in the code and is somewhat unfinished.

The Moodle SQL injection page outlines at least two broad steps that Moodle code should take to prevent a nefarious person from gaining inappropriate access. These are

Using appropriate parameter cleaning mechanisms on data coming from outside of Moodle (e.g. search terms entered into a form).
Using provided Moodle functions to retrieve data from the database (e.g. search the database for content in a Moodle book)

The Moodle book search block currently meets #1, but fails #2.

The following aims to explore and hopefully remedy this problem.

Current status is that some initial changes have been made to a local version of the block that borrows lessons from the forum search. Need to spend a bit more time on this, but it’s on the way.

The form

The form for the block currently passes the following information.

Added by the block code

courseid; and
page.

The user is able to enter data into: bsquery

The processing

Apart from standard processing the main searching is done in a function named search which

Deals with some apparent differences between flavours of SQL between databases.
Seeming to point to a problem in how it’s engaging with databases. DOES FORUM SEARCH DO THIS
Focuses attention on books the user is allowed to read.
Generates strings containing SQL statement
See below for the format.
- Supports + and –
Uses get_records_sql to retrieve any matches
Not using placeholders, which is a problem.

A search for “copyright” generates a SQL statement similar to the following

SELECT DISTINCT bc.* FROM mdl_book_chapters bc, mdl_book b
        WHERE b.course = 12 AND b.id IN (..long list of book ids)
        AND bc.bookid = b.id AND bc.hidden = 0 AND
        (( bc.title ILIKE '%copyright%' ) OR
         ( bc.content ILIKE '%copyright%' ) )
        ORDER BY bc.bookid, bc.pagenum

Each word added to the search phrase adds options to the “ilikes”

( bc.title ILIKE ‘%copyright%’ AND bc.title ILIKE ‘%creative%’
AND bc.title ILIKE ‘%commons%’ )

Questions include

Can get_records_sql be replaced with get_records
Can the database dependency be removed
Can placeholders be used

Comparison with forum search

The forum search is an accepted part of Moodle. Checking how it works might provide some inspiration to copy.

Has a slight be compartmentalised structure

[code lang=”php”] $forums = forum_get_readable_forums($USER->id, $courseid);[/code]

Has to deal with a lot more complexity than the book search.

Uses $DB->get_in_or_equal

Makes use of a method search_generate_SQL to do as the name suggests. This is something that should be worked into the Book search block.

Initial testing and that is working. Dig a bit and it makes sense. Also seems to tidy up the code a fair bit.

Book github tool: producing some HTML5

By admin

On December 14, 2015

In thesis

Following on from the work late last week and the lovely feedback provided by @rolley it’s time to convert some plans into action.

The aim is to modify the under-development Moodle Book github tool so that when it concatenates the chapters from a Moodle Book resource into a single HTML file, the HTML file is structured HTML 5 semantic elements.

An initial version is apparently working. Though it still needs some checking and likely tweaking.

You can view the output on GitHub (at least you can at the moment) as raw HTML.

Side benefits

As part of my testing of the code, I ran the HTML produced by the tool through this free online HTML 5 outliner. The outliner parses the HTML and its semantic elements and constructs an outline of the content. In much the same way that the Moodle Book module currently does via other means.

Here’s what the Moodle Book interface looks like for my test “book”.

It shows the Moodle Book design feature in terms of hierarchy. The Moodle Book only has two levels: chapter and sub-chapter. Won’t do three levels within a single book.

The image below is a screen capture of the output generated by the HTML 5 outliner on the single HTML file produced by the Moodle Book github tool.

There are more than two levels here.

The Untitled Section

First, there’s the “Untitled Section”. I need to identify where that’s coming from something wrong with:

the HTML 5 my code is producing; or,
the outliner tool.

I tried a few variations to address #1, didn’t work. May need to find another auto-outliner of HTML5.

More levels

HTML 5 outlining doesn’t stop at the use of the section and article elements I’ve added in. It also makes use of the heading elements and other parts of the HTML.

e.g. the first page in this book (titled “The perils of copyright”) includes a H3 title for a section of that page. The section is titled “How do you ask?”. Which is the second part of the outline produces by the auto-outliner.

The next heading “It can be embarrassing if you don’t” is a sub-chapter in the Moodle book. The trouble here is that both are showing up at the same level. Not quite the behaviour the best fits.

However, there is some potential in this auto generation of the outline. Might be useful for other purposes

Need to

Test out what’s working and what’s not here.
Explore a bit more with what other tools produce HTML5 with semantic elements.
Update the Moodle book github tool so it imports this content.

Moodle book to a single file: which format?

By admin

On December 11, 2015

In moodleopenbook, thesis

The Moodle Book github tool allows the import/export (pull/push) of Book content from/to GitHub. The content from the Moodle Book is stored as a single file on GitHub. One of the many unanswered questions about the tool is the format of the exported file. The current format is a bit of dodgy HTML with divs, classes, and ids. Aim here is to figure out if and how the HTML 5 semantic elements might provide a more useful method.

Given that my last serious web development role and interest was 10+ years ago, some of the following is likely to be a bit silly. It’s not helped by the fact that the online explanation of some of these elements differ and the whole semantic element thing appears to be somewhat less than widely supported and used.

HTML 5 semantic elements

Are a collection of elements/tags in HTML 5, including <article> <aside> <header> etc that are intended to help define different parts of a web page and thus make it easier to share/reuse data across applications. Sounding exactly like what is needed here.

All are probably relevant, but the more immediately relevant include:

<article> – Specifies independent, self-contained content.
Good match for an individual book. Includes sections.
<section> – A section in a document.
Good match for a book chapter (or sub-chapter)? Can be nested.
<header> – for either a document or section
Could be used within a section to specify the title of the chapter from the book, but also appear on the page when viewing the single file.
<footer> – for either document or section
Could be used within a section to hold icons for next/previous (getting optional here)
<nav> – defines a set of navigation links
Thinking this could be added to the github file by the tool to add navigation links within the file. Ability to jump to specific chapters etc.
<aside>
<details>
<main>

Structure for the single github file

Which brings me to to the following. Note: the Moodle book calls every page a chapter or a sub-chapter. A sub-chapter(s) is nested within a chapter.

[code lang=”html”]
<html>
<header>
<title>Title of book in Moodle</title>
</header>
<body>

<article data-title="Title of book in Moodle" data-introformat="1" data-customtitles="0" data-numbering="1" data-navstyle="1">
<header>
<h1>Title of book in Moodle</h1>
<div>Introduction to the book from Moodle</div>
</header>

<section data-pagenum="1" data-contentformat="1" data-title="Title of 1st chapter from book">
<header>
<h1>Title of 1st chapter from book</h1>
</header>
CONTENT OF THE CHAPTER from the book

<section data-pagenum="2" data-contentformat="1" data-title="Title of 2nd chapter (and a sub-chapter) from book">
<header>
<h1>Title of 2nd chapter (and a sub-chapter) from book</h1>
</header>
Content of the sub-chapter from the book
</section>
</section>

<section data-pagenum="2" data-contentformat="1" data-title="Title of 3rd chapter from book">
<header>
<h1>Title of 3rd chapter from book</h1>
</header>
CONTENT OF THE 3rd CHAPTER from the book

</section>
</article>
</body>
</html>
[/code]

Entering HTML like that to get pretty coloured in WordPress is harder than it looks.

Testing the structure is quite easy given this online outliner.

Questions

My first question is whether or not the above is “valid” HTML 5? The outliner seemed to like it.

The second question is whether or not the above will work? Not from a HTML 5 perspective, but my code.

Which picks up the following questions

Do the data attributes play nicely with semantic elements?
Should the title be both a data attribute and in the header element?
Should I rely on the parsing code to auto-generate pagenum?
Should I rely on the parsing code to identify chapters and sub-chapters?
Are there any nice existing javascript resources that will auto-generate the navigation between chapters? (Or do I have to write something?)
Are there any nice CSS resources that will style semantic elements nicely? (Or do I have to write something?)
Will the PHP parsing code handle semantic elements (and data attributes)?

Moodle book and GitHub: working together

By admin

On December 3, 2015

In moodle, moodleopenbook, thesis

A major aim of the Moodle Open Book project has been to connect the Moodle Book module with GitHub. The intent was that such a connection would enable the easy sharing of content that is currently largely locked within the LMS, not to mention improving the authoring process for the Moodle Book module. Earlier this week I gave a presentation in which I demonstrated a working connection between the Book module and GitHub. The following post illustrates how this connection works.

This connection is implemented as a Moodle book tool, i.e. an extension to the Moodle Book module that can be installed on any current version of Moodle. The code for the GitHub tool is available from this GitHub repository. The current status of this code is that it works, but is ugly (as the screenshots below will illustrate) and incomplete. The intent is that to get a working first version contributed to the Moodle Plugins database by early 2016 (end of January hopefully).

This work is funded by the USQ Open Textbook Initiative.

What is github?

If you don’t know what GitHub is, then I suggest you take the time to read the following or anyone of the many other resources on the web that explain GitHub.

What is GitHub and what do geeks use it for?
Basic intro into git and github.
Embracing participatory culture in education.
Blog post summarising some findings of a research study that examined why and how University academics are using GitHub, includes links to the published article.

Summary of how it works

The Moodle Book github tool currently works by

Connecting a single “book” (a collection of web pages) created using the Moodle book with a single HTML file in a GitHub repository.
The tool keeps a track of the relationship between the “book” and the HTML file and tells you if they are the same or different.
Provides the ability to
1. push the content of the “book” onto GitHub, and
2. pull the content of the GitHub file back into the “book”.

Once the content of the book is on GitHub, this means it can be shared, modified, and updated by anyone via any means.

Hopefully it might become common for other people using Moodle to use the Book github tool to import books authored by someone else from GitHub into their Moodle course.

I’m certainly looking forward to being able to create and modify Moodle books outside of Moodle and using GitHub to migrate my changes back into Moodle.

Demonstration of how it works

The following contains a range of cropped screen-shots illustrating how the tool currently works. Click on any of the images to see a larger version.

A Moodle Book

First, let’s start with a Moodle book. Here’s what one looks like in my course site.

It’s just a collection of web pages. But it does provide the Table of Contents and the “next page” and “previous page” navigation. It’s also a full part of Moodle hence services like activity completion can be used.

Make a change

Let’s make a change to this Moodle book.

Can you see the rather pointless change (“**** SHOWING OFF GITHUB TOOL ****”) that I made to that page? Let’s assume that this change is important and responds to the experience of learners.

Is the Book github tool installed?

I want to save this change and the book to GitHub. To do this the Book github tool needs to be installed. Is it?

To find this out I look at the Book adminstration menu, which on my institutions Moodle theme looks like this. Can you see evidence of the GitHub tool?

Create the connection between book and github

To create (or check) the connection between the book and github I click on the GitHub link.

Authorise with your GitHub account

The first time you click on the GitHub link within Moodle, you will be redirected to GitHub and will see something like the following

The github tool assumes that you have a user account on GitHub. This step is the github tool asking you for permission to use your GitHub account. Everything the github tool does on github will be done using this account.

If you agree to this you will see the connection page. This page allows you to configure the connection between the book and github, and also to view the status of that connection. Here’s some of what I see.

Under construction: The current interface for the tool is very much under development. What you see is the minimal interface necessary to get all this working.

In this case a connection has already been established.

In it’s current state the github tool expects you to provide two components for the connection

the name of the GitHub repository; and,
the full path to the specific file within the repository to connect with.

Currently the tool then combines these two bits of information with your GitHub username to arrive at the location of the file within GitHub.

I could change the connection to point another file on GitHub, but I’ll stick with this one.

Under construction: At the very least the ability to specify the github username and perhaps the branch (or similar) for the github file needs to be added. Perhaps the option to copy and paste a github URL and have it checked and parsed?

View the file on GitHub

My username on GitHub is djplaner which means that the URL for the file that this book is connected to is http://github.com/djplaner/ICT-and-Pedagogy/Copyright.html.

If you click on that link, you can see the current status of the file on GitHub. When I wrote this, the file on GitHub looked like the following

The book is a sequence of web pages. When pushing a book onto GitHub the Book github tool combines all of those web pages into a single HTML file. That HTML file includes some additional HTML to help the Book github tool pull the content back into the book.

Under Construction: The format/structure of the HTML produced by the Book github tool’s export/import is still undergoing some refinement. Use of HTML5 semantic tags is on the list.

View the file as a web page

If you look closely at the image above of the github tool showing the connection you should see

View the file as a web page.

If you click on that link you will see something like the following (depending on what changes I’ve made since I took this screenshot)

Under Construction: This uses a free service to display a GitHub file as a web page. How this is done also requires a bit more work.

Under Construction: At the moment the HTML is a simple concatenation of the book pages. Very soon this will be modified to include some additional markup and some basic style sheets. The aim is that when you view this HTML page you will see a table of contents and be able to navigate it like a book.

What’s the status of the connection?

Underneath the details of the connection the Book github tool page shows a simple summary of the status of the connection. In the image above, the status is

The book has been revised since the last push.

This is because of the change I made to the book earlier in this post. That change means that the GitHub file is now out of date. It’s not the latest version of Book.

Update the GitHub file

At this stage I can decide whether or not I want to update the file. When and if I update GitHub file will be entirely up to me, the source of the book I’ve edited, the changes I’ve made etc.

But if I do wish to update the GitHub file, I hit the “Push” link and see something like the following.

First, there’s a brief warning just to make sure that you know that pushing will probably make the content of the book open to all to see.

Second, there is a space to enter some details (a comment) about the changes you are about to push onto GitHub. The details about the push are visible in both GitHub and the Book github tool. The details about the push are useful for understanding what changes are being made.

Once I’ve entered my comment, I hit the push button and hopefully see a report of a successful push.

This means that the GitHub file has been changed.

View the file on GitHub

If I view the file on GitHub, that change should be visible in the change below

The first change is that the comment/details I added about the push is visible in the row with the blue background (“Just showing off for the blog post”). That row includes my username, avatar, and how long ago the change was made. The second change is that the HTML for the file now contains the change I made in the Moodle book up above. I’ve highlighted it in green to highlight it.

View the change history

If I return to the Book github tool to view the connection, I can see the following

The Status has been updated to indicate that the Book and the GitHub file are now the same.

You can also see that the “Change History” for the connection now includes the same comment/details (“Just showing off for the blog post”) that showed up on GitHub. Can you see the link “commit details” in the Change History?

The link takes me to GitHub and shows me the following colour coded summary of the changes that were made to the file by this commit.

The green and red colours are used to indicate the additions (79) and deletions (128) made by this commit. This is much higher than you’d expect from the simple change I made. This is because I’ve been playing with the code.

Note: you should be able to click on the link and see the same page. Even though I’ve subsequently made changes to the file on GitHub, I (and you) can always take a look at what the file looked like at this particular point in time.

View the file on my computer

So far we’ve been using the Moodle book and GitHub to view and change the file. There are GitHub clients for a wide array of software and hardware. For example, there is a GitHub application for Mac OS X that I can use to make a local copy of the GitHub repository on my computer.

The following image is an example of a Mac finder window showing my local copy of the repository. It shows that the version of the Copyright.html file (the one we’re using for the Moodle book) was created and modified yesterday.

With the repository files on my computer I can then use all my normal applications to edit and view the file. If I double click on the Copyright.html file in Finder, this is what I see.

Note that the “SHOWING OFF” message is missing. That’s because the copy on my computer is behind that on GitHub.

Update it

To fix this I use the GitHub desktop tool to pull the latest content from GitHub to my computer. Having done that I see the following when I view the Copyright.html file on my computer

All up to date

Make a change

The change I made to the file is silly. I can’t leave it there, I need to remove it. There are currently three methods I could use to make this change:

Directly on GitHub.
GitHub provides a means by which to directly edit the files via the GitHub website.
Using the Moodle Book.
I could go back to Moodle book where I first made the change, delete what I added, and then use the Book github tool to push it back to GitHub (and then pull the change to my computer).
On my computer.
Change the file on my computer, use the Desktop GitHub tool to push that change back to GitHub, and then use the Book github tool to pull the change back into the Book

I’m going to use the last option.

Due to my age and background, I use the vim editor to edit HTML

But you could use any HTML editing tool you wished to make the change.

Push it back

Time to push these changes back to GitHub using the Mac GitHub application.

Note how the application does a very nice job of highlighting the change I’ve made. The text I removed is highlighted by the dark red background.

Just like with the Book github tool, I get the chance to enter a some details/comment about the change I made. In this case, “Remove the showing off”

Check GitHub

On GitHub the file now looks like the following

Can you see that the message within the blue background row has changed to “Remove the showing off”. The message I used on my Mac to commit the change.

What’s the status of the connection

Let’s head back into Moodle and the Moodle book github tool to check the status of the connection between the book and the github file

As you can see “The GitHub file is ahead of the book.” and you can also see that the “Change History” is now headed by the message “Remove the showing off”. Matching the message shown on GitHub in the previous image.

Pull

In order to update the Moodle book with this new content, I need to pull the data from GitHub into the book. Click on Pull and see the following warning

The pull process will replace the existing content of the book with the content from GitHub, hence the need to be sure. I’m happy with that so go ahead and Pull

If I check the status via the Book github tool it will show green – the book and the github file are the same.

View the change

And back to look at the book to see that the change has been made

What’s yet to be done

Coding

More work is required to get a version 1 ready use. There are a few “under construction” indicates listed above and a list of issues on the tool’s GitHub repository.

Anything missing? Then let me know, or better yet, fork the tool repository, make the change, and generate a pull request

Use

Once version 1 is complete, the task will be to get it installed within the institution and then start working with people who want to use it. In particular, explore how it might be used within my institution to transform current practices.