A new day
📅 23 May 2025 in: colophon
Introduction#
A new day dawns. The planned technology migration of my blog is complete. Goodbye to "big tech" Wordpress and its (IMHO) horrid authoring environment and bloated performance.
Hello to a markdown based static site generated through Python and numerous open source tools. The blog is reasonably functionally complete, but tinkering will continue. After all, enabling tinkering was a key motivation for the migration.
This post celebrates that move and:
- Introduces the new technology stack;
- Shares the migration method;
- Outlines the new authoring process;
- Reflects on the process; and,
- Lists some initial ideas for next steps.
Linking the stream and the garden
Many of the links below demonstrate a linkage between my "garden and the stream". Another key motivation for the migration.
The published version of this blog post is part of my stream. A serialisation of more complete ideas I'm explicitly sharing with others.
Some of the links below point to some of the formative and still under construction ideas/work that underpin this published post. These formative ideas exist in my memex site. My garden of ideas. The space where I iteratively develop, refine and share ideas in ways that are tentative and interconnected. A space where I document experiments and slowly grapple with ideas. Experiments like this migration.
The new technology stack#
First, a quick overview of the technology stack for the new blog.
Content and authoring#
The new blog starts as a collection of markdown files saved in a git repository (currently hosted on GitHub). There markdown files are created/edited using VSCode leveraging the Foam extension.
Foam is an assemblage of existing VSCode extensions configured to provide Roam Research like functionality. I've been using Foam for Memex and authoring Wordpress blog posts since 2020. The aim is to continue/improve this process for the new blog - see below for more details.
Publishing the blog#
The markdown files are converted into a collection of HTML files using Material for MkDocs. A template (extension) of the Python based MkDocs static site generator (SSG). MkDocs reads the markdown files and generates a collection of HTML files. The HTML files are added to another git repository hosted on my personal web server. From which you've retrieved what you're reading.
Material for MkDocs also runs additional Python scripts I've written to generate additional HTML (inserted into existing HTML files) and new HTML files. The archive and category pages are generated by these Python scripts. A Python script also generates the "blog stats" table on the About page. "Blog stats" is implemented as a macro that can be inserted into any page. For example, here it is on this page.
| Statistic | Value |
|---|---|
| # posts | 1442 |
| # pages | 66 |
| First post | Wednesday, 1 March 2006 14:29:17 |
| Last post | Friday, 23 May 2025 14:17:25 |
| Year | # Posts |
|---|---|
| 2006 | 85 |
| 2007 | 57 |
| 2008 | 120 |
| 2009 | 334 |
| 2010 | 215 |
| 2011 | 142 |
| 2012 | 97 |
| 2013 | 105 |
| 2014 | 89 |
| 2015 | 104 |
| 2016 | 77 |
| 2017 | 34 |
| 2018 | 8 |
| 2019 | 12 |
| 2020 | 15 |
| 2021 | 6 |
| 2022 | 2 |
| 2023 | 3 |
| 2025 | 3 |
This use of Python is an expansion of the computational-components approach I've started using in Memex.
Serving the blog#
As the published blog is just HTML, it is served by the standard web server (Apache) used by my hosting service and consumer by your browser (and numerous crawlers etc).
The migration method#
The task of converting my Wordpress blog to Markdown worked through the following steps:
-
Export the contents of the Wordpress blog to markdown.
- Use Wordpress' in-built export functionality to download an XML file with all of the blog's content.
- Use "Wordpress export to Markdown" to generate a structured collection of markdown files.
-
Modify the markdown files to mirror the old blog structure, fix outdated content, and create blog functionality.
In order to retain the same URLs for individual blog posts, the markdown files produced by step 1 had to be placed into a different folder structure. The content of the markdown files was modified to tidy up and correct outdated content (e.g. Slideshare links I now longer use, outdated links etc) and to add blog functionality to MkDocs (e.g. category pages, archives, RSS feeds, comments/pingbacks).
-
Tweak the default SSG template to better suit the design of my new blog.
Mostly involving tweaking Material for MkDocs HTML and CSS templates.
-
Compare and refine the new blog against the old and expectations.
Analyse the speed and accessibility of the new blog HTML to compare with Wordpress. Confirming it is faster, but not without some issues to address.
-
Replace Wordpress.
Modify the the web server configuration to move the Wordpress blog to its new location and replace it with the new blog.
The authoring process#
Too big for a merged garden and stream
I had hoped to merge my stream (blog) and garden (memex) into a single Foam workspace. Sadly the size of the combined content exceeded the limitations of Foam's wikilink autocompletion. Waiting tens of seconds for link suggestions was not conducive to a smooth writing experience.
Hence the stream and garden exist in separate Foam workspaces.
Given that I wanted to continue my existing practice of writing blog posts in my garden, and Memex and the blog are in separate repositories I needed a script to "publish" a post from Memex to the blog. In essence keeping the process I've using since 2020 to "publish" posts from Memex to Wordpress. In the absence of Wordpress the new pubish script also needed to add some blog functionality to the Memex markdown files.
Reflections#
Overall, the migration has met my expectations. The blog is faster. The content is in a more interoperable format. The authoring process remains embedded in Memex. The linkages between the blog and Memex have improved. The MkDocs ecosystem seems to be a good foundation for extensibility and personal tinkering.
Hopefully that translates to a bit more writing.
Misc. other reflections follow.
Everything old is new again - static sites#
Moving away from Wordpress to static site generation using a scripting language generate nostalgia for my early days publishing on the web with Webfuse. A bespoke "LMS" that formed much of my work in the late 1990s and early 2000s and was the basis of my PhD. Unlike other LMS, Webfuse generated static HTML for speed reasons. A very different approach to my recent, no longer active forays into Javascript frameworks and dynamic sites.
The nostalgia for the good old days may be one reason why I liked the return to static sites. However, another big factor has to be the ever-changing fluidity of the Javascript ecosystem. The Svelte projects I worked on a few years ago have become obsolete due (amongst other things) to changes in the Svelte ecosystem. Python and MkDocs aren't static, but they aren't running pell mell into the future either.
All models (e.g. software) are wrong#
Not that all is perfect, all models are wrong. As mentioned above, Foam has limitations on the number of documents. Material for MkDocs also has a similar limit (though much larger). MkDocs was designed to manage online documentation. Stretching it for blogs and Memex starts to reveal more of the mismatch between capability and need. For example, the blog isn't as responsive/fast as I'd like. A major part of that is how Material for MkDocs implements its search engine (via a large JSON file).
There are other problems (e.g. I haven't yet been able to get the URL for the Wordpress RSS feed to point to the new blog's feed).
For now, I'm happier to live with this collection of wrongness, rather than the wrongness of Wordpress. Largely because...
Some assemblage required and enabled, fighting the reusability paradox#
As the tag line for this blog says, some assemblage is always required. The migration and authoring sections above describe some of the assemblage required so far. The wrongness is a pointer to some further required assemblage. My ability to engage in that assemblage has been better enabled by the migration. Foam and MkDocs are extensible open source projects in technologies I'm familiar with. Having my blog content in Markdown also enables this ongoing assemblage.
Which provides the opportunity to engage more directly with the reusability-paradox. The notion that the more generic a tool/resource (e.g. MkDocs, Wordpress) the less useful it will be for a specific purpose (e.g. my personal knowledge management). However, the ability to leverage interoperability and the extensibility of some tools means I can add some personal utility. At the cost of genericity.
House of cards, Web of complexity#
That personal utility also comes at the cost of complexity. The extensibility of the tools I'm using is not really just inherent in the tool. It's an affordance. A relationship between the tool and my background, knowledge, and context. If I weren't retired (time) with a background in scripting languages (knowledge) and engaging in PKM on the web (purpose), I wouldn't be able to do this.
Next steps#
The tinkering - assemblage - will continue. Some existing ideas follow. Suggestions welcome.
Further reduce reliance on big tech#
The migration has reduced my reliance on big tech, but not eliminated it. The git repositories for both this blog and Memex are hosted on GitHub. I'm editing this post in VSCode.
GitHub can be easily replaced to any other server (including self-hosted) supporting git. The only issue is likely to be the connection with the Foam project. Not a big issue.
Since Foam is a VSCode extension, replacing it could be significantly harder. However, the relationship between VSCode and VSCodium means it could be quite straight forward.
The main challenge being the mundane nature of this type of change, versus something involving more coding.
Comments and pingbacks#
Interaction is a key argument for a blog. At the moment, the new blog has a record of past comments and pingbacks, but has not functionality to accept new comments and pingbacks. Enabling this is important.
Some possible approaches:
-
Mastodon and ActivityPub
An approach Tim uses on some posts is to ask for comments as replies to a matching Mastodon toot. It does seem somewhat limiting to require comments via a specific platform, but it is a good platform (and protocol). There appears to be code and advice on how to do this.
-
Material for Mkdocs has direct support for Gicuss. An open source system built on top of GitHub discussions. Given my interest in moving away from big tech, increasingly my reliance on GitHub is a concern. Not to mention requiring commenters to have a GitHub account.
-
Another 6 alternatives, none of which seems to be a perfect fit.
In terms of ping(track)backs, need to think about both sending and receiving pingbacks.
Statistics and analytics#
Knowing more about the structure of the blog and its content (analytics) and what people are reading from the blog (logging) strike me as useful additions.
The blogStats macro shown above is a start on logging. Jim Nielsen has a nice collection of statistics that might be interesting to explore. In particular, the internal links visualisation and tracking the most referenced internal links.
In the past, I've taken the easy route and relied either on Wordpress analytics or Google Analytics. Moving away from big tech suggests something like these suggestions for small analytics. Or, maybe I should just start analysing log files.
Visual design#
The visual design is functional (at best). Being prettier would be nice, especially if it was combined with interesting functionality. But this is far into the future.