Remaking My Blog From Scratch

I’ve been blogging in one form or another for 5 years now.

All blogs and other online publishing adventures had one thing in common: I barely knew what I was doing, using components I never learned properly.

I wanted to change that! And with the benefit of hindsight I will now take you on this adventure with me.

What I Had Before

The latest iteration of my blog lived at blog.marco.ninja and was a hugo site using an open-source theme¹.

Both my version of hugo and the theme were very outdated and over the years I added many customizations and hacks to the theme that were not compatible with the fundamental decisions made by the developers.

Besides this pretty standard hugo setup I also run some custom scripts to publish content from my second brain to my blog.

Technically all of this was implemented in one repository called brain.

.
├── .publish
│   ├── archetypes
│   ├── content
│   ├── layouts
│   ├── public
│   ├── resources
│   ├── static
│   ├── themes
│   └── hugo.yaml
├── .scripts
│   ├── publish_brain.py
│   └── ...
├── technology
│   ├── ansible
│   ├── argocd
│   ├── awk
│   ├── bash
│   ├── caddy
│   ├── ...
├── .gitlab-ci.yml
├── ...

In .publish we find a standard hugo site which is published to GitLab Pages using GitLab CI and some custom post-processing, nothing fancy here.

technology/ is exemplary for a area-folder² in my second brain. It contains sub-folders and finally .md files with short notes on various technology related things.

.scripts/publish_brain.py is where the magic happens. Based on frontmatter (published: True) notes get copied from technology/<path>/<to>/<note>.md to .publish/content/notes/technology/<path>/<to>/<note>.md where hugo picks them up and publishes them.

Since I did not want to run this script manually I ended up implementing a two step CI pipeline like this:

  flowchart TB
    subgraph brain-server
        brain_repo["brain.git"]
    end

    subgraph "gitlab brain.git"
        commit["New commit on main"]
        collect_notes["CI Job runs .scripts/publish_brain.py"]
        pages["CI Job runs hugo"]
        deploy["GiLab takes over and deploys to pages"]

        commit ==> collect_notes
        collect_notes == passes folder of publishable notes ==> pages
        pages == passes output of hugo build ==> deploy
    end

    brain_repo == push ==> commit

One could argue the script is an additional step that adds no value since hugo could just read and publish notes from the second brain directly, but I have my reasons, most importantly:

Security: For something internal to be published it must be both in an area-folder setup for publishing through the script and marked for publishing in frontmatter. I feel much safer for my more sensitive area-folders like journal or people to be separated from hugo in this way.

Transformations: The script does not just check frontmater and copies the files, it also performs a number of transformations. It removes tags I do not wish to publish, replaces wikilinks with public links where possible, adds date & lastmod based on git information, extracts the title from markdown to frontmatter and copies assets referenced by the note to be published as well.

The Template

One could argue this is putting the cart before the horse and in a professional project I would agree.

I was unhappy with the look and feel of my site for a while already, so finding PicoCSS lead, in combination with my shiny object syndrome, to me writing a basic html template for all major page types.

At this point I would also like to answer the question of why, as in why did I create my own theme instead of using one of the many available?

The answer is not that I don’t like any of the themes available, they are all awesome! It is rooted in my experience customizing or overwriting things to work the way I want, which was never easy and sometimes required effectively forking from upstream.

Additionally I enjoy learnign new things and building a theme from scratch allowed me to think deeply about HTML and CSS.

For now, this is where we will leave the theme.

Requirements

Very early in the process of checking out the latest and greatest hugo features I wrote down these requirements for my new blog:

• Blog scoped to /blog/ so I can add more things beside it cleanly

• All scripts, images and styles hosted locally

• Better handling of per post assets

  With my old blog I had the convention of placing assets in static/<path>/<to>/<post>/ which was never great.

• Source available repository

• Fully containerized local dev experience

Hugo & Structure

There is nothing fancy about my hugo folder structure, the biggest change for me was using page bundles³ instead of single file pages to co-located assets with the posts they belong to.

The most complicated part was achieving the URL structure I wanted.

/blog/  <- Everything
/blog/posts/ <- Just posts
/blog/posts/<year>/<month>/<day>/<page slug>
/blog/notes/ <- Just notes
/blog/notes/<path/<to>/<note>
/blog/tags/ <- Tag list
/blog/tags/<tag>/

From how I understand hugo it does not like different kinds of content existing beneath the same sub-path, but I got it working the way I wanted in the end.

# ...
permalinks:
  posts: /blog/posts/:year/:month/:day/:contentbasename/
  notes: /blog/notes/:sections[1:]/:contentbasename/
  # ...
  term:
    tags: /blog/tags/:contentbasename/

.
├── _index.md
├── blog
│   └── _index.md
├── notes
│   ├── _index.md
│   ├── misc
│   └── technology
├── posts
│   ├── _index.md
│   ├── bash-utilities-in-powershell
│   ├── copy-paste-is-dangerous
│   ├── ...
│   └── year-review
└── tags
    ├── _index.md
    ├── 1password
    ├── ansible
    ├── ...
    └── yamllint

Some of the _index.md files also add url overwrites in their frontmatter, you can check them out in the repo⁴.

Creating The Hugo Theme

Going from my plain html template to a hugo theme was a challenge for me, I had to familiarize myself with the hugo theme system and template engine at a much deper level then I knew beforehand.

All in all I enjoyed this very much and ended up with a couple of patterns I find quite interesting, which I will write about separately in the future. (Hint, hint: Go subscribe to my RSS)

You can have a look at the theme at the time of writing or at the latest version.

Migrating Posts

Previously all posts were single .md files, with my decision of using page bundles I had to jiggle files around a little bit.

It looked something like this:

find . -type f -name "*.md" -exec sh -c '
  for f; do
    dir="$(dirname "$f")/$(basename "${f%.md}")"
    mkdir -p "$dir"
    cat "$f" > "$dir/index.md"
    rm $f
  done
' sh {} +

Beyond this simple step I manually migrate assets used by these posts into the page bundles as well.

Migrating Notes

Since notes are pushed from the brain by a script I had to only slightly adapt the existing script to output page bundles.

More by accident I also created _index.md leaf bundles in all note folders, which now makes the published notes behave like a directory you can browse on every level.

Not Breaking URLs

I already support old URLs for my previous domains ps1.guru and marco.kamner.eu, which I do through a combination of hugo aliases and a small redirection service I created myself years ago.

Under the hood these become netlify style redirects⁵, which GitLab also supports⁶.

Additionally I added a global redirects functionality that allows me to do this instead of creating aliases for all tags and posts manually:

params:
  # Custom redirects
  redirects:
    # These are needed to make the old blog.marco.ninja URLs work
    - path: /notes/*
      target: /blog/notes/:splat
      permanent: false
    - path: /tags/*
      target: /blog/tags/:splat
      permanent: false
    - path: /posts/*
      target: /blog/posts/:splat
      permanent: false

Which I inject into an otherwise standard redirect template like this:

{{ range $p := .Site.AllPages }}
{{- range .Aliases -}}
{{ . }} {{ $p.RelPermalink }} 301
{{ end }}
{{- end -}}
{{- range .Site.Params.redirects -}}
{{ .path }} {{ .target }} {{ if .permanent }}301{{ else }}302{{ end }}
{{ end }}

Adding Some New Things

After all of this I honestly felt a little burned out by this project, so I decided to work on two fun features I wanted to experiment with.

This was certainly something I felt much more at home with and effectively re-ignited my hunger for finishing this project.

make_social.py

One thing I always disliked about my old blog was the default social preview image, which was just my face.

I wanted something that feels more professional and shows per-post information.

About an hour of vibe-coding later I had a python function to generate an image that looks much better than anything I could have done myself in that time. After tying it into a larger script to automatically generate these social preview images from frontmatter of page bundles I’m quite happy with the resulting script⁷.

A bit later I also added a standalone CLI to generate single images manually⁸

make_tags.py

The idea was simple, I wanted to create the page bundle for each tag used by posts and also warn me if a tag was on disk but not used in any post.

The resulting script⁹ is pretty straight forward.

Again one may ask why this is needed given hugo renders tag pages without this as well. The answer is related to social metadata, which I will write a separate post about.

Tooling

I’m a big believer in good user experience makes you use the tool more often, which is one of the goals of this whole project.

So to make things more usable for me after a simple git checkout I added a couple of things:

• A container image¹⁰ with hugo and all dependencies I need, automatically built in CI¹¹
• A serve.sh¹² script that starts said image with the hugo project mounted
• A hugo.sh¹³ that gives me an interactive shell inside a container to do other things with like hugo new
• A container image¹⁴ for my tools with all dependencies pre-installed, automatically built in CI¹¹
• A tools.sh¹⁵ that gives me an interactive shell inside a container to do things like python -m tools.make_social

Tying It All Together

At this point I have what amounts to a theoretically working blog consisting of a hugo project and my brain repository with a script to select and transform notes to be published into the hugo project.

As I wanted my blog repository to be source available the old way of hosting the hugo project inside the brain was no longer an option.

In the end I opted for two separate repositories which I will need to checkout separately and a script to automate the pushing of new notes to the hugo repository.

The process now looks like this:

  flowchart TB
    subgraph brain-server
        brain_repo["brain.git"]
        hugo_repo["hugo.git"]
        pre_commit["pre commit make_tags & make_social"]

        brain_repo == publish_ninja.py ==> hugo_repo
        hugo_repo ==> pre_commit
        pre_commit ==> hugo_repo
    end

    subgraph "gitlab hugo.git"
        commit["New commit on main"]
        pages["CI Job runs hugo"]
        deploy["GiLab takes over and deploys to pages"]

        commit ==> pages
        pages == passes output of hugo build ==> deploy
    end

    hugo_repo == push ==> commit

Conclusion

I’ve done my best to go into some of the fun, challenging or interesting things I had to do and figure out during this project, but there is still much left I want to write in more detail about:

• Integration with pagefind for search
• How I handle social metadata
• How and why I do the CI/CD the way it is
• The script used to publish notes from my second brain
• Some hugo theme system quirks and best practices
• Using shortcodes

If you want to stay up to date on future posts consider subscribing to my RSS feed.