libertytoolsio/README.md

Liberty Tools (libertytools.io) — Hugo static site

A Hugo site built on the hugo-book theme, with Pagefind client-side search and lychee-based dead-link checking. This document is the single reference for understanding the architecture, building, previewing, searching, and maintaining the site.

If you only need to build and preview, jump to Build & preview (§4). Everything you normally type is a make target.

---

Table of contents

• Architecture
• 1. Clone
• 2. Prerequisites (install once)
• 3. Configuration
• 4. Build & preview (make commands)
• 5. Search (Pagefind)
• 6. Dead links
• 7. Troubleshooting
• 8. Do not commit
• Repository layout

---

Architecture

The site is a fully static output: no server-side rendering at request time, no database, no runtime. Hugo renders the markdown to plain HTML files at build time; a search index is generated from those files; and a trivial static file server (delivering the built directory) is all that runs in front of a browser.

The three moving parts

Component	Role
Hugo	The static site generator. Reads content/*.md + the theme, emits public/.
hugo-book	The theme (vendored under themes/hugo-book/). Layouts, SCSS styling, menu/TOC.
Pagefind	Client-side full-text search. Indexes the built public/ into public/pagefind/.
lychee	Dead-link checker (used only by the link-check/link-fix/external-check targets).

Only Hugo and Node (for Pagefind's npx wrapper) are required to build. lychee is needed only if you run the dead-link targets.

Content → build → serve, data flow

        ┌──────────────────────────────────────────────┐
        │                    content/                  │
        │        *.md pages + images (the source)      │
        └──────────────────────┬───────────────────────┘
                               │  make build   (hugo --gc)
                               ▼
        ┌──────────────────────────────────────────────┐
        │                   public/                    │
        │  rendered static HTML + fingerprinted CSS/JS │
        │  (hugo-book SCSS compiled by the extended    │
        │   build; each asset hash-named + SRI-checked)│
        └──────────────────────┬───────────────────────┘
                               │  make index   (pagefind --site public)
                               ▼
        ┌──────────────────────────────────────────────┐
        │              public/pagefind/                │
        │   Pagefind search index (WASM + JSON)        │
        │   — ships inside public/, so any static      │
        │     server can serve a working search        │
        └──────────────────────┬───────────────────────┘
                               │  make serve / make serve-lan
                               ▼
                     browser (any machine on the network)

Two consequences worth internalizing:

1. public/ is the single source of truth for what is served. Both the search index and every preview server read from it. If the site "isn't showing your change," the build hasn't run — re-run make index.
2. The Pagefind bundle lives inside public/. That's why make serve-lan (a plain Python static server) serves working search with zero extra setup — there is no special search server required.

The theme SCSS pipeline

html-head.html compiles the theme's stylesheet on every build:

book.scss  →  resources.ExecuteAsTemplate  →  css.Sass  →  resources.Minify  →  resources.Fingerprint

css.Sass is why the Hugo extended edition is mandatory (the standard build has no SCSS compiler). resources.Fingerprint hashes the output into the filename (book.min.<sha>.css) and embeds an SRI integrity attribute on the <link>; a single byte change in the CSS produces a new filename, which is why old cached stylesheets never collide with new ones.

Where your customizations live

Two files hold all site-level customization — you rarely need to touch anything else:

• themes/hugo-book/layouts/_partials/docs/search.html — the entire search UI: the Pagefind new PagefindUI({...}) config, the #search-scoped <style> block that fits the UI to the sidebar, and the keyboard-navigation script. (See §5.)
• themes/hugo-book/assets/_variables.scss — site-wide dimensions and font. This file is imported after the theme's _defaults.scss, so the values here override the theme defaults with plain (non-!default) assignments.

  Variable	Current value	Effect
  :root { --font-size }	17px	Site-wide base font (theme default is 16px). All rem sizes scale off this.
  $container-max-width	100rem	Max width of the centered content area (~1700px at 17px root).
  $menu-width	18.75rem	Width of the left sidebar — the category list and the search box and the search-results pane are all this one column.
  $mobile-breakpoint	(recomputed)	Must be recomputed whenever you change $menu-width/$body-min-width/$toc-width — it is derived as $menu-width + $body-min-width * 1.2 + $toc-width.

Search index scoping (data-pagefind-body)

By default Pagefind indexes the entire <body> of each page — including the sidebar, menu, and footer — which pollutes results. Instead, the article element in each layout is tagged with data-pagefind-body:

<article class="markdown book-article" data-pagefind-body>

Pagefind then indexes only that element and skips any page that lacks the attribute. The attribute is present in layouts/baseof.html (default single + home), layouts/term.html (list pages), and layouts/posts/{single,list}.html. 404.html is intentionally left untagged (it has its own standalone <html>). Do not remove these attributes or the menu text will start appearing in search results. After any change, confirm the indexed page count in the make index log — a sharp drop means a page lost its tag (usually the home page).

The #search CSS cascade trick

pagefind-ui.css is loaded in <body> (by the search partial) — after the theme's head stylesheet. A CSS rule in the head at equal specificity therefore loses the cascade. All search overrides live in a #search-scoped <style> block in search.html; the #search id + class gives them higher specificity than Pagefind's class-only rules regardless of load order. Keep this pattern when you tune the search UI.

Pagefind 1.5.2 quirks we work around (all in search.html)

These are non-obvious behaviors of the Default UI in Pagefind 1.5.2 (the version npx -y pagefind resolves to). They cost real debugging time; they are documented here so the next person doesn't rediscover them:

• The results drawer is an in-flow child of the form. Once results render, .pagefind-ui__form stretches to the full height of the results list (measured ~1800px for a dozen rows). The fixed-height sidebar (.book-menu-content, position:fixed; overflow-y:auto) simply scrolls it — that's intended. Consequence: any %-based height on the absolutely-positioned clear button would resolve against that stretched form, so the clear button uses fixed pixel dimensions keyed to the input.
• There is no keyboard navigation, and stock Enter is a silent no-op. The built pagefind-ui.js handles Enter with only preventDefault() — typing a query and pressing Enter navigates nowhere. A custom capture-phase keydown listener on #search implements ↑/↓ highlighting and Enter-to-open (see §5).
• An empty thumbnail occupies the left of every row. Even with zero images in the index, Pagefind renders an ~81px .pagefind-ui__result-thumb placeholder, which pushes result text ~90px right (a "right-justified" look). We hide it (display:none) because this index has no images.
• Matched terms render in harsh UA-yellow. Pagefind's CSS reset does mark { all: revert }, so <mark> falls back to the browser default (yellow). We restyle #search .pagefind-ui__result mark to a subdued slate.

Dead-link automation

Two independent checks, deliberately split by scope and cost:

• Internal (make link-check / make link-fix): offline. lychee checks the source markdown against the built site (--root-dir public, --scheme file), so no network is involved. Confirmed-dead internal page links are auto-commented out by scripts/fix_dead_links.py; external/bare-domain/empty links are routed to a manual review file, never auto-edited.
• External (make external-check): network. scripts/check_external_links.py pings every http(s) link in content/. Report-only, always exits 0. A daily cron job runs it (see §6).

The removal mechanism for internal links is commenting, not deleting ([Old](/gone/) → <!-- [Old](/gone/) -->). An HTML comment survives the Hugo build verbatim, is invisible on the rendered page, is excluded from the search index, and is no longer detected by lychee — which makes make link-fix idempotent (safe to re-run; it never loops).

---

1. Clone

git clone https://dev.libertytools.io/adminuser/libertytoolsio.git
cd libertytoolsio

2. Prerequisites (install once)

All tools install into ~/.local/bin (no sudo). Make sure ~/.local/bin is on your PATH (it usually is; if not: export PATH="$HOME/.local/bin:$PATH").

• Hugo extended (v0.167.0 or newer). The theme compiles SCSS with css.Sass, so the extended build is required. From the repo root:

  cd "$(mktemp -d)"
  curl -sL -o h.tgz https://github.com/gohugoio/hugo/releases/download/v0.167.0/hugo_extended_0.167.0_linux-amd64.tar.gz
  echo "0163f5c3deddac1f494a1629ddc40c65d18de9d5794facd98f7f96ac2c7d8957  h.tgz" | sha256sum -c -
  tar xzf h.tgz
  install -m 0755 hugo "$HOME/.local/bin/hugo"
  hugo version        # must print ...+extended...
  

• Node (v18+). Pagefind runs through npx -y pagefind, which auto-installs the Pagefind CLI the first time (resolves to 1.5.x; the UI quirks in Architecture are for 1.5.2). Verify: node --version && npx --version.
• lychee (v0.24.2) — link checker, needed only for the dead-link targets:

  cd "$(mktemp -d)"
  curl -sL -o ly.tgz https://github.com/lycheeverse/lychee/releases/download/lychee-v0.24.2/lychee-x86_64-unknown-linux-musl.tar.gz
  tar xzf ly.tgz
  install -m 0755 lychee-x86_64-unknown-linux-musl/lychee "$HOME/.local/bin/lychee"
  lychee --version    # lychee 0.24.2
  

3. Configuration

• config.toml — site-level settings: theme = "hugo-book", BookSearch = true, BookTheme = 'dark', plausibleDomain, and markup.goldmark.renderer.unsafe = true (required so raw HTML — including the commented-out dead links and inline <style> blocks — passes through the markdown renderer unharmed).
• themes/hugo-book/assets/_variables.scss — site dimensions + font (see the table in Architecture). This is the file to edit to change the base font size, page width, or sidebar width.
• themes/hugo-book/layouts/_partials/docs/search.html — the search UI (see §5).

4. Build & preview (make commands)

Run from the project root. public/, linkcheck/, and resources/_gen/ are generated and gitignored.

Target	What it does
make build	Build Hugo content into public/ (hugo --gc).
make index	build + build the Pagefind search index into public/pagefind/. This is the normal "I edited content, re-make it" command.
make serve / make run	index + serve the built site with search at http://localhost:1414/ (local only). Uses Pagefind's own server, which binds 127.0.0.1.
make serve-lan	Serve the built site on all interfaces so another machine on the network can open it: http://<LAN-IP>:8090/. Python stdlib only; build first with make index.
make link-check	Offline scan of internal links in content/ → linkcheck/internal.json. Fast, no network.
make link-fix	Comment out confirmed dead internal links, then rebuild content + index.
make external-check	Network scan of all external links in content/ → console report + linkcheck/external_dead.txt. Report-only; always exits 0.
make clean	Remove public/ and linkcheck/.
make all	build + index (the default target).

Notes on the targets:

• build runs hugo --gc — --gc garbage-collects orphaned resources from the asset cache.
• serve / serve-lan do not rebuild. They serve whatever is already in public/. Always run make index first (or use make serve, which chains index), or you'll be serving a stale build.
• Why serve uses pagefind --serve, not hugo server: hugo server renders in memory and may not write public/, leaving Pagefind with nothing to index. Serving the built public/ through Pagefind's own static server is what makes the preview match production.
• serve-lan is a plain static file server (scripts/serve_lan.py, stdlib only) rooted at public/ on 0.0.0.0:8090. It exists specifically because make serve binds to 127.0.0.1 and is unreachable from other machines.

Viewing changes locally

make serve

Open http://localhost:1414/. There is no live reload — re-run make serve after editing content.

Viewing from another machine on the same network

make index        # build first so public/pagefind/ exists
make serve-lan

serve-lan prints this machine's LAN IP at startup (also: hostname -I | head -1). On the other machine, open http://<that-printed-IP>:8090/. The IP is dynamic (DHCP); re-check it if it changes. Search works identically under this plain static server as under make serve (the Pagefind bundle is part of public/, so no special server is needed for cross-machine access).

5. Search (Pagefind)

The search box lives in the left sidebar on every page (injected via themes/hugo-book/layouts/_partials/docs/menu.html → docs/search.html).

Using it

• Press / anywhere on a page to jump to the search box (focusOnSlash).
• Type a query; results appear below the box, scoped to page content (see data-pagefind-body in Architecture).
• Navigate with the keyboard:
  • ↓ / ↑ move a highlight through the results, in visual order. Top-level hits and their sub-topics (Pagefind "sub-results") each get their own highlight — e.g. a query under "Hacking" can step through Hacking, then its Anonymity / OSINT / Security Courses sub-entries, one at a time. Wraps around at the ends.
  • Enter opens the highlighted result. If nothing is highlighted, it opens the top result. (Stock Pagefind does neither — this is the custom listener described in Architecture.)
• The matched terms inside each result are shown in a subdued slate (not the browser's default yellow).

Customizing the look

Everything is in the #search-scoped <style> block and the new PagefindUI({...}) call in themes/hugo-book/layouts/_partials/docs/search.html. Always keep the #search prefix on overrides — see the cascade trick. Current values (verified against the built page):

Element	Current value
Input font / height / padding	font-size: 0.85em; height: 2.4em; min-height: 2.4em; padding: 0 2.6em 0 2.4em;
Magnifier icon (.pagefind-ui__form:before)	top: 7px; left: 12px; width/height: 14px; — fades to opacity: 0 while the input has text
Clear button (.pagefind-ui__search-clear)	top: 3px; right: 3px; height: 23px; font-size: 11px; padding: 0 6px; (fixed px, not % — see form-stretch quirk)
Results layout (.pagefind-ui__drawer)	flex-direction: column; gap: 0.4em; (stack, don't wrap)
Shrink guards (.results-area, .result-excerpt)	min-width: 0;
Empty thumbnail (.result-thumb)	display: none; (no images in this index)
Result title / link	font-size: 1em; overflow-wrap: anywhere;
Result excerpt	font-size: 0.88em; overflow-wrap: anywhere; word-break: break-word;
"N results for …" message	font-size: 0.8em;
Keyboard-selected row (.pf-kbd-selected)	background: rgba(100,116,139,0.5); outline: 2px solid #cbd5e1; outline-offset: -2px; color: #fff;
Matched term (mark)	background: rgba(100,116,139,0.45); color: #fff; font-weight: 600; border-radius: 2px;
Excerpt length (words)	excerptLength: 20 in the new PagefindUI({...}) call

The legacy Fuse.js search was removed (it double-loaded with Pagefind). A backup of the old head partial is at themes/hugo-book/layouts/_partials/docs/html-head.html.bkup.

6. Dead links

Internal links (fast, offline)

make link-check     # writes linkcheck/internal.json
make link-fix       # comments out confirmed dead internal page links, then rebuilds

How it works: lychee checks the source markdown against the built site (--root-dir public). A link whose target page does not exist is a dead internal link and scripts/fix_dead_links.py wraps it in an HTML comment: [Old](/gone/) → <!-- [Old](/gone/) -->. A commented link disappears from both the rendered page and the search index, and is not re-detected, so make link-fix is idempotent and safe to run repeatedly.

Safety rule: only internal page links are auto-commented. External https:// failures, bare-domain links, and empty hrefs are written to linkcheck/manual_review.txt and left for you to fix by hand (a bare domain may be a typo to fix, not a link to delete).

External links (network, daily)

make external-check    # pings every http/https link in content; report-only

It lists each dead external link with its file:line and writes linkcheck/external_dead.txt. It does not edit anything. A scheduled job runs this once a day (below).

The daily external-link job

A cron job named libertytools-external-link-check runs scripts/check_external_links.py every day at 6:00am. It is report-only and always exits 0. Its output is saved and viewable with:

hermes cronjob list            # confirm it exists and is enabled

• To run it manually right now: hermes cronjob run <job_id> (from hermes cronjob list).
• To pause it: hermes cronjob pause <job_id>; to resume: hermes cronjob resume <job_id>.
• To change the time, edit the job's schedule in config.yaml (or ask the agent).

7. Troubleshooting

• hugo: command not found → you're using the non-extended build or it isn't on PATH. Re-run the Hugo install above and confirm hugo version says +extended.
• Search shows no results → the index wasn't built. Run make index, confirm the log prints Indexed N pages and public/pagefind/pagefind-ui.js exists, then reload. If the page count looks low, a layout lost its data-pagefind-body attribute.
• pagefind / lychee not found → re-run the relevant install in §2; both live in ~/.local/bin.
• make serve says port 1414 in use → a previous server is still running. Stop it (Ctrl+C in its terminal) or find it: ss -ltnp | grep 1414.
• Other machine can't reach http://<LAN-IP>:8090/ → confirm you're on the same network and using the IP serve-lan printed (hostname -I | head -1 on the host). The host's firewall may need to allow inbound 8090.
• Search is too big/small for the sidebar → tune the #search block in themes/hugo-book/layouts/_partials/docs/search.html (see §5).
• Search results look right-justified / have a big left gap → the empty thumbnail isn't hidden; ensure #search .pagefind-ui__result-thumb { display: none; } is present (see the quirk in Architecture).
• Up/down arrows don't highlight results → the keyboard-nav script in search.html is missing or its rowFor() no longer matches the result DOM. Sub-results are nested inside the top-level <li> (see the quirk notes).

8. Do not commit

All changes from a maintenance pass are left in the working tree for review. Commit only after you have validated the result (make build exits 0 and you've checked the rendered output). The following are generated and gitignored — never commit them:

• public/ — built site output
• linkcheck/ — link-check reports
• resources/_gen/ — Hugo asset-generation cache
• .hermes/ — agent working files

Repository layout

content/            # site pages (markdown + images)
themes/hugo-book/   # the theme (vendored). Do not edit themes/hugo-book.backup/.
  assets/_variables.scss                       # SITE overrides: font, page width, sidebar width, breakpoint
  layouts/baseof.html                          # root layout; <article data-pagefind-body>
  layouts/term.html                            # list/term pages (data-pagefind-body)
  layouts/posts/{single,list}.html             # blog post pages (data-pagefind-body)
  layouts/_partials/docs/search.html           # Pagefind UI + sidebar-fit CSS + keyboard nav
  layouts/_partials/docs/menu.html             # injects search.html into the sidebar
  layouts/_partials/docs/html-head.html        # (legacy fuse search removed; .bkup kept)
  layouts/_partials/docs/html-head.html.bkup   # backup of the pre-Pagefind head
scripts/
  fix_dead_links.py        # internal dead-link comment-out (idempotent)
  check_external_links.py  # external dead-link report (report-only, daily cron)
  serve_lan.py             # static LAN preview server (serves public/ on 0.0.0.0:8090)
Makefile              # build / index / serve / run / serve-lan / link-check / link-fix / external-check / clean
config.toml           # theme + BookSearch + BookTheme + goldmark.unsafe