Markdown and code

Every feature that stores written material renders it through one function. Not for tidiness: the step that cuts a stored body down to an allowlist is a security decision, and a second copy of it is a second thing to remember the day the first one is wrong.

Table of contents

• Using it
• What it does, in this order
  • The dialect
  • The allowlist
• Code highlighting
  • What it never does
  • Styling it

---

Using it

from splent_framework.markdown import render_markdown

@bp.route("/page/<slug>")
def page(slug):
    page = service.get(slug)
    return render_template("page.html", body_html=render_markdown(page.body_md))

The return value is Markup, so a template renders it with {{ body_html }} and no filter. Never mark a stored body safe any other way: | safe on raw body_md runs whatever it happens to contain.

---

What it does, in this order

parse       markdown, GitHub-flavoured, raw HTML allowed
highlight   code blocks, by Pygments, into spans
sanitise    the whole result, against one allowlist

The order is the point. Highlighting happens before sanitising, so the spans Pygments produces go through nh3 with everything else rather than being trusted because the framework produced them.

The dialect

GitHub-flavoured markdown, because that is what written material tends to be: tables, strikethrough and bare URLs already appear in it, and a stricter CommonMark reader shows them as literal text. Raw HTML is enabled for the same reason, since bodies migrated from another wiki contain it, and is cleaned afterwards rather than dropped, which would silently lose content.

Four plugins, the ones written material actually uses: heading anchors, so a long page can be linked to by section; definition lists; footnotes; task lists.

The allowlist

Built from nh3’s own list rather than written out here, so the day an element turns out to be dangerous the library takes it away too. On top of that:

• id and class are allowed on anything. They carry the heading anchors, the plugins’ markup, the highlighter’s token names and whatever a migrated body was styled with. Neither can execute anything.
• <input> is allowed, and only as a checkbox, which is what a task list is. Anything else would be a form pointing somewhere of the author’s choosing.
• URLs may be http, https or mailto.

---

Code highlighting

Server side, at render time, by Pygments. Nothing is vendored and no JavaScript is loaded: the markup already goes through this pipeline, so there is no moment where the code is on screen uncoloured, and a page still works with JavaScript off, when printed, and when pasted somewhere else.

A fenced block keeps the class markdown-it gives it, so the markup still says what the block is:

<pre><code class="language-bash">vagrant init ubuntu/trusty32 <span class="tok-c1">#comentario</span>
</code></pre>

What it never does

Raise. A language nobody has a lexer for is the ordinary case, not an error: a body is written by a person and may say console, or Bash, or a typo. The block renders plainly and still reaches the reader.

Colour terminal output. Blocks tagged text, txt or plaintext are left alone. In one migrated wiki a tenth of the blocks were output pasted from a terminal rather than source, and colouring those invents structure that is not there.

Emit markup for whitespace. Pygments writes a span for every token it produces, whitespace included, and whitespace has no colour in any palette. Whitespace is folded into plain text and adjacent plain runs merged, so an ordinary command line is one string rather than a dozen fragments.

Styling it

Token classes carry a tok- prefix, because Pygments’ own names are one or two letters (k, nb, s1) and a migrated body arrives carrying whatever classes the wiki it came from used.

splent_feature_theme maps those classes to eight custom properties, tuned for a pale code panel. A skin whose panel is dark overrides the eight in one block and inherits every rule:

.cms-public {
    --code-comment: #7c8797;
    --code-keyword: #ff7bbf;
    --code-string:  #8ddb8d;
    --code-number:  #ffb86c;
    --code-name:    #c9a6ff;
    --code-builtin: #79c0ff;
    --code-added:   #8ddb8d;
    --code-removed: #ff8080;
}

What a skin decides is the palette, not which token gets which colour. Check the contrast against the panel it will sit on: 4.5:1 is the threshold, and a palette that reads on white is unreadable on near-black.

Only what carries meaning is coloured. Punctuation and plain names inherit, because a block where everything is a different colour reads as decoration and stops helping anyone find the line they wanted.