Static Indieweb pt2: Using Webmentions

In last week's post, I talked about syndicating content from a static site to Twitter. But getting content out is only half the challenge.

The real value of social media (apart from the massive ad revenue and dystopian data mining) is in the reactions we get from other people. The likes, reposts and replies - they’re what makes it “social”. To gain control over our own content, we need to capture these interactions as well and pull them back to our sites. In indieweb terms, that’s known as “backfeed”.

Hello Webmentions

A Webmention is an open standard for a reaction to something on the web. It’s currently in W3C recommendation status. When you link to a website, you can send it a Webmention to notify it.

It’s comparable to pingbacks, except that webmentions contain a lot more information than a simple “ping”. They can be used to express likes, reposts, comments or other things.

To make a site support webmentions, it needs to declare an endpoint to accept them. That endpoint can be a script hosted on your own server, or in the case of static sites, a third-party service like webmention.io.

Webmention.io is a free service made by indieweb pioneer Aaron Parecki that does most of the groundwork of receiving, storing and organizing incoming webmentions for you. It’s awesome!

To use it, sign up for a free account there using the IndieAuth process, then include a link tag in the head of your site:

<link rel="pingback" href="https://webmention.io/mxb.dev/xmlrpc">
<link rel="webmention" href="https://webmention.io/mxb.dev/webmention">

Turning social media interactions into webmentions

Cool. So that’s all very nice, but the real party is still over at [currently hip social network], you say. Nobody ever sends me any webmentions.

Well, while your platform of choice is still around, you can use a tool to automatically turn social media interactions into beautiful open webmentions. Bridgy is another free service that can monitor your Twitter, Facebook or Instagram activity and send a webmention for every like, reply or repost you receive.

So if you were to publish a tweet that contains a link back to your site, and somebody writes a comment on it, Bridgy will pick that up and send it as a webmention to your endpoint!

The resulting entry on webmention.io then looks something like this:

    {
"type": "entry",
"author": {
"type": "card",
"name": "Sara Soueidan",
"photo": "https://webmention.io/avatar/pbs.twimg.com/579a474c9b858845a9e64693067e12858642fa71059d542dce6285aed5e10767.jpg",
"url": "https://sarasoueidan.com"
},
"url": "https://twitter.com/SaraSoueidan/status/1022009419926839296",
"published": "2018-07-25T06:43:28+00:00",
"wm-received": "2018-07-25T07:01:17Z",
"wm-id": 537028,
"wm-source": "https://brid-gy.appspot.com/comment/twitter/mxbck/1022001729389383680/1022009419926839296",
"wm-target": "https://mxb.dev/blog/layouts-of-tomorrow/",
"content": {
"content-type": "text/plain",
"value": "This looks great!",
"text": "This looks great!"
},
"in-reply-to": "https://mxb.dev/blog/layouts-of-tomorrow/",
"wm-property": "in-reply-to",
"wm-private": false
}

But wait, there’s more!

The beauty of webmentions is that unlike with regular social media, reactions to your content are not limited to users of one site. You can combine comments from Facebook and Twitter with replies people posted on their own blogs. You can mix retweets and shares with mentions of your content in newsletters or forum threads.

You also have complete control over who and what is allowed in your mentions. Content silos often only allow muting or blocking on your own timeline, everyone else can still see unwanted or abusive @-replies. With webmentions, you’re free to moderate reactions however you see fit. Fuck off, Nazis!

Including webmentions in static sites

Once the webmention endpoint is in place, we still need to pull the aggregated data down to our site and display it in a meaningful way.

The way to do this depends on your setup. Webmention.io offers an API that provides data as a JSON feed, for example. You can query mentions for a specific URL, or get everything associated with a particular domain (allthough the latter is only available to site owners.)

My site uses Eleventy, which has a conventient way to pull in external data at build time. By providing a custom function that queries the API, Eleventy will fetch my webmentions and expose them to the templates when generating the site.

// data/webmentions.js
const API_ORIGIN = 'https://webmention.io/api/mentions.jf2'

module.exports = async function() {
const domain = 'mxb.dev'
const token = process.env.WEBMENTION_IO_TOKEN
const url = `${API_ORIGIN}?domain=${domain}&token=${token}`

try {
const response = await fetch(url)
if (response.ok) {
const feed = await response.json()
return feed
}
} catch (err) {
console.error(err)
return null
}
}

The feed can now be accessed in the {{ webmentions }} variable.

Here’s the complete function if you’re interested. Other static site generators offer similiar methods to fetch external data.

Parsing and Filtering

Now that the raw data is available, we can mold it into any shape we’d like. For my site, the processing steps look like this:

  • Filter the raw data for each post, only include mentions targeting that URL.
  • Only allow “mentions” and “replies” in the comment section. Likes and Reposts go somewhere else.
  • Remove entries that dont have any content to display.
  • Sanitize the output - strip HTML tags, truncate long content, etc.
// filters.js
const sanitizeHTML = require('sanitize-html')

function getWebmentionsForUrl(webmentions, url) {
const allowedTypes = ['mention-of', 'in-reply-to']

const hasRequiredFields = entry => {
const { author, published, content } = entry
return author.name && published && content
}
const sanitize = entry => {
const { content } = entry
if (content['content-type'] === 'text/html') {
content.value = sanitizeHTML(content.value)
}
return entry
}

return webmentions
.filter(entry => entry['wm-target'] === url)
.filter(entry => allowedTypes.includes(entry['wm-property']))
.filter(hasRequiredFields)
.map(sanitize)
}

In Eleventy’s case, I can set that function as a custom filter to use in my post templates.
Each post will then loop over its webmentions and output them underneath.

<!-- webmentions.njk -->
{% set mentions = webmentions | getWebmentionsForUrl(absoluteUrl) %}
<ol id="webmentions">
{% for webmention in mentions %}
<li class="webmentions__item">
{% include 'webmention.njk' %}
</li>
{% endfor %}
</ol>

You can see the result by scrolling down to the end of this post (if there are any replies 😉).

Client-Side Rendering

Because static sites are, well, static - it’s possible that new mentions have happened since the last build. To keep the webmention section up-to-date, there’s an extra step we can take: client side rendering.

Remember I said the webmention.io API can be used to only fetch mentions for a specific URL? That comes in handy now. After the page has loaded, we can fetch the latest mentions for the current URL and re-render the static webmention section with them.

On my site, I used Preact to do just that. It has a very small (~3kB) footprint and lets me use React’s mental model and JSX syntax. It would probably also have been possible to re-use the existing nunjucks templates, but this solution was the easiest and most lightweight for me.

I essentially used the same logic here as I did in the static build, to ensure matching results. The rendering only starts after the API call returned valid data though - if anything goes wrong or the API is unavailable, there will still be the static content as a fallback.

// webmentions/index.js
import { h, render } from 'preact'
import App from './App'

...
const rootElement = document.getElementById('webmentions')
if (rootElement) {
fetchMentions()
.then(data => {
if (data.length) {
render(<App webmentions={data} />, rootElement)
}
})
.catch(err => {
console.error(err)
})
}

And that’s it! There are of course still some missing pieces, most notably the ability to send outgoing webmentions to URLs linked to in your own blog posts. I might have to look into that.

Update: Outgoing Webmentions!

Remy Sharp has recently published a very useful new tool that takes care of handling outgoing webmentions for you. Webmention.app is a platform agnostic service that will check a given URL for links to other sites, discover if they support webmentions, then send a webmention to the target.

You can use that service in a number of ways, including your own command line. If you host your site on Netlify though, it’s also very straightforward to integrate it using deployment webhooks!

Eleventy Starter

I made an Eleventy Starter Template with basic webmention support, using some of the techniques in this post. Check it out!

Jekyll Plugin

My implementation was heavily inspired by Aaron Gustafson’s excellent Jekyll Plugin (link below), which goes even further with customization and caching options. If you’re running a Jekyll site, use that for almost instant webmention support 👍.

Further Resources

Webmentions

  1. Phil Hawksworth
    Brilliant! I've been wanting to do this for some time... but then @mxbck implements Webmentions beautifully on his @eleven_ty site and shares exactly how with more clarity than I could have hoped for. Yoink! mxb.at/blog/using-web…
  2. Calum Ryan | calumryan.com
    Static Indieweb pt2: Using Webmentions by @mxbck » (mxb.at/blog/using-web…) (calumryan.com/note/2352)
  3. Netlify
    Surely a static site can't include dynamic content like Webmentions, right? Wrong. 😎 @mxstbr shows us how he added Webmentions to his #JAMstack site in the second post of his Static Indie Web series.
  4. Šime Vidas
    It may be a good idea to show the avatars here, in a horizontal list, so that visitors can see who’s in there before opening the webmentions.
  5. Michael Scharnagl
    Static Indieweb pt2: Using Webmentions mxb.at/blog/using-web…
  6. Brian Z
    @mxbck you article on using #webmentions with @eleven_ty is awesome! Thank you for that detailed look on how to make that work with the #JAMstack. So cool! mxb.at/blog/using-web…
  7. Scott Mathson
    Wonderful article on incorporating webmentions on static sites! mxb.at/blog/using-web…
  8. Lobsters
    Static Indieweb pt2: Using Webmentions via @flyingfisch_ lobste.rs/s/hvk0ob #web mxb.at/blog/using-web…
  9. lobste.rs bot
    Static Indieweb pt2: Using Webmentions mxb.at/blog/using-web… lobste.rs/s/hvk0ob/stati… #web
  10. Nicolas Hoizey
    One of the features I feared missing if I left @jekyllrb for @eleven_ty was Webmention. It looks like it's pretty easy, after all, thanks Max for showing the way!
  11. Bryan Robinson
    Don’t mind me, just over here researching @eleven_ty because I’ve been writing Gulp tasks for things that @mxbck did here mxb.at/blog/using-web… by writing code for his SSG. If only Jekyll were JS instead of Ruby ;)
  12. There's a Jekyll plugins for #Webmention support! I use it on jacky.wtf github.com/aarongustafson…
  13. Šime Vidas
    Implementing Webmentions on a static site with Webmentions.io and Brid.gy. @mxbck’s post: mxb.at/blog/using-web… My notes: webplatform.news/issues/2019-01…
  14. Chris Aldrich
    Replied to a tweet by Tom Critchlow (Twitter) “@jgmac1106 Thanks - been trying to get my head around webmentions for a while and still haven't figured it out....” Tom, for the basics of what Webmention is you might try this intro article Webmentions: Enabling Better Communication on the Internet. To get started quickly, just to have the notifications, you might try creating an account with Webmentions.io and put the endpoint into the <head> of your site so you can receive them in the erstwhile on a separate service and worry about direct integration at a later date. As I recall Aaron Gustafson has a Webmention.io Jekyll Plugin for display and some of the outline is covered in this recent article by Max Böck. If necessary, you can get help in the #Dev channel of the IndieWeb chat. Syndicated copies to: Twitter icon
  15. Zach Leatherman
    After a super memorable @indiewebcamp last weekend, I’m experimenting with adding webmentions to my personal site (using @eleven_ty). Example: zachleat.com/web/google-fon… Super huge thanks to @mxbck for his blog posts that got me going: mxb.dev/blog/using-web…
  16. Yyyyyess!
  17. Max Böck
    Glad it was helpful. Looking forward to see what you'll come up with!
  18. danfascia
    @paulrobertlloyd totally did this before any of us on his @eleven_ty based personal blog github.com/paulrobertlloy…
  19. Zach Leatherman
    Ha! Of course he did 🏆
  20. danfascia
    His repo should be made the official reference for @eleven_ty I've learned more from browsing that than anywhere else... Docs included!
  21. Gregory Covfefe
    What an amazing article, API and service! 🤯🤩😎 mxb.dev/blog/using-web…
  22. Pelle Wessman
    Nice! And for a solution that works independently of static site generator one can use my webmention.herokuapp.com Love that more and more Webmentions finds its ways into static sites. Also intrigued by @eleven_ty, may perhaps switch from Jekyll to that on voxpelli.com
  23. Jeremy Swinnen
    That’s definitely on my list as well. Building with Eleventy has been a blast so far!
  24. CSS-Tricks
    Webmentions are very cool. They are an actual standard for collecting what other websites have to say about your particular URLs and aggregating them. Like a proof-based collection of commentary. mxb.dev/blog/using-web…
  25. Pierpaolo Tommasi
    Webmentions are very cool. They are an actual standard for collecting what other websites have to say about your particular URLs and aggregating them. Like a proof-based collection of commentary. mxb.dev/blog/using-web…
  26. The moving walkway is cancelled
    word.
  27. What a fantastic write up @mxbck! Adding web mentions to my personal site now 🚀. mxb.dev/blog/using-web…
  28. Max Böck
    Thanks! Glad you found it useful.
  29. nystudio107
    ✅ Craft CMS tip 🎉 You can perform external API queries and decode the JSON response all via Twig: {% set res = craft.app.api.client.get(url).getBody().__toString() | json_decode %} …like querying webmentions 🎩 @zachleat @mxbck #craftcms mxb.dev/blog/using-web…
  30. Mark
    This is downright terrifying 🤪. Makes me wonder if a `helper`-like tag that evaluates to a Twig function would clean this up… {% helper webmentions(url) %} // real PHP here $url = “${url}?token=foo”; // … return Craft::$app->api->client->get(… {% endhelper %}
  31. nystudio107
    It's not stupid if it works, Mark! 😃
  32. Giovanni Bellocchio
    This is terrifying and useful.
  33. Max Böck
    woah, never seen an API call written in a template language before...
  34. shawn swyx wang 🇨🇦
    ah WebMentions & Bridgy for tweet mentions! really cool!
  35. devMode.fm podcast
    Might be fun to have @swyx on to talk @Netlify sometime!
  36. knut
    …and here's mine about how to do it with @gatsbyjs and @Netlify knutmelvaer.no/blog/2019/06/g… (also blatantly stolen, like all great art)
  37. shawn swyx wang 🇨🇦
    🤗 pick a time
  38. Max Böck in 🇬🇪
    Works very well together IMHO! Also worth checking out webmention.app to handle outgoing webmentions. via @rem
  39. Trezy 💫
    I'm reading @mxbck's article on Webmentions and loving it. I dig your writing style. mxb.dev/blog/using-web…