Using Webmentions in Eleventy

In last week's post, I talked about syndicating content from a static site to Twitter. But getting content out is only half the challenge.

The real value of social media (apart from the massive ad revenue and dystopian data mining) is in the reactions we get from other people. The likes, reposts and replies - they’re what makes it “social”. To gain control over our own content, we need to capture these interactions as well and pull them back to our sites. In indieweb terms, that’s known as “backfeed”.

Hello Webmentions

A Webmention is an open standard for a reaction to something on the web. It's currently in W3C recommendation status. When you link to a website, you can send it a Webmention to notify it.

It's comparable to pingbacks, except that webmentions contain a lot more information than a simple "ping". They can be used to express likes, reposts, comments or other things.

To make a site support webmentions, it needs to declare an endpoint to accept them. That endpoint can be a script hosted on your own server, or in the case of static sites, a third-party service like webmention.io.

Webmention.io is a free service made by indieweb pioneer Aaron Parecki that does most of the groundwork of receiving, storing and organizing incoming webmentions for you. It’s awesome!

To use it, sign up for a free account there using the IndieAuth process, then include a link tag in the head of your site:

<link rel="pingback" href="https://webmention.io/mxb.dev/xmlrpc">
<link rel="webmention" href="https://webmention.io/mxb.dev/webmention">

Turning social media interactions into webmentions

Cool. So that’s all very nice, but the real party is still over at [currently hip social network], you say. Nobody ever sends me any webmentions.

Well, while your platform of choice is still around, you can use a tool to automatically turn social media interactions into beautiful open webmentions. Bridgy is another free service that can monitor your Twitter, Facebook or Instagram activity and send a webmention for every like, reply or repost you receive.

So if you were to publish a tweet that contains a link back to your site, and somebody writes a comment on it, Bridgy will pick that up and send it as a webmention to your endpoint!

The resulting entry on webmention.io then looks something like this:

    {
"type": "entry",
"author": {
"type": "card",
"name": "Sara Soueidan",
"photo": "https://webmention.io/avatar/pbs.twimg.com/579a474c9b858845a9e64693067e12858642fa71059d542dce6285aed5e10767.jpg",
"url": "https://sarasoueidan.com"
},
"url": "https://twitter.com/SaraSoueidan/status/1022009419926839296",
"published": "2018-07-25T06:43:28+00:00",
"wm-received": "2018-07-25T07:01:17Z",
"wm-id": 537028,
"wm-source": "https://brid-gy.appspot.com/comment/twitter/mxbck/1022001729389383680/1022009419926839296",
"wm-target": "https://mxb.dev/blog/layouts-of-tomorrow/",
"content": {
"content-type": "text/plain",
"value": "This looks great!",
"text": "This looks great!"
},
"in-reply-to": "https://mxb.dev/blog/layouts-of-tomorrow/",
"wm-property": "in-reply-to",
"wm-private": false
}

But wait, there’s more!

The beauty of webmentions is that unlike with regular social media, reactions to your content are not limited to users of one site. You can combine comments from Facebook and Twitter with replies people posted on their own blogs. You can mix retweets and shares with mentions of your content in newsletters or forum threads.

You also have complete control over who and what is allowed in your mentions. Content silos often only allow muting or blocking on your own timeline, everyone else can still see unwanted or abusive @-replies. With webmentions, you’re free to moderate reactions however you see fit. Fuck off, Nazis!

Including webmentions in static sites

Once the webmention endpoint is in place, we still need to pull the aggregated data down to our site and display it in a meaningful way.

The way to do this depends on your setup. Webmention.io offers an API that provides data as a JSON feed, for example. You can query mentions for a specific URL, or get everything associated with a particular domain (allthough the latter is only available to site owners.)

My site uses Eleventy, which has a conventient way to pull in external data at build time. By providing a custom function that queries the API, Eleventy will fetch my webmentions and expose them to the templates when generating the site.

// data/webmentions.js
const API_ORIGIN = 'https://webmention.io/api/mentions.jf2'

module.exports = async function() {
const domain = 'mxb.dev'
const token = process.env.WEBMENTION_IO_TOKEN
const url = `${API_ORIGIN}?domain=${domain}&token=${token}`

try {
const response = await fetch(url)
if (response.ok) {
const feed = await response.json()
return feed
}
} catch (err) {
console.error(err)
return null
}
}

The feed can now be accessed in the {{ webmentions }} variable.

Here’s the complete function if you’re interested. Other static site generators offer similiar methods to fetch external data.

Parsing and Filtering

Now that the raw data is available, we can mold it into any shape we’d like. For my site, the processing steps look like this:

  • Filter the raw data for each post, only include mentions targeting that URL.
  • Only allow “mentions” and “replies” in the comment section. Likes and Reposts go somewhere else.
  • Remove entries that dont have any content to display.
  • Sanitize the output - strip HTML tags, truncate long content, etc.
// filters.js
const sanitizeHTML = require('sanitize-html')

function getWebmentionsForUrl(webmentions, url) {
const allowedTypes = ['mention-of', 'in-reply-to']

const hasRequiredFields = entry => {
const { author, published, content } = entry
return author.name && published && content
}
const sanitize = entry => {
const { content } = entry
if (content['content-type'] === 'text/html') {
content.value = sanitizeHTML(content.value)
}
return entry
}

return webmentions
.filter(entry => entry['wm-target'] === url)
.filter(entry => allowedTypes.includes(entry['wm-property']))
.filter(hasRequiredFields)
.map(sanitize)
}

In Eleventy’s case, I can set that function as a custom filter to use in my post templates.
Each post will then loop over its webmentions and output them underneath.

<!-- webmentions.njk -->
{% set mentions = webmentions | getWebmentionsForUrl(absoluteUrl) %}
<ol id="webmentions">
{% for webmention in mentions %}
<li class="webmentions__item">
{% include 'webmention.njk' %}
</li>
{% endfor %}
</ol>

You can see the result by scrolling down to the end of this post (if there are any replies 😉).

Client-Side Rendering

Because static sites are, well, static - it’s possible that new mentions have happened since the last build. To keep the webmention section up-to-date, there’s an extra step we can take: client side rendering.

Remember I said the webmention.io API can be used to only fetch mentions for a specific URL? That comes in handy now. After the page has loaded, we can fetch the latest mentions for the current URL and re-render the static webmention section with them.

On my site, I used Preact to do just that. It has a very small (~3kB) footprint and lets me use React’s mental model and JSX syntax. It would probably also have been possible to re-use the existing nunjucks templates, but this solution was the easiest and most lightweight for me.

I essentially used the same logic here as I did in the static build, to ensure matching results. The rendering only starts after the API call returned valid data though - if anything goes wrong or the API is unavailable, there will still be the static content as a fallback.

// webmentions/index.js
import { h, render } from 'preact'
import App from './App'

...
const rootElement = document.getElementById('webmentions')
if (rootElement) {
fetchMentions()
.then(data => {
if (data.length) {
render(<App webmentions={data} />, rootElement)
}
})
.catch(err => {
console.error(err)
})
}

I also made an Eleventy Starter Template with basic webmention support, using some of the techniques in this post. Check it out!

There are of course still some missing pieces, most notably the ability to send outgoing webmentions to URLs linked to in your own blog posts. I might have to look into that.

Update: Outgoing Webmentions!

Remy Sharp has recently published a very useful new tool that takes care of handling outgoing webmentions for you. Webmention.app is a platform agnostic service that will check a given URL for links to other sites, discover if they support webmentions, then send a webmention to the target.

You can use that service in a number of ways, including your own command line. If you host your site on Netlify though, it’s also very straightforward to integrate it using deployment webhooks!

Jekyll Plugin

My implementation was heavily inspired by Aaron Gustafson’s excellent Jekyll Plugin (link below), which goes even further with customization and caching options. If you’re running a Jekyll site, use that for almost instant webmention support 👍.

Further Resources

Webmentions

What’s this?
  1. Zach Leatherman
    After a super memorable @indiewebcamp last weekend, I’m experimenting with adding webmentions to my personal site (using @eleven_ty). Example: zachleat.com/web/google-fon… Super huge thanks to @mxbck for his blog posts that got me going: mxb.dev/blog/using-web…
  2. Yyyyyess!
  3. Max Böck
    Glad it was helpful. Looking forward to see what you'll come up with!
  4. danfascia
    @paulrobertlloyd totally did this before any of us on his @eleven_ty based personal blog github.com/paulrobertlloy…
  5. Zach Leatherman
    Ha! Of course he did 🏆
Show All Webmentions (53)
  1. danfascia
    His repo should be made the official reference for @eleven_ty I've learned more from browsing that than anywhere else... Docs included!
  2. Gregory Covfefe
    What an amazing article, API and service! 🤯🤩😎 mxb.dev/blog/using-web…
  3. Pelle Wessman
    Nice! And for a solution that works independently of static site generator one can use my webmention.herokuapp.com Love that more and more Webmentions finds its ways into static sites. Also intrigued by @eleven_ty, may perhaps switch from Jekyll to that on voxpelli.com
  4. Jeremy Swinnen
    That’s definitely on my list as well. Building with Eleventy has been a blast so far!
  5. CSS-Tricks
    Webmentions are very cool. They are an actual standard for collecting what other websites have to say about your particular URLs and aggregating them. Like a proof-based collection of commentary. mxb.dev/blog/using-web…
  6. Pierpaolo Tommasi
    Webmentions are very cool. They are an actual standard for collecting what other websites have to say about your particular URLs and aggregating them. Like a proof-based collection of commentary. mxb.dev/blog/using-web…
  7. What a fantastic write up @mxbck! Adding web mentions to my personal site now 🚀. mxb.dev/blog/using-web…
  8. Max Böck
    Thanks! Glad you found it useful.
  9. nystudio107
    ✅ Craft CMS tip 🎉 You can perform external API queries and decode the JSON response all via Twig: {% set res = craft.app.api.client.get(url).getBody().__toString() | json_decode %} …like querying webmentions 🎩 @zachleat @mxbck #craftcms mxb.dev/blog/using-web…
  10. Mark
    This is downright terrifying 🤪. Makes me wonder if a `helper`-like tag that evaluates to a Twig function would clean this up… {% helper webmentions(url) %} // real PHP here $url = “${url}?token=foo”; // … return Craft::$app->api->client->get(… {% endhelper %}
  11. nystudio107
    It's not stupid if it works, Mark! 😃
  12. Giovanni Bellocchio
    This is terrifying and useful.
  13. Max Böck
    woah, never seen an API call written in a template language before...
  14. ah WebMentions & Bridgy for tweet mentions! really cool!
  15. devMode.fm podcast
    Might be fun to have @swyx on to talk @Netlify sometime!
  16. knut
    …and here's mine about how to do it with @gatsbyjs and @Netlify knutmelvaer.no/blog/2019/06/g… (also blatantly stolen, like all great art)
  17. 🤗 pick a time
  18. Works very well together IMHO! Also worth checking out webmention.app to handle outgoing webmentions. via @rem
  19. Trezy 💫
    I'm reading @mxbck's article on Webmentions and loving it. I dig your writing style. mxb.dev/blog/using-web…
  20. IndieWeb.Life
    Static Indieweb pt2: Using Webmentions | Max Böck - Frontend Web Developer #indieweblife mxb.dev/blog/using-web…
  21. Bryan Robinson
    Finally got around to pushing my Webmentions functionality live on my blog. Almost 100% ripped from @mxbck's excellent tutorial: mxb.dev/blog/using-web…
  22. Corey McCarty
    What a coincidence, I just pushed a bunch of changes to my blog after ripping chunks out of your templates.
  23. Danny 🚀
    Thanks! I’m going to give it a try and let you know if it works!
  24. Alex Fenton
    If you’re scratching your head wondering: “what the hell are web mentions” 🤯 these two articles helped massively. Thanks @swyx & @mxbck swyx.io/writing/client… mxb.dev/blog/using-web…
  25. Oli
    Getting a bit of attention on my last couple of posts finally motivated me to get webmentions working on my site. Shoutout to webmention.io for handling all the hard parts and @mxbck for the inspiration/guide mxb.dev/blog/using-web…
  26. Max Böck
    Awesome, looks great! 👍
  27. 💫 Josh
    I forget if I've told you this before, or simply thought it to myself, but your site/blog is beautiful 😍 Also, agree, static sites are perfect for emergency sites like that. Can handle sudden unexpected traffic surges with no problems 💯
  28. Max Böck
    Oh thank you - likewise! 😅 Yeah it's amazing how much traffic a single server with static HTML can handle.
  29. Lobsters
    Indieweb pt2: Using Webmentions in Eleventy lobste.rs/s/qqwy71 #javascript #web mxb.dev/blog/using-web…
  30. Andy Bell
    Yeh I used this great resource for my site. I ended up actually removing them because I managed to attract a bit of spam with them, unfortunately.
  31. Kevin Marks
    Spam from twitter? We haven't had much native webmention spam, though we have been looking out for it.
  32. Andy Bell
    Bit of both unfortunately. I'm not overly fond of having comments etc on my site, so I already had a pretty dim view.
  33. fluffy 💜 🎂
    You could always receive webmentions and not display them. There are several sites which do that to gauge reactions around the web without making those reactions public.
  34. Andy Bell
    Yeh for sure. Making my own handler for them would be cool!
  35. marclittlemore
    This is great! Thanks for sharing the article. 👍🏻
  36. Stefan Natter
    Thank you! I gotta read that asap :) very interesting.
  37. Ankur💚JavaScript
    Interesting bookmarked
  38. Ed Summers
    @celia thank you for the pointer! I had missed your about/site which is also very helpful. I'm contemplating moving from Jekyll so it's interesting because you migrated as well.
  39. Thanks Amber, super helpful
  40. Aliaksei Chapyzhenka
    Does webmentions work in @observablehq ? @mbostock
  41. Emm, not sure about this but will find out soon since ill be doing a project in @observablehq