If you go to one of my more popular articles, you will see some social media activity at the bottom, including likes, reposts, and comments. This data comes from Twitter, Mastodon, and other places via webmentions. There is more to webmentions than I actually make use of, but they are a good mechanism for collecting this activity for each article on my site.
I should mention that I learned much of this from Sebastian De Deyne’s excellent post on the same topic, and a lot of the code is based on his examples.
Add RelMeAuth links
The first step to get webmentions on your site is to ensure that your site has a link to either your GitHub or Twitter profile (or both) and that the link has the rel="me" attribute. This is how Webmention.io will verify that you are the owner of your site.
<!-- Something like this should work. -->
<a href="https://github.com/your-username" rel="me">GitHub</a>
Set up Webmention.io
Go to Webmention.io and create an account. Per the previous step, you will select either GitHub or Twitter as your identity provider.
Once you have an account, you will be provided with two <link> tags to add to the <head> of your site. I have added these to my BaseLayout.astro component.
<!-- BaseLayout.astro -->
<head>
<!-- Do not copy these. Use the ones from your Webmention.io settings. -->
<link rel="webmention" href="https://webmention.io/example.com/webmention" />
<link rel="pingback" href="https://webmention.io/example.com/xmlrpc" />
</head>
Now you are ready to start receiving webmentions for your site.
Set up Bridgy
In order to have Mastodon and Twitter activity related to your site sent as webmentions, you will need to use a service called Bridgy. From their site, click on the button for each account you want to set up (e.g. Twitter and Mastodon). You will need to authorize Bridgy to access each account.
Once you are logged in using one of those accounts, you can see:
- when your account was last polled,
- when your site was last crawled for webmention targets, and
- any sent webmentions (under “Responses”).
Now, if you create a post on Twitter or Mastodon that links to a page on your site, any activity on that post will be sent as webmentions to your site!
The overall pipeline
Here is how the webmentions get added to my static site:
- A GitHub action executes a script to fetch any new webmentions (see the next section).
- If there are any new webmentions, the action commits them to JSON files in my repo.
- The changes automatically trigger Netlify to rebuild and deploy my site.
If you would like a different approach that fetches the webmentions from within Astro component code, check out Webmentions in Astro (for blog posts) by Steve Frenzel.
Getting data from the Webmention.io API
Hooray, it’s time for some JavaScript. We need a script to fetch all of our webmentions from Webmention.io’s API. You will need the API Key from the Settings page on their site.
You can add your API key to the .env file in the root of your project.
# .env
WEBMENTION_API_KEY=your-api-key
Alternatively, you can set the API key whenever you run the script.
WEBMENTION_API_KEY=your-api-key node ./webmentions.js
Here is the webmentions.js script. It fetches all of my webmentions, and it creates a JSON file for each post. Ensure that you have a data/webmentions directory already created in your project before running the script.
// webmentions.js
import fs from 'fs';
import https from 'https';
const DOMAIN = 'example.com'; // Change this!
const webmentions = await fetchWebmentions();
webmentions.forEach(writeWebMention);
function fetchWebmentions() {
const url =
'https://webmention.io/api/mentions.jf2' +
`?domain=${DOMAIN}` +
`&token=${process.env.WEBMENTION_API_KEY}` +
'&per-page=999';
return new Promise((resolve, reject) => {
const req = https.get(url, (res) => {
let body = '';
res.on('data', (chunk) => (body += chunk));
res.on('end', () => {
try {
const response = JSON.parse(body);
if (res.statusCode !== 200) reject(body);
resolve(response.children);
} catch (error) {
reject(error);
}
});
});
req.on('error', (error) => reject(error));
});
}
function writeWebMention(webmention) {
// Each post will have its own webmentions json file, named after the slug
const slug = webmention['wm-target']
.replace(`https://${DOMAIN}/`, '')
.replace(/\/$/, '')
.replace('/', '--');
const filename = `./data/webmentions/${slug || 'home'}.json`;
// Create the file if it doesn't exist
if (!fs.existsSync(filename)) {
fs.writeFileSync(filename, JSON.stringify([webmention], null, 2));
return;
}
// If the file already exists, append the new webmention while also deduping
const entries = JSON.parse(fs.readFileSync(filename))
.filter((wm) => wm['wm-id'] !== webmention['wm-id'])
.concat([webmention]);
entries.sort((a, b) => a['wm-id'] - b['wm-id']);
fs.writeFileSync(filename, JSON.stringify(entries, null, 2));
}
When you run node ./webmentions.js, if you have any webmentions, you will see a new JSON file for each relevant post in your data/webmentions directory.
Send a test webmention
If you do not already have webmentions, you can send one using Webmention.rocks. Their Receiver Test #1 will do the trick. You will need to log in using IndieAuth (just like Webmention.io). Once you are logged in, you can send a webmention to any URL on your site, including the home page.
After you send the webmention, you should see it on your Webmention.io dashboard. You can always delete the webmention from the dashboard at any time.
Run node ./webmentions.js, and you should also see it in a JSON file in your data/webmentions directory. It will look something like this:
[
{
"type": "entry",
"author": {
"type": "card",
"name": "Webmention Rocks!",
"photo": "https://webmention.io/avatar/webmention.rocks/e08155b03da96cb1bdfd161ea24efdfad8d85d06afcee540ec246f1f613eb5a9.png",
"url": ""
},
"url": "https://webmention.rocks/receive/1",
"published": "2023-03-10T10:37:38-08:00",
"wm-received": "2023-03-10T18:37:38Z",
"wm-id": 1638150,
"wm-source": "https://webmention.rocks/receive/1/892e9ebcaaef2ec6823ac3b93c13cdb8",
"wm-target": "https://example.com/",
"name": "Receiver Test #1",
"content": {
"html": "<p>This test verifies that you accept a Webmention request that contains a valid source and target URL. To pass this test, your Webmention endpoint must return either HTTP 200, 201 or 202 along with the <a href=\"https://www.w3.org/TR/webmention/#receiving-webmentions\">appropriate headers</a>.</p>\n <p>If your endpoint returns HTTP 201, then it MUST also return a <code>Location</code> header. If it returns HTTP 200 or 202, then it MUST NOT include a <code>Location</code> header.</p>",
"text": "This test verifies that you accept a Webmention request that contains a valid source and target URL. To pass this test, your Webmention endpoint must return either HTTP 200, 201 or 202 along with the appropriate headers.\n If your endpoint returns HTTP 201, then it MUST also return a Location header. If it returns HTTP 200 or 202, then it MUST NOT include a Location header."
},
"in-reply-to": "https://example.com/",
"wm-property": "in-reply-to",
"wm-private": false
}
]
Poll for webmentions using a GitHub action
Now that you have a working script to fetch webmentions, you can set up a GitHub Action to run the script on a schedule. The action will only commit changes if there are new webmentions, so if your site automatically deploys whenever the main branch changes, you will not have to worry about unnecessary deployments.
Add the API key as a secret
You will need to add your Webmention.io API key to your project’s action secrets. Go to your repository’s Settings page, and then click on “Actions” under “Secrets and variables”. Click on “New repository secret”, and add the key WEBMENTION_API_KEY with the value of your API key.
Add webmentions.yml to your project
Here is the YAML workflow file I use for my site. Be sure to change the email address, name, and project URL near the bottom of the file, and save it as .github/workflows/webmentions.yml in your project.
name: Webmentions
on:
schedule:
- cron: '*/30 * * * *'
jobs:
webmentions:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@master
- name: Set up Node.js
uses: actions/setup-node@master
with:
node-version: 18.x
- name: Fetch webmentions
env:
WEBMENTION_APK_KEY: ${{ secrets.WEBMENTION_API_KEY }}
run: node ./webmentions.js
- name: Commit to repository
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMMIT_MSG: |
add webmentions
skip-checks: true
run: |
git config user.email "your-email@example.com"
git config user.name "Your Name"
git remote set-url origin https://x-access-token:${GITHUB_TOKEN}@github.com/your-username/your-project.git
git checkout main
git add .
git diff --quiet && git diff --staged --quiet || (git commit -m "${COMMIT_MSG}"; git push origin main)
This action will run every 30 minutes. It will check out the repository, set up Node.js, run the webmentions.js script, and then commit any changes to the main branch. You should see the action run in the “Actions” tab of your GitHub repository.
Part two
In a future article, I will go over the various Astro components I have created to display webmentions on my site. Honestly, some of them could probably use a bit of work before I share them publicly. Until then, why not send me some webmentions on this post?
Webmentions
Jesse Skinner :javascript:, nathanreyes, Kieran McGuire, jon ⚝, and Marc Littlemore liked this.
Kieran McGuire reposted this.
Steve Frenzel mentioned this.
@yala I'm not sure if Bridgy will pick up posts that link to an article. I know it will pick up boosts as "reposts". ????♂️ As far as your reply not showing up on my article, apparently Bridgy stopped fetching new activity on this post because of its age. I entered the URL into the "Resend for post" field on the Bridgy dashboard to manually force discovery, and now it shows up. Thanks for the heads up on that.
@kevinleedrum I'm not exactly sure how this works. Embedding the link into a post did not yield a mention being displayed on the article's URL. Is it answers to this toot, which show up there? https://degrowth.social/@yala/112084414365968434 jon ⚝ (@yala@degrowth.social)
@kevinleedrum hey, great article! I'm gonna try to follow along... but I'm starting with a templated theme (Cactus) and I don't know what I'm doing. Your site and setup looks great so if I can muddle my way to something like that, I'd be very happy!