Blame


1 93390e9f 2024-03-14 o # Apas: ActivityPub via email
2 93390e9f 2024-03-14 o
3 93390e9f 2024-03-14 o By Oliver Lowe <[o@olowe.co](mailto:o@olowe.co)> ([otl@apubtest2.srcbeat.com](https://apas.srcbeat.com/otl/actor.json))
4 93390e9f 2024-03-14 o
5 93390e9f 2024-03-14 o > Every program attempts to expand until it can read mail.
6 93390e9f 2024-03-14 o > Those programs which cannot so expand are replaced by ones which can.
7 93390e9f 2024-03-14 o
8 93390e9f 2024-03-14 o —[Zawinski's Law of Software Envelopment]
9 93390e9f 2024-03-14 o
10 93390e9f 2024-03-14 o **[Source code]** | **[GoDoc]**
11 93390e9f 2024-03-14 o
12 93390e9f 2024-03-14 o [Source code]: https://git.olowe.co/apub
13 93390e9f 2024-03-14 o [GoDoc]: https://godoc.org/olowe.co/apub
14 93390e9f 2024-03-14 o
15 93390e9f 2024-03-14 o ---
16 93390e9f 2024-03-14 o
17 93390e9f 2024-03-14 o The most popular systems on the [ActivityPub]-speaking [Fediverse] are imitations of non-federated platforms.
18 93390e9f 2024-03-14 o [Mastodon] and [Misskey] are [Twitter] clones;
19 93390e9f 2024-03-14 o [Lemmy] is a [Reddit] clone.
20 93390e9f 2024-03-14 o But ActivityPub –
21 93390e9f 2024-03-14 o the protocol connecting all these systems together –
22 93390e9f 2024-03-14 o is often said to be similar to email,
23 93390e9f 2024-03-14 o which involves exchanging messages.
24 93390e9f 2024-03-14 o In the case of at least Mastodon and Lemmy,
25 93390e9f 2024-03-14 o ActivityPub was implemented after the bulk of each sofware was designed.
26 93390e9f 2024-03-14 o Message exchange – federation by ActivityPub – is arguably a second-class citizen for these
27 93390e9f 2024-03-14 o traditional [CRUD] web applications backed by SQL databases and fronted by web browser UIs.
28 93390e9f 2024-03-14 o
29 93390e9f 2024-03-14 o Apas is an experiment in exposing ActivityPub in a familiar and popular interface: email.
30 93390e9f 2024-03-14 o Its primary goal is to clarify how ActivityPub and the Fediverse work for the broader community.
31 93390e9f 2024-03-14 o A number of secondary goals are detailed later.
32 93390e9f 2024-03-14 o
33 93390e9f 2024-03-14 o [CRUD]: https://en.wikipedia.org/wiki/Create,_read,_update_and_delete
34 93390e9f 2024-03-14 o [upas]: http://doc.cat-v.org/bell_labs/upas_mail_system/
35 93390e9f 2024-03-14 o ## 1. Motivation
36 93390e9f 2024-03-14 o
37 93390e9f 2024-03-14 o As a fan of [Plan 9] and a weirdo who likes to fiddle with network protocols for fun,
38 93390e9f 2024-03-14 o I was disappointed with what using Mastodon, Lemmy et al. felt like.
39 93390e9f 2024-03-14 o
40 93390e9f 2024-03-14 o What excites me is *communication!*
41 93390e9f 2024-03-14 o Exchanging messages *between* people, systems, and places we can't think of yet!
42 93390e9f 2024-03-14 o It's what makes receiving even just a single email from a random person such a viscerally distinct experience from
43 93390e9f 2024-03-14 o thousands reading your post you uploaded somewhere.
44 93390e9f 2024-03-14 o We're communicating!
45 93390e9f 2024-03-14 o
46 93390e9f 2024-03-14 o Implementing a subset of the Mastodon and Lemmy HTTP APIs in a couple of languages was relatively straightforward.
47 93390e9f 2024-03-14 o After writing some small clients and tooling
48 93390e9f 2024-03-14 o it felt like I was just dealing with platforms,
49 93390e9f 2024-03-14 o not a federated universe.
50 93390e9f 2024-03-14 o The pattern was familiar for many software developers:
51 93390e9f 2024-03-14 o
52 93390e9f 2024-03-14 o * You create a post,
53 93390e9f 2024-03-14 o * that gets written to a database,
54 93390e9f 2024-03-14 o * you get an ID back, indicating success.
55 93390e9f 2024-03-14 o
56 93390e9f 2024-03-14 o But the whole federation bit is obscured.
57 93390e9f 2024-03-14 o You hope that others can see that post... somehow...?
58 93390e9f 2024-03-14 o The platform thinking is evident in the language we see around these systems:
59 93390e9f 2024-03-14 o "I saw this on Lemmy", or "this is trending on Mastodon", or "find me on Akkoma".
60 93390e9f 2024-03-14 o Nobody says "find me on email" or "someone sent this on email".
61 93390e9f 2024-03-14 o
62 93390e9f 2024-03-14 o Interoperability efforts fall flat when expertise in one system does not translate to another.
63 93390e9f 2024-03-14 o Moderation and tooling discussions are artificially limited to a particular system.
64 93390e9f 2024-03-14 o Should a plugin for Friendica filtering posts containing racist language only work with Friendica when all the systems work together?
65 93390e9f 2024-03-14 o Should it even be a plugin tied to one particular system in a particular programming language at all?
66 93390e9f 2024-03-14 o
67 93390e9f 2024-03-14 o Finally, interoperability and portability is the "killer feature" of ActivityPub systems and any significant software system.
68 93390e9f 2024-03-14 o We know software developers can write standalone Twitter clones day-in, day-out.
69 93390e9f 2024-03-14 o But no amount of funding to Instagram or any other incumbent commercial platform
70 93390e9f 2024-03-14 o will ever make it available for
71 93390e9f 2024-03-14 o shitty government systems,
72 93390e9f 2024-03-14 o space stations,
73 93390e9f 2024-03-14 o embedded devices,
74 93390e9f 2024-03-14 o your grandmother,
75 93390e9f 2024-03-14 o charities,
76 93390e9f 2024-03-14 o snail mail,
77 93390e9f 2024-03-14 o VR headsets, and
78 93390e9f 2024-03-14 o automatic vacuum cleaners
79 93390e9f 2024-03-14 o *all at once*.
80 93390e9f 2024-03-14 o
81 93390e9f 2024-03-14 o Writing better software systems often means communicating better.
82 93390e9f 2024-03-14 o That means understanding ActivityPub better.
83 93390e9f 2024-03-14 o
84 93390e9f 2024-03-14 o ## 2. Overview
85 93390e9f 2024-03-14 o
86 93390e9f 2024-03-14 o **Apas** is mostly inspired by the [upas] email system available with [Plan 9];
87 93390e9f 2024-03-14 o a collection of small programs operate on files and streams,
88 93390e9f 2024-03-14 o relaying messages out to the Internet or
89 93390e9f 2024-03-14 o delivering to mailboxes on the filesystem.
90 fb77ab16 2024-03-15 o (But it's so much more limited and poorly designed than upas that I was hesitant to even write this bit!)
91 93390e9f 2024-03-14 o
92 fb77ab16 2024-03-15 o An important difference from existing Fediverse software is that
93 fb77ab16 2024-03-15 o **apas** only represents Activities as messages.
94 fb77ab16 2024-03-15 o There are no application-specific data structures like
95 fb77ab16 2024-03-15 o posts, toots, comments, or pages.
96 fb77ab16 2024-03-15 o Messages can be files in a filesystem,
97 fb77ab16 2024-03-15 o which simplifies implementation significantly.
98 fb77ab16 2024-03-15 o
99 93390e9f 2024-03-14 o ### 2.1 Messages
100 93390e9f 2024-03-14 o
101 93390e9f 2024-03-14 o **Apas** marshals ActivityPub objects into [RFC5322] messages and vice-versa.
102 93390e9f 2024-03-14 o
103 93390e9f 2024-03-14 o The [Note Activity] is probably the most recognisable object exchanged by ActivityPub servers.
104 93390e9f 2024-03-14 o They are represented as comments by Lemmy, and posts (toots?) by Mastodon.
105 93390e9f 2024-03-14 o For instance, imagine a reply from Alex to Bowie talking about motorcycle tyres.
106 fb77ab16 2024-03-15 o It's passed around the Fediverse as JSON-encoded data like this:
107 93390e9f 2024-03-14 o
108 93390e9f 2024-03-14 o {
109 93390e9f 2024-03-14 o "type": "Note"
110 93390e9f 2024-03-14 o "id": "https://apub.example.com/alex/12345678",
111 93390e9f 2024-03-14 o "attributedTo": "https://apub.example.com/alex"
112 93390e9f 2024-03-14 o "to": "https://apas.test.example/bowie/87654321",
113 93390e9f 2024-03-14 o "cc": "https://apas.test.example/bowie/followers",
114 93390e9f 2024-03-14 o "inReplyTo": "https://apas.test.example/bowie/87654321",
115 93390e9f 2024-03-14 o "name": "Thoughts on 50/50 tyres"
116 93390e9f 2024-03-14 o "content": "But what if you don't know when you want to ride off-road?",
117 93390e9f 2024-03-14 o }
118 93390e9f 2024-03-14 o
119 93390e9f 2024-03-14 o For **apas** this is equivalent to the mail message:
120 93390e9f 2024-03-14 o
121 fb77ab16 2024-03-15 o From: "Alex " <alex@apub.example.com>
122 93390e9f 2024-03-14 o To: "Bowie" <bowie@apas.test.example>
123 93390e9f 2024-03-14 o CC: "Bowie (followers)" <bowie+followers@apas.test.example>
124 93390e9f 2024-03-14 o Message-ID: <https://apub.example.com/alex/12345678>
125 93390e9f 2024-03-14 o In-Reply-To: <https://apas.test.example/bowie/87654321>
126 93390e9f 2024-03-14 o Subject: Thoughts on 50/50 tyres
127 93390e9f 2024-03-14 o
128 93390e9f 2024-03-14 o But what if you don't know when you want to ride off-road?
129 93390e9f 2024-03-14 o
130 fb77ab16 2024-03-15 o Unlike other Fediverse software,
131 fb77ab16 2024-03-15 o the message to be distributed is written and read by people; not just machines.
132 93390e9f 2024-03-14 o For developers, administrators, and advanced users, seeing data like
133 fb77ab16 2024-03-15 o this builds familiarity with the behaviour between systems,
134 fb77ab16 2024-03-15 o and facilitates communication.
135 fb77ab16 2024-03-15 o We go from "why can I see my toot on Kbin but not on Pleroma?" to
136 fb77ab16 2024-03-15 o "why didn't your Pleroma server receive my message?"
137 fb77ab16 2024-03-15 o which is a much easier question to answer; it's what the systems are actually doing.
138 93390e9f 2024-03-14 o
139 93390e9f 2024-03-14 o If there was only one thing to take away from **apas**, it's that
140 fb77ab16 2024-03-15 o familiarity with data over an API is hugely helpful for
141 fb77ab16 2024-03-15 o troubleshooting. Especially when typical bug reports consist of URLs
142 fb77ab16 2024-03-15 o to irrelevant web apps (or even just screenshots!) trying to explain
143 fb77ab16 2024-03-15 o what was sent versus what was received. That's before we we even
144 fb77ab16 2024-03-15 o address what could be in a database, itself requiring its own query
145 fb77ab16 2024-03-15 o language and tightly-controlled administrative access to read.
146 93390e9f 2024-03-14 o
147 93390e9f 2024-03-14 o [Note Activity]: https://www.w3.org/TR/activitystreams-vocabulary/#dfn-note
148 93390e9f 2024-03-14 o
149 93390e9f 2024-03-14 o ### 2.2. Sending
150 93390e9f 2024-03-14 o
151 93390e9f 2024-03-14 o Presenting posts, comments, notes, etc. as a mail message immediately
152 93390e9f 2024-03-14 o clarifies a big source of confusion with existing systems:
153 93390e9f 2024-03-14 o why isn't my post showing up?
154 93390e9f 2024-03-14 o It becomes easier to reason about this when it is obvious where is a message is sent.
155 93390e9f 2024-03-14 o An email lists recipients explicitly.
156 93390e9f 2024-03-14 o When replying to a Kbin comment via Mastodon,
157 93390e9f 2024-03-14 o it takes knowledge of how each system is implemented to know who the
158 93390e9f 2024-03-14 o recipients are, if any.
159 93390e9f 2024-03-14 o
160 93390e9f 2024-03-14 o Regular email clients
161 93390e9f 2024-03-14 o (or even any old text editor!)
162 93390e9f 2024-03-14 o provide an interactive way to test other AP systems.
163 93390e9f 2024-03-14 o For instance, we can easily test how the message is received if we address the recipient in the `CC` field instead of `To`,
164 fb77ab16 2024-03-15 o or if we list the same recipient 20 times in both fields.
165 fb77ab16 2024-03-15 o **Apas** could report deliverability errors either:
166 93390e9f 2024-03-14 o
167 93390e9f 2024-03-14 o * immediately, or
168 93390e9f 2024-03-14 o * as a bounced message
169 93390e9f 2024-03-14 o
170 fb77ab16 2024-03-15 o At the moment, error messages are returned immediately.
171 93390e9f 2024-03-14 o This has provided a pleasant enough testing experience
172 93390e9f 2024-03-14 o that makes learning ActivityPub an interactive process,
173 fb77ab16 2024-03-15 o directly from any mail client, especially compared with the
174 93390e9f 2024-03-14 o usual drudgery of sifting through logs of big web applications.
175 93390e9f 2024-03-14 o
176 fb77ab16 2024-03-15 o Sending is involves two programs each playing its own role:
177 93390e9f 2024-03-14 o
178 fb77ab16 2024-03-15 o 1. Asubmission program accepts messages from authorised users, and
179 fb77ab16 2024-03-15 o 2. A mailer handles sending messages to the Internet.
180 fb77ab16 2024-03-15 o
181 93390e9f 2024-03-14 o ![](send.png)
182 93390e9f 2024-03-14 o
183 93390e9f 2024-03-14 o #### 2.2.1. Submission
184 93390e9f 2024-03-14 o
185 93390e9f 2024-03-14 o Messages are submitted to a server running `apsubmit`.
186 93390e9f 2024-03-14 o `apsubmit` is a SMTP server.
187 93390e9f 2024-03-14 o It listens for SMTP connections,
188 93390e9f 2024-03-14 o authenticates the session,
189 93390e9f 2024-03-14 o then passes the received message to the mailer `apsend`.
190 93390e9f 2024-03-14 o
191 93390e9f 2024-03-14 o SMTP is a widely implemented protocol.
192 93390e9f 2024-03-14 o `apsubmit` enables
193 93390e9f 2024-03-14 o existing mail clients,
194 93390e9f 2024-03-14 o embededed devices,
195 93390e9f 2024-03-14 o and systems that I don't even know exist,
196 93390e9f 2024-03-14 o to publish to the Fediverse.
197 93390e9f 2024-03-14 o
198 93390e9f 2024-03-14 o For personal use,
199 93390e9f 2024-03-14 o it has been fine using [mutt] via SSH on a Linux server,
200 93390e9f 2024-03-14 o [Sylpheed] on my OpenBSD laptop,
201 93390e9f 2024-03-14 o [MailMate] on a shared iMac,
202 93390e9f 2024-03-14 o and the built-in Mail app on my iPhone for replies.
203 93390e9f 2024-03-14 o I'll leave others to come up with more ideas;
204 93390e9f 2024-03-14 o keep in mind weather stations, printers, video records can usually
205 93390e9f 2024-03-14 o send email but definitely cannot speak ActivityPub!
206 93390e9f 2024-03-14 o
207 fb77ab16 2024-03-15 o For fast feedback,
208 93390e9f 2024-03-14 o `apsubmit` takes advantage of the `RCPT` stage of the SMTP transaction.
209 93390e9f 2024-03-14 o It verifies that listed recipients exist and have inboxes we can target.
210 93390e9f 2024-03-14 o This is in contrast with e.g. Mastodon,
211 93390e9f 2024-03-14 o which will always accept creating the following post:
212 93390e9f 2024-03-14 o
213 93390e9f 2024-03-14 o @john@nowhere.invalid @deleteduser@example.org what do you think?
214 93390e9f 2024-03-14 o
215 93390e9f 2024-03-14 o There's several possible error conditions here. For `john`, perhaps:
216 93390e9f 2024-03-14 o
217 93390e9f 2024-03-14 o * their server is down and messages are undeliverable,
218 93390e9f 2024-03-14 o * their Actor is misconfigured and is missing an `inbox` endpoint,
219 93390e9f 2024-03-14 o * the address is totally invalid
220 93390e9f 2024-03-14 o
221 93390e9f 2024-03-14 o For `deleteduser`, perhaps the account no longer exists.
222 93390e9f 2024-03-14 o Mastodon never notifies of any delivery errors.
223 93390e9f 2024-03-14 o We could ask the server administrators to trawl through the server's logs for us,
224 93390e9f 2024-03-14 o or ask `johnny` and `deleteduser` out-of-band if they got our message.
225 fb77ab16 2024-03-15 o Accounting for some common errors at submission time obviates that extra work.
226 93390e9f 2024-03-14 o
227 93390e9f 2024-03-14 o #### 2.2.2 Mailer
228 93390e9f 2024-03-14 o
229 93390e9f 2024-03-14 o Sending messages is performed by a command-line utility called `apsend`.
230 93390e9f 2024-03-14 o `apsend` reads a message from standard input and disposes of it based on the recipients.
231 fb77ab16 2024-03-15 o If the above message from Alex to Bowie was in a file called "note.eml",
232 93390e9f 2024-03-14 o we could send it with the following shell command:
233 93390e9f 2024-03-14 o
234 93390e9f 2024-03-14 o apsend -t < file.eml
235 93390e9f 2024-03-14 o
236 fb77ab16 2024-03-15 o In general, `apsend` is not intended to be executed directly.
237 fb77ab16 2024-03-15 o Instead, a frontend like an email client (sending via SMTP)
238 fb77ab16 2024-03-15 o or a tool like Plan 9's [marshal(1)]
239 fb77ab16 2024-03-15 o should be used which take care of adding entries to and formatting the
240 fb77ab16 2024-03-15 o header correctly.
241 93390e9f 2024-03-14 o
242 fb77ab16 2024-03-15 o [marshal(1)]: https://9p.io/magic/man2html?man=marshal&sect=1
243 fb77ab16 2024-03-15 o
244 93390e9f 2024-03-14 o ### 2.3 Receiving
245 93390e9f 2024-03-14 o
246 fb77ab16 2024-03-15 o Core to **apas** is handling ActivityPub objects as files in a filesystem.
247 fb77ab16 2024-03-15 o This reveals there are many different ways to retrieve Activitity from the Fediverse
248 fb77ab16 2024-03-15 o beyond the typical process of servers sending Activity to an Actor's inbox.
249 fb77ab16 2024-03-15 o **Apas** of course supports this (see 2.3.2),
250 fb77ab16 2024-03-15 o but it's worth mentioning other techniques to show how flexible
251 fb77ab16 2024-03-15 o working with the Fediverse can be.
252 fb77ab16 2024-03-15 o It may also help clarify discussions on user privacy.
253 fb77ab16 2024-03-15 o
254 fb77ab16 2024-03-15 o #### 2.3.1 Direct
255 fb77ab16 2024-03-15 o
256 fb77ab16 2024-03-15 o This was the first implementation of receiving ActivityPub objects for **apas**.
257 fb77ab16 2024-03-15 o The command `apget` fetches the Activity at a URL, then writes it to the standard output.
258 fb77ab16 2024-03-15 o Throughout testing, I ran the tool in shell scripts like the below to deliver messages to my inbox:
259 fb77ab16 2024-03-15 o
260 fb77ab16 2024-03-15 o apget https://apub.example.com/alex/12345678 | apsend otl
261 fb77ab16 2024-03-15 o
262 fb77ab16 2024-03-15 o Little shell scripts can fetch a series of posts:
263 fb77ab16 2024-03-15 o
264 fb77ab16 2024-03-15 o for i in `seq 12345671 12345678`
265 fb77ab16 2024-03-15 o do
266 fb77ab16 2024-03-15 o apget -m https://apub.example.com/alex/$i
267 fb77ab16 2024-03-15 o done
268 fb77ab16 2024-03-15 o
269 fb77ab16 2024-03-15 o Obviously this is inefficient compared to other methods,
270 fb77ab16 2024-03-15 o but we're not Google.
271 fb77ab16 2024-03-15 o Handy ad-hoc testing.
272 fb77ab16 2024-03-15 o
273 fb77ab16 2024-03-15 o #### 2.3.2 Targeting the ActivityPub inbox
274 fb77ab16 2024-03-15 o
275 fb77ab16 2024-03-15 o This is the typical ActivityPub process.
276 fb77ab16 2024-03-15 o For example, someone could mention us in a Mastodon post:
277 fb77ab16 2024-03-15 o
278 fb77ab16 2024-03-15 o @bowie@apas.test.example hope apas is going OK!
279 fb77ab16 2024-03-15 o
280 fb77ab16 2024-03-15 o Which results in the Mastodon server sending Activity to bowie's Actor inbox.
281 fb77ab16 2024-03-15 o In **apas**,
282 93390e9f 2024-03-14 o `apserve` provides a typical HTTP server for a minimal ActivityPub service.
283 93390e9f 2024-03-14 o It is responsible for:
284 93390e9f 2024-03-14 o
285 93390e9f 2024-03-14 o * receiving Activity over HTTP (ActivityPub inbox)
286 93390e9f 2024-03-14 o * serving users' sent Activity for other servers to fetch (ActivityPub outbox)
287 93390e9f 2024-03-14 o * serving each user's Actor
288 93390e9f 2024-03-14 o * resolving WebFinger lookups
289 93390e9f 2024-03-14 o
290 93390e9f 2024-03-14 o ![](receive.png)
291 93390e9f 2024-03-14 o
292 93390e9f 2024-03-14 o Delivery is not handled by `apserve`.
293 93390e9f 2024-03-14 o Instead, `apserve` converts Activities to mail messages,
294 93390e9f 2024-03-14 o and passes them on to `apsend` for delivery.
295 93390e9f 2024-03-14 o
296 fb77ab16 2024-03-15 o #### 2.3.2 Following
297 93390e9f 2024-03-14 o
298 fb77ab16 2024-03-15 o [Follows] can be sent using `apsend`.
299 fb77ab16 2024-03-15 o Because Follows are not represented clearly as mail,
300 fb77ab16 2024-03-15 o the Follow needs to be written as JSON directly.
301 fb77ab16 2024-03-15 o For example, for user bowie to follow alex:
302 93390e9f 2024-03-14 o
303 fb77ab16 2024-03-15 o {
304 fb77ab16 2024-03-15 o "@context": "https://www.w3.org/ns/activitystreams"
305 fb77ab16 2024-03-15 o "actor": "https://apas.example.org/bowie",
306 fb77ab16 2024-03-15 o "type": "Follow",
307 fb77ab16 2024-03-15 o "object": "https://apub.example.com/alex"
308 fb77ab16 2024-03-15 o }
309 fb77ab16 2024-03-15 o
310 fb77ab16 2024-03-15 o then piped to `apsend`:
311 fb77ab16 2024-03-15 o
312 fb77ab16 2024-03-15 o apsend -j alex@apub.example.com < follow.json
313 fb77ab16 2024-03-15 o
314 fb77ab16 2024-03-15 o Wrapped up in X-line shell script named `apfollow`,
315 fb77ab16 2024-03-15 o following and unfollowing is equivalent to running the commands:
316 fb77ab16 2024-03-15 o
317 fb77ab16 2024-03-15 o apfollow alex@apub.example.com
318 fb77ab16 2024-03-15 o apfollow -u alex@apub.example.com
319 fb77ab16 2024-03-15 o
320 fb77ab16 2024-03-15 o #### 2.3.3 RSS/Atom feeds
321 fb77ab16 2024-03-15 o
322 fb77ab16 2024-03-15 o Many ActivityPub servers also make content available via [web feeds].
323 fb77ab16 2024-03-15 o This could be an efficient way to fetch content using a battle-tested format
324 fb77ab16 2024-03-15 o from resource-constrained servers.
325 fb77ab16 2024-03-15 o
326 fb77ab16 2024-03-15 o One possible tool is something that manages reading new entries in a feed.
327 fb77ab16 2024-03-15 o For each entry, it extracts the ActivityPub object ID from the
328 fb77ab16 2024-03-15 o `<guid>` or `<link>` tag for RSS and Atom respectively.
329 fb77ab16 2024-03-15 o
330 fb77ab16 2024-03-15 o <entry>
331 fb77ab16 2024-03-15 o <title>Atom-Powered Robots Run Amok</title>
332 fb77ab16 2024-03-15 o <link href="https://apub.example.com/alex/12345678"/>
333 fb77ab16 2024-03-15 o <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
334 fb77ab16 2024-03-15 o <updated>2023-12-13T18:30:02Z</updated>
335 fb77ab16 2024-03-15 o <summary>hello, world!</summary>
336 fb77ab16 2024-03-15 o </entry>
337 fb77ab16 2024-03-15 o
338 fb77ab16 2024-03-15 o #### 2.3.4 Fediverse software HTTP APIs
339 fb77ab16 2024-03-15 o
340 fb77ab16 2024-03-15 o Many existing systems provide a HTTP API which provides a convenient
341 fb77ab16 2024-03-15 o way of finding content based on some application-specific logic e.g.
342 fb77ab16 2024-03-15 o a group, full-text search, or time created.
343 fb77ab16 2024-03-15 o
344 fb77ab16 2024-03-15 o An early **apas** prototype was really just a Python script which
345 fb77ab16 2024-03-15 o synchronised my [GoToSocial] (like Mastodon) timeline with a directory on disk.
346 fb77ab16 2024-03-15 o In short:
347 fb77ab16 2024-03-15 o
348 fb77ab16 2024-03-15 o for status in timeline():
349 fb77ab16 2024-03-15 o note = apget(status.source_id)
350 fb77ab16 2024-03-15 o with open(status.id) as f:
351 fb77ab16 2024-03-15 o apub2email(f, note)
352 fb77ab16 2024-03-15 o
353 fb77ab16 2024-03-15 o #### 2.3.5 Takeaway
354 fb77ab16 2024-03-15 o
355 fb77ab16 2024-03-15 o As mentioned already,
356 fb77ab16 2024-03-15 o core to **apas** is handling ActivityPub objects
357 fb77ab16 2024-03-15 o as text streams and
358 fb77ab16 2024-03-15 o files in a filesystem.
359 fb77ab16 2024-03-15 o It's not meant to be the most performant system (not to say that it's slow),
360 fb77ab16 2024-03-15 o but it lets us develop an understanding of the ActivityPub protocol
361 fb77ab16 2024-03-15 o and focus on the data over APIs via quick prototyping.
362 fb77ab16 2024-03-15 o
363 fb77ab16 2024-03-15 o [web feeds]: https://www.rfc-editor.org/rfc/rfc4287
364 fb77ab16 2024-03-15 o [Follows]: https://www.w3.org/TR/activitystreams-vocabulary/#dfn-follow
365 fb77ab16 2024-03-15 o [GoToSocial]: https://gotosocial.org
366 fb77ab16 2024-03-15 o
367 fb77ab16 2024-03-15 o #### 2.x TODO Filtering, spam?
368 fb77ab16 2024-03-15 o
369 fb77ab16 2024-03-15 o - text streams
370 fb77ab16 2024-03-15 o - small portable programs instead of plugins to growing systems
371 fb77ab16 2024-03-15 o
372 fb77ab16 2024-03-15 o ### 2.4 Reading
373 fb77ab16 2024-03-15 o
374 fb77ab16 2024-03-15 o Messages are stored in the [Maildir] format; one message per file.
375 fb77ab16 2024-03-15 o This is not an important part of the system.
376 fb77ab16 2024-03-15 o Maildir is used only because of the easy implementation for `apsend`;
377 fb77ab16 2024-03-15 o it just neads to create create files.
378 fb77ab16 2024-03-15 o
379 fb77ab16 2024-03-15 o How messages are presented to users –
380 fb77ab16 2024-03-15 o no matter how they are stored –
381 fb77ab16 2024-03-15 o is a job for which software has been written for decades already.
382 fb77ab16 2024-03-15 o
383 fb77ab16 2024-03-15 o Here are some that are being used or
384 fb77ab16 2024-03-15 o
385 fb77ab16 2024-03-15 o #### 2.4.1 `read()`
386 fb77ab16 2024-03-15 o
387 fb77ab16 2024-03-15 o No, really.
388 fb77ab16 2024-03-15 o
389 fb77ab16 2024-03-15 o During development, being able to just run cat(1) on a file to debug Content-Type encoding bug
390 fb77ab16 2024-03-15 o was a breath of fresh air when compared to what is more common in web development.
391 fb77ab16 2024-03-15 o
392 fb77ab16 2024-03-15 o That is, running a unit test which queries a relational database running in a container in a virtual machine hopefully all configured correctly, then marshalling that into the application's unique data structure, to ActivityPub, then finally JSON-encoded (half-joking).
393 fb77ab16 2024-03-15 o
394 fb77ab16 2024-03-15 o #### 2.4.2 Existing solutions
395 fb77ab16 2024-03-15 o
396 fb77ab16 2024-03-15 o **Maildir**. Some clients can interact with Maildir directly, like [mutt].
397 fb77ab16 2024-03-15 o
398 fb77ab16 2024-03-15 o **IMAP**. The obvious and most popular method for accessing mailboxes over the network. Dovecot works well. IMAP is very widely supported by mail clients.
399 fb77ab16 2024-03-15 o
400 fb77ab16 2024-03-15 o **NNTP/Usenet**.
401 fb77ab16 2024-03-15 o Throughout this document I've referred to "mail messages".
402 fb77ab16 2024-03-15 o But the so-called "Internet Message Format" described in RFC 5322 is also used by
403 fb77ab16 2024-03-15 o [Usenet] via a protocol known as [NNTP].
404 fb77ab16 2024-03-15 o The protocol is a simple line-based text protocol
405 fb77ab16 2024-03-15 o with many open-source libraries available.
406 fb77ab16 2024-03-15 o Serving Fediverse messages from a filesystem over NNTP would be a fun project.
407 fb77ab16 2024-03-15 o Similar to how the Linux Kernel Mailing list is available over NNTP at `nntp.lore.kernel.org`.
408 fb77ab16 2024-03-15 o
409 fb77ab16 2024-03-15 o **Mailing list archive web interfaces**.
410 fb77ab16 2024-03-15 o Finally yet another opportunity to give those old Perl scripts another lease of life.
411 fb77ab16 2024-03-15 o
412 fb77ab16 2024-03-15 o **upasfs(4)**.
413 fb77ab16 2024-03-15 o [upas] is the system I studied to implement **apas**.
414 fb77ab16 2024-03-15 o Messages could be relayed from `apsend` to `upas/send`,
415 fb77ab16 2024-03-15 o or the Maildir could be converted to mdir(6).
416 fb77ab16 2024-03-15 o Then we would have a *real* filesystem interface over 9P.
417 fb77ab16 2024-03-15 o Another project for another time.
418 fb77ab16 2024-03-15 o
419 fb77ab16 2024-03-15 o [Maildir]: https://en.wikipedia.org/wiki/Maildir
420 fb77ab16 2024-03-15 o [Usenet]: https://en.wikipedia.org/wiki/Usenet
421 fb77ab16 2024-03-15 o [NNTP]: https://datatracker.ietf.org/doc/html/rfc3977
422 fb77ab16 2024-03-15 o
423 93390e9f 2024-03-14 o ## 3. Workarounds & limitations
424 93390e9f 2024-03-14 o
425 93390e9f 2024-03-14 o The mapping between Activity objects, mail messages,
426 93390e9f 2024-03-14 o ActivityPub HTTP methods, and SMTP transactions
427 93390e9f 2024-03-14 o has a number of limitations.
428 93390e9f 2024-03-14 o **Apas** uses some workarounds internally to fill some gaps.
429 93390e9f 2024-03-14 o
430 93390e9f 2024-03-14 o The [Mention] Activity,
431 93390e9f 2024-03-14 o used by Mastodon for notifications,
432 93390e9f 2024-03-14 o is implemented by reading the To field of submitted messages.
433 93390e9f 2024-03-14 o Recipients in `To` are added as Mentions.
434 93390e9f 2024-03-14 o For example, the message:
435 93390e9f 2024-03-14 o
436 93390e9f 2024-03-14 o To: "Oliver Lowe" <otl@hachyderm.io>
437 93390e9f 2024-03-14 o
438 93390e9f 2024-03-14 o Hello!
439 93390e9f 2024-03-14 o
440 93390e9f 2024-03-14 o results in an entry in `tags` in an ActivityPub Note:
441 93390e9f 2024-03-14 o
442 93390e9f 2024-03-14 o {
443 93390e9f 2024-03-14 o "type": "Note"
444 93390e9f 2024-03-14 o ...
445 93390e9f 2024-03-14 o "tags": {[
446 93390e9f 2024-03-14 o "type": "Mention",
447 93390e9f 2024-03-14 o "href": "https://hachyderm.io/users/otl",
448 93390e9f 2024-03-14 o "name": "@otl@hachyderm.io"
449 93390e9f 2024-03-14 o ]}
450 93390e9f 2024-03-14 o }
451 93390e9f 2024-03-14 o
452 93390e9f 2024-03-14 o There is not an easy way to address an Actor's followers using the `acct:` mail address syntax.
453 93390e9f 2024-03-14 o `apas` understands a non-standard syntax using "plus addressing".
454 93390e9f 2024-03-14 o For example to address the followers of user@example.com
455 93390e9f 2024-03-14 o the address user+followers@example.com may be used.
456 93390e9f 2024-03-14 o These followers addresses cannot be resolved by WebFinger.
457 93390e9f 2024-03-14 o
458 93390e9f 2024-03-14 o Likes and Dislikes are silently dropped by `apserve`.
459 93390e9f 2024-03-14 o The reader can decide whether this is a workaround, feature, or bug.
460 93390e9f 2024-03-14 o
461 fb77ab16 2024-03-15 o Accept and Rejects from Follow requests can be received via ActivityPub
462 fb77ab16 2024-03-15 o and delivered as mail but for notifications only.
463 fb77ab16 2024-03-15 o The reverse does not work;
464 fb77ab16 2024-03-15 o **apas** cannot read a Follow request from a mail message.
465 fb77ab16 2024-03-15 o
466 93390e9f 2024-03-14 o To simplifly delivery to local mailboxes,
467 93390e9f 2024-03-14 o Actors served by `apserve` have no shared inbox/outbox.
468 93390e9f 2024-03-14 o Fortunately shared inbox endpoints are inteded as a performance opimitisation for
469 93390e9f 2024-03-14 o servers hosting many Actors, which is beyond the scope of **apas**.
470 93390e9f 2024-03-14 o
471 93390e9f 2024-03-14 o [Zawinski's Law of Software Envelopment]: https://en.wikipedia.org/wiki/Jamie_Zawinski#Zawinski's_Law
472 93390e9f 2024-03-14 o [NetNewsWire]: https://netnewswire.com
473 93390e9f 2024-03-14 o [mutt]: http://www.mutt.org
474 93390e9f 2024-03-14 o [MailMate]: https://freron.com
475 93390e9f 2024-03-14 o [Sylpheed]: https://sylpheed.sraoss.jp/en/
476 93390e9f 2024-03-14 o
477 93390e9f 2024-03-14 o [ActivityPub]: https://en.wikipedia.org/wiki/ActivityPub
478 93390e9f 2024-03-14 o [Misskey]: https://misskey-hub.net
479 93390e9f 2024-03-14 o [Twitter]: https://en.wikipedia.org/wiki/Twitter
480 93390e9f 2024-03-14 o [Reddit]: https://en.wikipedia.org/wiki/Reddit
481 93390e9f 2024-03-14 o
482 93390e9f 2024-03-14 o [Mention]: https://www.w3.org/TR/activitystreams-vocabulary/#microsyntaxes
483 93390e9f 2024-03-14 o
484 93390e9f 2024-03-14 o [2023 Reddit API controversy]: https://en.wikipedia.org/wiki/2023_Reddit_API_controversy
485 93390e9f 2024-03-14 o [3rd Party Twitter Apps]: https://www.theverge.com/2023/1/22/23564460/twitter-third-party-apps-history-contributions
486 93390e9f 2024-03-14 o [Lemmy]: https://join-lemmy.org
487 93390e9f 2024-03-14 o [Mastodon]: https://joinmastodon.org
488 93390e9f 2024-03-14 o [RFC5322]: https://www.rfc-editor.org/rfc/rfc5322
489 93390e9f 2024-03-14 o [Plan 9]: http://9p.io/plan9/