RFC: Encrypted Feeds and Twt Signatures #770

Problem:

As a User I want to be able to optionally encrypt Twts I post so only my friends/family can see the contents.

Supporting the notion of "Private Feeds" is something I've wanted to explore for a while now (as mentioned in Future of Yarn.social) -- And over the course of this project @xuu has always wanted at least to be able to Sign and Verify Twts.

A discussion took place on our IRC channel #yarn.social on Libera Chat between @xuu and I and the following is a draft proposal.

Proposal:

Saltpack is used as the underlying cryptography protocol and format of encrypted message as well as detached signatures.
A Feed Author's Public Key is stored and presented in the Feed's Metadata preamble as # pubkey = ...
It is up to clients to verify and validate public key fingerprints.
Private keys are stored client-side, and it is encouraged to store the key securely as well as password protect it asymmetric encryption and a passphrase or a key derivation function (KDF).
All encryption and signatures are generated client-side.
- Twt Signatures on "post" are processed and stored as a comment preceding the Twt itself.
- Encrypted Twts are encoded as Saltpack messages and stored as the contents of the Twt. Only the Twt's timestamp is in cleartext and possibly (TBD) the Twt Subject Hash (#xxxxxxx).

Flows:

Encrypt:

Client encrypts the contents of the Twt to the target recipients identified as @-mentions and signs it with their Private Key.
Client posts the encrypted Twt to their feed as well as the Signature as a comment preceeding the Twt.

Example:

# signature 6628f12374ce7c2a55d1c864e71206296e380586c9e61c3aa3ac1b5f11674bd53b895705183ff54d00fdeb5534b412569f58cb22dc6b3673b9a265e3bffe470d
2022-01-30T14:27:47+10:00	BEGIN SALTPACK ENCRYPTED MESSAGE. kiNJamlTJ29ZvW4 RHAOg9hm6h1xJua c8FYYLmpqcI15I6 a0494Bs38GI8xpn 5uMQ9KoUPsFdUmJ 1eWr369H5gRcvFV zeN4UQcv72qDUiw FhNCpO7w4LpuhHJ q9WqDpYTtdipccS hRF3YxLXWbUQfUD s8qJLeQfkwofTia ckHbhORxrFktGdj EctkqZshbETTHkq MMQJCuoOcz9JDwv iouSZ1cw1XcbkwS aMEUNxvl6mH0vKF Tqm53x7RoxSYWNq Ng42CnKZwTZR2nW DlsIlI2GHOATEBP VMhVSEU3A78Q5QA aGmaLRGe7jbxYs0 aTeBDsKl6jApfHV UXZQHRkDgAplfN3 1ZI6BEkdc5ZWY3a vnAswh2KKIrWDRq 2wNwFt6YTb0OlWw htMJ3o23P1iTNyo VsPQZl1x8Fb6RyR 7xQIqD9nD95Cwzt GhvUjcjaefpHvOU MfASiyjmbrOm7V1 lHVDCvxL40mkT3K eYXAves35ei41I3 . END SALTPACK ENCRYPTED MESSAGE.

Decrypt:

Client sees a Twt that starts wiht BEGIN SALTPACK ENCRYPTED MESSAGE and attempts to decrypt it with its Private Key
- If successful, it is displays.
- If unsuccessful the client ignores this Twt and does not display it.

Verify:

The client as it parses a Feed would identify comments preceeding a Twt with # signature ... and attempt to validate those signatures against the Feed's Public Key (as specified in the Feed's Metadata Preamble)

Questions:

What do we do about Twt Subject(s) / Twt Subject Hashes?
Do we build a "web of trust" model, a PKI or do we let users verify public key fingerprints and "mark as trusted" in their clietns?

**Problem:** > As a User I want to be able to optionally encrypt Twts I post so only my friends/family can see the contents. Supporting the notion of "Private Feeds" is something I've wanted to explore for a while now (as mentioned in [Future of Yarn.social](https://www.prologic.blog/2021/12/19/future-of-yarnsocial.html)) -- And over the course of this project [@xuu](https://txt.sour.is/user/xuu/twtxt.net) has always wanted _at least_ to be able to Sign and Verify Twts. A discussion took place on our IRC channel #yarn.social on Libera Chat between @xuu and I and the following is a draft proposal. **Proposal:** - [Saltpack](https://saltpack.org) is used as the underlying cryptography protocol and format of encrypted message as well as detached signatures. - A Feed Author's Public Key is stored and presented in the Feed's Metadata preamble as `# pubkey = ...` - It is up to clients to verify and validate public key fingerprints. - Private keys are stored client-side, and it is encouraged to store the key securely as well as password protect it asymmetric encryption and a passphrase or a key derivation function (KDF). - All encryption and signatures are generated client-side. - Twt Signatures on "post" are processed and stored as a comment preceding the Twt itself. - Encrypted Twts are encoded as Saltpack messages and stored as the contents of the Twt. Only the Twt's timestamp is in **cleartext** and possibly (_TBD_) the Twt Subject Hash (#xxxxxxx). **Flows:** Encrypt: - Client encrypts the contents of the Twt to the target recipients identified as @-mentions and signs it with their Private Key. - Client posts the encrypted Twt to their feed as well as the Signature as a comment preceeding the Twt. Example: ``` # signature 6628f12374ce7c2a55d1c864e71206296e380586c9e61c3aa3ac1b5f11674bd53b895705183ff54d00fdeb5534b412569f58cb22dc6b3673b9a265e3bffe470d 2022-01-30T14:27:47+10:00 BEGIN SALTPACK ENCRYPTED MESSAGE. kiNJamlTJ29ZvW4 RHAOg9hm6h1xJua c8FYYLmpqcI15I6 a0494Bs38GI8xpn 5uMQ9KoUPsFdUmJ 1eWr369H5gRcvFV zeN4UQcv72qDUiw FhNCpO7w4LpuhHJ q9WqDpYTtdipccS hRF3YxLXWbUQfUD s8qJLeQfkwofTia ckHbhORxrFktGdj EctkqZshbETTHkq MMQJCuoOcz9JDwv iouSZ1cw1XcbkwS aMEUNxvl6mH0vKF Tqm53x7RoxSYWNq Ng42CnKZwTZR2nW DlsIlI2GHOATEBP VMhVSEU3A78Q5QA aGmaLRGe7jbxYs0 aTeBDsKl6jApfHV UXZQHRkDgAplfN3 1ZI6BEkdc5ZWY3a vnAswh2KKIrWDRq 2wNwFt6YTb0OlWw htMJ3o23P1iTNyo VsPQZl1x8Fb6RyR 7xQIqD9nD95Cwzt GhvUjcjaefpHvOU MfASiyjmbrOm7V1 lHVDCvxL40mkT3K eYXAves35ei41I3 . END SALTPACK ENCRYPTED MESSAGE. ``` Decrypt: - Client sees a Twt that starts wiht `BEGIN SALTPACK ENCRYPTED MESSAGE` and attempts to decrypt it with its Private Key - If successful, it is displays. - If unsuccessful the client ignores this Twt and does not display it. Verify: - The client as it parses a Feed would identify comments preceeding a Twt with `# signature ...` and attempt to validate those signatures against the Feed's Public Key (_as specified in the Feed's Metadata Preamble_) **Questions:** 1. What do we do about Twt Subject(s) / Twt Subject Hashes? 2. Do we build a "web of trust" model, a PKI or do we let users verify public key fingerprints and "mark as trusted" in their clietns?

Keeping the timestamp in clear makes this somehow compatible with the twtxt format, however leaks meta information of course. Not sure whether this is a concern to you.

Other than that one could think about whether twtxt is the right vehicle for that.

Keeping the timestamp in clear makes this somehow compatible with the twtxt format, however leaks meta information of course. Not sure whether this is a concern to you. Other than that one could think about whether twtxt is the right vehicle for that.

Keeping the timestamp in clear makes this somehow compatible with the twtxt format, however leaks meta information of course. Not sure whether this is a concern to you.

True, but this has the same argument as Delta.Chat that leaks the RFC 2822 ehaders

Other than that one could think about whether twtxt is the right vehicle for that.

Again, same argument can be applied to Encrypted Email using GPG

@xuu Feel free to chime in 😅

One thing to keep in mind however is; this is an "extension" and 100% opt-in.

> Keeping the timestamp in clear makes this somehow compatible with the twtxt format, however leaks meta information of course. Not sure whether this is a concern to you. True, but this has the same argument as [Delta.Chat](https://delta.chat) that leaks the [RFC 2822 ehaders](https://www.google.com/search?q=email%20rfc%20headers&btnK) > Other than that one could think about whether twtxt is the right vehicle for that. Again, same argument can be applied to Encrypted Email using GPG @xuu Feel free to chime in 😅 One thing to keep in mind however is; this is an "extension" and 100% opt-in.

One thing I did not make very clear, which came to light in discussions between @lyse and I on IRC on #Yarn.social on Libera Chat:

Private Keys are stored with the client.

To be super clear here; This means the Private Key is stored with (for example):

a Browser, using client-side Javascript and localStorage.
the yarnc command-line client in say ~$HOME/.yarn/`
the Yarn.social Mobile APp in "secure storage"
in "whereever" twtr, tt or jenny decides sensitive material/keys goes.

Under no circumstances would a Yarn.social pod ever see, know about or have posession of a Pricate Key.

One thing I did not make very clear, which came to light in discussions between @lyse and I on IRC on #Yarn.social on Libera Chat: > Private Keys are **stored** with the client. To be **super clear** here; This means the Private Key is stored with (_for example_): - a Browser, using client-side Javascript and localStorage. - the `yarnc` command-line client in say ~$HOME/.yarn/` - the Yarn.social Mobile APp in "secure storage" - in "whereever" `twtr`, `tt` or `jenny` decides sensitive material/keys goes. Under **no** circumstances would a Yarn.social pod ever see, know about or have posession of a Pricate Key.

I think encrypted twts is a great idea, especially as opt-in. The way I personally would find this useful is if this is opt-in within the twt while composing it, rather than a blanket opt-in per account (maybe both so people can encrypt all of their twts without having to mark them individually?).

Use cases that this would be very handy for:

Sharing personal information with friends.
Sharing pictures with friends that you'd rather not have in public view.

I don't see any downsides to this as it would be opt-in and the end-user wouldn't know encrypted twts even exist.

I think encrypted twts is a great idea, especially as opt-in. The way I personally would find this useful is if this is opt-in within the twt while composing it, rather than a blanket opt-in per account (maybe both so people can encrypt all of their twts without having to mark them individually?). Use cases that this would be very handy for: - Sharing personal information with friends. - Sharing pictures with friends that you'd rather not have in public view. I don't see any downsides to this as it would be opt-in and the end-user wouldn't know encrypted twts even exist.

Do we build a "web of trust" model, a PKI or do we let users verify public key fingerprints and "mark as trusted" in their clietns?

I’d be very much in favor of TOFU here. All the other models have always caused headaches.

What do we do about Twt Subject(s) / Twt Subject Hashes? [...]
Only the Twt's timestamp is in cleartext and possibly (TBD) the Twt Subject Hash (#xxxxxxx).

I’d say that it’s important to not include the subject hash in cleartext.

If the hash was out in the open, an observer might end up seeing two people exchanging messages in a thread. Those messages would all be encrypted, yes, but the observer could still know who’s talking to whom.

Private keys are stored client-side, and it is encouraged to store the key securely as well as password protect it asymmetric encryption and a passphrase or a key derivation function (KDF).

I usually handle secrets by asking my password manager. So, for example, the config file of jenny would include a line like secret_key = "pass show twtxt-privkey". jenny invokes this command and uses its output on stdout as the secret key.

By offering a generic bridge, you can basically support whichever password manager you like and you never have to care about store those delicate secrets.

This doesn’t have to be added to the spec, I just want to spread the idea. 😅

My personal take on this:

At the moment, I think I wouldn’t implement encryption in my client. While this feature would be very useful for Yarn.social (because I suspect that users expect a “send a private message” feature), it is out of scope for my use cases. For me, twtxt is a public platform – if you want to send private messages, then just send me a GPG encrypted e-mail. Or even an unencrypted one, which might be good enough if you’re not sharing stuff like your bank account.

Signing my feed is a bit more interesting to me. I did that using GPG for a while, you probably remember. 😅 I’m not sure if it’s really necessary, though. If someone hacks my server, they have better things to do than to fiddle with my twtxt file. Basically, my twtxt file has the same level of “trust” as my blog posts, which aren’t signed, either (maybe they should be?).

(Would I prefer GPG over Saltpack? At least as an additional option just for signing feeds? Hmm. Dunno. I like that I have already set up GPG and, honestly, it’s not that hard to use, especially when you’re just verifying signatures. But I wouldn’t want to implement GPG key handling in a twtxt client, hell no – that would have to be an individual component, which would work fine for jenny, but that’s probably not an option for Yarn.social.)

> Do we build a "web of trust" model, a PKI or do we let users verify public key fingerprints and "mark as trusted" in their clietns? I’d be very much in favor of TOFU here. All the other models have always caused headaches. > What do we do about Twt Subject(s) / Twt Subject Hashes? [...] > Only the Twt's timestamp is in cleartext and possibly (TBD) the Twt Subject Hash (#xxxxxxx). I’d say that it’s important to *not* include the subject hash in cleartext. If the hash was out in the open, an observer might end up seeing two people exchanging messages in a thread. Those messages would all be encrypted, yes, but the observer could still know who’s talking to whom. > Private keys are stored client-side, and it is encouraged to store the key securely as well as password protect it asymmetric encryption and a passphrase or a key derivation function (KDF). I usually handle secrets by asking my password manager. So, for example, the config file of `jenny` would include a line like `secret_key = "pass show twtxt-privkey"`. `jenny` invokes this command and uses its output on stdout as the secret key. By offering a generic bridge, you can basically support whichever password manager you like and you never have to care about store those delicate secrets. This doesn’t have to be added to the spec, I just want to spread the idea. 😅 --- My personal take on this: At the moment, I think I wouldn’t implement *encryption* in my client. While this feature would be very useful for Yarn.social (because I suspect that users *expect* a “send a private message” feature), it is out of scope for my use cases. For me, twtxt is a *public* platform – if you want to send *private* messages, then just send me a GPG encrypted e-mail. Or even an unencrypted one, which might be good enough if you’re not sharing stuff like your bank account. *Signing my feed* is a bit more interesting to me. I did that using GPG for a while, you probably remember. 😅 I’m not sure if it’s really necessary, though. If someone hacks my server, they have better things to do than to fiddle with my twtxt file. Basically, my twtxt file has the same level of “trust” as my blog posts, which aren’t signed, either (maybe they should be?). (Would I prefer GPG over Saltpack? At least as an additional option just for signing feeds? Hmm. Dunno. I like that I have already set up GPG and, honestly, it’s not that hard to use, especially when you’re just verifying signatures. But I wouldn’t want to implement GPG key handling in a twtxt client, hell no – that would have to be an individual component, which would work fine for `jenny`, but that’s probably not an option for Yarn.social.)

I’d be very much in favor of TOFU here. All the other models have always caused headaches.

Agreed, wasn't aware it was called TOFU.

> I’d be very much in favor of TOFU here. All the other models have always caused headaches. Agreed, wasn't aware it was called `TOFU`.

What do we do about Twt Subject(s) / Twt Subject Hashes? [...]
Only the Twt's timestamp is in cleartext and possibly (TBD) the Twt Subject Hash (#xxxxxxx).

I’d say that it’s important to not include the subject hash in cleartext.

If the hash was out in the open, an observer might end up seeing two people exchanging messages in a thread. Those messages would all be encrypted, yes, but the observer could still know who’s talking to whom.

Thd downside of this is we lose (_I think I mentioned this somewhere-) the capability to group Twts together into a collection (we call a Yarn).

If we decide everything must be encrypted, including even say the timestamp, then perhaps a per-Twt encryption model isn't suitable here and instead a whole-Feed.

> > What do we do about Twt Subject(s) / Twt Subject Hashes? [...] > > Only the Twt's timestamp is in cleartext and possibly (TBD) the Twt Subject Hash (#xxxxxxx). > > I’d say that it’s important to *not* include the subject hash in cleartext. > > If the hash was out in the open, an observer might end up seeing two people exchanging messages in a thread. Those messages would all be encrypted, yes, but the observer could still know who’s talking to whom. Thd downside of this is we lose (_I think I mentioned this somewhere-) the capability to group Twts together into a collection (_we call a Yarn_). If we decide everything must be encrypted, including even say the timestamp, then perhaps a per-Twt encryption model isn't suitable here and instead a whole-Feed.

@movq I agree re the management of Private Keys. Defer this to the User and/or the User's Key Management software.

@movq Some final thoughts on this form me based on yor own final thoughts 😅

I don't expect clients to implement this as readily as any of the other extnesions to date.
GPG is "okay" but as you rightfully point out working with it in any software is "hard" -- Iv'e tried a few times in the past, it's painful 😅

And finally -- I agree Twtxt/Yarn was and is meant for public consmption. That being said though I see no reason Yarn.social specifically can't have encrypted feeds as long as we a) do it correctly and b) it doesn't get in the way of others that don't use or don't spport encryption.

Signing on the OTOH I see as a "good thing" and this is probably something we can do as a "first step".

You are right abot the "Just send me a GPG encrypted email. Do I ever have your GPG key? Hmm 🤔" or "Just send me a normal email", however there are lots of good/valid reasons to try to bidl this, even if it's only a feature of yarnd.

@movq Some final thoughts on this form me based on yor own final thoughts 😅 - I don't expect clients to implement this as readily as any of the other extnesions to date. - GPG is "okay" but as you rightfully point out working with it in any software is "hard" -- Iv'e tried a few times in the past, it's painful 😅 And finally -- I agree Twtxt/Yarn _was_ and **is** meant for public consmption. That being said though I see no reason Yarn.social specifically can't have encrypted feeds as long as we a) do it correctly and b) it doesn't get in the way of others that don't use or don't spport encryption. Signing on the OTOH I see as a "good thing" and this is probably something we can do as a "first step". You are right abot the "Just send me a GPG encrypted email. Do I ever have your GPG key? Hmm 🤔" or "Just send me a normal email", however there are lots of good/valid reasons to try to bidl this, even if it's only a feature of `yarnd`.

@xuu / @lyse / @movq Shall we park this and move it to 0.17? What do we want to do here? Get signed Twts going at least? 🤔

I don't have any need for either of them, so all decisions are okay. :-)

Oof. 😅 I pretty much forgot about this topic. I'm with lyse on this one. 🥴

I'd like it as Proof of Concept or something to play with, but I don't think we'll use it that much.

Then let's close this. Salty IM exists anyweay and is better suited to "private chats".

Then let's close this. [Salty IM](https://salty.im) exists anyweay and is better suited to "private chats".

Labels Milestones

RFC: Encrypted Feeds and Twt Signatures #770