Postfix is built without TLSRPT support

added priority3-normal severity4-low statusunconfirmed labels

assigned to @freswa, @toolybird, @antiz, and @gromit

added statusconfirmed label and removed statusunconfirmed label

assigned to @dvzrv and unassigned @freswa, @toolybird, and @gromit

Adding TLSRPT support sounds reasonable to me, any objection @dvzrv?

@christx Would you be okay testing a postfix package built with TLSRPT support to verify everything works as intended if I give this a shot? I could either push it to [extra-testing] or upload a prebuilt package in my https://pkgbuild.com repo.

Yeah, let's do it!

Moving a rebuild to extra-testing should be no problem :)

Absolutely. I got a full Postfix, MTA-STS, DANE, Dovecot etc. setup running on my server.

Cool! Thanks a lot for the suggestion and the help
I'm taking a look at packaging https://github.com/sys4/libtlsrpt & https://github.com/sys4/tlsrpt-reporter and will give a shot at rebuilding postfix with TLSRPT support. I'll let you know ;)

That being said, one has to have either postfix-tlspol or postfix-mta-sts-resolver running as a plugin. The latter is in an outdated version in AUR, the former is available as a Docker image. I'm using postfix-tlspol from Docker now, I would reckon this would be the then preferred way to set this up on Arch. postfix-mta-sts-resolver has the issue that it disables DANE if MTA-STS is also present, which postfix-tlspol doesn't have. postfix-tlspol could also be built normally; they also have a systemd template in their repository.

Hummm, okay. I could take a look at packaging postfix-tlspol afterwards eventually, but from my understanding it is not a blocker right now (since one could use docker for it), right?

Yes, it would just be extra. Using it with Docker is perfectly fine, and I've been having no problems with this so far. I was just adding some more context.

Turns out both libtlsrpt and tlsrpt-reporter currently only have rc releases. According to our packaging standards, we do not push non-stable releases (e.g. alpha, beta, release candidates) into our repositories.

I opened an upstream issue to ask if an ETA / expected target for a stable release has been established yet.

While waiting for upstream input, I still prepared a GitLab repo for both packages:

However, as explained earlier, the packages will not be pushed to our repositories before a stable release is out.

@christx In the mean time, if you're interested, I can eventually provide you a prebuilt package for postfix with TLSRPT support enabled, built against the latest rc release of libtlsrpt for test purposes.

@christx In case you want to give it a go, you can find prebuilt packages for libtlsrpt (which needs to be installed before the rest), postfix built with TLSRPT support and tlsrpt-reporter at https://pkgbuild.com/~antiz/packages/

If you give it a try, feel free to tell us if it works as intended! If so, I'll open a draft MR to the postfix package with the required changes while waiting for a stable upstream release for libtlsrpt (and tlsrpt-reporter).

I'll also take a look at packaging postfix-tlspol later this week. I'll let you know.

I hadn't realised that they both were still pre-releases, sort of slipped me as I just read the Postfix documentation and the readmes.

Anyway, thanks for the work already. I've installed the packages, and there's some caveats with tlsrpt-reporter. It's not super well documented, so I had to dive a bit through their code to figure things out.

There are man pages, they're found in the doc and are built with asciidoctor, which is already packaged.

The package will also always require both curl and sqlite. curl is needed for posting TLSRPT to HTTPS endpoints and set as default here. It would be possible for a user to change that, though, if they somehow wanted to. sqlite is the only storage option currently supported and thus needed to run any component at all.

There need to be some sort of systemd service for this to work. I've created something really trivial like the following working already:

[Unit]
Description=tlsrpt-report agent for creating TLSRPT reports
After=network.target

[Service]
ExecStart=/usr/bin/tlsrpt-reportd
PrivateDevices=true
PrivateTmp=true
ProtectSystem=true
Restart=always
User=tlsrpt
LogsDirectory=tlsrpt
StateDirectory=tlsrpt
RuntimeDirectory=tlsrpt
ConfigurationDirectory=tlsrpt

[Install]
WantedBy=multi-user.target

(another one for tlsrpt-collectd in the same manner would be needed). I've combined this with a sysusers.d entry for the tlsrpt user. Note that Postfix currently says in their docs that the recommended socket path isn't fixed yet, which is due to chroot being an option. I'm not using that myself - I just use AppArmor - so /run/tlsrpt is fine for my setup, and I believe a reasonable default for a package.

There are some configs needed, and the default location the three components look for collectd.cfg, fetcher.cfg and reportd.cfg is /etc/tlsrpt (see here for example). They can be absolutely minimal:

[tlsrpt_collectd]
storage = sqlite:///var/lib/tlsrpt/tlsrpt-collectd.sqlite
socketname = /run/tlsrpt/tlsrpt-collectd.socket
logfilename = /var/log/tlsrpt/tlsrpt-collectd.log

[tlsrpt_fetcher]
storage = sqlite:///var/lib/tlsrpt/tlsrpt-collectd.sqlite
logfilename = /var/log/tlsrpt/tlsrpt-fetcher.log

[tlsrpt_reportd]
dbname = /var/lib/tlsrpt/tlsrpt-reportd.sqlite
logfilename = /var/log/tlsrpt/tlsrpt-reportd.log
fetchers = tlsrpt-fetcher
organization_name = EXAMPLE.COM
contact_info = tlsrpt@EXAMPLE.COM
sender_address = noreply@EXAMPLE.COM

Upstream only has an example config, but basically all the settings that are documented in the man pages can be specified either via command line argument or via config entry.

It seems operational now, but unfortunately postfix-tlspol had broken something in their Docker container's startup. I've reported it to them, and they just now fixed it, so I should be able to test the whole combined setup together later today.

As for how to best package this, maybe upstream is willing to make the tlsrpt-reporter packaging a bit easier, or willing to carry systemd service files for us? It's not a major hassle to bundle the aforementioned things in the package sources though, I reckon.

Thanks a lot for the detailed report!

I updated the tlsrpt-reporter PKGBUILD with your feedback (see tlsrpt-reporter@301704cb) and rebuilt the package accordingly (pushed in https://pkgbuild.com/~antiz/packages/).

I'm afraid we cannot ship relevant default collectd.cfg, fetcher.cfg and reportd.cfg configuration files with the package though... Upstream doesn't provide any (apart from the example config that would need to be parsed and split accordingly, which is kinda painful to handle downstream) and there are too much system / case specific parameters to be able to ship sensible defaults in my opinion.
If anything, I think that pointing / documenting reasonable default should be wiki material, rather than trying to handle that (probably poorly unfortunately) on the packaging side of things. What do you think?

That means users will still need to provide their own config files (for what it's worth, the package now provides the man pages as well as the example config under /usr/share/doc/tlsrpt-reported), but the rest should now all be shipped / covered by the package itself

As for how to best package this, maybe upstream is willing to make the tlsrpt-reporter packaging a bit easier, or willing to carry systemd service files for us? It's not a major hassle to bundle the aforementioned things in the package sources though, I reckon.

It would indeed definitely be a plus if we could get upstream to provide systemd service unit files in the sources (and eventually the sysusers.d configuration file as well). I also added a downstream pkg-config integration (.pc file) to the libtlsrpt package which would great having provided by upstream as well (see libtlsrpt@9e27c370).
I plan to submit all of those to upstream once we validate that everything works as intended

Feel free to give a go to the new packages at https://pkgbuild.com/~antiz/packages/ and tell us how it went

I'm afraid we cannot ship relevant default collectd.cfg, fetcher.cfg and reportd.cfg configuration files with the package though...

That is true, and the example config they're providing seems to be more targeted at development or debugging rather than productive usage. I was just suggesting providing minimal configs with the locations for data, logs, etc. that would be consistent with the package, all other options that they might want I'd leave for a user to figure out. The three additional lines I added to the reportd config are just the absolute bare minimum that any user has to set for this to work at all.

If anything, I think that pointing / documenting reasonable default should be wiki material

I'm with you on that; this definitely needs a wiki article. E-mail in general is currently a bit weak documentation wise. TLSRPT only makes sense if you're using DANE and/or MTA-STS anyways, and that's not touched in the wiki at all. It would of course be possible to make an isolated wiki entry about TLSRPT, but I'm not sure how useful that is. Unfortunately, the current wiki guides about setting up virtual mail systems as a whole aren't that useful as Yahoo and Google require DMARC, SPF, DKIM and under certain circumstances ARC even for small scale senders. They're also all very much focused around giving config examples to copy paste (e.g. for postfixadmin), without explaining what is being done. The issue overall is that Postfix and friends can require quite individual configuration that would be vastly different from the wiki. So maybe, an isolated guide on TLSRPT would be appropriate? I'm not quite sure what the best approach here is.

I plan to submit all of those to upstream once we validate that everything works as intended

That sounds very reasonable, I would prefer some more guidance upstream about this rather than having to bundle a huge bunch of files downstream. That just sounds like a recipe for headaches with updates.

Feel free to give a go to the new packages at https://pkgbuild.com/\~antiz/packages/ and tell us how it went

I will do that as soon as I got postfix-tlspol sorted out. Unfortunately, the Docker container specifically has some more issues. Thus, I was thinking to quickly make a PKGBUILD for it to avoid having to deal with that, although I reported the issues upstream. It's just a Go binary, one config file and a systemd service, so that should be quick to do. I'll get on that after I had some sleep.

I was just suggesting providing minimal configs with the locations for data, logs, etc

Fair enough. Unfortunately though, in the current state of things, that means 3 more files to maintain downstream
But indeed, having consistent locations for data, logs and stuff for the package might be worth it. I updated the PKGBUILD accordingly and pushed the new package to https://pkgbuild.com/~antiz/packages/. It would be nice to verify if the tmpfiles.d config is relevant / correct / needed though.

So maybe, an isolated guide on TLSRPT would be appropriate? I'm not quite sure what the best approach here is.

I'll be completely honest here, I'm not knowledgeable enough on the TLSRPT part to have a precise opinion... Although, given how specific (and kinda complex) it seems to be, I'd say that a dedicated page could make sense (which could be linked anywhere relevant).

That sounds very reasonable, I would prefer some more guidance upstream about this rather than having to bundle a huge bunch of files downstream. That just sounds like a recipe for headaches with updates.

Indeed, hopefully they'll be inclined to guide / help us on that front :)

Thus, I was thinking to quickly make a PKGBUILD for it to avoid having to deal with that, although I reported the issues upstream. It's just a Go binary, one config file and a systemd service, so that should be quick to do. I'll get on that after I had some sleep.

I will try to take a look at packaging it soonish, but feel free to beat me to it :D

I created a PKGBUILD / repo for postfix-tlspol: https://gitlab.archlinux.org/archlinux/packaging/packages/postfix-tlspol
Prebuilt package available in https://pkgbuild.com/~antiz/packages/

WARNING: The default / template systemd unit file provided by upstream has quite some superfluous options and doesn't seem to be designed for a System-Wide usage.
I added a patch to remove some superfluous options from it and adjust it for System-Wide usage but I haven't tested / validated it nor I dug through every options to see which one are actually relevant to keep or not.
All of this to say that the systemd service might require some adjustments. Don't hesitate to report required changes or suggestions so I can apply them :)

@christx Good news, libtlsrpt & tlsrpt-reporter now both have stable releases
I pushed them both to [extra]:

I also pushed postfix-tlspol in [extra]:

https://archlinux.org/packages/extra/x86_64/postfix-tlspol/

Additionally, I pushed a postfix package built with TLSRPT support to [extra-testing]:

7b2cc03a

If using the testing repositories on your server to make some tests is a problem, I built the postfix packages against [extra] and uploaded them at https://pkgbuild.com/~antiz/packages/

As said earlier, there is probably some adjustments to make on the systemd services side. Let us know how it went

Wow, you've been busy I only just got back on my PC, and you've already got everything packaged up.

Thanks a lot for the postfix-tlspol package. I'll get to set up my server with this in a bit and see how it works. I've got a friend who has gotten TLSRPT set up on his network, so I can test the functionality.

I'll be completely honest here, I'm not knowledgeable enough on the TLSRPT part to have a precise opinion...

So, TL;DR is that for publicly reachable E-mail servers one always has to accept plaintext in addition to TLS. There's MTA-STS (similar to HSTS for HTTPS just for E-mail) and DANE (publish certificates in DNS with DNSSEC protection) to signal that a server should always be reached via TLS. An E-mail server aware of either would know to demand TLS and not transmit via plaintext if for example a man-in-the-middle attacker was attempting to downgrade to block TLS. postfix-tlspol is a plugin to support MTA-STS for Postfix, as it has only DANE built in (Exim is in the same situation).

TLSRPT is a mechanism to report TLS connectivity to a server. If for example an e-mail server cannot establish a connection, the mail it tried to send would just get lost. TLSRPT is then used to inform about successes and failures regarding TLS. It's pretty much a system to make deploying MTA-STS and DANE less risky by having a reporting system that would make it possible to be informed about any problems, be it due to configuration issues or attacks.

Either way, TLSRPT goes hand in hand with deploying MTA-STS and DANE. So, it would probably be best to have some article about these three technologies together. Those three things together are a pretty isolated topic for Postfix configuration, and I think it's most appropriate to have a wiki article on this specifically, as it doesn't really interfere or depend on the specifics on how the rest of the setup looks like. I hope that makes sense?

WARNING: The default / template systemd unit file provided by upstream has quite some superfluous options and doesn't seem to be designed for a System-Wide usage.

I did notice that already yesterday, I'll report back on whether the current solution works. It's not super pretty to patch their template, but maybe once we got this figured out, we can work with upstream to simplify packaging.

From what I just explained about TLSRPT - wider availability of it also in other distros would be good to help make more people and organisations deploy MTA-STS or DANE and thus make plaintext mail less of a thing.

I'll report back once I got everything set up and tested.

Wow, you've been busy I only just got back on my PC, and you've already got everything packaged up.

Thanks a lot for the postfix-tlspol package. I'll get to set up my server with this in a bit and see how it works. I've got a friend who has gotten TLSRPT set up on his network, so I can test the functionality.

Those three things together are a pretty isolated topic for Postfix configuration, and I think it's most appropriate to have a wiki article on this specifically, as it doesn't really interfere or depend on the specifics on how the rest of the setup looks like. I hope that makes sense?

Makes sense to me at least

I did notice that already yesterday, I'll report back on whether the current solution works. It's not super pretty to patch their template, but maybe once we got this figured out, we can work with upstream to simplify packaging.

Yes, that's the idea! Patching the template downstream is indeed not so elegant, but the point effectively is to work with upstream once we got a working setup on our side

From what I just explained about TLSRPT - wider availability of it also in other distros would be good to help make more people and organisations deploy MTA-STS or DANE and thus make plaintext mail less of a thing.

Indeed! I'm packaging a few stuff for Alpine as well, but can't do much for other distributions on my side
Hopefully they'll bring TLSRPT support once they all move to postifx 3.10!

I'll report back once I got everything set up and tested.

Thanks a lot! That's really appreciated

added scopefeature label

mentioned in commit tlsrpt-reporter@301704cb

mentioned in commit 7b2cc03a

Hi - just to add small note re postfix 3.10.0-2

Postfix package needs a depends on libtlsrpt otherwise updating postfix will fail with:

error: failed to prepare transaction (could not satisfy dependencies)
:: unable to satisfy dependency 'libtlsrpt.so=0-64' required by postfix

Hummm, it does already: 7b2cc03a

Right and I even explicitly installed it and I am still getting the error

Ahhhh, the package is missing the provides statement on libtlsrpt.so, my bad.

Fixing this right now ;)

Yes indeed thanks

Should be fixed with libtlsrpt@0cae0c22

Yep that fixed the update just fine thank you.

I currently use DANE and receive tlsreports for my domains ( I made a little python script to create human reports from these). I used to also use MTA-STS but ended up turning it off (which needed care to due to the max-age part of policy).

It will no doubt take me some time, but I will learn how to use postfix + tlsprt - and hopefully report back once its working and in production.

@genesso

Yep that fixed the update just fine thank you.

Cool, thanks for the confirmation!

It will no doubt take me some time, but I will learn how to use postfix + tlsprt - and hopefully report back once its working and in production.

Sure, don't hesitate to report back here, that'd be really appreciated!

Quick update, tlsrpt-reporter got a typo in the PKGBUILD here - the last /systemd should be /system in both lines.

I agree that a separate dedicated doc on tlsrpt would be super helpful.

We also still ship opendkim which is essentially a dead project. I switched to dkimpy [1] a long time back and found it to be far better and actually is supported. I would also like to replace openarc/opendmarc at some point too for same reason.

[1] https://git.launchpad.net/dkimpy

We also still ship opendkim which is essentially a dead project.

That is true, OpenDKIM and OpenDMARC are not really updated (there has been very slight development activity in 2020-2022 and the last commit is 3 years old) and Arch ships some beta release out of necessity because the release versions have known security vulnerabilities. Red Hat refuses to ship them for this reason. I would strongly advise against using them and recommend rspamd instead, which can handle DKIM, DMARC and ARC for both incoming and outgoing mail. It's also in active development and has no known security vulnerabilities.

It will no doubt take me some time, but I will learn how to use postfix + tlsprt - and hopefully report back once its working and in production.

If you've got any questions regarding that, feel free to hit me up.

Now on topic: I've got everything set up now, and it seems like it's working. tlsrpt-collectd correctly assembles data for domains that have TLSRPT for me. There's the oddity that because tlsrpt-reporter works in the way that it uses the previous day's database that it's spewing fetcher errors on the first day you've got it set up, and I'll need to wait for a day so that it will actually send reports.

Issues wise, there's the problem that the tlsrpt-collectd socket needs to be readable by Postfix. I edited the service file with an override so that it runs in the postfix group and set socketmode to 0660 so that postfix can write to it. I'm not really sure how we could package this better?

The other pitfall that I'll bring up with upstream is that because the sender address can only be configured with one sender address that it's not really suitable for multi-domain setups. I have several e-mail domains and am not really sure how to best deal with this. For a single domain setup, it seems operational.

Indeed! I'm packaging a few stuff for Alpine as well, but can't do much for other distributions on my side

Hehe, I rather meant that if we can get upstream to clean things up and make it easier to package, it will also be easier for other distributions to ship TLSRPT support.

All of this to say that the systemd service might require some adjustments. Don't hesitate to report required changes or suggestions so I can apply them

postfix-tlspol seems to run fine with the shipped service. Postfix queries it and gets its reply properly. Of course, a user needs to have a DNS server capable of DNSSEC validation for DANE to work, but that's a requirement for DANE with any mail server anyway. I'm using it together with Unbound, but PowerDNS etc. should also work I suppose. I've sent mails to my friend whose server has both MTA-STS and DANE, and unlike postfix-mta-sts-resolver it correctly prioritises DANE in this case.

From what I've tested so far, there shouldn't be any issues except with maybe tlsrpt-reportd, which I can only test after it collected data for a day. The postfix build and postfix-tlspol I can confirm to work though.

Thanks @christx - have had rspamd on my todo list for a while now - and appreciate the offer on tlsrpt too.

I'm on IRC with the same nick, feel free to message me.

I've got everything set up now, and it seems like it's working.

Nice, thanks for the report!

Issues wise, there's the problem that the tlsrpt-collectd socket needs to be readable by Postfix.

You mean "writeable" right?

I edited the service file with an override so that it runs in the postfix group and set socketmode to 0660 so that postfix can write to it. I'm not really sure how we could package this better?

I mean... I could reflect that in the package (e.g. by making /run/tlsrpt writeable to the postfix group via sysusers.d and / or by reflecting the changes you made to the systemd service), but that would probably mean adding postfix as a dependency for it (so that we ensure the postfix group actually exists). I'm not sure how much sens that would make (e.g. how much is tlsrpt-reporter tied up to postfix)?

Hehe, I rather meant that if we can get upstream to clean things up and make it easier to package, it will also be easier for other distributions to ship TLSRPT support.

Ahhhh, yeah sure! Makes sense ;)

Of course, a user needs to have a DNS server capable of DNSSEC validation for DANE to work, but that's a requirement for DANE with any mail server anyway. I'm using it together with Unbound, but PowerDNS etc. should also work I suppose.

Maybe we should list DNS server capable of DNSSEC available in the Arch repository as optional dependencies?

From what I've tested so far, there shouldn't be any issues except with maybe tlsrpt-reportd, which I can only test after it collected data for a day. The postfix build and postfix-tlspol I can confirm to work though.

Again, thanks a lot for the testing / reporting!

Maybe we should list DNS server capable of DNSSEC available in the Arch repository as optional dependencies?

Could be an idea, but any actual mail setup needs recursive resolution with DNSSEC anyway as DNS blocklists that the likes rspamd or SpamAssassin use are rate limited and won't work with public DNS thus. We could also handle this in a Wiki article on the subject. The thing with an opt dependency is that quite a lot of solutions exist, like Unbound, PowerDNS-recursor, etc. - so I'm not really sure what the dependency would be on?

You mean "writeable" right?

Yes, sorry. I use 0660 as socketmode.

The thing with an opt dependency is that quite a lot of solutions exist, like Unbound, PowerDNS-recursor, etc. - so I'm not really sure what the dependency would be on?

I ended up going this route, which I think is more elegant than an extensive list of opt-depends.

Postfix is built without TLSRPT support

Child items ...

Activity

Admin message

Postfix is built without TLSRPT support

Activity