Use packages instead of ports ?

[bozhinov]

REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER:

• iRedMail version (check /etc/iredmail-release): 1.7.1
• Deployed with iRedMail Easy or the downloadable installer? No
• Linux/BSD distribution name and version: Freebsd 14.2
• Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
• Web server (Apache or Nginx): Nginx
• Manage mail accounts with iRedAdmin-Pro? No
• [IMPORTANT] Related original log or error message is required if you're experiencing an issue.

Hello,

I have not used ports in ages. I did not used to mind it but

• 3 Gb of ports take a while to compile
• I chose LDAP for a backend and it ended up downloading MySQL & PGSQL any way along with all of their dependencies

If all is going to be installed anyway - wouldn't it be much better to use packages ?

[iredmail]

We tried years ago, but pre-built packages don't have required compile options. e.g. LDAP support, MySQL support, etc.
Not sure how it works nowadays.

[bozhinov]

the same say
it should be possible to install cmake, mariadb and rust at least from pkg.
that will save like couple of hours and my disks.

[iredmail]

Mixing ports and packages is not good idea.
if you don't want to wait for compile, try Linux or OpenBSD instead. :(

[bozhinov]

I have considered OpenBSD but there is a lot more running on that machine and who has the time to patch Linux boxes as often as needed.

I m sure most admins would deploy iRedMail in a jail that will be used for that purpose alone.
Long term maintenance using ports is no picnic either

[iredmail]

Hi @bozhinov

Would you like to give it a try? Update functions/packages_freebsd.sh to install packages with pkg instead of ports tree.

[bozhinov]

Hi,

I have it all setup for trying out the packages and the sequence is done and tested.
Post new years I just ve got few things to take care of first.

[bozhinov]

I have added you to my private repo.
apart from that stupid SSL bug it is OK
says fixed in FreeBSD 15-dev branch

[iredmail]

Hi @bozhinov

could you fork the iRedMail repo directly instead of copying repo and make it private? So that we can easily create pull request after finished.

[bozhinov]

Hi,
I m definitely doing that. Need few days. My schedule is hectic these days.

[xbeaudouin]

Hello there,
On ports there is now flavors.
As a FreeBSD ports commiter I can adapt (if needed) the ports to have such flavors already done.
Also I can help to have a mail/iredmail port as well.
What do you think of that?
Kind regards

[iredmail]

Hi @xbeaudouin

That's wonderful.

Flavors should be enough. We need few packages with MySQL, PostgreSQL, OpenLDAP support. For example, postfix, Dovecot, Amavisd. If you can help make it happen, we can work together. :)

[xbeaudouin]

Well dovecot has already flavors. I am trying to understand functions/packages_freebsd.sh and have a check about current port tree.
Really I would like this project works to replace my old mail server.

[xbeaudouin]

By the way why should you need openssl from ports? Since on FreeBSD 14.2 we already have openssl v3.0.15:

$ openssl version
OpenSSL 3.0.15 3 Sep 2024 (Library: OpenSSL 3.0.15 3 Sep 2024)

[iredmail]

Hi @xbeaudouin,

Do you mean we should remove ssl=openssl from /etc/make.conf?
https://github.com/iredmail/iRedMail/blob/master/functions/packages_freebsd.sh#L54

[iredmail]

Hi @xbeaudouin,

According to functions/packages_freebsd.sh, these ports requires flavors:

• mail/spamassassin: need flavors for mysql and postgresql support.
• mail/roundcube: need flavors for openldap, mysql, postgresql. Note: for iRedMail OpenLDAP backend, it requires both openldap and mysql support.
• mail/postfix: need flavors for openldap, mysql, postgresql. Note: for iRedMail OpenLDAP backend, it requires both openldap and mysql support.
• security/amavisd-new: need flavors for openldap, mysql, postgresql. Note: for iRedMail OpenLDAP backend, it requires both openldap and mysql support.
• devel/apr1: need flavors for openldap, mysql, postgresql. Note: for iRedMail OpenLDAP backend, it requires both openldap and mysql support.
• devel/sope: need flavors for openldap, mysql, postgresql. Note: for iRedMail OpenLDAP backend, it requires both openldap and mysql support.
• databases/py-sqlalchemy14: need flavors for mysql, postgresql.

[iredmail]

Currently the installation involves compiling Perl / Python / PHP / Rust / Go / LLVM, you can imagine how much time we waste.

[xbeaudouin]

Hi,
Well excepted if you need some special cyphers on openssl, I don't see any good reason to add this dependency. Postfix, nginx, dovecot can be run on freebsd stock openssl AFAIK. So I think we can remove ssl=openssl in make.conf.

About ports with flavors, I will try to push some of them. Issue will be "dual" flavors which is not supported by the port infrastructure

[iredmail]

If dual flavors is impossible, then we still have to install all packages from ports tree, to avoid mixing binary packages and ports.
Is such mixing recommended or not?

I see Dovecot has LDAP flavor on by default, and you can choose additional MySQL or PGSQL flavor. Is it possible that we have LDAP flavor on by default too for other ports?
For example, MySQL flavor of mail/postfix supports both LDAP and MySQL, PGSQL flavor of mail/postfix supports both LDAP and PGSQL.

[iredmail]

Or, besides standard ldap, mysql, pgsql flavers, can we create ldap-mysql, ldap-pgsql flavor?
I mean, ldap-mysql is just one flavor name, except it enables two compile options instead of one.

• in FLAVORS=, append ldap-mysql as a flavor name:
  https://cgit.freebsd.org/ports/tree/mail/postfix/Makefile?id=d4e949bafdb4e842868267d26a357c9921d3efeb#n25
• Add ldap_mysql_PKGNAMESUFFIX= -ldap-mysql
• and so on ...

[xbeaudouin]

the '-' is used to sperate the flavors... I will discuss with other maintainer for that...

[xbeaudouin]

I think we can mix both packages and compiled with port ones. There is no problems about that.
Also the LDAP can be the exception.
Maybe we should do a iredmail ports, that will handle that too.

[iredmail]

I think "dual" (or even more) favors is the key. I prefer waiting for you discussion with other maintainers. :)

OpenBSD has multi flavors support and it works well, no need to use ports tree at all for normal users.

[xbeaudouin]

I think we can make some "special flavors" eg -mysqlldap -pgsqlldap for such ports.

[iredmail]

I will wait for your update. Thank you very much.

Special flavor might be a trick or temporary solution, for example,
What if someone needs ldap + mysql + pgsql? I wish FreeBSD ports tree supports multi flavors like OpenBSD in the future. :)

[bozhinov]

Not sure what happened here but I m not compiling this from ports ever again.

bozhinov@66bb775

Turned out MariaDb is not possible as a backend on FreeBSD
There is no postfix-mariadb package

P.S. Tested using the latest packages and not the quarterly ones

[bozhinov]

Got it working 100% with PGSQL via packages
https://github.com/bozhinov/iRedMail-pub/commits/master/

[iredmail]

Got it working 100% with PGSQL via packages https://github.com/bozhinov/iRedMail-pub/commits/master/

Great news. 👍
Could you please use only one pkg install command to install all packages?

[bozhinov]

Done!
@iredmail I initially cloned the masted branch. at some point \n became \r\n
could have been me though. I run on Windows.

@xbeaudouin adding support back for MariaDb does not seem like a lot of work
these 3 packages could easily 3 turn into new ones for MariaDb:
dovecot-MySQL
dovecot-pigeonhole-MySQL
postfix-mysql

[xbeaudouin]

@bozhinov Nice :) Well did you made a PR ?

@iredmail if @bozhinov made a PR, can you merge it ? Then I will update for mysql/mariadb pkg ?

[iredmail]

How about I merge it in a new git branch then we continue in that branch instead of master?

[bozhinov]

Can't push with those new lines all messed up. thanks to the windows desktop client
@iredmail whatever works. the dialog on FreeBSD will still show MariaDb & ldap as supported

[iredmail]

Hi @bozhinov

Could you please grant me write privilege in your repo? I made some changes locally and waiting for push to your repo.

[bozhinov]

sent invite

[iredmail]

Hi @bozhinov

Not necessary.
I created branch pkg with all your changes, and my improvements. Thanks.
#280

Your changes dropped support for other backends, i have to clean it up.

[iredmail]

@bozhinov

Why do you prefer quarterly instead of latest packages?

[bozhinov]

I used latest for dev test and I use latest in production

[iredmail]

Only Latest packages offer packages like dovecot-mysql, dovecot-pgsql.
We have to use Latest for now, maybe switch back to quarterly next quarter.

[iredmail]

Improved with MariaDB backend support in PR #280
Currently installation goes well for both mariadb and pgsql backends, but didn't have time to test applications yet.
Please help test.

[bozhinov]

Hi, postfix-mysql this package forces the removal of MariaDb tested already won’t work Momchil

[xbeaudouin]

Well this is mostly because -mysql packages on the ports USE=mysql in Mk/bsd.default-version.mk use :

# Possible values: 8.0, 8.4, 9.0, 9.1, 10.5m, 10.6m, 10.11m, 11.4m
MYSQL_DEFAULT?=		8.0

If the poudriere that make this set mysql:10.6m then all packages are set to mariadb...
This is also one of the reasons I want to make a port of the installer.

[bozhinov]

I imagine a port would break these guyses' business model

Also what happens when MySQL reaches v10 in another 5 to 10 years ?

[iredmail]

postfix-mysql this package forces the removal of MariaDb

Thanks for helping. I found this conflict moment ago during testing, yesterday it's kind of rush and didn't notice the conflict message reported by pkg.

[iredmail]

Well this is mostly because -mysql packages on the ports USE=mysql in Mk/bsd.default-version.mk use :

# Possible values: 8.0, 8.4, 9.0, 9.1, 10.5m, 10.6m, 10.11m, 11.4m
MYSQL_DEFAULT?=		8.0

Hi @xbeaudouin

All packages with -mysql flavor require MySQL client package (e.g. mysql80-client), but iRedMail uses MariaDB instead for a long time.

Possible solutions:

1. Switch to MySQL on FreeBSD in iRedMail installer. No plan for other linux/openbsd.
2. Tune FreeBSD pkg to use MariaDB instead of MySQL. I didn't find any doc, guess it's impossible. Ideally packages with -mysql flavor should work with either mariadb or mysql, but mysql is now forced, and many people don't think mariadb is a drop-in replacement for mysql in some cases.
3. Improve FreeBSD ports tree to add -mariadb flavor for all these packages. for example, postfix-mariadb, dovecot-mariadb.

Currently, 1) is the easiest solution and it's what we can do, but I personally think no. 3 is the best solution, but i cannot make it happen myself.

[iredmail]

Switched to MySQL in PR moment ago.

• iRedMail installation for all 3 backends went fine.
• Roundcube works fine with all 3 backends.

@xbeaudouin Need your help:

• SOGo doesn't work due to missing MySQL support. Error in log file: sogod[637:100423] EOAdaptor: cannot find adaptor bundle: 'MySQL'.

  1. We need mysql and pgsql flavors for port devel/sope. https://github.com/iredmail/iRedMail/blob/master/functions/packages_freebsd.sh#L1025-L1029
  2. When iRedMail is installed with OpenLDAP backend, we need sope to support both OpenLDAP and MySQL.

• Dovecot package dovecot supports LDAP, but we need Dovecot to support both LDAP and MySQL when iRedMail is installed with OpenLDAP backend. Dovecot queries mail accounts from LDAP server, and stores other data in SQL db.

  • Error: dovecot[2282]: dict(2520): Error: acl: Failed to initialize dictionary 'acl': Unknown dict module: mysql
  • As discussed earlier, this could be solved by adding new flavor like -ldapmysql.

We're very close to success.

[iredmail]

@xbeaudouin @bozhinov
Any update?

[bozhinov]

Sorry, I was not aware there was something pending on me. What is it ?
Frankly, I believe Postgres is enough for the BSD folks. There probably aren't that many out there.

[iredmail]

• Mostly pending on @xbeaudouin :) We need help to improve 2 ports.
• hi @bozhinov could you help test other backends too?

[bozhinov]

Definitely. Let me know once ready for testing
I checked on the mysql option and it is incomplete.
though if it is worth something in the current state I will test it.

I m not sure we are getting 6 more packages just cause of iredmail.
I mean, the pkg build process is probably long enough as it is.

[iredmail]

I checked on the mysql option and it is incomplete.

The incomplete part is the missed package flavors (sope, dovecot).
Can we get some help from other FreeBSD ports commitors?

[xbeaudouin]

So we need:

• sope@mysql
• sope@pgsql
• sope@ldapmysql
• dovecot@ldapmysql

Is that all we need?

[iredmail]

So we need:

• sope@mysql
• sope@pgsql
• sope@ldapmysql
• dovecot@ldapmysql

Is that all we need?

Exactly. :) @xbeaudouin

[xbeaudouin]

Sope : https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=284900 https://reviews.freebsd.org/D49056

[xbeaudouin]

For dovecot, I don't think we need a ldapmysql flavor since:

OPTIONS_DEFAULT=        GSSAPI_NONE LDAP LIBWRAP

And the flavor -mysql add just MYSQL to OPTIONS_DEFAULT:

.if ${FLAVOR:U} == mysql
OPTIONS_DEFAULT+=       MYSQL
.endif

[iredmail]

Hi @xbeaudouin

Thanks for helping.

For dovecot, I don't think we need a ldapmysql flavor since:

You made my day: 660ebb9
Now all software except SOGo works in latest iRedMail git branch pkg.

About devel/sope, i found it supports ldap by default (same as Dovecot), so maybe we just need to build its mysql flavor to get both mysql and ldap support? for example sope-mysql. In this case your patch should be updated and drop ldapmysql flavor.

[xbeaudouin]

Well I have to add every flavors otherwise we can have an issue for sope. Waiting for maintainer feedback !

[iredmail]

Hi @xbeaudouin

In case you didn't notice yet, there's a reply left in review: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=284900
Could you help reply?