Score:0

procmail's logging helped me get one account's .procmailrc working, but two other accounts don't work and don't log); having difficulty diagnosing

ru flag

As a site that has a number of users who are "technologically challenged", and as the system mangler, it's fallen on me to solve the spam problem for us all. So, SpamAssassin's doing a fine job of identifying the spam, now it's just a matter of moving it into a directory where people can find it and identify the ham now and then - as the SA people call it!

I'd been using an IMAP based reader for that, but this needs a more systemic approach, and the SA people recommended I use procmail, which had lain fallow for a long time but has recently been revived by the original author, Stephen R. van den Berg. Thus, bugs fixed in the modern procmail v 3.22, ONWARD! (Tripleee noted that this is still the OLD procmail - the new one comes out with "rawhide," Fedora 38, which is in testing now - we're using Fedora Server 37, updated in January, with Postfix 3.7, and SpamAssassin 3.4 - more on that if anyone cares).

I figured I'd be the test subject, and I have quite a few accounts for various reasons, as system's staff often do. My first one I chose because it's seldom got inbound mail, and so if it goes wrong, it's unlikely to be an issue. And, I had trouble; I got crickets.

When I added to ~/.procmailrc the logging entries (seen below), I got the data that pointed me along, and when I added what were supposed to be some default environment variable definitions, most notably defining MAILDIR, it perked right up! It was ALREADY working fine! YAY! To help others behind me, KEY were these settings:

USER=<the_username>
LOGFILE=/var/log/procmail.$USER
VERBOSE='yes'
LOGABSTRACT='all'

This let me find the issues - there weren't many, but a few - and solve them. While other variables helped, the critical one was setting MAILDIR, which, only later did I discover would be set wrong by default for most / many Fedora systems. ... You CAN just "hard code" the file paths to the "folders", but this helped a lot:

MAILDIR=$HOME/mail

I have just ONE recipe:

:0
* ^X-Spam-Status: *Yes
Spam

The plan is that 'Spam' is where 'known spam' goes, as identified by SA, but it could be wrong. Spam that SA doesn't catch I want users to put in their 'spam' folder (note the case difference) for use in training SA. But, I digress.

But when I copied and re-chowned this simple config to a more used, and vital, account, I can see in /var/mail/maillog that it's calling the (also identical content) .forward script I'm using, and so I know procmail (the command) is getting called. But from there, NOTHING is going to the configured LOGFILE. And, it's worth pointing out that that's similar to what happened with the first account, which I solved by using the logging (in retrospect, it was only the wrong MAILDIR entry). But here, the logging isn't getting any content into it!

So, I tried another one - same thing. And, of course, I've checked file permissions and so forth... I must be "overlooking something", but, well, it was clear that the log writing was (and remains) setup just fine for the first account, so I've focused on that, and it seems absolutely "stellar" - I'd take just getting that far (logging working) on these other two accounts as a win!

(I even tried putting execute privs on .forward, but of course it wasn't needed before and didn't help now!)

Thanks.

Additional info:

It may be helpful to others to know that if you create a log file in /var/log, say, /var/log/procmail, and set the log env to point there, ALL users can use it if the permissions on the file are simply 777. No directory tree issues were found.

Also, while procmail is indeed called from postfix "without additional help," experimentation showed that the entry in .forward is required on this Fedora system - exactly how the .forward is called / read I haven't had time to fully investigate.

in flag
Permissions on the log file? Is the system user allowed to create files in /var/log?
ru flag
@GeraldSchneider I thought of that... I created the logs with the root account and set the permissions to 777. ...However, it occurred to me that MAYBE I should try putting it in the user's tree. BUT, that wasn't needed for the first account, and, ironically, one of the two others not only has SUDU privs but was the origin account of the OS when it was first installed... It's enough to make you go Hmmmm!
tripleee avatar
ar flag
Uh, 3.22 is still the old buggy release. If you want to play it safe, wait until 3.24 is officially available for your platform.
tripleee avatar
ar flag
My suspicion would be that Procmail refuses to run because of a permission or ownership problem. You should receive a bounce message, and your Postfix log should contain a report about the error.
tripleee avatar
ar flag
You should probably not be overriding `USER`; and indeed, the best practice is probably to store the logs in the user's home directory. Once things are working properly, maybe set up some sort of log rotation regime for the accumulating log files.
tripleee avatar
ar flag
With Postfix, you should not need an explicit `.forward`; it should be configured out of the box to run Procmail and look for a `.procmailrc` even without one.
ru flag
@tripleee Thanks, triplee. I don't usually get up this early, but feel compelled to look into this all bleary-eyed, BC - before coffee! ... Postfix may well be running procmail 'without help'; early testing suggested it wasn't but now that I have ONE account working, I have a "test bed." As for USER, it was a last-ditch effort to get it to do SOMETHING! ;-) ...And maybe I can build the 3.24 on the box? Hmmm... Would be nice to have some idea of the timing of its RPM release for Fedora 37...
tripleee avatar
ar flag
Sorry, not much of a Fedora person. [Rawhide seems to have it](https://fedora.pkgs.org/rawhide/fedora-x86_64/procmail-3.24-1.fc38.x86_64.rpm.html) but I have no idea how complex it would be to get it from there to a regular Fedora.
ru flag
@tripleee Thanks again... Some results: 1) I had already removed the explicit call to procmail from Postfix - it didn't have it from the original distribution, I tried it, not needed. And 2) When I moved the working .forward to .forward_works, the movement of spam stopped and the mail just went to the inbox. And when I restored it, it resumed going to the desired "folder." ... SO... the .forward entry is required, though I do understand I can do this in, say, the cram file or maybe in aliases to do it system-wide, however, that removes the autonomy for personal forwarding.
tripleee avatar
ar flag
If you don't manage to resolve with the information you have in the comments so far it might be worth exploring how your Postfix config differs from the standard distribution, and perhaps [edit] to include a small passage about how exactly Postfix delivery is configured. But I would expect the Postfix log to be your first stop and possibly sufficient for figuring out exactly what's wrong.
Score:0
ru flag

I would imagine that systems that don't have an email server being banged on so hard that the rate of new lines being added to the mail server's log is "a waterfall" would have an easier time finding this problem; because our system's logging rate is often over 100 lines a second (sometimes WAY higher), if you don't have something to search for - via a tail -f /var/log/maillog | grep "some search text" - you haven't a prayer of finding anything.

But, I gave it a shot, with ONE working account to go by. And, from that I found a procmail[7965]: Suspicious rcfile entry. OK, something to bite into!

Pursuit showed no obvious errors, BUT, well, the parent directory correctly has a different group entry than the username - different because of VERY legitimate reasons. Changing the group would have unknowable consequences without substantial investigation.

None of the usual advice applies, such as chmod 0640, so, I dug deeper.

I found this tidbit hiding somewhere:

Procmail usually gives this error if: [...] The directory is both world-writable and world executable but not ‘sticky’ (mode +t).

And, so, a simple chmod +t on the parent directory did the trick!

ru flag
@tripleee Thanks again, triplee, your encouragement pushed me forward this morning, and, as you can see, I found it. ... I had NOT suspected the parent directory and, frankly, I think procmail is being a bit too cautious as it wasn't world writeable. A simple check on the user being in the group would have been nice! Or, a config option root sets maybe. Oh well! At least it was a simple enough solution! ;-) OH, BY THE WAY, my "over-use" of "code" is a personal protest against having more means of emphasis, like underscore, that visually call something out without yelling!
tripleee avatar
ar flag
There is a build-time option to let Procmail assume the group is valid, but apparently the Fedora build didn't specify this. See e.g. https://www.iki.fi/era/procmail/mini-faq#group-writable
tripleee avatar
ar flag
You can use `**bold**` (which renders as **bold**) and `*italics*` (which renders as *italics*) in Markdown for emphasis.
ru flag
@tripleee thanks again! (BTW, took many of your edit suggestions, though not all.) ... I'd imagine I ought download it with the source and turn that on as I really didn't like having to add the sticky bit! ... I just added a bit of what was learned to the question so others may be spared the testing. ;-) Oh, and italics and bolding?! Italics here are close to useless as they just make it harder to read and aren't pulling your eyes to them, visually, while bolding has its place, but other than titles is akin to yelling! HEY KID, GET OFF MY LAWN! :-O
tripleee avatar
ar flag
Why does _SpamAssassin_ or _spam_ or _ham_ need emphazis, though? **Holy Batman,** your question reads a bit like a **comic book speech bubble!**
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.