OTRS not handling procmail

Moderator: crythias

Post Reply
User avatar
crythias
Moderator
Posts: 9980
Joined: 04 May 2010, 18:38
OTRS Version?: 5.0.x
Location: SouthWest Florida, USA
Contact:

OTRS not handling procmail

Post by crythias » 11 Jan 2018, 17:20

Summary: .procmail on OTRS version 5.0.24 (git) isn't generating tickets.

Any Ideas? I can't seem to get this working ... it makes spool tickets and then I have to run the spool process. This isn't good because I can't check if the process spool is already running and it doubles, triples up tickets.

I'm on CentOS 7.4.1708
mariadb (not relevant)
factory (slightly modified) .procmailrc with logging enabled.

Code: Select all

# --
# Copyright (C) 2001-2017 OTRS AG, http://otrs.com/
# --
# This software comes with ABSOLUTELY NO WARRANTY. For details, see
# the enclosed file COPYING for license information (AGPL). If you
# did not receive this file, see http://www.gnu.org/licenses/agpl.txt.
# --

#SYS_HOME=/opt/otrs/
SYS_HOME=$HOME

UMASK=003
SHELL=/bin/sh
PATH=/bin:/usr/bin:/usr/local/bin
MONTHFOLDER=`date +%Y-%m`
YEARFOLDER=`date +%Y`
LOGFILE=$SYS_HOME/var/log/procmail-$MONTHFOLDER.log
VERBOSE=on


# Remove all X-OTRS Header (allow this only for trusted email)
# e. g. from *@example.com

:0 fhw :
* !^From.*@example.com
| grep -vi '^X-OTRS-'

# or just no emails with "X-OTRS-Ident: ahfiw2Fw32r230dddl2foeo3r" header!
#:0 fhw :
#* !^X-OTRS-Ident: ahfiw2Fw32r230dddl2foeo3r
#| grep -vi '^X-OTRS-'



# Examples for queue presorting.

:0 fhw :
* TO:.*@example.com
| formail -I "X-OTRS-Queue: example"

:0 fhw :
* ^List-Id:.*OpenAntiVirus
| formail -I "X-OTRS-Queue: OpenAntiVirus"


# Example for add free form key and value (max 3).

:0 fhw :
* TO:.*@example.com
| formail -I "X-OTRS-ArticleKey1: Test" |  formail -I "X-OTRS-ArticleValue1: True!"


# Example for assigning the "email domain as customer ID" automaticaly.

#:0hc
#CUSTOMERID=|formail -X "From:"|perl -e '$i=<STDIN>; $i=~s/^From: //;$i=~s/\(.*?\)//;$i=~s/<(.*?)>/$1/;$i=~s/ //g;$i=~s/.*@(.*)/$1/;print $i'
#
#:0 fhw :
#| formail -I "X-OTRS-CustomerNo: $CUSTOMERID"

# Backup of all incoming emails.
# It's always better to have a backup of all incoming emails!

:0 c :
$SYS_HOME/var/INBOX.Backup.$MONTHFOLDER


# Pipe all email into the PostMaster process.

:0 :
| $SYS_HOME/bin/otrs.Console.pl Maint::PostMaster::Read


# spool all the rest (which the otrs.PostMaster.pl can't process!)
# If the database is down or the otrs.PostMaster.pl exit was not '0'!

:0 :
$SYS_HOME/var/spool/.


# end of .procmailrc
The permissions on .procmailrc:

Code: Select all

-rw-r-----  1 otrs otrs     1859 Jan 11 08:01 .procmailrc
Error message in Syslog when messages arrive:

Code: Select all

Thu Jan 11 08:02:08 2018	error	OTRS-otrs.Console.pl-Maint::PostMaster::Read-10	mkdir /opt/otrs/var/tmp/CacheFileStorable/Valid/2: Permission denied at /opt/otrs/Kernel/System/Cache/FileStorable.pm line 87.
The folder exists:

Code: Select all

drwxrws---  3 apache apache 4096 Jan  4 07:44 2
procmail log:

Code: Select all

procmail: [12028] Thu Jan 11 08:10:38 2018
procmail: Match on ! "^From.*@example.com"
procmail: Couldn't determine implicit lockfile from "grep"
procmail: Locking ".lock"
procmail: Executing "grep,-vi,^X-OTRS-"
procmail: Unlocking ".lock"
procmail: No match on "TO:.*@example.com"
procmail: No match on "^List-Id:.*OpenAntiVirus"
procmail: No match on "TO:.*@example.com"
procmail: Locking "/opt/otrs/var/INBOX.Backup.2018-01.lock"
procmail: Assigning "LASTFOLDER=/opt/otrs/var/INBOX.Backup.2018-01"
procmail: Opening "/opt/otrs/var/INBOX.Backup.2018-01"
procmail: Acquiring kernel-lock
procmail: Unlocking "/opt/otrs/var/INBOX.Backup.2018-01.lock"
procmail: Couldn't determine implicit lockfile from "/opt/otrs/bin/otrs.Console.pl"
procmail: Locking ".lock"
procmail: Executing "/opt/otrs/bin/otrs.Console.pl,Maint::PostMaster::Read"
ERROR: OTRS-otrs.Console.pl-Maint::PostMaster::Read-10 Perl: 5.16.3 OS: linux Time: Thu Jan 11 08:10:39 2018

 Message: mkdir /opt/otrs/var/tmp/CacheFileStorable/Valid/2: Permission denied at /opt/otrs/Kernel/System/Cache/FileStorable.pm line 87.


 Traceback (12035):
   Module: Kernel::System::Console::Command::Maint::PostMaster::Read::Run Line: 138
   Module: (eval) Line: 460
   Module: Kernel::System::Console::BaseCommand::Execute Line: 454
   Module: Kernel::System::Console::InterfaceConsole::Run Line: 86
   Module: /opt/otrs/bin/otrs.Console.pl Line: 38

procmail: [12028] Thu Jan 11 08:10:39 2018
procmail: Non-zero exitcode (75) from "/opt/otrs/bin/otrs.Console.pl"
procmail: Assigning "LASTFOLDER=/opt/otrs/bin/otrs.Console.pl Maint::PostMaster::Read"
procmail: Unlocking ".lock"
procmail: Locking "/opt/otrs/var/spool/..lock"
procmail: Assigning "LASTFOLDER=/opt/otrs/var/spool/8"
procmail: Opening "/opt/otrs/var/spool/8"
procmail: Acquiring kernel-lock
procmail: Unlocking "/opt/otrs/var/spool/..lock"
procmail: Notified comsat: "otrs@0:/opt/otrs/var/spool/8"
From name.redacted@domainredacted.com  Thu Jan 11 08:10:38 2018
 Subject: RE: Subject Redacted
  Folder: /opt/otrs/var/spool/8                                           26873
I can't get procmail command

Code: Select all

| $SYS_HOME/bin/otrs.Console.pl Maint::PostMaster::Read
to debug. even if I add --debug to the entry.
OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
Please edit your signature to include your OTRS version, Operating System, and database type.
Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
Need help? Before you ask

root
Moderator
Posts: 1214
Joined: 18 Dec 2007, 12:23
OTRS Version?: 4/5/6
Real Name: Roy Kaldung
Company: Znuny Inc.
Contact:

Re: OTRS not handling procmail

Post by root » 11 Jan 2018, 17:44

Hi,

The user should be otrs with the primary group apache when using CentOS. I assume you use the RPM.

This is how it looks on my reference system, fresh installed:

Code: Select all

[root@support ~]# ls -ld /opt/otrs/.procmailrc
-rw-r-----. 1 otrs root 1892 Nov  1  2016 /opt/otrs/.procmailrc
Did you tried it after clearing the cache and running otrs.SetPermissions.pl

- Roy
OTRS 4/5/6 CentOS / RHEL / Debian / SLES / MySQL / PostgreSQL / Oracle / OpenLDAP / Active Directory / SSO

You need professional services? Check out https://www.znuny.com/

User avatar
crythias
Moderator
Posts: 9980
Joined: 04 May 2010, 18:38
OTRS Version?: 5.0.x
Location: SouthWest Florida, USA
Contact:

Re: OTRS not handling procmail

Post by crythias » 11 Jan 2018, 18:34

I didn't use RPM. I installed via source.
The only difference for your procmailrc and mine is group root. Which ... shouldn't really be a thing, but okay.

Code: Select all

> groups otrs
otrs : otrs apache
> ls -la .procmailrc
-rw-r----- 1 otrs root 1859 Jan 11 08:51 .procmailrc

Code: Select all

Locking ".lock"
procmail: Executing "/opt/otrs/bin/otrs.Console.pl,Maint::PostMaster::Read"
ERROR: Permission denied: /opt/otrs/Kernel/Config/Files/ZZZAAuto.pm
ERROR: Permission denied: /opt/otrs/Kernel/Config/Files/ZZZAuto.pm
ERROR: OTRS-otrs.Console.pl-Maint::PostMaster::Read-10 Perl: 5.16.3 OS: linux Time: Thu Jan 11 09:31:55 2018

 Message: mkdir /opt/otrs/var/tmp/CacheFileStorable/Valid/2: Permission denied at /opt/otrs/Kernel/System/Cache/FileStorable.pm line 87.


 Traceback (5174):
   Module: Kernel::System::Console::Command::Maint::PostMaster::Read::Run Line: 138
   Module: (eval) Line: 460
   Module: Kernel::System::Console::BaseCommand::Execute Line: 454
   Module: Kernel::System::Console::InterfaceConsole::Run Line: 86
   Module: /opt/otrs/bin/otrs.Console.pl Line: 38

procmail: [5167] Thu Jan 11 09:31:55 2018
procmail: Non-zero exitcode (75) from "/opt/otrs/bin/otrs.Console.pl"
I'm guessing this is because it can't make the CacheDirectory that exists but doesn't.
OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
Please edit your signature to include your OTRS version, Operating System, and database type.
Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
Need help? Before you ask

User avatar
crythias
Moderator
Posts: 9980
Joined: 04 May 2010, 18:38
OTRS Version?: 5.0.x
Location: SouthWest Florida, USA
Contact:

Re: OTRS not handling procmail

Post by crythias » 11 Jan 2018, 19:12

root wrote:Roy
Are you available to address this? I'm really in need of looking at this ... I can't seem to get Postmaster Read to debug.
OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
Please edit your signature to include your OTRS version, Operating System, and database type.
Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
Need help? Before you ask

wurzel
OTRS guru
Posts: 2440
Joined: 08 Jul 2010, 22:25
OTRS Version?: 6.0.x

Re: OTRS not handling procmail

Post by wurzel » 11 Jan 2018, 19:24

Hi,

my .procmailrc is
932287 4 -rw-r----- 1 otrs root 1913 Jan 10 09:53 .procmailrc

It seems you have a permission issue and on your /opt/otrs filesystem, too

Regards
Florian
    ((OTRS)) Community Edition 6.0.x, LAMP LIVE auf Debian 9
    OTRS 6 Managed Silver

    -- Ich beantworte keine Forums-Fragen PN - No PN please

    I won't answer to unfriendly users any more. A greeting and regards are just polite.

    User avatar
    crythias
    Moderator
    Posts: 9980
    Joined: 04 May 2010, 18:38
    OTRS Version?: 5.0.x
    Location: SouthWest Florida, USA
    Contact:

    Re: OTRS not handling procmail

    Post by crythias » 11 Jan 2018, 19:57

    wurzel wrote:It seems you have a permission issue and on your /opt/otrs filesystem, too
    But my .procmailrc runs. it's not a permission with procmailrc. I get logs. It's OTRS not doing the thing.

    I would guess so, but there's no particular difference in permissions between ticket cacheing and postmaster read cacheing.
    OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
    Please edit your signature to include your OTRS version, Operating System, and database type.
    Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
    Need help? Before you ask

    wurzel
    OTRS guru
    Posts: 2440
    Joined: 08 Jul 2010, 22:25
    OTRS Version?: 6.0.x

    Re: OTRS not handling procmail

    Post by wurzel » 11 Jan 2018, 20:18

    Hi,

    OK. Fine.

    I just wanted to mention my permissions on the file :)

    in any case... your log shows

    ERROR: Permission denied: /opt/otrs/Kernel/Config/Files/ZZZAuto.pm

    so it is actually a permission issue.
    Either your ./Files directory is not allowed to be touched by otrs user (this usually happens if you perform the Rebuild Config via root user (should not be possible in versions 5+6)

    Or your otrs user is in wrong permission group (apache/httpd/www-data/your-apache-group)
    or your apache webserver is running with wrong user/group (and writes files with wrong permissions)


    I doubt that my tipps will help you, because you ususally are more expert than me...

    Sorry that I can not help you further


    but maybe you can show your file permissions of
    /opt/otrs
    /opt/otrs/Kernel/Config/Files
    /opt/otrs/var/tmp/ + subdirectories

    and your group membership of your otrs user.

    Regards
    Florian
      ((OTRS)) Community Edition 6.0.x, LAMP LIVE auf Debian 9
      OTRS 6 Managed Silver

      -- Ich beantworte keine Forums-Fragen PN - No PN please

      I won't answer to unfriendly users any more. A greeting and regards are just polite.

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 11 Jan 2018, 21:14

      wurzel wrote:in any case... your log shows

      ERROR: Permission denied: /opt/otrs/Kernel/Config/Files/ZZZAuto.pm

      so it is actually a permission issue.
      Either your ./Files directory is not allowed to be touched by otrs user (this usually happens if you perform the Rebuild Config via root user (should not be possible in versions 5+6)
      And this is what ticks me off regarding OTRS (not you).

      Code: Select all

      [root@sp Config]# ls -lad /opt/otrs/Kernel/Config
      drwxrwsr-x 3 otrs apache 4096 Dec 22 15:22 /opt/otrs/Kernel/Config
      [root@sp Config]# ls -lad /opt/otrs/Kernel/Config/Files
      drwxrwsr-x 2 otrs apache 4096 Jan 11 09:27 /opt/otrs/Kernel/Config/Files
      [root@sp Config]# groups otrs
      otrs : otrs apache
      []ls -la /opt/otrs/Kernel/Config/Files/ZZZ*
      -rw-rw---- 1 apache apache 371535 Jan 11 09:27 /opt/otrs/Kernel/Config/Files/ZZZAAuto.pm
      -rw-rw---- 1 apache apache  10368 Jan 11 09:26 /opt/otrs/Kernel/Config/Files/ZZZAuto.pm
      
      Every time apache rewrites ZZZ*.pm it changes the owner from otrs to apache. and uses 660 permissions when I need (at least) 644

      Code: Select all

       ls -la
      total 2128
      drwxrwsr-x 2 otrs   apache   4096 Jan 11 09:27 .
      drwxrwsr-x 3 otrs   apache   4096 Dec 22 15:22 ..
      -rw-rw---- 1 otrs   apache   3795 Nov 25 15:06 CloudServices.xml
      -rw-rw---- 1 otrs   apache  37825 Nov 25 15:06 Daemon.xml
      -rw-r--r-- 1 apache apache 111045 Nov 25 15:36 FAQ.xml
      -rw-rw---- 1 otrs   apache 351711 Nov 25 15:06 Framework.xml
      -rw-r--r-- 1 apache apache   1636 Nov 25 15:42 FrameworkITSMCore.xml
      -rw-r--r-- 1 apache apache   3208 Nov 25 15:41 GeneralCatalog.xml
      -rw-rw---- 1 otrs   apache  27122 Nov 25 15:06 GenericInterface.xml
      -rw-r--r-- 1 apache apache 255612 Nov 25 15:44 ITSMChangeManagement.xml
      -rw-r--r-- 1 apache apache  51564 Nov 25 15:44 ITSMConfigItem.xml
      -rw-r--r-- 1 apache apache   5722 Nov 25 15:44 ITSMConfigItemGenericInterface.xml
      -rw-r--r-- 1 apache apache   3711 Nov 25 15:44 ITSMConfigItemTicketLink.xml
      -rw-r--r-- 1 apache apache  21863 Nov 25 15:42 ITSMCore.xml
      -rw-r--r-- 1 apache apache  11836 Nov 25 15:42 ITSMService.xml
      -rw-r--r-- 1 apache apache  40896 Nov 25 15:46 ITSMTicket.xml
      -rw-r--r-- 1 apache apache   1682 Nov 25 15:47 ImportExport.xml
      -rw-r--r-- 1 apache apache  29327 Nov 25 15:37 OTRSMasterSlave.xml
      -rw-r--r-- 1 apache apache   2380 Nov 25 15:49 PasswordPolicy.xml
      -rw-rw---- 1 otrs   apache  25448 Nov 25 15:06 ProcessManagement.xml
      -rw-r--r-- 1 apache apache   8851 Nov 25 15:38 SystemMonitoring.xml
      -rw-rw---- 1 otrs   apache 637050 Nov 25 15:06 Ticket.xml
      -rw-r--r-- 1 apache apache    675 Nov 25 15:42 TicketITSMService.xml
      -rw-r--r-- 1 apache apache   6859 Nov 25 15:46 TicketITSMTicket.xml
      -rw-r--r-- 1 apache apache  21090 Nov 25 15:38 TimeAccounting.xml
      -rw-rw---- 1 apache apache 371535 Jan 11 09:27 ZZZAAuto.pm
      -rw-rw---- 1 apache apache  10368 Jan 11 09:26 ZZZAuto.pm
      -rw-r--r-- 1 apache apache   4135 Nov 25 15:48 ZZZZZZnuny4OTRSRepo.pm
      -rw-r--r-- 1 apache apache  24785 Nov 25 15:48 Znuny4OTRSRepo.xml
      -rw-r--r-- 1 apache apache   2369 Nov 25 15:50 Znuny4OTRSShowPendingTimeIfNeeded.xml
      
      What's interesting is that the procmail dump of Backup emails is otrs/apache:

      Code: Select all

      -rw-rw-r--  1 otrs   apache  56485107 Jan 11 12:10 INBOX.Backup.2018-01
      
      procmailrc is running as otrs. It knows $HOME is /opt/otrs.
      what's strange is that even as procmail runs otrs ... stuff, otrs seems not to run as otrs user. It seems to run as something that's not otrs and not apache.
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      jojo
      Moderator
      Posts: 14289
      Joined: 26 Jan 2007, 14:50
      OTRS Version?: Git Master
      Contact:

      Re: OTRS not handling procmail

      Post by jojo » 11 Jan 2018, 22:46

      SELINUX?
      "Production": OTRS™ 6, STORM powered by OTRS
      "Testing": ((OTRS Community Edition)) git Master

      Never change Defaults.pm! :: Blog
      Professional Services:: http://www.otrs.com :: enjoy@otrs.com :: Share your ideas

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 12 Jan 2018, 01:49

      Code: Select all

      sestatus
      SELinux status:                 disabled
      
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      root
      Moderator
      Posts: 1214
      Joined: 18 Dec 2007, 12:23
      OTRS Version?: 4/5/6
      Real Name: Roy Kaldung
      Company: Znuny Inc.
      Contact:

      Re: OTRS not handling procmail

      Post by root » 12 Jan 2018, 07:13

      What't the primary group of the OTRS user? Should be apache on a CentOS.

      - Roy
      OTRS 4/5/6 CentOS / RHEL / Debian / SLES / MySQL / PostgreSQL / Oracle / OpenLDAP / Active Directory / SSO

      You need professional services? Check out https://www.znuny.com/

      seberget2
      OTRS expert
      Posts: 71
      Joined: 17 May 2013, 09:30
      OTRS Version?: 6.0.4
      Real Name: Stein Erik Berget

      Re: OTRS not handling procmail

      Post by seberget2 » 12 Jan 2018, 15:02

      crythias wrote:
      wurzel wrote:Every time apache rewrites ZZZ*.pm it changes the owner from otrs to apache. and uses 660 permissions when I need (at least) 644
      So what is your umask setting for the apache user?
      OTRS 6.0.4 on Ubuntu with MySQL DB, and various plug-ins and a hack or two :-D

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 12 Jan 2018, 16:12

      seberget2 wrote:So what is your umask setting for the apache user?

      Code: Select all

       su -s /bin/bash apache
      bash-4.2$ umask
      0022
      
      /etc/profile:

      Code: Select all

      # By default, we want umask to get set. This sets it for login shell
      # Current threshold for system reserved uid/gids is 200
      # You could check uidgid reservation validity in
      # /usr/share/doc/setup-*/uidgid file
      if [ $UID -gt 199 ] && [ "`/usr/bin/id -gn`" = "`/usr/bin/id -un`" ]; then
          umask 002
      else
          umask 022
      fi
      
      /etc/passwd

      Code: Select all

      apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
      
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      seberget2
      OTRS expert
      Posts: 71
      Joined: 17 May 2013, 09:30
      OTRS Version?: 6.0.4
      Real Name: Stein Erik Berget

      Re: OTRS not handling procmail

      Post by seberget2 » 12 Jan 2018, 17:29

      And the otrs user is member of the apache group?
      OTRS 6.0.4 on Ubuntu with MySQL DB, and various plug-ins and a hack or two :-D

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 12 Jan 2018, 18:02

      I've already posted this twice, but since it's asked again:

      Code: Select all

       groups otrs
      otrs : otrs apache
      
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 13 Jan 2018, 23:03

      removing
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 00:36

      Code: Select all

      # Pipe all email into the PostMaster process.
      
      :0 :
      | $SYS_HOME/bin/otrs.Console.pl Maint::PostMaster::Read
      
      procmail says the current running user is otrs.
      $SYS_HOME certainly resolves to /opt/otrs

      So far, I've narrowed this down to
      Kernel/System/PostMaster.pm:

      Code: Select all

              # add Dynamic Field headers
              my $DynamicFields = $DynamicFieldObject->DynamicFieldList(
                  Valid      => 1,
                  ObjectType => [ 'Ticket', 'Article' ],
                  ResultType => 'HASH',
              );
      
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 00:57

      Something more interesting:
      Changes to DynamicField.pm for debugging:

      Code: Select all

                  $Kernel::OM->Get('Kernel::System::Log')->Log(
                      Priority => 'info',
                      Message  => 'CacheGet!',
                  );
          my $Cache = $CacheObject->Get(
              Type => 'DynamicField',
              Key  => $CacheKey,
          );
      
                  $Kernel::OM->Get('Kernel::System::Log')->Log(
                      Priority => 'info',
                      Message  => 'CacheGot!',
                  );
          if ($Cache) {
      
                  $Kernel::OM->Get('Kernel::System::Log')->Log(
                      Priority => 'info',
                      Message  => 'is Cache!',
                  );
      
      If procmail does this, it doesn't Cache. When cleanup does it, it does Cache.
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 01:18

      Code: Select all

                  $SQL .= ' WHERE valid_id IN (' . join ', ', $ValidObject->ValidIDsGet() . ')';
      This is where it stops. I'm bouncing to the next place.
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 01:28

      In some places Queue.pm, it's

      Code: Select all

      " st.valid_id IN ( ${\(join ', ', $ValidObject->ValidIDsGet())} )"
      
      or Salutation.pm:

      Code: Select all

      WHERE valid_id IN ( ${\(join ', ', $Kernel::OM->Get('Kernel::System::Valid')->ValidIDsGet())} )
      in MailAccount it's

      Code: Select all

      'WHERE valid_id IN ( ' . join ', ', $ValidObject->ValidIDsGet() . ' )'
      in LinkObject it's

      Code: Select all

      my $ValidIDs = join ', ', $Kernel::OM->Get('Kernel::System::Valid')->ValidIDsGet();
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 02:29

      DynamicField doesn't want to grab ValidIDsGet (it fails on Cache-ing)
      And it's having trouble Cache-ing DynamicField Hash data

      So there's a problem where procmail doesn't try to use the cache and spooler does.
      ValidIDsGet doesn't return values from line 732 (the call fails with permissions to cache):

      Code: Select all

          else {
              my $SQL = 'SELECT id, name, field_order FROM dynamic_field';
      
              # get database object
              my $DBObject = $Kernel::OM->Get('Kernel::System::DB');
      
              if ($Valid) {
      
                  # get valid object
                  my $ValidObject = $Kernel::OM->Get('Kernel::System::Valid');
      
                  $SQL .= ' WHERE valid_id IN (' . join ', ', $ValidObject->ValidIDsGet() . ')';
      
      So, why is that ValidObject->ValidIDsGet() failing? It's exactly the same near 982.
      https://github.com/OTRS/otrs/blob/460b0 ... ld.pm#L732
      https://github.com/OTRS/otrs/blob/460b0 ... ld.pm#L982

      Somehow, only when the procmail engages, 732 kicks in and crashes. When SpoolMailsReprocess does it, 732 is ignored, and everything runs.

      Anyone have any idea?
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 17:08

      The difference between procmailrc and spool:

      When procmailrc requests a CacheKey:

      Code: Select all

      DynamicFieldList::Valid::1::ObjectType::Article_Ticket::ResultType::HASH
      When Spool reprocess requests a CacheKey:

      Code: Select all

      DynamicFieldList::Valid::1::ObjectType::Ticket::ResultType::HASH
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 17:38

      So here's the fun part:

      Both of these have permissions issues:

      Valid.pm:
      https://github.com/OTRS/otrs/blob/a79e8 ... lid.pm#L84

      Code: Select all

          while ( my @Row = $DBObject->FetchrowArray() ) {
              $Data{ $Row[0] } = $Row[1];
          }
      
          # set cache
          $Kernel::OM->Get('Kernel::System::Cache')->Set(
              Type  => $Self->{CacheType},
              TTL   => $Self->{CacheTTL},
              Key   => $CacheKey,
              Value => \%Data
          );
      DynamicField.pm:
      https://github.com/OTRS/otrs/blob/a79e8 ... ld.pm#L774

      Code: Select all

                  while ( my @Row = $DBObject->FetchrowArray() ) {
                      $Data{ $Row[0] } = $Row[1];
                  }
      
                  # set cache
                  $CacheObject->Set(
                      Type  => 'DynamicField',
                      Key   => $CacheKey,
                      Value => \%Data,
                      TTL   => $Self->{CacheTTL},
                  );
      
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 17:50

      I gotta find a different way to address. all the Cache Sets have permission issues via procmail, even though procmail is running (as far as I can tell) in the context of otrs.

      Spool reprocess is also running via crontab under the otrs user and has no problem processing. But also doesn't seem to touch cache.
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 17:53

      From SpoolMailsReprocess.pm
      # Here we use a system call because Maint::PostMaster::Read has special exception handling
      # and will die if certain problems occur.
      my $Result = system("$^X $Home/bin/otrs.Console.pl Maint::PostMaster::Read < $File ");
      Maybe this is a hint? If so, then procmailrc doesn't handle this the same way.
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 18:01

      welp, I went the wrong way I guess:
      running as otrs,
      procmail .procmailrc <test-email.box
      works exactly as designed.

      So it's a CentOS permissions issue.
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      User avatar
      crythias
      Moderator
      Posts: 9980
      Joined: 04 May 2010, 18:38
      OTRS Version?: 5.0.x
      Location: SouthWest Florida, USA
      Contact:

      Re: OTRS not handling procmail

      Post by crythias » 14 Jan 2018, 18:46

      After all this, I'm throwing my towel in the ring.

      I changed the crontab to every minute to clean up the spool and flock to not run this multiple times and duplicate tickets.

      Code: Select all

      * * * * *  /usr/bin/flock -n /tmp/noynim-otrs-mail.lockfile  $HOME/bin/otrs.Console.pl Maint::PostMaster::SpoolMailsReprocess >> /dev/null
      
      OTRS 5.0.x (private/testing/public) on Linux with MySQL database. Also on github.
      Please edit your signature to include your OTRS version, Operating System, and database type.
      Click Subscribe Topic below to get notifications. Consider amending your topic title to include [SOLVED] if it is so.
      Need help? Before you ask

      Post Reply