OTRS6 unable to fetch/process "larger" e-mail messages

Moderator: crythias

Post Reply
frennkie
Znuny newbie
Posts: 9
Joined: 20 Jan 2014, 16:05
Znuny Version: 6.0.7

OTRS6 unable to fetch/process "larger" e-mail messages

Post by frennkie »

I recently update from OTRS 5 to OTRS 6.0.7.
I now have to (major) problem that message that are bigger than a few KB are no longer successfully fetched.
This blocks the processing of all (incoming) messages!

When I send a text only message it's working fine.
A message with a small attached (e.g. 40kb png) it works.
But something like a 32kb file breaks it.

I tried to find a possible cause for this and ran strace (as I didn't find anything helpful in the logs )

strace ./bin/otrs.Console.pl Maint::PostMaster::MailAccountFetch --debug --force-pid

Code: Select all

[...]
 read(4, "", 8192)                       = 0
close(4)                                = 0
rt_sigaction(SIGALRM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
rt_sigaction(SIGALRM, {0x4a13b0, [], SA_RESTORER, 0x7f83fd7dc390}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
alarm(600)                              = 0
sendto(3, "\1\0\0\0\1", 5, 0, NULL, 0)  = 5
shutdown(3, SHUT_RDWR)                  = 0
close(3)                                = 0
write(1, "\n\33[33mSpawning child process to "..., 83
Spawning child process to fetch incoming messages from mail accounts...

) = 83
lseek(10, 184005, SEEK_SET)             = 184005
lseek(10, 0, SEEK_CUR)                  = 184005
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f83fe1019d0) = 4760
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({0, 100000000}, mail.example.com (POP3)...
POP3: Message 1/1 (my_user/mail.example.com)
NULL)         = 0
wait4(4760, 0x7ffc924c5dc4, WNOHANG, NULL) = 0
nanosleep({0, 100000000}, NULL)         = 0
wait4(4760, 0x7ffc924c5dc4, WNOHANG, NULL) = 0
nanosleep({0, 100000000}, NULL)         = 0
wait4(4760, 0x7ffc924c5dc4, WNOHANG, NULL) = 0
nanosleep({0, 100000000}, NULL)         = 0
wait4(4760, 0x7ffc924c5dc4, WNOHANG, NULL) = 0
nanosleep({0, 100000000}, NULL)         = 0
wait4(4760, 0x7ffc924c5dc4, WNOHANG, NULL) = 0
[...] (endless)

This wait4/nanosleep loop goes on forever (I killed this after 10 minutes.. but it was hanging for +24h earlier this week).

I already tried switching from POP3 to IMAP (and back) -> no change.

Any suggestions would be highly appreciated..! :-)

Mail Server: Dovecot 2.1

OTRS System: Ubuntu 16.04.4
Perl: v5.22.1

Code: Select all

$ ./bin/otrs.CheckModules.pl
z  o Apache::DBI......................ok (v1.12)
  o Apache2::Reload..................ok (v0.13)
  o Archive::Tar.....................ok (v2.04)
  o Archive::Zip.....................ok (v1.56)
  o Crypt::Eksblowfish::Bcrypt.......ok (v0.009)
  o Crypt::SSLeay....................ok (v0.73_04)
  o Date::Format.....................ok (v2.24)
  o DateTime.........................ok (v1.21)
  o DBI..............................ok (v1.634)
  o DBD::mysql.......................ok (v4.033)
  o DBD::ODBC........................ok (v1.52)
  o DBD::Oracle......................Not installed! (optional - Required to connect to a Oracle database.)
  o DBD::Pg..........................ok (v3.5.3)
  o Digest::SHA......................ok (v5.95)
  o Encode::HanExtra.................ok (v0.23)
  o IO::Socket::SSL..................ok (v2.024)
  o JSON::XS.........................ok (v3.01)
  o List::Util::XS...................ok (v1.41)
  o LWP::UserAgent...................ok (v6.26)
  o Mail::IMAPClient.................ok (v3.38)
    o IO::Socket::SSL................ok (v2.024)
    o Authen::SASL...................ok (v2.16)
    o Authen::NTLM...................Not installed! Use: 'apt-get install -y libauthen-ntlm-perl' (optional - Required for NTLM authentication mechanism in IMAP connections.)
  o ModPerl::Util....................ok (v2.000009)
  o Net::DNS.........................ok (v0.81)
  o Net::LDAP........................ok (v0.65)
  o Template.........................ok (v2.24)
  o Template::Stash::XS..............ok (undef)
  o Text::CSV_XS.....................ok (v1.21)
  o Time::HiRes......................ok (v1.9726)
  o XML::LibXML......................ok (v2.0123)
  o XML::LibXSLT.....................ok (v1.94)
  o XML::Parser......................ok (v2.44)
  o YAML::XS.........................ok (v0.41)
Last edited by frennkie on 19 May 2018, 10:34, edited 3 times in total.
fcasal
Znuny wizard
Posts: 336
Joined: 21 Apr 2014, 16:14
Znuny Version: 6.0.18

Re: OTRS6 unable to fetch/process "larger" e-mail messages

Post by fcasal »

If you try to upload an attachment manually, it works? Because it could be database configuration
frennkie
Znuny newbie
Posts: 9
Joined: 20 Jan 2014, 16:05
Znuny Version: 6.0.7

Re: OTRS6 unable to fetch/process "larger" e-mail messages

Post by frennkie »

If you try to upload an attachment manually, it works? Because it could be database configuration
You mean e.g. in a note? Just tested and works fine.

I kind of get the feeling that the PERL IMAP/POP library might be part of the problem..
frennkie
Znuny newbie
Posts: 9
Joined: 20 Jan 2014, 16:05
Znuny Version: 6.0.7

Re: OTRS6 unable to fetch/process "larger" e-mail messages

Post by frennkie »

Damn.. it's not directly related to the size.. It just successfully send in a message with an attached 1,4 MB PNG and another one with a 1,7 MB PDF.

So it affects messages from a certain customer (they use Outlook) and I assume it's something with the (MIME?!) format. Maybe their embedded images in the signature.. any hints on how to debug the Perl mail parser?
Last edited by frennkie on 21 May 2018, 11:06, edited 1 time in total.
fcasal
Znuny wizard
Posts: 336
Joined: 21 Apr 2014, 16:14
Znuny Version: 6.0.18

Re: OTRS6 unable to fetch/process "larger" e-mail messages

Post by fcasal »

In OTRS 6 you have the communication logs, which is a very useful utility to find the problem. Copy and paste that log.
frennkie
Znuny newbie
Posts: 9
Joined: 20 Jan 2014, 16:05
Znuny Version: 6.0.7

Re: OTRS6 unable to fetch/process "larger" e-mail messages

Post by frennkie »

Attached are screenshot from the Communication Log.

The last entry before it "hangs" is "Checking if is a Bounce e-mail."
Therefore I went into System Configuration and disabled "Kernel::System::PostMaster::Filter::DetectBounceEmail".
Now it's working..! :-D

So I guess this is either something weird in the mail setup (either on my system or on the customer side) or it's a bug in OTRS.

Thanks a lot anyway!
You do not have the required permissions to view the files attached to this post.
frennkie
Znuny newbie
Posts: 9
Joined: 20 Jan 2014, 16:05
Znuny Version: 6.0.7

Re: OTRS6 unable to fetch/process "larger" e-mail messages

Post by frennkie »

The headers "Received" and "Message-ID" both contain the FQDN of the sending mail server.
This FQDN is "<something>.<something else>.local (I replaced the actual names).
Could this ".local" be the part where the bounce module fails to parse it correctly?
frennkie
Znuny newbie
Posts: 9
Joined: 20 Jan 2014, 16:05
Znuny Version: 6.0.7

Re: OTRS6 unable to fetch/process "larger" e-mail messages

Post by frennkie »

Post Reply