Troubleshooting IMAP

Troubleshooting IMAP.

Unless you’re working at an ISP chances are you may never use IMAP if you’re a straight MS shop since outlook uses MAPI for its connections.  However, if you want to connect an application to your mail environment then you’ll probably be using IMAP.  This is because IMAP connections provide flexibility for connecting to subfolders, monitoring for new message arrival and downloading the messages that is not available in a POP connection. 

Enabling IMAP in Exchange 2007 is very straightforward.  One thing to keep in mind is that the Exchange IMAP service works just like the Exchange Transport Service Any changes to the configuration will require that the service be restarted.

My goal last week was to get our Siebel system connected to a mailbox on my Exchange 2007 server.  Sounds like a simple process.  I have another application using IMAP to import mail into an archive system. It works just fine.  So far Siebel does not play well with others. So I needed to convince myself and my consultants that the problems were specific to the application and not IMAP

Troubleshooting IMAP begins with a telnet session just like troubleshooting SMTP.  However, the similarities stop there.  If you have TLS enforced for IMAP then you’ll need to telnet to port 993. For non-encrypted connections port 143 is used.

This is really important - Just connecting is not enough.  You’re connecting to a mailbox, not relaying anonymous mail.  Some type of authentication is required so you must at least test that part of the process. Then, once you’ve authenticated, you’ll want to verify that you can see the folders in the mailbox. Often the application process will be able to connect but cannot authenticate or cannot list the items in the mailbox. Your job is to follow the same process the application is using to attempt to duplicate the problem.

You also need to know that IMAP supports multiple connections so each command must start with a unique connection tag.  It can be something as simple as a period or you can get fancy and use an alpha-numeric tag like A01. I won’t go into the details of the commands.  Rather I’ll give you a link to one of the best pages I’ve found explaining IMAP, here. He also covers POP. You’ll want to review it too since there’s a good chance the application will want to send mail using POP.

At a minimum I’d suggest the following:

telnet <server> 143

. login mailbox@domain.com password

. list "" "*"

. status INBOX (messages)

These commands will test the basic connectivity.  Again, I suggest that you duplicate the process your application is uses using the commands to fetch a message and read it.

If a MIME uses email does anyone read it?

The challenge a couple weeks back was to finish moving the remaining applications that are using my Exchange 2003 bridgehead servers for SMTP services to one of my message hygiene servers.

The last remaining application is in Oracle.  When pointing to the Exchange 2003 server it works.  When pointed to the Exchange 2007 server or to my Trend IMSS server it drops the first line of data,

Here’s the code they’re using.

Oracle code

l_mail_conn := utl_smtp.open_connection(l_mailhost, l_smtp_port);

-- SMTP on port 25

-- l_mailhost ex2003SRV

utl_smtp.Helo(l_mail_conn,l_mailhost);

utl_smtp.Mail(l_mail_conn,l_msg_from);

utl_smtp.Rcpt(l_mail_conn,p_msg_to);

utl_smtp.Open_Data(l_mail_conn);

utl_smtp.Write_Data(l_mail_conn, 'Date: '||to_char(sysdate,'Dy, DD Mon YYYY hh24:mi:ss')|| utl_tcp.crlf);

utl_smtp.Write_Data(l_mail_conn, 'From: Table Switching'||utl_tcp.crlf);

utl_smtp.Write_Data(l_mail_conn, 'To: '||p_msg_to||utl_tcp.crlf);

utl_smtp.Write_Data(l_mail_conn, 'Subject: ' ||p_msg_subject||utl_tcp.crlf);

utl_smtp.Write_Data(l_mail_conn, p_msg_text);

-- p_msg_text is the message

utl_smtp.Close_Data(l_mail_conn);

utl_smtp.Quit(l_mail_conn);

I was sure that this was a 7 to 8 bit MIME conversion issue.  In the olden days all mail consisted of straight ASCII characters. Email has evolved to handle attachments and rich text items such a font changes and even HTML code but SMTP still remains a 7 bit system. So email with this type of data is typically encoded in 8bitmime and then encapsulated in 7-bit packets for transmission.  The receiving server then converts it back to 7-bit format.  I first looked at this in depth when troubleshooting an issue with receiving this type of mail from a client. Trend IMSS does not aways complete the conversion process correctly with IMSS version 7.0.  Their workaround is to turn off the 8bitmime advertisement.  I’m not willing to do this because it adds about a 33% increase in overhead on the server and in the size of a mail.

Checking the IMSS server, the 2003 and 2007 servers I found that they all three advertise 8bitmime.  Since the 2003 server is advertising 8bitmime I doubt it's a cnversion issue. Looking at the code I also see that they are issuing a HELO command not an EHLO command so the ESMTP 8bitmime advertisement shouldn’t be negotiated. I need to look elsewhere.

The challenge is that I need to see the raw mail to understand why that line is dropped.  If this was an HTML formatted mail I could view the source, but this is text.  Outlook doesn’t provide a method for view a raw message, just the internet headers.

I need to grab the mail before Outlook does any conversion. I decided to try MFCMAPI.  This allows a direct MAPI connection to a mailbox without using Outlook.

 Raw Mail

Looking at the raw mail with MFCMAPI the body of the corrupt mail looks like this:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">

<HTML>

<HEAD>

<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<META NAME="Generator" CONTENT="MS Exchange Server version 08.01.0240.003">

<TITLE>Table Switching </TITLE>

</HEAD>

<BODY>

<!-- Converted from text/plain format -->

<BR>

<P><FONT SIZE=2>23-FEB-2011 04:26:04</FONT>

</P>

</BODY>

</HTML>

The body of the correctly displayed mail looks like this:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">

<HTML>

<HEAD>

<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<META NAME="Generator" CONTENT="MS Exchange Server version 08.01.0240.003">

<TITLE>Table Switching</TITLE>

</HEAD>

<BODY>        

<!-- Converted from text/plain format -->

<P><FONT SIZE=2>23-FEB-2011 04:40:43 - Database: PRACDSD.KCI</FONT>

<BR><FONT SIZE=2>SUCCESS - DL_ZPD48943DM9403.RPT_PATIENT </FONT>

</P>

<P><FONT SIZE=2>23-FEB-2011 04:40:47</FONT>

</P>

</BODY>

</HTML>

I’ve highlighted the problem in red.  MFCMAPI is a thin client so this is displayed in an HTML rendering. <BR> is an HTML equivalent of ASCII CHAR 10 13 (carriage return line feed).  According to RFC 2045 8bit email transfer supports "up to 998 octets per line with CR and LF (codes 13 and 10 respectively)" but this is only allowed to appear as part of a CRLF line ending.

What’s happening is the mail servers are interpreting the CHAR 10 13 as the end of the line and dropping the remaining characters. So why does it work when the mail is sent to the Exchange 2003 server?  It's been my experience that the Exchange team sometimes plays a little fast and loose with their compliance to the current RFC standards.  In this case we were able to get away with being a little sloppy with our coding until the compliance was tightened in a newer version of Exchange. Changing the placement of the CHAR 10 13 resolved the issue.

Here are some links to more detailed MIME conversion and ESMTP articles.

http://searchexchange.techtarget.com/tutorial/Part-3-How-Extended-SMTP-works-and-common-ESMTP-commands

http://technet.microsoft.com/en-us/library/dd535395%28EXCHG.80%29.aspx

http://download.oracle.com/docs/cd/E19636-01/819-2650/bgamw/index.html

http://technet.microsoft.com/en-us/library/bb232174.aspx

http://support.microsoft.com/?id=323489

http://support.microsoft.com/?id=836555

http://en.wikipedia.org/wiki/MIME

http://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol

Content conversation2010 –

http://technet.microsoft.com/en-us/library/bb232174.aspx