Saturday, March 12, 2011

If a MIME uses email does anyone read it?

The challenge a couple weeks back was to finish moving the remaining applications that are using my Exchange 2003 bridgehead servers for SMTP services to one of my message hygiene servers.
The last remaining application is in Oracle.  When pointing to the Exchange 2003 server it works.  When pointed to the Exchange 2007 server or to my Trend IMSS server it drops the first line of data,
Here’s the code they’re using.
Oracle code
l_mail_conn := utl_smtp.open_connection(l_mailhost, l_smtp_port);
-- SMTP on port 25
-- l_mailhost ex2003SRV
utl_smtp.Helo(l_mail_conn,l_mailhost);
utl_smtp.Mail(l_mail_conn,l_msg_from);
utl_smtp.Rcpt(l_mail_conn,p_msg_to);
utl_smtp.Open_Data(l_mail_conn);
utl_smtp.Write_Data(l_mail_conn, 'Date: '||to_char(sysdate,'Dy, DD Mon YYYY hh24:mi:ss')|| utl_tcp.crlf);
utl_smtp.Write_Data(l_mail_conn, 'From: Table Switching'||utl_tcp.crlf);
utl_smtp.Write_Data(l_mail_conn, 'To: '||p_msg_to||utl_tcp.crlf);
utl_smtp.Write_Data(l_mail_conn, 'Subject: ' ||p_msg_subject||utl_tcp.crlf);
utl_smtp.Write_Data(l_mail_conn, p_msg_text);
-- p_msg_text is the message
utl_smtp.Close_Data(l_mail_conn);
utl_smtp.Quit(l_mail_conn);

I was sure that this was a 7 to 8 bit MIME conversion issue.  In the olden days all mail consisted of straight ASCII characters. Email has evolved to handle attachments and rich text items such a font changes and even HTML code but SMTP still remains a 7 bit system. So email with this type of data is typically encoded in 8bitmime and then encapsulated in 7-bit packets for transmission.  The receiving server then converts it back to 7-bit format.  I first looked at this in depth when troubleshooting an issue with receiving this type of mail from a client. Trend IMSS does not aways complete the conversion process correctly with IMSS version 7.0.  Their workaround is to turn off the 8bitmime advertisement.  I’m not willing to do this because it adds about a 33% increase in overhead on the server and in the size of a mail.
Checking the IMSS server, the 2003 and 2007 servers I found that they all three advertise 8bitmime.  Since the 2003 server is advertising 8bitmime I doubt it's a cnversion issue. Looking at the code I also see that they are issuing a HELO command not an EHLO command so the ESMTP 8bitmime advertisement shouldn’t be negotiated. I need to look elsewhere.
The challenge is that I need to see the raw mail to understand why that line is dropped.  If this was an HTML formatted mail I could view the source, but this is text.  Outlook doesn’t provide a method for view a raw message, just the internet headers.
I need to grab the mail before Outlook does any conversion. I decided to try MFCMAPI.  This allows a direct MAPI connection to a mailbox without using Outlook.
 Raw Mail
Looking at the raw mail with MFCMAPI the body of the corrupt mail looks like this:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 08.01.0240.003">
<TITLE>Table Switching </TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<BR>

<P><FONT SIZE=2>23-FEB-2011 04:26:04</FONT>
</P>

</BODY>
</HTML>

The body of the correctly displayed mail looks like this:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 08.01.0240.003">
<TITLE>Table Switching</TITLE>
</HEAD>
<BODY>        
<!-- Converted from text/plain format -->

<P><FONT SIZE=2>23-FEB-2011 04:40:43 - Database: PRACDSD.KCI</FONT>

<BR><FONT SIZE=2>SUCCESS - DL_ZPD48943DM9403.RPT_PATIENT </FONT>
</P>

<P><FONT SIZE=2>23-FEB-2011 04:40:47</FONT>
</P>

</BODY>
</HTML>

I’ve highlighted the problem in red.  MFCMAPI is a thin client so this is displayed in an HTML rendering. <BR> is an HTML equivalent of ASCII CHAR 10 13 (carriage return line feed).  According to RFC 2045 8bit email transfer supports "up to 998 octets per line with CR and LF (codes 13 and 10 respectively)" but this is only allowed to appear as part of a CRLF line ending.
What’s happening is the mail servers are interpreting the CHAR 10 13 as the end of the line and dropping the remaining characters. So why does it work when the mail is sent to the Exchange 2003 server?  It's been my experience that the Exchange team sometimes plays a little fast and loose with their compliance to the current RFC standards.  In this case we were able to get away with being a little sloppy with our coding until the compliance was tightened in a newer version of Exchange. Changing the placement of the CHAR 10 13 resolved the issue.
Here are some links to more detailed MIME conversion and ESMTP articles.
Content conversation2010 –
http://technet.microsoft.com/en-us/library/bb232174.aspx

No comments:

Post a Comment