iy3xk ftc9ky RLamb

The problem on Windows Outlook

Setup address book (no GAL). e.g., ldapx.ends2ends.com [2]

Try to send encrypted email to someone in the address book that you do not have keys for already. e.g., natsec@dc.org (me)

After Outlook connects to ldap server and requests and gets userSMIMECertificate;binary attribute, it fails

Above works for userCertificate;binary attribute if intermiedate certs are pre-installed (follow AIA cert chain) or not used (eg 9Folders email client, etc..). Also works for Outlook for Mac address book.

Solution (thanks to Gary Nebbett)

Many thanks to Gary Nebbett for taking pity on me and lending his considerable experience tracking this down. Also many thanks to Metin Savignano for the weeks he also spent helping me on this problem and trying different configurations. Without them, I would have continued to bang my head on the wall for another few years with no progress. I hope Microsoft fixes this Windows Outlook bug. If they do, s/mime (+public ldap servers, e.g.,[2], like pgp key servers) can become more ubiquitous than pgp...and CA authenticated.

Outlook EMABLT32.DLL bug

Bug in Outlook (Office15, Office16, Microsoft365) EMABLT32.DLL incorrectly looks for "userSMIMECertificate" instead of "userSMIMECertificate;binary" in LDAP responses. 2-byte patch developed by Gary Nebbett (and Ghidra rev engineering by me) changes string pointer to correct ";binary" version. Here are the patches to EMABLT32.DLL. Compare ("cmp -lb") with your original to make sure you are changing the right binary. BEWARE:I noticed windows seems to put the old one back sometimes. Took a stab at a batch script to do this in Notes below.

Microsoft 365 (Office16) (129416 bytes sha256..77bb81a3e C:\Program Files (x86)\Microsoft Office\root\Office16\EMABLT32.DLL )
Patched sha256..2b62c7c751011

$ cmp -lb original patched
 12419  70 8     14 ^L
 12420 150 h    154 l
Hex:
00003083 38 0C
00003084 68 6C
Office16 (165272 bytes sha256..47561d10dcbce C:\Program Files\Microsoft Office\root\Office16\EMABLT32.DLL )
Patched sha256..322292f932
$ cmp -lb original patched
 16586 173 {    313 M-K
 16587 174 |    200 M-^@
Hex:
000040CA 7B CB
000040CB 7C 80
Office15 (125728 bytes sha256..f2b158f8f4db C:\Program Files\Microsoft Office 15\root\office15\EMABLT32.DLL )
Patched sha256..f310013071
$ cmp -lb original patched
 15857 200 M-^@ 130 X
 15858  26 ^V    32 ^Z
Hex:
00003DF1 80 58
00003DF2 16 1A

Now I finally have an answer to the question:

"What exactly is the contents of the LDAP userSMIMECertificate attribute" (see
stackexchange and doc.windows.com)
openssl smime -sign -in empty.txt -text -outform der -out userSMIMECertificate.der -nodetach -signer mycert.pem -inkey mykey.pem -certfile intermediatecerts.pem
The contents of userSMIMECertificate.der is what goes on the wire (in the OCTET). Sample pcap dump is here. Note that this is slightly different from what the RFC says and matches MS's own documentation here.

Alternative Fix

An alternative solution that un-does Outlook disabling AIA chaining like what browsers do anyway. Kinda strange for MSFT to shoot itself in the foot. This is also thanks to Gary's keen sense and eye. Here and here are the links. e.g. for office16
[HKEY_CURRENT_USER\software\policies\microsoft\office\16.0\outlook\security]
"enableaiacertextension"=dword:00000001
Please read caveats on those links regarding security. Batch regedit file here (rename to outlookaiaregfix.bat to run).

Making it all work.

If you are reading this, I assume you have a working s/mime setup and certificate already. FYI: Actalis.it still offers free s/mime certificates.
  1. Either 1) apply the 2-byte patch to your EMABLT32.DLL to see userSMIMECertificate;binary; or 2) allow Outlook to chain (follow AIA path) from the cert in userCertificate;binary by making the Registry fix.
  2. Start Outlook
  3. Add your ldap server to the Outlook Address Books. You can use ldapx.ends2ends.com[2] (no SSL and default Base) for testing.
  4. Restart Outlook
  5. If you installed the patch correctly or alternate fix, you should be able to send an encrypted email to natsec@dc.org (me) or checkme@ends2ends.com. You should also be able to send encrypted emails to anyone using [2] or whatever ldap/AD server you have set.
Notes: Refs:
May 2021 Rick Lamb