The Linux Electronic Mail HOWTO Vince Skahan, v1.12, 29 November 1995 This document describes the setup and care+feeding of Electronic Mail (e-mail) under Linux. You need to read this if you plan to communi­ cate locally or to remote sites via electronic mail. You probably do *not* need to read this document if don't exchange electronic mail with other users on your system or with other sites. 1. Introduction The intent of this document is to answer some of the questions and comments that appear to meet the definition of 'frequently asked questions' about e-mail software under Linux. This document and the corresponding UUCP and News 'HOWTO' documents collectively supersede the UUCP-NEWS-MAIL-FAQ that has previously been posted to comp.os.linux.announce. 1.1. New versions of this document New versions of this document will be periodically posted to comp.os.linux.announce, comp.answers, and news.answers. They will also be added to the various anonymous ftp sites who archive such information including sunsite.unc.edu:/pub/Linux/docs/HOWTO. In addition, you should be generally able to find this document on the Linux WorldWideWeb home page at http://sunsite.unc.edu/mdw/linux.html. 1.2. Feedback I am interested in any feedback, positive or negative, regarding the content of this document via e-mail. Definitely contact me if you find errors or obvious omissions. I read, but do not necessarily respond to, all e-mail I receive. Requests for enhancements will be considered and acted upon based on that day's combination of available time, merit of the request, and daily blood pressure :-) Flames will quietly go to /dev/null so don't bother. In particular, the Linux filesystem standard for pathnames is an evolving thing. What's in this document is there for illustration only based on the current standard at the time that part of the document was written and in the paths used in the distributions or 'kits' I've personally seen. Please consult your particular Linux distribution(s) for the paths they use. Feedback concerning the actual format of the document should go to the HOWTO coordinator - Greg Hankins (gregh@sunsite.unc.edu). 1.3. Copyright Information The Mail-HOWTO is copyrighted (c)1994 Vince Skahan. A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it. Short quotes may be used without prior consent by the author. Derivative work and partial distributions of the Mail-HOWTO must be accompanied with either a verbatim copy of this file or a pointer to the verbatim copy. Commercial redistribution is allowed and encouraged; however, the author would appreciate being notified of any such distributions (as a courtesy). In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents. We further want that ALL information provided in the HOWTOS is disseminated. If you have questions, please contact Greg Hankins, the Linux HOWTO coordinator, at gregh@sunsite.unc.edu. 1.4. Standard Disclaimer Of course, I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk. 1.5. Other sources of information 1.5.1. LINUX HOWTO Documents: There is plenty of exceptional material provided in the other Linux HOWTO documents and from the Linux DOC project. In particular, you might want to take a look at the following: · the Serial Communications HOWTO · the Ethernet HOWTO · the Linux Networking Administrators' Guide 1.5.2. USENET: comp.mail.elm the ELM mail system. comp.mail.mh The Rand Message Handling system. comp.mail.mime Multipurpose Internet Mail Extensions. comp.mail.misc General discussions about computer mail. comp.mail.multi-media Multimedia Mail. comp.mail.mush The Mail User's Shell (MUSH). comp.mail.sendmail the BSD sendmail agent. comp.mail.smail the smail mail agent. comp.mail.uucp Mail in the uucp environment. 1.5.3. Books The following is a non-inclusive set of books that will help... · "Managing UUCP and USENET" from O'Reilly and Associates is in my opinion the best book out there for figuring out the programs and protocols involved in being a USENET site. · "Unix Communications" from The Waite Group contains a nice description of all the pieces (and more) and how they fit together. · "Sendmail" from O'Reilly and Associates looks to be the definitive reference on sendmail-v8 and sendmail+IDA. It's a "must have" for anybody hoping to make sense out of sendmail without bleeding in the process. · "The Internet Complete Reference" from Osborne is a fine reference book that explains the various services available on Internet and is a great source for information on news, mail, and various other Internet resources. · "The Linux Networking Administrators' Guide" from Olaf Kirch of the Linux DOC Project is available on the net and is also published by (at least) O'Reilly and SSC. It makes a fine one-stop shopping to learn about everything you ever imagined you'd need to know about Unix networking. Shameless plug mode ON - the sendmail+IDA descriptions below have been very much expanded and more fully explained in Chapter 15 of the Linux Networking Administrators' Guide. I strongly recommend you grab a copy and read it. 1.5.4. Periodic USENET Postings Also worth mentioning, is Chris Lewis' periodic posting on unix e-mail software, which is available on rtfm.mit.edu in /pub/usenet/comp.mail.misc/UNIX_Email_Software_Survey_* 1.6. Where *NOT* to look for help There is nothing "special" about configuring and running mail under Linux (any more). Accordingly, you almost certainly do *NOT* want to be posting generic mail-related questions to the comp.os.linux.* newsgroups. Unless your posting is truly Linux-specific (ie, "please tell me what routers are already compiled into the SLS1.03 version of smail3.1.28") you should be asking your questions in one of the newsgroups or mailing lists referenced above. Let me repeat that. There is virtually no reason to post anything mail-related in the comp.os.linux hierarchy any more. There are existing newsgroups in the comp.mail.* hierarchy to handle *ALL* your questions. IF YOU POST TO COMP.OS.LINUX.* FOR NON-LINUX-SPECIFIC QUESTIONS, YOU ARE LOOKING IN THE WRONG PLACE FOR HELP. THE ELECTRONIC MAIL EXPERTS HANG OUT IN THE PLACES INDICATED ABOVE AND GENERALLY DO NOT RUN LINUX. POSTING TO THE LINUX HIERARCHY FOR NON-LINUX-SPECIFIC QUESTIONS WASTES YOUR TIME AND EVERYBODY ELSE'S...AND IT FREQUENTLY DELAYS YOU FROM GETTING THE ANSWER TO YOUR QUESTION. 2. Hardware Requirements There are no specific hardware requirements for mail under Linux. You'll need some sort of 'transport' software to connect to remote systems, which means either tcp-ip or uucp. This could mean that you need a modem or ethernet card (depending on your setup). 3. Getting the software In general, I grab my sources from ftp.uu.net and the other fine archive sites on Internet. In addition, Linux-specific binary ports are found in the usual Linux distrbutions and on the usual Linux anonymous ftp sites (sunsite.unc.edu and tsx-11.mit.edu in particular). The newspak-2.5.tar.z distribution contains config files and readme files related to building uucp, news, and mail software under Linux from the various freely-available sources. It can usually be found in sunsite.unc.edu:/pub/Linux/system/Mail/news. If you can't find it on sunsite, please send me mail and I'll make sure you get a copy of it. 4. Mail 'Transport Agents' This section contains information related to 'transport agents', which means the underlying software that connects your local system to remote systems. 4.1. Smail v3.1 Smail3.1 seems to be a de-facto standard transport agent for uucp-only sites and for some smtp sites. It compiles without patching from the sources. In addition, smail is provided in binary form in the SLS distribution of Linux. The newspak distribution contains config files for smail3.1.28 under Linux that you can use to start with. If you're building smail from sources, you need to have the following in your os/linux file so that 'sed' gives you shell scripts that work properly. CASE_NO_NEWLINES=true For a uucp-only system that has a MX-record and that wants a domainized header (who goes through a smart-host for everything), these are the entire config files you'll need: · replace 'subdomain.domain' with your domain name · replace 'myhostname' with you un-domainized hostname · replace 'my_uucp_neighbor' with the uucp name of your upstream site #-------- /usr/local/lib/smail/config ----------------- # # domains we belong to visible_domain=subdomain.domain:uucp # # who we're known as (fully-qualified-site-name) visible_name=myhostname.subdomain.domain # # who we go through smart_path=my_uucp_neighbor # #---------- /usr/local/lib/smail/paths -------------- # # we're a domainized site, make sure we accept mail to both names myhostname %s myhostname.subdomain.domain %s # #------------------------------------------------------------------- To run smail as a smtp daemon, add the following to /etc/inetd.conf: smtp stream tcp nowait root /usr/bin/smtpd smtpd Outgoing mail gets sent automatically, when using elm. If your inter­ net link is down when you send mail, then the mail sits in "/usr/spool/smail/input". When the link next comes up, "runq" is run which causes the mail to be sent. 4.2. Sendmail+IDA I run a ppp and uucp site and generally use sendmail5.67b+IDA1.5 instead of smail3.1.28 due to the incredible ease of use. There is a binary distribution in sunsite.unc.edu:pub/Linux/system/Mail/delivery. To install it: · you'll probably want to remove (or rename) all the files from smail (see the /install/installed directory if you are SLS) to be safe. · cd to / then "gunzip -c sendmail5.67b+IDA1.5.tpz | tar xvf -" If you have a "modern" tar from a recent Slackware (for example) you can probably just do a "tar -zxvf filename.tgz" and get the same results. · cd to /usr/local/lib/mail/CF and copy the sample.m4 local.m4 file to "yourhostname.m4". Edit out the distributed hostname, aliases, and smarthost and put in the correct one for your site. The default file is for a uucp-only site who has domainized headers and who talks to a smart host. Then "make yourhostname.cf" and move the resulting file to /etc/sendmail.cf · if you are uucp-only, you do *NOT* need to create any of the tables mentioned in the README.linux file. You'll just have to touch the files so that the Makefile works. Just edit the .m4 file, make sendmail.cf, and start testing it. · if you're uucp-only and you talk to sites in addition to your "smart-host", you'll need to add uucpxtable entries for each (or mail to them will also go through the smart host) and run dbm against the revised uucpxtable. · if you use my sendmail5.67b+IDA1.5 distribution you should not use a "freeze file". · If you run Rich Braun's original binary distribution of 5.67a, you'll need to freeze the configuration if you change your .cf file with "/usr/lib/sendmail -bz" to make the changes take effect. You should also update your version to at least 5.67b since there is a nasty security hole in 5.67a and earlier. Another nice thing is that if you have mail.debug set and you run syslogd, your incoming and outgoing mail messages will get logged. See the /etc/syslog.conf file for details. The sources for sendmail+IDA may be found at vixen.cso.uiuc.edu. They require no patching to run under Linux if you're running something like a kernel of 1.00. If you're running a current kernel of around 1.1.50 or later, you get the fun of reversing most of the Linux-specific patches that are now in the vanilla sources. It's extremely obvious where this needs to be done. Just type make and when it blows up, go to that line in the sources and comment out the Linux-specific code that's in there. Sometime after things settle down, I'll send the 'unpatches' to the sendmail+IDA authors and ask'em to remove the now unnecessary patches. If you're going to run sendmail+IDA, I strongly recommend you go to the sendmail5.67b+IDA1.5 version since all required Linux-specific patches are now in the vanilla sources and several security holes have been plugged that WERE (!!!) in the older version you may have grabbed or built before about December 1st, 1993. The May/June 1994 edition of Linux Journal has an extensive article on the care and feeding of sendmail+IDA. The new edition of the Linux DOC Project Networking Administrator's Guide has an even more detailed and complete version. 4.2.1. The sendmail.m4 file Sendmail+IDA requires you to set up a sendmail.m4 file rather than editing the sendmail.cffile directly. The nice thing about this is that it is simple to set up mail configurations that are extremely difficult (if not totally impossible for most people to set up correctly) in smail or traditional sendmail. The sendmail.m4 file that corresponds to the above smail example looks like the following: dnl #------------------ SAMPLE SENDMAIL.M4 FILE ------------------ dnl # dnl # (the string 'dnl' is the m4 equivalent of commenting out a line) dnl # dnl # you generally don't want to override LIBDIR from the compiled in paths dnl #define(LIBDIR,/usr/local/lib/mail)dnl # where all support files go define(LOCAL_MAILER_DEF, mailers.linux)dnl # mailer for local delivery define(POSTMASTERBOUNCE)dnl # postmaster gets bounces define(PSEUDODOMAINS, BITNET UUCP)dnl # don't try DNS on these dnl # dnl #------------------------------------------------------------- dnl # dnl # names we're known by define(PSEUDONYMS, myhostname.subdomain.domain myhostname.UUCP) dnl # dnl # our primary name define(HOSTNAME, myhostname.subdomain.domain) dnl # dnl # our uucp name define(UUCPNAME, myhostname)dnl dnl # dnl #------------------------------------------------------------- dnl # define(UUCPNODES, |uuname|sort|uniq)dnl # our uucp neighbors define(BANGIMPLIESUUCP)dnl # make certain that uucp define(BANGONLYUUCP)dnl # mail is treated correctly define(RELAY_HOST, my_uucp_neighbor)dnl # our smart relay host define(RELAY_MAILER, UUCP-A)dnl # we reach moria via uucp dnl # dnl #-------------------------------------------------------------------- dnl # dnl # the various dbm lookup tables dnl # define(ALIASES, LIBDIR/aliases)dnl # system aliases define(DOMAINTABLE, LIBDIR/domaintable)dnl # domainize hosts define(PATHTABLE, LIBDIR/pathtable)dnl # paths database define(GENERICFROM, LIBDIR/generics)dnl # generic from addresses define(MAILERTABLE, LIBDIR/mailertable)dnl # mailers per host or domain define(UUCPXTABLE, LIBDIR/uucpxtable)dnl # paths to hosts we feed define(UUCPRELAYS, LIBDIR/uucprelays)dnl # short-circuit paths dnl # dnl #-------------------------------------------------------------------- dnl # dnl # include the 'real' code that makes it all work dnl # (provided with the source code) dnl # include(Sendmail.mc)dnl # REQUIRED ENTRY !!! dnl # dnl #------------ END OF SAMPLE SENDMAIL.M4 FILE ------- 4.2.2. Defining a local mailer Unlike most Unix distributions, Linux does not come with a local mail delivery agent by default. I recommend using the commonly available deliver program, which is an optional package in a number of the usual Linux distributions. In order to do so, you need to define a LOCAL_MAILER_DEF in the sendmail.m4 file that points to a file that looks like: # -- /usr/local/lib/mail/mailers.linux -- # (local mailers for use on Linux ) Mlocal, P=/usr/bin/deliver, F=SlsmFDMP, S=10, R=25/10, A=deliver $u Mprog, P=/bin/sh, F=lsDFMeuP, S=10, R=10, A=sh -c $u There is a also built-in default for deliver in the Sendmail.mc file that gets included into the sendmail.cf file. To specify it, you would not use the mailers.linux file but would instead define the following in your sendmail.m4 file: dnl --- (in sendmail.m4) --- define(LOCAL_MAILER_DEF, DELIVER)dnl # mailer for local delivery Unfortunately, Sendmail.mc assumes deliver is installed in /bin, which is not the case with Slackware1.1.1 (which installs it in /usr/bin). In that case you'd need to either fake it with a link or rebuild deliver from sources so that it resides in /bin. 4.2.3. The Sendmail+IDA dbm Tables Setting up special behavior for sites or domains is done through a number of optional dbm tables rather than editing the sendmail.cf file directly. Refer to the July-1994 issue of Linux Journal, to the docs in the sources, or to the sendmail chapter in the newest version of the Linux DOC Project Networking Administration Guide which will be available real-soon-now for more details. · mailertable - defines special behavior for remote hosts or domains. · uucpxtable - forces UUCP delivery of mail to hosts that are in DNS format. · pathtable - defines UUCP bang-paths to remote hosts or domains. · uucprelays - short-circuits the pathalias path to well-known remote hosts. · genericfrom - converts internal addresses into generic ones visible to the outside world. · xaliases - converts generic addresses to/from valid internal ones. · decnetxtable - converts RFC-822 addresses to DECnet-style addresses. 4.2.4. So Which Entries are Really Required? When not using any of the optional dbm tables, sendmail+IDA delivers mail via the RELAY_HOST and RELAY_MAILER) defined in the sendmail.m4 file used to generate sendmail.cf. It is easily possible to override this behavior through entries in the domaintable or uucpxtable. A generic site that is on Internet and speaks Domain Name Service, or one that is UUCP-only and forwards all mail via UUCP through a smart RELAY_HOST, probably does not need any specific table entries at all. Virtually all systems should set the DEFAULT_HOST and PSEUDONYMS macros, which define the canonical site name and aliases it is known by. If all you have is a relay host and relay mailer, you don't need to set these defaults since it works automagically. UUCP hosts will probably also need to set UUCPNAME to their official UUCP name. They will also probably set RELAY_MAILER, and RELAY_HOST which enable smart-host routing through a mail relay. The mail transport to be used is defined in RELAY_MAILER and should usually be UUCP-A for UUCP sites. If your site is SMTP-only and talks `Domain Name Service', you would change the RELAY_MAILER. If you're a SLIP site, you might want to take the easy way out and just forward all outgoing mail to your service provider to do the right thing with. To do so, you'd want to define ISOLATED_DOMAINS and VALIDATION_DOMAINS to be your domain, you'd also want to define RELAY_HOST to be your service provider and RELAY_MAILER to be TCP. Of course, you want to ask permission before you set any system up as your general purpose relay. 4.3. Sendmail 8.7 Sendmail 8.7.x from Berkeley is the latest major revision after sendmail5. It has wonderful built-in support for building under Linux. Just "make linux" and you'll be all set. You'll probably be best served by grabbing one of the various binary distributions off of the usual Linux archive sites rather than fighting things like Berkeley dbm yourself. There's a nice distribution of sendmail 8.6.12 from Jason Haar - j.haar@lazerjem.demon.co.uk on sunsite.unc.edu in /pub/Linux/system/Mail/delivery/sendmail-8.6.12-bin.tgz that has the source documentation and a very nice quickie description of how to run sendmail v8 for common configurations. Bottom line with sendmail v8 is that you want to configure the bare minimum necessary to get the job done. The following is an example that should get you close at least. 4.3.1. A Sample 8.7.x mc file Much like sendmail+IDA, sendmail v8 uses m4 to process a config file into a full sendmail.cf that sendmail uses. The following is my current mc file for my site (ppp to Internet for outgoing mail, uucp for incoming mail). dnl divert(-1) #--------------------------------------------------------------------- # # this is the .mc file for a linux host that's set up as follows: # # - connected to Internet for outbound mail (ppp here) # - connected via UUCP for incoming mail # - domainized headers # - no local mailer (use 'deliver' instead) # - no DNS running so don't canonicalize outgoing via DNS # - all non-local outbound mail goes to the RELAY_HOST over smtp # (we run ppp and let our service provider do the work) # # vds 3/31/95 # #--------------------------------------------------------------------- include(`../m4/cf.m4') VERSIONID(`linux nodns relays to slip service provider smarthost')dnl Cwmyhostname.myprimary.domain myhostname.UUCP localhost OSTYPE(linux) FEATURE(nodns)dnl FEATURE(always_add_domain)dnl FEATURE(redirect) FEATURE(nocanonify) dnl MAILER(local)dnl MAILER(smtp)dnl MAILER(uucp)dnl define(`RELAY_HOST', smtp:my.relay.host.domain) define(`SMART_HOST', smtp:my.relay.host.domain) define(`UUCP_RELAY', smtp:my.relay.host.domain) define(`LOCAL_MAILER_PATH', `/bin/deliver') define(`LOCAL_MAILER_ARGS', `deliver $u') 4.3.2. Sendmail v8 tidbits There are a few differences I suppose to the 'IDA bigots' among us. So far, I've found the following. · Instead of 'runq', you type 'sendmail -q' to run the queue 4.4. Other "transport agents" The following also are known to run under Linux. Consult "archie" for details regarding how to find them... · smail2.5 - very simple UUCP-based smail 4.5. Local Delivery Agents Unlike most operating systems, Linux does not have mail "built-in". You'll need a program to deliver the local mail. One good program is Rich Braun's "lmail" program, but I've switched to using the more commonly available "deliver" program. Documentation for how to use either for local delivery is in the sendmail5.67b+IDA1.5 binary release (on sunsite) mentioned above. 5. Mail "User Agents" This section contains information related to "user agents", which means the software the user sees and uses. This software relies on the "transport agents" mentioned above. 5.1. Elm Elm compiles, installs, and runs flawlessly under Linux up to and through Slackware 1.1.1 (gcc2.4.5, gcclib 4.4.4). For more information, see the elm sources and installation instructions. The only thing to know is that Elm's Configure script incorrectly sets the "ranlib" and "stdarg" variables in config.sh. The Elm Development Team has been informed of this little problem, so please don't bother them with it (again). · (from Chip Rosenthal - chip@chinacat.unicom.com ) The easiest way to deal with this is to create a file called config.over at the top of you Elm source tree and include the line: ranlib='ranlib' stdarg='define' · Elm and filter need to be mode 2755 (group mail) with /usr/spool/mail mode 775 and group mail. If you use a binary distribution, you'll need to create a /usr/local/lib/elm/elm.rc file to override the compiled-in hostname and domain information: · replace "subdomain.domain" with your domain name replace · "myhostname" with you un-domainized hostname replace #---------- /usr/local/lib/elm/elm.rc ------------------ # # this is the unqualified hostname hostname = myhostname # # this is the local domain hostdomain = subdomain.domain # # this is the fully qualified hostname hostfullname = myhostname.subdomain.domain # #-------------------------------------------------------- One thing you want to be aware of is that if you have Elm compiled to be MIME-able, you need metamail installed and in your path or Elm will not be able to read MIME mail you've received. Metamail is available on thumper.bellcore.com and of course via "archie". We have heard reports that gcc and gcclib newer than v2.4.5 and v4.4.4 respectively are rather strict and fail to compile Elm. Here's the scoop as reported by ccnp@unitrix.utr.ac.za (Neil Parker) who forwarded a posting by longyear@netcom.com (Al Longyear). · ELM is using internal fields in the FILE structure in an effort to bypass the standards. (The _flag, _IOERR, and _IOEOF are old fields for the pre-POSIX runtime package. While POSIX doesn't say that you can't define these fields, it does not say that you _must_. Linux does not. It does say that programs should not be written to use them, even if they are in the implementation.) where it does if (fp->_flag & _IOERR) ... change it to if (ferror(fp)) .... where it does if (fp->_flag & _IOEOF) ... change it to if (feof(fp)) ... These are the ANSI/POSIX definitions for the same function. · Also, there is a nice set of diffs for Elm 2.4.23 that works just fine (for me at least) located on nic.funet.fi in the file /pub/OS/Linux/kernel/src/net-source/mail/elm-2.4.24.dif that works around some Linux-isms. While just grabbing the vanilla sources and patching with these diffs works fine, I've personally had success with just setting up the config.over file mentioned above and only patching opt_utils.c and lock.c in the Elm sources. (I run kernel 1.3.33 over Slackware 2.1.0) · In the "too cool to be true" category, there is a distribution of Elm-2.4.24 that is "PGP-aware". To try it, grab the file ftp://ftp.viewlogic.com/pub/elm-2.4pl24pgp3.tar.gz, which is elm2.4.24 with PGP hooks added. You configure and build it the same way you do normal Elm, which means you probably need to add the patches mentioned above. For what it's worth, I run it here and like it a lot. · While this item is not Linux-specific, it's perceived (wrongly) to be a nagging Elm bug nevertheless. We've heard that Elm sometimes fails with a message that it's unable to malloc() some massive number of bytes. The identified workaround is to remove the post- processed global mail aliases (aliases.dir and aliases.pag). THIS IS NOT A BUG IN ELM. It's an error in configuration of Elm by whomever you got your binary distribution of Elm from. Elm has an enhanced, and non-compatible, format for aliases. You need to ensure that the path Elm uses for aliases is different from the path sendmail/smail uses. From the volume of reports of this problem, it's apparent that at least one major distribution 'on the street' has in the past been misconfigured. The current Slackware does it correctly. · (from scot@catzen.gun.de (Scot W. Stevenson) ) The current metamail package requires csh for some of its scripts. Failure to have csh (or tcsh) will cause most interesting errors... 5.2. Mailx Safe yourself the pain. Just go and grab the mailx kit from Slackware 2.1.0 or later, which has a nice implementation of mailx5.5. If you're into building from sources, mailx v5.5 compiles without patching under Linux if you have "pmake" installed. If anybody is still using it, I strongly recommend removing the old "edmail" stuff from SLS1.00 and replacing it with mailx. 5.3. Other user agents The following also are known to run under Linux. Consult "archie" for details regarding how to find them... · Pine - from the Univ. of Washington · Metamail - allows MIME support · mh - yet another way to handle mail · deliver - file/process mail based on rules · procmail - file/process mail based on rules · Majordomo - manages e-mail lists · Mserv - provide files-by-mail 6. Acknowledgements The following people have helped in the assembly of the information (and experience) that helped make this document possible: Steve Robbins, Ian Kluft, Rich Braun, Ian Jackson, Syd Weinstein, Ralf Sauther, Martin White, Matt Welsh, Ralph Sims, Phil Hughes, Chip Rosenthal, Scot Stevenson, Neil Parker If I forgot anybody, my apologies...