Email archival without mailing list software?

I believe that the lack of archives in business communication is a serious enough problem to solve. Especially in these times, when a lot of brainstorming happens on email amongst team members located in different geographies.

I've tried using Basecamp for all business communication and succeeded only partially. Although Basecamp can serve as a good archive for email communication, it just doesn't beat the simplicity of adding any random email address to the CC list when you hit "Reply all". You have to first add a user to a project, then add him to the message, and then reply to the message. This, of course, will work only if the person knows how to use Basecamp. Otherwise, add the additional overhead of evangelizing Basecamp over regular email before getting him to accept this new mode of communication. [1] If something confidential is being discussed, there's also this fear of archiving your trade secrets on someone else's servers.

Probably there's a good use-case to add a new feature in SMTP servers -- an archive header and an email archiving module. If an incoming email has the archive header (X-Archive: true) the SMTP server will archive the email by storing a copy of that email and making it accessible on a web page. All replies to that email (and further replies to those emails) will automatically get archived, regardless of whether they have the archive header or not [2]. The server can also add a footer to each email in the thread to direct the user to the archive. Optionally, the archiving module may also archive email attachments and make them available on the company intranet (or the web).

Various mail clients will also need to be modified to present a user with a checkbox while composing the mail that sets/unsets the archive header.

I think this feature can be easily hacked into postfix (or sendmail) and a couple of open source email clients (like Evolution and Thunderbird).

Alternatively, this can also be achieved by having a record-keeper email ID (something like archive@somecompany.com). Any email CCed to the record-keeper will be archived and available on the company intranet (or the web). The only drawback of this is that the archival chain can break very easily if someone removes the record-keeper from the CC list. This approach can probably be an easier path towards implementation, both from the client's as well as the server's side. (Aren't such archival modules already available?)

If anyone wants to collaborate on this project with me, please drop me an email at saurabhnanda (at) gmail (dot) com. I've been wanting to work on a decent tech project for quite some time now.

[1] Given, recipients of Basecamp messages can now simply reply to messages just like normal email, it's still not perfect. All formatting (even line-breaks) are stripped. And you still have to log on to Basecamp to compose a completely new message.

[2] I think it's reasonable to assume that since the parent wanted the email thread to be archived, it must be archived in its entirety.

In Support of "Top Posting"

All over the web, on mailing lists, especially in the open source geek circles [1], top posting is frowned upon. Even while bottom posting, you're not considered cultured if you don't truncate the parent post to the exact specific part(s) your reply pertains to. I, too, used to follow this religiously until recently.

After three years of communicating with non-geeks (you know, the business and marketing types), I realized that a lot of business communication was simply not possible in the typical quote-reply-quote-reply format that is prevelant on mailing lists. This is because of a couple of reasons, which I'll try to enumerate in the following paragraphs.

One of the reasons given for not including the parent post in its entirety is that anyone wanting to refer to the thread can always look up the mailing list archives. Cultured folk should not waste bandwidth and screen real estate by repeating what is available a click away. This is not the case with typical business communication, which starts between two people (with usually their bosses on the CC list -- to be "kept in the loop"). Along the course of a reasonably sized email discussion, the CC list grows exponentially (believe me)! First the bosses' bosses are added -- if the small fry realize that they'll need to save their ass if something goes wrong. Suddenly something is under the purview of a chappy in some other department and him and his boss and boss' boss are added to the list. Later someone realizes that the legal department also needs to be "kept in the loop". Within no time you're having an orgy on email! With each new addition to the CC list you need a place where the latecomers can read-up on the discussion till then. In normal business communication there are no "archives". The latest email, itself, serves as an archive. I've had a tough time catching up on the discussion where each email was regularly truncated by the participants. I had to piece together more than 25 different emails forwarded to me as the "archive" -- believe me, it wasn't an easy task. Therefore it's much better to preserve the entire thread in the body and add your reply to the top of the email.

A side-effect of not having an archive is that you can't really follow the quote-reply-quote-reply pattern ("Comments inline...") very effectively. If five different people are communicating, you suddenly lose track of (a) who said what, (b) in reply to what, and (c) in reply to whom. With an archive, each post can easily be identified on all three parameters. Take away the archives, and within two or three replies you lose track of (a) and (c) very easily. Inline comments, even in business communication, are useful when each paragraph or point in your email can be answered independent of the others. For example, when you're seeking answers to a list of technical questions. However as soon as you want to reply to the reply of your questions you start losing track of (a) and (c). One decent solution I've noticed is prefixing your reply with your name in square brackets. So, a typical email after a couple of back and forth replies looks like:

* Can I pass the customer's shipping address instead of the billing address in the API?
[Alice] It's possible, but why would you want to do it? You already have the billing address in the DB, don't you?
[Bob] Yes, we do -- but we don't want to share it due to privacy concerns.
[Alice] Okay, then you can pass the shipping address.

I've observed a significant side effect of following the quote-reply-quote-reply model of communication -- especially on tech mailing lists. A lot of discussions regularly morph into flame-wars, mudslinging matches, or "preachers preaching the lesser mortals." Don't get me wrong -- the communication style is not the sole reason for discussions going off track. Obviously a lot depends on the attitude of the people involved in the discussion, but I feel the communication style definitely promotes the degeneration of the discussion. When you read a post you generally disagree with, it's very tempting to pick each line, quote it, and try to rebut it individually. It's very easy to get lost in the trees for the woods.

[1] These are the only circles which seem to have a respectable mailing list culture.

Overestimating users of technology

Most software products assume too much about their users. After reading Alan Cooper's "The Inmates Are Running The Asylum" (at Amazon and at IndiaPlaza) I realized how much I used to presume about a user's comfort level with technology while designing products. I now consciously switch off the developer/geek side of my brain while writing product specs or mocking up UI prototypes.

Here's a post by David Pogue on what everyone assumes that everyone else knows but is wrong. I thought I would know every single thing on the list but was surprised to find a keyboard shortcut that was new to me - using shift + space to scroll up in a web page.

You can tap the Space bar to scroll down on a Web page one screenful. Add the Shift key to scroll back up.

Yahoo! recently released their research findings about an OpenID usability report. They were not surprising:

None of the users had heard of OpenID before, and none of them even noticed the OpenID sign-in box displayed below the traditional email/password login form on the site. [...] Observing these tests was more than a bit frustrating for the Yahoo! OpenID team, and the test subjects may have been distracted by the sounds of the groans and head-pounding coming from the other side of the one-way mirror. Certainly there is a lot of work to be done on the OpenID UX (user experience) front.

(At the risk of over-simplification, OpenID is a common username/password using which you can login to multiple websites without creating individual username/passwords for each.)