Wokai Blog: Our Philanthropic Hacker Ben Benson's Engineering Report: How to Build an Innovative Template Email System: Wokai China Microfinance Blog

As the official philanthropic hacker at Wokai, I've been asked to provide regular updates on new product features, behind-the-scenes development, technology related to peer to peer lending, or even Internet engineering in general if I think it is particularly nifty.

Today's topic is pretty low level detail, and in fact usually overlooked as being too small to be worthy of any design consideration at all - though I hope to prove otherwise. Topic; the delivery of automated email messages. In the coming weeks, I'll be adding many more event-based or periodic email reports for Wokai lenders. Monthly financial accounting reports with featured micro-loan recipients, notifications of activities on loans (i.e., borrower has fully repaid the loan), etc. The intent is to make all such email generated from templates that our Marketing Director can manage.

The vast majority of websites send email messages by composing email messages inline (within their code) and handing content off to a function for email delivery. The more formal websites do it by writing code to assemble needed values, formatting these values, then passing a map of the values and a pointer to a specific email template off to a method to perform data merge and delivery. This is fairly clean and straightforward, but requires changes to the code every time someone makes a change to the email template that requires a new (or newly formatted) value.

I've assembled a much more powerful solution that leaves the code cleaner and gives the email template editor far more flexibility. Here's what the actual code looks like for sending Wokai's welcome letter after a registration:

	emailManager.addContext("user", user);
	emailManager.processEvent(EmailEventType.REGISTRATION);
	emailManager.deliver();

That's it! All the work is neatly handled by all the right parties. Now let's take a look at how this is achieved.

(1) The first line is very straight-forward. "user" is an object that represents the user who has been registered. We are adding it to the context of the email manager service for later use in parsing templates. If there were other root objects that were related to this event, we would add them all in the same manner.

(2) The next line is where the magic happens. ProcessEvent() retrieves every email template from the database that matches the given event type (in this case a REGISTRATION), then passes the current context along to the template and asks the template to generate an EmailDeliverable (a custom object I've written that represents the basic parts of an email message).

Here's the processEvent() code:

    List templates = this.emailTemplateDAO_.findActiveByEvent(event);

    Iterator iterator = templates.iterator();
    while (iterator.hasNext())
    {
        EmailTemplate template = iterator.next();
        EmailDeliverable deliverable = template.generate(this.contextMap_);
        this.deliverables_.add(deliverable);
    }

The Template.generate() method looks like this:

    @Transient
    public EmailDeliverable generate() throws Exception
    {
        // this can be done in a startup servlet or wherever you like
	Velocity.init();

	EmailDeliverable e = new EmailDeliverable();
	e.setFromName(this.fromName_);
        e.setFromEmail(this.fromEmail_);

	if (this.context_.getKeys().length > 0)
	{
	    StringWriter writer;
	    String templateRef = "emailTemplate: " + this.id_;

	    writer = new StringWriter();
	    Velocity.evaluate(this.context_, writer, templateRef, this.subject_);
	    e.setSubject(writer.toString());

	    writer = new StringWriter();
	    Velocity.evaluate(this.context_, writer, templateRef, this.html_);
	    e.setHtml(writer.toString());

	    writer = new StringWriter();
	    Velocity.evaluate(this.context_, writer, templateRef, this.text_);
	    e.setText(writer.toString());

	    writer = new StringWriter();
	    Velocity.evaluate(this.context_, writer, templateRef, this.addressList_);
	    e.setAddresses(writer.toString());
	}
	else
	{
	    e.setSubject(this.subject_);
	    e.setText(this.text_);
	    e.setHtml(this.html_);
	    e.setAddresses(this.addressList_);
	}

	return e;
    }

Using Velocity, the email templates content can now contain references to context objects at ANY level of depth. We can also harness the power of Velocity directives directly from our email template:

	Hello $user.firstName, 

	Here's a list of your current loans:
  
        #foreach( $loan in $user.loans )
		#if( $loan.repaid )
			[dynamic email content here]
		#end
	#end

You might have noticed that we never indicate the sender or recipient of a message. This work is now pushed off on the editor of the email template itself! The recipient of the welcome message is simply $user.email. At this point the power of this system should be evident. The author/editor of an email template now has access to pretty much any business values related to the associated web event and has the freedom to navigate and format these values as desired to compose a highly dynamic email.

(3) Lastly, returning to our original three line implementation, the third and last line of code is to deliver all emails that have been generated and queued. You could write any method you like. For immediate return of control, one can write a method to pass the queue to a background delivery agent using a thread manager like Quartz, and/or deliver mail directly to a mail server using JavaMail (or a wrapper). A number of things should be considered.

• 1. If using your server's local email service (Sendmail, Postfix, etc), the percentage of emails that get caught in Spam traps will largely depend on the history of your server's IP address. Any junk email sent by this IP address in past decades can have a bearing on its current treatment by ISPs who are trying to help their customers control Spam.

• 2. Define an SPF DNS record for your domain.

• 3. Be absolutely sure your server is accepting return/bounce messages, but is NOT accepting relay mail from the outside world.

• 4. Ensure your delivery process is applying a correct bounce/sender address - separate from the message sender address which can change more freely based on business needs.

As a builder of one of the Internet's largest outbound email systems, I can expound on the intricacies of email reliability and tracking at length. Covering the basics may be good enough for your service, but getting into opt-out management, archiving, routing and processing of return mail starts to get rather time consuming for in-house development. At a minimum, I recommend using a third party outbound email service with a good reputation (such as AuthSMTP) to carry out deliveries. Configuring your local server to forward mail to this mail transfer agent will provide performance improvement to the web application and an added layer of queuing/logging for more flexible control over the outbound mail.