How we built Twittermail in 48 hours

Disclaimer: This article was written in 2009 when Twittermail was one of the first major Twitter 3rd party apps. It’s a little outdated now, back in 2009 it was uncommon people had smartphones or a permanent internet connection. Also, this use case was focused on CakePHP 1.2, it’s very outdated now but still fun to read.

Okay, for those who don’t know Twittermail, Twittermail is an open idea by Boris Veldhuijzen van Zanten, he thought that it would be great if people could e-mail their tweets to Twitter.com.

This is potentially very useful for people who use mobile phones. Of course you can go to the mobile webpage of Twitter, but sending an e-mail is pretty much easier. Some older phones do not even contain a browser and only have e-mail functionality. Also a lot of businesses block Twitter.com on their corporate network but with Twittermail you can continually update your Twitter account even from your business e-mail address.

Those who register at Twittermail, get a secret unique email address like 1234abcde [AT] twittermail com. When you send an e-mail to this secret email address, it will get posted instantly to Twitter.com through the Twitter API.

The Environment

Twittermail is a small project so basically we only need a sign-up form, a settings page and we need to show some recent activity. Since we handle loads of visitors and more than 300.000 emails a day this project needed to be very scalable and thin. We managed to do this project in exactly 48 hours, thanks to CakePHP (and Coca Cola).

Custom Validation

Users supply us their Twitter credentials so that we can create a unique email address. Because we need to verify if the information they provide is valid, we created a custom validation which checks the Twitter API whether or not everything is valid. In our model we created a custom function called ‘isValidTwitter’, in this function we make a CURL call to the Twitter API, we parse the output from JSON to an array and we get back the user information or an error.

1
2
3
4
5
6
<?php
public $validate = array(
'rule' => array('isValidTwitter'),
'message' => 'The credentials you supplied are not valid! Please try again'
);
?>
1
2
3
4
5
6
7
<?php
function isValidTwitter($fielValue) {
// Validate the twitter credentials
return true;
}
?>

From now on you can do whatever you want with the results. In our project we encode the password, fetch some Twitter information and save it to the database.

Because we don’t need a very complex User Authentication, we didn’t use ACL.

Localization & Internationalization, l10n and i18n

At the moment our Website is English-only, but we already made it multi-language proof by using the __(‘Login’) function. For those who never worked with localization, in the cake console you can simply generate a language file by using the command ‘cake I18n’ here you get the option to extract all the gettext strings to POT files. You can read more about localization at the wonderful Cookbook at http://book.cakephp.org/1.2/en/The-Manual/Common-Tasks-With-CakePHP/Internationalization-Localization.html

Scaling

Caching

Because we served more than 175.000 users who sent over 300.000 emails per day, scaling is an serious subject. First of all, use Cache! You don’t need to get all data over and over again from the database. For example in the sidebar we have ‘Newest Twittermailers’. This is cached for a year, every time when someone is registering, we invalidate the cache. Cache isn’t a thing you should forget and for those who never worked with it please read the Cookbook.

1
2
3
4
<?php
// Invalidate cache, so next time the newest Twittermailers list will be re-generated.
Cache::delete('sidebar_newesttwitters');
?>

Containable Behavior

This is a very substantial improvement in the Cake 1.2 series, in the past I always used for my CakePHP projects ‘expect’ (For 1.1 users: http://bakery.cakephp.org/articles/view/keeping-bindmodel-and-unbindmodel-out-of-your-controllers). This model behavior allows you to filter and limit model find operations. Using Containable will help you cut down on needless wear and tear on your database, increasing the speed and overall performance of your application. The class will also help you search and filter your data for your users in a clean and consistent way.

It helps you to select only the data you really want, it’s recommended to read the full details about this at http://book.cakephp.org/1.2/en/The-Manual/Core-Behaviors/Containable.html

Think about security

Often developers forget to think about security, but security is a very important subject when you’re developing web-applications. Use the validations and double check them with the sanitize class from CakePHP.

1
2
3
4
<?php
App::import('Sanitize');
echo Sanitize::html($untrustedString, true);
?>

I was always taught: NEVER trust the input of your users!
Then there’s the Model::save function, that lacks a complete description in the cakebook, but in my opinion the $fieldList method is very important.

Like I said before, never trust the input of your users, supply $fieldList with an array of all the fields you’re supposed to fill.

For example:

1
2
3
<?php
$this->Twitter->save($this->data, true, array('username', 'password', 'email'));
?>

How we did it in 48 hours

Of course, this is a (very) small project, but because of Cake we were able to manage this project in 48 hours. It’s essential to stick to the CakePHP conventions. Use the built-in validation methods, use the helpers and use the components, it actually helps you with keeping your code clean and compact and it speeds up your developing time.

Bram Kok

Lennaert Ekelmans

http://www.twittermail.com