In the course of building the frontend application for Earbits, I’ve been constantly annoyed with the responsiveness of my local development server. As a pretty complex AJAX application, there are lots of little (and some bigger) calls to the Rails backend APIs to do lots of potentially slow things. This means that there are lots of calls that are originating in the browser and being served by one mongrel, so the server ends up handling requests too slowly. Which leads to me being frustrated. Which leads to me finding a nice solution to the problem.

Warning: This May Not Apply To You

If you have slow running requests that have to run in series, or if you’re not doing AJAX-y things, this isn’t going to help you much. This setup is only useful if your app sends a bunch of parallel requests to the backend. For instance, if after your main page loads, you need to do AJAX requests to a) load the user’s friends, b) get their comment history, c) calculate some suggestions, and d) check to see if you should show a popup promo to the user, then this might help you out. But if you need the results of request (a) to get (b) and the results of (b) to get (c), then you’re out of luck and this will be a waste of your time.

Ok, ready to do this?

Step 1 – Install Nginx

This is much simpler if you have macports installed. So go do that first if you haven’t already. If the first step below doesn’t do anything, you don’t have macports. Once you have macports, it’s a simple two-command process to install nginx.

$ sudo port -d selfupdate
$ sudo port install nginx +ssl +debug

Now, while nginx is installing you can open a new terminal window and keep going. The install will take a few minutes.

Step 2 – Configure Foreman

Foreman is a great tool released by David Dollar to make running web processes and workers crazy easy. It uses a Procfile to define the different things it controls and you should refer to his blog or a Google search to learn about the finer points of Foreman. We use a pretty simplistic file here as an example.

To get started just run:
$ gem install foreman

Then create a file called Procfile in the root of your project using the text editor of your choice. My Procfile has just one line that define a web process using the standard Rails server:

web:    bundle exec rails server -p $PORT

Step 3 – Configure Nginx

Now that nginx is probably finished installing, we can get it configured to act as a local Load Balancer for our Rails server. I’m going to set it up using port 80 locally. If you are running an HTTP server locally already on port 80, then you should adjust your configuration accordingly. But if you’re not using port 80, Rails redirects might cause you some headaches. So I suggest you stick with the standard port.

First, you’ll need to find your nginx.conf configuration file. If you used macports to install it, it’s probably in /opt/local/etc/nginx/nginx.conf but if you can’t find it, run nginx -V to see where it’s hiding.

If you’re not familiar with nginx at all, there are two main directives that you’ll be using, server and upstream. The server directive defines the front-end part of nginx (i.e. where it listens for incoming requests). The upstream directive defines the back-end (i.e. where to send requests so that Rails can handle them). Both of these snippets should exist within the http section of your config file. I’ll provide a complete nginx.conf at the end of this section so that you can just replace the default file and then tweak it if necessary.

Upstream section

This is the configuration that I’m using, I will explain it in a second:

upstream foreman4000 {
  server localhost:4000;
  server localhost:4001;
  server localhost:4002;
  server localhost:4003;
  server localhost:4004;
}

Here on line 1 we’re defining the upstream which we will reference later. Note that “foreman4000″ is just an arbitrary name used to reference it within the config. Then since we plan on running 5 web processes starting at port 4000, we list out each of the ports where our Foreman will be listening for requests.

Server section

This will define a server so that nginx will listen to requests on port 80 and direct them to the upstream we created above (line 6). We also pass the real IP of the request and the hostname along to the server (lines 7-9, in case you need either of these bits of data.

 server {
    listen       80;
    server_name  localhost;
    access_log  /opt/local/var/log/nginx/foreman4000.access.log  main;
    location / {
      proxy_pass http://foreman4000;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $remote_addr;
    }
 }

Test the config

After you’re done making the changes above (or if you download my nginx.conf example file), you should run $ sudo nginx -t to test the config and make sure it’s all good. If it fails, double check everything. If it’s good, you’re ready to start it up.

Step 4 – Start your Engines

Now all we have to do is start everything up. In your Rails project directory:
$ sudo nginx
$ foreman start -p 4000 -c web=5

Note: I use the non-standard port 4000 as the starting point so that other random servers can run on 3000 (rails default) and 5000 (foreman default) without conflicting with this setup.

If it’s working you should be able to open up http://localhost in your browser and your Rails site should load. In the terminal running Foreman, you should see it handing requests to each of the 5 web processes (web.1 – web.5) like this:

Now, enjoy your new, super-responsive AJAX server! I know I do.

Notes and Extras

There are a few reasons my local development environment is slow. Firstly, I’m using a hosted mongodb that is shared among a few of our developers so that we can more easily collaborate on work. It makes it possible for me to add a new feature and seed the database with some good example data, then hand it off to our UI guy to style it up with zero configuration on his end other than a git pull. Secondly, we make heavy use of the Facebook Graph APIs which can at times be sluggish over my mere-mortal internet connection. Getting someone’s friends might take a full second, posting something to a wall might take a couple seconds. You can imagine how with just one process running, it would get a bit slow.

The third reason our site is slow in development is the simple fact that it’s designed to be chatty. Instead of large monolithic views getting rendered for the user, we do a lot of client-side template rendering using underscore.js based on json data requested by the client app (which leverages backbone.js for much of the interaction logic). In production, this works great because we can fire up a bunch of web processes and everything scales on out. But in development, between the slow database access and the numerous calls, things can take seemingly forever to fully load. But not anymore!

BTW – if you haven’t tried Earbits out for your online radio needs, you absolutely should. Great way to discover new music, share with your friends and really just have a great time. And you should stop listening to so much Genesis anyway. It’s time for something fresher.

1) From shipment notification, hit the tracking number link.

2) Click the save button on the UPS site (you’ll need to be logged in, but they have a nice persistent login)

3) Open the iPhone app, see your tracking number

4) Nickname it for easy reference!

5) Refresh constantly.

Kudos to UPS for making it so simple and so right.

I’ve personally found that most non-makers are completely oblivious to how much having even short meetings during the day can disrupt our work process.  I can’t remember where I read it (maybe Joel Spolsky) but I recall a discussion about how if it takes 15 or 30 minutes for a developer to “get in the zone”, a couple 15 second distractions can ruin a whole day of productivity.

When you’re operating on the maker’s schedule, meetings are a disaster. A single meeting can blow a whole afternoon, by breaking it into two pieces each too small to do anything hard in. [...]

I find one meeting can sometimes affect a whole day. A meeting commonly blows at least half a day, by breaking up a morning or afternoon. But in addition there’s sometimes a cascading effect. If I know the afternoon is going to be broken up, I’m slightly less likely to start something ambitious in the morning. I know this may sound oversensitive, but if you’re a maker, think of your own case. Don’t your spirits rise at the thought of having an entire day free to work, with no appointments at all? Well, that means your spirits are correspondingly depressed when you don’t. And ambitious projects are by definition close to the limits of your capacity. A small decrease in morale is enough to kill them off.

[full essay here]

On a related note, why in the world does Paul Graham not have an RSS feed for his essays??  Dave Winer needs to make that happen.

For an innexplicable reason, Microsoft decided to kill the Export function from IIS7 in favor of this new feature.  But for those of us who don’t trust technology, we like to do things manually and to get a repeatable result that doesn’t update automatically when we least expect it.  Yes, I am the sort of person who wonders why the default Windows Update on servers is to Install and Reboot Automatically at 2am…

In any case, in a simple 3 step process you too can export and import your Internet Information Server 7 websites and app pools.

I will call the target server TWEB (this is the server where you want a duplicate configuration) and the source server SWEB (this is where the current configuration exists).

First, on TWEB make a backup copy of the files in C:\Windows\System32\inetsrv\config.  I just created a subfolder called “bak” and copied them.  This is very important.  If you forget or skip this step because backups are for sissies, you will be re-installing IIS7 in step 4.

Second, copy the AppliationHost.config from SWEB into the C:\Windows\System32\inetsrv\config folder on TWEB.  Also copy any application files like your c:\websites folder or whatever over to TWEB in the appropriate location if you haven’t already.

Third, on TWEB open both the new ApplicationHost.config and the backup ApplicationHost.config from step 1 and locate the <configProtectedData> node in the backup.  Copy that node and replace it into the new config file.

Fourth, if you didn’t backup the existing config file, remove the IIS role and add it back, then start at step 1.

Note: if you have custom accounts under which you’re running app pools (cuz you’re not using LocalSystem, right?) then you just need to go into the IIS Management Console and re-configure the passwords for those accounts.  They were encrypted with the other server’s AES keys so they won’t be valid on this server and the pools won’t start.

That’s it.  Hope that helps.  It should work the same for C# .NET applications (which is what I’m using) or just static websites or whatever.  But of course, you shouldn’t trust me.  Test it yourself first.

My new good-karma policy is that if I spend more than an hour tracking down a general bug or wacky configuration thing, I promise to blog the solution.

The way I set it up, the top wheel scrolls documents like any normal mouse and the left-right clicks are set to do Prev-Next in my iTunes playlist. The thumb wheel is set to do volume up-down and click to mute. Then it still has the browser back button, a middle click for tabbed browsing and an interesting Search button where you can highlight text and press it, then it opens a google search on that phrase. Cool. Very cool.