nerdtopia etal

Life has been crazy for a wide variety of reasons, but let me begin this post with a bit of arrogance:

I am, in fact, a prophet. My memcache predictions became realized (at least partially) in a number of projects, Voldemort being the most popular. I haven’t had much time to look at it, but it seems pretty good for the purposes.

More to come. I promise.

The vast majority of database queries (at least, in a web application) are SELECT statements. In my experience, unless you get Digg’ed, it’s usually somewhere between 80% and 90% reads vs writes. The exception to this rule is web forums and other software which tracks hits.  For every page query, you have to increment the thread’s ‘hit’ column. This involves opening a database connection, submitting your update query, and closing the socket once the transactions been completed. As you can imagine, this quickly becomes very expensive. vBulletin’s method of dealing with this is simply not reporting accurate hit stats. Unacceptable in my opinion.

Alternatively, what one could do is store the hits in an array and access the memory directly for every query. Any page render queries this array instead of the database. In practice, I’d probably store this in memcache. At a given interval, a call gets made to the memory (memcached store) and the number of hits incremented gets commited to the database. Since you’re only storing the hits incremented, this should scale pretty well if necessary between application servers. If the memcache happens to crash, in most cases no important data is lost.

The new Facebook layout change has been met with a certain amount of opposition. Up until recently, there was an option which allowed users to revert back to the “Classic” theme. All good things must come to an end, though. Lucky for us, Facebook has to keep backwards compatibility for older browsers. By forging our client to seem as though we’re using an antiquated browser, Facebook returns the Classic theme instead of the newer one. This only works for Firefox unfortunately, but I’m sure you could do something similar in Chrome.

1) Download the User Agent Switcher
2) Restart Firefox
3) Go to Tools -> Add-Ons
4) Click the Extensions tab, select User Agent Switcher, then click Options
5) Click User Agents, and then Add
6) Fill the fields with the following information

Description: IE5.5
User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0 )
App Name: Microsoft Internet Explorer
App Version: 4.0 (compatible; MSIE 5.5; Windows NT 5.0)

All set!

Edit, 1:54 AM EST 16 Sept 08
Seems like this may or not still be working. Varied accounts.
If it doesn’t work, try this link:
http://www.new.facebook.com/?fbnew_opt_out=1

I’ve been lucky enough to stumble my way into a decent amount of traffic on one of my endeavours. Suffice it to say I am impressed with Django’s ability even in developmental mode (yeah I know, bad practice- never go live without a perfect product) to handle multiple requests and handle them well.

Unfortunately, I’m not large enough to delegate my IT tasks to a team. This leaves me playing server administrator as well as developer. Everything I’ve deployed in the last year or so has been Django-FCGI based so I managed to fall out of touch with setting up a production PHP environment. I finally needed a PHP application, so I decided to do some research. I’ve been using nginx religiously for sometime now and it turned out I couldn’t just do the typical `mod_php5, AddHandler` etcetc that you can do with Apache 2. There was tons of misinformation being spread around and plenty of bad guides.

For my own archival purposes and perhaps your curiosity, this is my optimal server configuration for my image host site.

1) aot-get install mysql-server
2) apt-get install nginx (Alternatively, you can install from source.)
3) apt-get install subversion
4) apt-get install python-mysqldb
5) Install django:
python -c “from distutils.sysconfig import get_python_lib; print get_python_lib()”
svn co http://code.djangoproject.com/svn/django/trunk/ django-trunk
ln -s /home/andrew/django-trunk/django /usr/lib/python2.5/site-packages/django
ln -s /home/andrew/django-trunk/django/bin/django-admin.py /usr/local/bin
6) apt-get install python-imaging
7) apt-get install make gcc autoconf automake

The more difficult part is getting php installed with a fastcgi initiator. This was the best guide I ultimately managed to find. It fit my purposes because it compiled lighttpd, but didn’t make install it (which would have subsequently cluttered up my /usr/bin and /etc/init.d/, and more importantly raced with nginx to see which would first bind to :80).

If you have to install fcgi-php5 on Ubuntu with nginx, I highly suggest setting it first up in a test VMware environment a few times before actually installing on your prod server. (This is a good rule of thumb anyways, but sometimes configurations are so simple you don’t need to have a testbed. This is not the case in my experience).

I recently deployed a new instance server in order to run a production instance of one of my applications. I copied over my Django app folder from my development machine to the production environment, modified a few entries in urls.py and downloaded the Django trunk. After symlinking everything and getting nginx to proxy_pass to 8080, I execute `python manage.py runserver 8080` and query the webpage. I’m greeted with a yellow Django error page. How nice.

Turns out the trunk I pulled was 1.0-ALPHA, an upgrade the 0.97 trunk I was using on my dev machine. I spent 10 minutes trying to fix the app before giving up. At 6 A.M. everything becomes cumbersome. I just tarred up my old trunk and threw it on my new box.

For the record, Python/Django is my language/framework of choice. It does almost everything flawlessly and efficently. It handles url’s beautifully, slug integration is a godsend, the user/sessions management leaves nearly nothing to be desired, I love the ORM design, and it seems to scale well enough. It could use something like Migrations for Rails, and the Model-Template-View instead of standard MVC design is a bit counter-intuitive. Other than those two minor gripes, I’ll be contentious and say it’s the best environment available for 95 percent of the web.

One of the reasons why PHP3/4 was so successful, despite being a terribly constructed language, was an upgrade wouldn’t break backwards compatibility. Incidently, PHP5 is expected to break a quite few legacy apps. Oh well, I guess that’s good news for those developers who stuck with PHP; I expect they’ll be seeing a lot of work coming their way.