I have been so incredibly busy lately that I have not had anytime to write here. Anyway I just wanted to take some time to talk about something cool I have been doing for a project I have been working on outside of work.

By now everyone knows what Memcached is so I won't bore you with an introduction. Basically it is an insanely simple caching system designed to eliminate load on the database.

While in an ideal world you would just throw cache on everything and watch your application scale to infinity and beyond, things aren't always that simple. The problem I'm going to show you today has to do with pagination and cached database queries. I will try to make this pretty straightforward so anyone can follow.

There is a user named Pablo Picasso who is using your site. He makes Cap-Sacs for a living and has received 15 messages from people on the site who are interested in purchasing one from him. This means that page 1 displays messages 6 to 15 (the 10 most recent messages) and page 2 displays 1 to 5.

Let's now imagine you cache these pages of results. A new user comes to your site and wants to buy a pink Cap-Sac. They send Picasso a message so your application says, "Oooh Picasso just got a new message. That's pretty rad. Since the query to get his messages is Cached let's just drop the cache for page 1." Now what happens you ask?

The new message shows up on the top of page 1 as expected so what's the problem?? Picasso now has 16 messages. Page 1 will display messages 7 to 16 which is fine, but since page 2 was cached from before it is going to show messages 1 to 5 still. This means that as long as the cache lives poor message number 6 (the last message on page 1 before the new message was recieved) gets dropped from the result set completely. Not to mention that page 1 thinks Picasso has 16 messages while page 2 still thinks he has 15.

This can also allow duplicates of the same item to show up if for example the user had 22 messages and page 1 and 2 were cached but page 3 wasn't. In this case a new message would cause page 3 and page 1 to be correct but since page 2 is still hitting cache, the last item on page 2 would be the same as the first item on page 3.

So what can we do about this? I see a few options

1. Don't use memcached for paginated result sets
2. Live with missing and/or duplicate items from time to time in your pagination
3. Select everything at once and handle the pagination at the PHP level instead of in the database
4. Use Pseudo cache namespaces to invalidate all cache keys for paginated queries

And here is my analysis of the options:

Option 1

This seems to be the approach that is often taken where I work, and of course it will work, but what happens if the data rarely changes? In that case it seems rather silly to continue to hit the database over and over for the same data.

Option 2

Surprisingly I don't think option 2 is a terrible option because it will definitely scale pretty well. Facebook search results have duplicate items all over most likely for this very reason, but since, in my example, I am thinking about displaying accurate data to the user I would stay away from this. For search results I think this is a much more viable option.

Option 3

This option is another interesting one. It would arguably scale better than any of the other options I have proposed although the problem here is what if Lady Gaga is using your site and she has 15,000 messages from fans. We don't want to have to select 15,000 items from the database if this were to miss cache.

Option 4

Well considering the title of this post I think you all knew this is the road I was going to go down. You get the performance benefits of caching, and you get 100 percent data accuracy.

The approach here is really quite simple and similar to the example found on the memcached FAQ page.

What I have chosen to do is use the DAO name as my main namespace identifier along with the unique userId. Please understand this is a very crude implementation just to illustrate my idea. It is more or less procedural code so I would not recommend doing it like this. I have been using a Cache_Namespace class to handle common logic such as generating the key and grabbing/setting the generation version/incrementing the version. Anyway here goes:

I tried to comment that code pretty well. As you can see the version is used as part of the query cache key so incrementing its value by 1 will make all calls to that method not hit cache. If you have more than one method you want to invalidate you can give them the same namespace key as seen in line 11.

Okay that is all for now.

Just when you thought I had run out of material to write about I am back! I have a few topics still lined up, but I have been so busy that I haven't had time to write anything.

The project I have been working on at work has lots of pagination in it, and the other day I was really frustrated with how difficult it was to do simple pagination and how many lines of code it took in the controller. Here I am going to present to you a pagination class I wrote and a usage example.

I am aware that there is a Zend_Pagination class in Zend Framework, but I'm not crazy about it. It offers some nice view helpers to draw pagination on a page, however, it is over 900 lines long, contains a lot of code you probably will never need, and is missing simple functions that you may need such as determining the page offset or the first and last item on the current page.

Here is the class (download) I have written in its entirety (all of 239 lines including comments):

Keep in mind this is a rough draft, but it seems to work pretty well. All you need to tell the class is 3 things:

1. The current page number in the constructor
2. The number of items you would like to see per page in the constructor
3. The total number of items that you will be paginating in the setTotalItems() method

Now here is an example of this class in action (controller):

Here is the method in the User_Dao to get the users from the database:

There are a few things to note here. SQL_CALC_FOUND_ROWS is a totally awesome mysql function that calculates the number of rows that would be returned if you did not include a LIMIT on the query. This is what is used to tell the pager how many total items there are. Also the DAO here is returning a collection. To see how that works check out this post I wrote a little while ago.

Since PHP passes objects by reference by default we can now use the same Pager object that we instantiated in our controller in our view. That would look something like this:

I'm probably forgetting something here, but I think it is a good start and in my opinion it certainly eliminates any headaches caused from having to deal with pagination.

One thing I have realized in my journey across the internets is that there is no simple, clear way to make an overlay using only one div (at least that I could find). If you have no clue what I am talking about click on the "add comment" link, and you will see. The page dims, the curtain goes up and the audience applauds.

Just a small disclaimer: The CSS that I am about to show you will work just fine in all browsers except for Internet Explorer 6 (what a shock!). I will show you the necessary Javascript (using jQuery) needed to fix this.

The first step is to simply add one div inside the body tag like so:

The next step is to add the following to your stylesheet:

To go through this step by step:

display: none

This simply means don't show this div. The reason is obvious since we don't want our entire page to load with a colour on top of it. One thing to note is that we will have to use javascript to set the display here to block or something other than none when we want this to show up on the page.

position: fixed

This means that the top left point of the div is fixed at point (0,0) or whatever point you specify. This means intelligent browsers can scale the width and height to cover the whole window

z-index: 1000

You can think of the z-index as if you have many sheets of paper on top of another. Each layer is one above the previous. Unfortunately it doesn't always work as advertised. Setting this at 1000 just ensures that it is above all content on the page although it can really be any number greater than zero.

-moz-opacity: 0.5

This is a special css property that is specific to Mozilla Firefox. Like any of the other opacity settings: play around with it until it looks how you want it to.

opacity: .50

This is the default opacity property in css. Safari uses this.

filter: progid:DXImageTransform.Microsoft.Alpha(opacity=50)

This whole nonsense is specifically for Internet Explorer 7. As you can see Microsoft so slyly inserted their name here.

Now you may be thinking, "That's it? We are done? That was so simple!". If only the world were that simple. We still have to take Internet Explorer 6 into account.

I'm pretty sure if Internet Explorer 6 were a child in elementary school it would be the last kid picked in gym class. Hmmm ... Actually it might be the kid that skipped gym class all together and was in the nurse's office being treated for its terrible diseases.

Anyway, the first thing we have to do is create a stylesheet specifically for IE6. To do that we use a CSS conditional statement in our html like so:

Now inside of overlay_ie6.css we add the following styles:

This is because IE6 does not like position: fixed or the Microsoft filter used for IE7. Therefore we must do this so that it will begin to play nicely. The final step is adding a few lines of Javascript only for IE6. I am using jQuery cause it is my framework of choice and is very speedy and light weight. This is definitely a hack, and I do not really like hacks so if anyone has come across this problem before and has a more elegant solution I urge you to leave me a comment.

This code is fairly straight forward, but basically what it is doing is figuring out a pixel width and pixel height to set the overlay to for IE6. IE6 does not like using percentages in this case. First we get the height and width of the current window. Then we get the height of the window with content (assuming all of the horizontal content fits within the width of the window - no side scrolling). Whichever height is greater is the height we use at the end to explicitly set the height and width of the #overlay div.

That's all for now!

I dare anyone to tell me a better way to spend the evening after an 11 hour workday staring at code than writing a blogpost about code. Haha, you'll have to forgive me if any of my writing seems scatterbrained that is most likely because my head is spinning like one of those amusement park rides that goes completely upside down and then stays there for 10 seconds before moving again.

Anyway I have only written a couple serious posts here and already I have had a few people ask me how I did my syntax highlighting. It is no secret online syntax highlighting solutions are not super amazing. They are pretty good, sure, but not awesome. That is where TextMate comes in. TextMate has a little known feature found in the Bundles menu under TextMate: Create HTML From Document.

That's it! Well not quite. The HTML TextMate generates is not perfect. For one it uses the <pre> tag which has been deprecated in XHTML 1.0 Strict. To fix that is easy just replace <pre with <div in the html as well as the styles. The next thing to do is move your styles out into their own stylesheet. One bizarre thing is that any user generated colours will appear in your stylesheet in the format color: rgba(255, 0, 9, 0.87); which will not work in Internet Explorer so you'll have to convert those to their hex values.

Also removing the <pre> tag will break the code in Internet Explorer:

The fix is super simple, however. This is the one I am using on this site:

I added the extra margin in there just to give it some breathing room. Also if you want to make a scrollable box you can just add the "limit-height" class to the main div that holds your code. Another thing to note is that for some reason the "line-num" span class is in single quotes. It still validates, but you might as well make it double quotes for consistency. I am probably forgetting something, but I can barely keep my eyes open at the moment so I will add it if I think of it, but yeah very cool. Enjoy!