An experiment in my writing: parsimony of language. I’ll be trying to say more with fewer words because I’ve picked up a bad habit of mid-level writers: upon learning four ways to state an idea, use all four instead of choosing the best! Such writing is a recipe for boring communication.
I leapt over a leg press machine so I could quickly move to another part of the weight room floor.
“How old are you, man?”
A random guy at the gym, one who I’ve talked to only once before, surprised me with his question.
After laughing for a bit to regain my composure, I answered: “I’m 26.” Twenty-six, I thought. That sounds… old. I’m over halfway through my twenties. That was quick.
With a mixture of admiration and puzzlement, he continued: “Well the way you move, you look like a fuckin’ teenager.”
I took it as a compliment. And really, that’s the thing I love most about exercise — enjoying the miraculous machine of the human body. Look what these limbs, with focused effort and practice, look what they can do! Look how we can jump and move ourselves, how we can push ourselves to go faster, how we can ask our mechanics to carry out a task and how they oblige.
Srikumar Rao gave a talk a few TEDs back (2009) called “Plug into your hard-wired happiness”. I’ve transcribed parts of it in a kind of note format, because it was really interesting:
There is NOTHING you have to get, do, or be in order to be happy.
An if-then model: “If this happens, THEN we will be happy.” E.g. If I were to get a better job, if I were to get more money, if only I was married. The thing that’s different between you now and you 10 years ago is what is the particular “if” you are focusing on.
Think about your life 10 years ago. Odds are pretty good that many of the things you wanted 10 years ago, you now have. Where has that left you? In exactly the same place! What we don’t realize is that the model itself, the if this happens then I will be happy model, is flawed. Instead, we spend an enormous amount of time changing the “if”.
Actions are within your control, the outcome is completely outside your control. “When it’s over and you look in the mirror, did you do the best that you were capable of? If you did the best that you were capable of, the score doesn’t matter. But I suspect that if you did the best you were capable of, you will find the score to your liking.” That is investing in our process. What we do, is the exact opposite — THIS is what I want, and I want it so desperately. You are always investing in the outcome. Focusing on the outcome is fine, but investing in the outcome is a surefire recipe for failure. Invest in the process. Focus on the outcome only to the extent that it gives you direction; then you invest completely in the process. If you succeed, wonderful. If you don’t succeed, STILL wonderful.
I think this can also be applied to the narrative versus experiential that I KEEP coming back around to. Narrative: you keep telling yourself the story of yourself, losing track of what matters. Experiential: you allow things to happen to you, because you are present in the moment.
Here’s a quick summary if you haven’t time to read the whole thing:
Solaris 5.11 (virtual: Joyent SmartMachine)
PHP 5.3.6 with PHP-FPM: 4 instances running, 10meg APC cache
nginx 0.8.53
Pax 1.0 (my silly self-coded website software… and yes, oops there’s already software with that name)
120 megs of RAM used
Load tested using blitz.io: 9 million+ daily hit capability
The point: I’m not doing anything exotic. I’m doing this as a hobby. This type of performance should be the rule, not the exception, for small websites. Many sites need some improvement to get to that point.
History
My site has only been linked to by John Gruber’s Daring Fireball twice. In 2007, I wrote a piece about Wozniak’s Prius’ top speed. In 2009, I wrote about the sad state of statistical analysis in tech journalism.
He even liked my site’s design! Geek excitement! Sorry. Anyhow…
While Mr. Gruber’s site does tend to crash those he links, my server was thankfully spared the full onslaught of the Daring Fireball audience — the topics I addressed were minor, transient little additions to the dialog between Mr. Gruber and his readers. So, I survived those bursts of traffic. But early this year, I got to thinking: what if my muse humored me and I actually produced something popular? Could my server get the required number of pages onto people’s screens without melting or exploding?
So, in January, I began to refocus my coding efforts on the software powering this website.
Thank You, Shopify
My first goal was to get my PHP execution time down into the realm of Daring Fireball’s. If you pull up the markup on DF’s front page, you’ll notice a commented bit about how long it took to produce:
<!-- 0.0003 seconds -->
After checking this for about 9 months, I can tell you this almost always reads that number: 300 microseconds. This is about one third the time a camera flash illuminates. That’s, well, pretty quick. When I started, my software was taking about 0.25 seconds (250 000 microseconds) to produce the front page of my website. I needed to improve performance by over 800x.
I had already written a nice little PHP class to cache using either APC or memcached, but I had been stymied by how to expire things correctly. Doing this for a hobby, and therefore not being steeped in the best practices of caching, Tobias Lütke’s article The Secret to Memcached hit me like a FREAKIN’ THUNDERBOLT:
At the beginning of each request we load a shop object which we pick depending on the incoming host name. We use the fact that we always load this shop model anyways and add versioning to it. This version column is incremented every time we want to sweep all caches.
AHA. And it works beautifully. Whenever anything in the DB is updated, I update the cache by incrementing that version number; because it is incorporated into all cache ids, all cache ids change. Expired cache items are never explicitly marked as such, they are simply no longer accessed and rotated out when the cache fills up.
Of course, in retrospect, it makes sense to let the cache itself manage rotating out expired items, but it took me a while to realize that. And of course, you don’t understand something until you think it’s obvious. Anyhow, requests that come in while the versioning is being updated still load the stale version. A new cache id is produced because the incremented version number is hashed in.
My blog is quite light on input… and traffic, so worrying about cache stampedes is a bit much right now. After a few weeks running, PHP’s APC gives a nice hit/miss ratio:
This caching (I’m using APC right now) got my page load times down to about 170 microseconds for most pages, and 400 microseconds for the front page, which takes some time to set a cookie or two. The reason for those cookies follows.
Faking Dynamic Features Using Inline Caching
The title of this section could be yet another preposterous acronym: FDFUIC. What it means is, caching can be aggressive but still deliver dynamic features to your visitor.
This was the challenge: I wanted to give each user a personal update on what was added to my website since they last visited. But I wanted to cache only one version of the front page… and serve it to everyone. These two goals seem mutually exclusive. They aren’t. Here’s the solution (scalability notes after the implementation):
<time datetime="2011-08-27T19:15:38Z" pubdate style="display:none"><time> of a node is after the cookie time, then change the style to make that comment visible.<time>s before the cookie time.<article> node). Mark nodes with a red dot if their <time>s are after the cookie time.Interesting tidbit here. I actually used a modified version of John Resig’s “Pretty Date” code snippet, one he put together to live update time on nodes in a twitter clone he was thinking about. The final function I ended up with is available here.
An image follows to explain how it all comes together:
So, if you want to show someone what is new since they have been gone, your first instinct may be to do it dynamically. My point here: for small sites, that’s not always the best solution. Here, we use the fact that the site is small to our advantage: we can easily prepend 60 days worth of comments, but we don’t have too much spare processing power or RAM to dynamically assemble the front page for every user AND maintain robust performance.
Scalability note: if you have a higher traffic website, perhaps you should only set the expiration time to 5 days. Then you won’t be prepending your front page with a lot of unnecessary data/comments (from the other 55 days). If the user visits less frequently than every 5 days, well, then they have a lot to catch up on anyway, and you might as well not overload them with new stuff.
Trial Run
This past May, Hacker News picked up a long piece I wrote about my efforts to improve infinite scroll. I was thrilled that it was pretty popular. I was thrilled my server didn’t melt! However, it came close.
I checked my running processes and found that Apache’s MaxClients parameter was not at all the right fit for my little 256-megs-of-RAM server.
nginx, PHP-FPM
After a few days of research, I installed nginx and PHP-FPM. Unlike the Apache client explosion that happened under load, I get much better control over processes with this set-up. PHP-FPM is set to a max_children of 6 and as I write this has 4 processes running.
nginx, of course, is a beast (in the best possible way: rock solid, low memory usage).
A little tidbit about how PHP & nginx communicate: instead of using a port (with the corresponding overhead), nginx is communicating with PHP over a Unix socket. The relevant part of the config files are as follows.
PHP-FPM:
listen = /tmp/php5-fpm.sock
nginx’s PHP location block:
fastcgi_pass unix:/tmp/php5-fpm.sock;
Fast fast fast.
Memory Use
With a little prstat -Z -s size (remember, this is Solaris), my RSS is currently at 115megs. I’ve run the following rush at blitz.io:
--pattern 1-250:60 -T 4000 -r california http://tumbledry.org/
Yes, the timeout is increased. Give me a break: I can’t make miracles!
I never knew servers could be this efficient. I have a lot to learn.
George Monbiot’s “The Lairds of Learning” (via HN) is exceedingly important:
But the academic publishers get their articles, their peer reviewing (vetting by other researchers) and even much of their editing for free. The material they publish was commissioned and funded not by them but by us, through government research grants and academic stipends. But to see it, we must pay again, and through the nose.
As a future dentist, I don’t actually know how I will stay up-to-date without access to the hundreds of specialty journals available at the U of M. Without institutional access, it costs too much to stay abreast of the current research. As a result, one’s continuing education money goes toward funding established speakers (“Continuing Education” credits) — this is not preferable to getting information first-hand from primary sources.
This is an experiment. I will now post the following to my Facebook account:
The only thing I want more than a really good Democratic presidential candidate is a Republican one.
Likely response: apathy. Time will tell, though.
Sewing project #1 completed: just finished stitching up the crotch hole in my 3 year old pair of scrub bottoms! Let’s see if we can’t get them to last for 9 more months.
Today, I saw a four year old (I didn’t get a look at her teeth so I can’t tell you for sure, but I’m pretty sure she was four) on a cell phone. Having a conversation. While walking next to her father. I’m always making sarcastic remarks about how young kids are starting earlier and earlier with cell phones, but confronted with the reality of it in person, it made me feel more sad than smug.
What could someone that age have to talk about that was so important it couldn’t wait to be said in person?
Three patients left until my last dental school summer break EVER. This is extremely exciting! This fall, it is the beginning of the end. It is a little odd to be starting summer break in August, though. In high school, I was lamenting the twilight of summer at this point in the year.
Since the humidity and heat decided to die down for a day, it has been feeling downright cool outside — 70° with a pleasant breeze. Things smell different — there’s a crispness that isn’t fall but isn’t the oppressive July heat, either.
Halfway through yet another rotation (pediatric dentistry), I’m beginning to realize that there is a point in my life when I’ll be done with dental school. At that point, I’ll have a world of options in front of me. Like a river delta opening into the ocean, my life will have 1000 directions where there once was one. Invigorating, right? Well, I suppose. More on that in a minute. Here’s something I wrote almost four years ago, on the private changelog for my software that powers tumbledry:
23 Jan 2008
Dental school begins in 7 months. If tumbledry is to survive four years of dental school and beyond, I must now take steps to make it ultra robust and easily maintainable.
I remember thinking at that time how long four years seemed. I mean, I was going to be hard at it for four years! (Not entirely true: I did a redesign this spring I’d never have thought possible in 2008). Despite that, I’m happy I took the steps I did, because I’m realizing that the “ultra robust” (ha!) code I worked on needs to work for MUCH longer than dental school. It’s all just beginning.
Now, about those 1000 choices. Happiness studies have produced interesting results: show a person 3 paintings and tell them they get 1 for free, no exchanges or substitutions. Now, show another person those same 3 paintings and tell them they get 1 for free and that they can exchange it for another in a week, and they’ll never be as happy as the first person. Living with limitations on our choices helps us find happiness. You lose the burden of imagining greener grass on the other side when you know you can’t go to the other side. Case in point: I LOVE where we live. The lack of air conditioning, old bathroom, ancient windows… I don’t mind them. I know, with only 1 car and me biking to school, we can’t move. Until school is over.
Then what? Suddenly, we can live wherever we want. How can you be happy in a job when you’re always imagining how you could change jobs? Same with a house. It’s a little scary. You have to put your own limitations on your choices, I guess. Bit of a challenge.
↓ More