Serving Static Files Could Be Slowing Down Your NodeJS Express Server
Express is a great web server and I have used it extensively in many of our projects. It is fast, scalable and pretty powerful. Express rocks.
Every now and then we find something interesting to learn in Express. Mostly a quirk in how we may be using it and how to use it better. This is a story of one such quirk.
It was a regular monsoon day at work. Our server cluster was performing its heavy duty load serving over 10K requests per minute, per node. A typical high throughput server built with NodeJS and express. It had experienced some jamming the day before due to a new product launch.
It was great outside and our office overlooked this beautiful bridge on an overcast day.
There was absolutely no cause for this jamming. Our server was perfect, perfect I tell you. Everything was cache served, no errors etc. So reluctantly, we decided to pull up New Relic.
Now if Express is the truant in this story, then New Relic is the protagonist trying to bring it back to the normal world. Kudos to the New Relic team for their NodeJS APM monitor which is free to use for last 24 hours data. Let's credit them, lest we forget them on our way.
So here is what we found in New Relic. Every GET request consisted of a stat system call. Which means that every GET request was trying to find something from the disk.
Now disk IO is expensive and this was hurting performance of all GET calls. So we checked our code, there was no reason for the disk calls. We were not trying to look up anything at all from the disk, everything was cached.
We also tangentially explored if we were printing something to console.log which is a clear No No as per Express's Performance Guide, as such writes are synchronous. Also since these were stat calls which are typical for lookup and not disk writes so it was an unlikely culprit. So, we kept digging.
You must have heard that serving static files is best handled at the level of Nginx or CDN where the root webserver handles all static files. However, sometimes we need to serve some occasional files via our Nginx Express server which is behind our Nginx's reverse proxy. These static files were pretty basic, logo images and such. The number of requests for these files were also minuscule. You know, the kind of static files nobody notices you are serving from the in-built static file server in Express using
app.use(express.static(path.join(__dirname, 'public')));
After some digging we found that apparently all GET requests configured in the app after this line go through this static middleware. Think about it. For every GET request, Express needs to check, is this request thing /clearlyAGetRequest something on disk and should I serve it from disk ? So it goes and stat's it from the disk, finds that it's not there and then serves it via the router configured for this route.
Now thats a serious performance hit. We pulled this line in our app.js to come below all configured routes and BAM!, we saw a 15% jump in performance.
Who would have thought a carefully monitored, aged server could gain 15% with a single line of code change and yet there it was.
And thats the story of that rainy day. Server has been humming along nicely since then.
How much is a great User Experience worth to you?
Browsee helps you understand your user's behaviour on your site. It's the next best thing to talking to them.
