We have lost count of the number of failed products that we have built. However, there is one kind of failure I particularly hate.
The first product that I built eight years ago was hosted on AWS's micro tier. So the application, database and the server had merely a GB of ram to share. We were lucky to get some good PR for our product but unfortunately we didn't know in advance and guess what most of the people who landed on our site saw.
Pingdom to the rescue and we have not looked back. That obviously didn't work for our databases and services only accessible from our internal servers. So we hacked a script on the same idea and surprisingly its still pretty handy.
Why Do We Need Another Agent
You might have heard of the term MVP or minimum viable products. We have made a bunch of those. The idea is that you build something very basic and minimal and see if you can add at least some value to someone's life. Most MVP's don't live very long and that's a good thing.
There is a particular dimension in which MVP's are bad or should ideally be bad. Scale and stability is probably one of them. It's okay to write non scalable code but what I learnt from my first product is that if by some stroke of luck, your MVP does get some attention, it should probably not crash. This is where this brain dead simple free script has helped us and still helps us!
We built a simple script (now formatted into this npm command) which lists the currently running processes and simply alert us if the target process is missing.
In a nutshell, it does:
ps ax | grep mongod
If the output lists a mongodb process, it exits successfully, else it sends an email alert. We put this process in our cron tab and we have monitoring.
Code and Repository
We have published the agent here
You just need NodeJS running on your service and an active SES account (or any other email provider) configured with your from email address.
In order to get going, just
- Clone the repo
npm installto install the dependencies.
config.jswith your SES keys.
- Add handof to your crontab.
Now everytime your target process crashes, handof will send an alert.
Monitoring Production Setups
We don't recommend this method for production setups which require monitoring more metrics or require historical data.
There are some great server monitoring tools like
- Data Dog
- New Relic
that you should evaluate if many customers depend on your product.
You could also consider setting up your monitoring using open source tools like
- Grafana for your dashboards and Prometheus or InfluxDb as your time series database.
However, for your experimental setups handof, a free open source script should suffice. We would love to hear from you and improve handof!