05
May
I sat down with Mark Imbriaco to talk about life as a server admin at 37signals. Mark was the first official server admin that 37signals hired and has overseen a lot of their current architeture.
Show Notes
- 37signals has four main apps: Basecamp, Backpack, Campfire, and Highrise.
- They have a ton of small apps including: Job board, Signal vs Noise, Changelog, and Queen bee.
- 37signals has 40 servers, and a total of 125 OSes running.
- 3 Server admins work at 37signals.
- They use capistrano and chef to automate a ton of stuff.
- Mark tweeted it is hard to automate with chef when he can do it quickly on his own.
- Bare metal servers showed a 15% increase using production traffic, over KVM.
- 37signals uses Cisco Load Balancer, Nginx, Haproxy, Unicorn, and MySQL.
- Campfire uses redis now.
- Jamis Buck and Mark Imbracio switched the Campfire poller to Erlang.
- Erlang allowed them to switch from 300 fcgi processes to 3 erlang processes.
- Josh uses Eventmachine and RabbitMQ to power tweethopper
- Randy wanted to know if 37signals was hiring, and they aren’t right now.
- 37signals uses Nagios, Website Pulse, Circonus, and Ganglia to monitor their apps.
- 37signals uses a weekly on-call rotation now.
- Downtime is managed by fixing the app first and analysis later, sometimes hard for a developer to do.
- 37signals has around 60TB of data in Amazon S3.
- Mark says the cloud gets expensive when you need to scale up.
- They purchased 150TB San setup to start replacing their S3 setup
- Deploys push to the web servers and rails servers, so Nginx has the static content.
- They use mogileFS, they switched from NFS
- Josh Owens has used GlusterFS
- Dreamhost has cheap bandwidth!
- They are using Schooner MySQL appliances, they are seeing significant performance increase - around 300%
- 37signals uses the Percona flavor of MySQL (extraDB).
- Mark needs to do more Nuts and Bolts videos on SvN!
