Using Varnish to scale NYtimes.com
I had a chat with Jacob Harris to talk about New York Times, working on the interactive newsroom team, and how they use varnish to scale their apps without breaking a sweat.
Show Notes
The interactive newsroom team makes interactive apps that go up on the NYTimes home page: elections, oscars, olymipics, etc.
News editing staff with done thing such as election, water polluters, football player, travel, face a lot of charms and challenges but its fun.
They are using Rails Apps for 3 years running with EC2 tools couple of proxy come in and ice of varnish of cache in the traffic.com
They use MySQLfor databases and it's awesome according to Jacob
The Mango IT is very useful for photo uploads forms, data that could anything, pictures, and colors. Also used in Twitter hacker but mostly in New York Times
In easy tunes, they have few backend server applications a loader and bouncer and 1 varnish cache. Miscellaneous machine, Mysql- RDS mostly use.
They looking for using Red-Hot Proxy Season and
Proxy Spot
If there's a events like election and Oscar award night they use proxy, bc2, 3 or 4 ec 2, 4 additional web server, for traffic and a large of ec2 proxy different from day to day.
New York times have a bunch of 40 live application, mango, proxy configuration, for market folks sever. They use varnish for cache, it’s a good deal to the traffic.
Varnish is easy to set up it has VCL language, it’s a DSL, and main feature of Varnish is its configuration language. When the program comes in it takes action and if your retrieving on the backend it easy to store cache.
Varnish is technically single point of failure. But it’s a way of correcting. Central point cache goes to application and it’s very quick, do not need to go further.
Varnish is very powerful to decomposing the site in a small sub page in the substituting together in cache with separate time outs, different TTPL, can send full dynamic in the cache of the site.
Olympics apps, New York Times have a graphic group; they are people good in JavaScript and do a lot of work with Flash Apps,and maps wizard for IPod.
They have 1 varnish to all of the traffic.
New York Times Request per day, 30 to 60 hits for seconds for traffic.
Varnish can used for open source C 10K caches.
Varnish cache control is private stuff.
Varnish VSL, browser can be cache things and sometimes its use by JavaScript or a flash thing by adding step on request.
In general, varnish follows the same logic a browser cache, it’s a browser header. It cache the full URL, can check longer, with particular info like PPIT, CSC, Google, CC Sql, which cached keys.
Jacob said that Varnish is his favorite cached tools right now.
