Jeff Longland

Relax, don't worry – have a home brew!

#OSCON – AMQP in Production: Building a High Performance Compute Cluster

leave a comment »

  • Presenter: Nicholas Silva, Box
  • Not going to discuss how they decided on RabbitMQ at Box.  But briefly: low latency, high throughput, high reliability, open source, and AMQP libraries for pretty much every platform.
  • Topic exchanges allow for more complicated routing to queues, rather than pushing directly to a queue.
  • Demo of how easily you can get up and running RabbitMQ – yup, that’s easy.
  • For PHP folks making long-poll requests, take a look at rabbitmq_shovel and just run another rabbitmq instance on the webserver.
  • Monitoring with Nagios plugin for rabbitmq
  • Metrics and trending with OpenTSDB
  • Using an exchange for analytics info
  • Over the last year, have introduced clustering and HA queues
  • Need to distribute over more datacentres? Federation Exchanges and Shovel
  • Worker crash recovery mostly works…  but if you have a huge job that kills a worker, it’ll kill the rest of the workers too.
  • Messages are opaque, so you can’t see into them – which you may find problematic at times.
Advertisements

Written by jlongland

July 20, 2012 at 10:37 am

Posted in OSCON 2012

Tagged with

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: