asynchronous nginx plugins with python and tornado

Очень лаконичная и очень технически интересная статья:

by returning X-Accel-Redirects and using multiple tornado processes you can use python as an async plugin language for nginx.

I've been using an architecture at work for building video streaming servers that I've not seen anybody else describe. I'm combining the ease of writing, reading and testing python with the performance of nginx. I've been pretty happy with the maintainability of the result. It's been taken from separate cache and origin to cache+origin and from Smooth Streaming to HLS; and I can hit 1000 qps and 8 Gbps of video with it (with pretty beefy hardware though).

This setup wouldn't serve 1000 qps, missing are:

• You'd want to use more tornado front-end processes so that python isn't the bottleneck
• If there is any cache of any kind (e.g., vfs) in the backend, you want to use content-aware backend selection
• You'd cache the mysql results rather than getting them out of the db every time
• In the front-end, you'd want to merge all simultaneous requests for the same URL into one
• Nginx should cache the service responses

http://blog.mikie.iki.fi/2014/10/writing-asynchronous-nginx-plugins-with.html

Смысл в том, что запросы через Nginx направляются в аппликуху Tornado, где уже раскидываются по бэкендам. Нет, смысл в том, что логику распределения нагрузки можно написать на Python, вот:

http://linuxfiddle.net/f/5f96743c-3d62-40f7-89ef-f64460e904c1

Я больше получаса разглядывал эти 200 строк кода, пока разобрался, кто на ком стоял. Очень интересно.

Кстати, хотите поиграть с виртуализацией в стиле Docker? Пожалуйте в http://linuxfiddle.net/

original post http://vasnake.blogspot.com/2014/11/asynchronous-nginx-plugins-with-python.html