Rate Limiting

How it works

Rate limiting is implemented on the router app’s nginx configuration. See the configuration for the current limits. A user exceeding the limit is returned a static error page with a 429 (Too Many Requests) status code.

Rate limiting being turned on depends on the environment variables present when supervisor compiles the templates and launches nginx using the (digitalmarketplace-router/nginx.sh bash script).

In the case of rate limiting the DM_RATE_LIMITING_ENABLED environment variable is parsed and supplied to the templates as rate_limiting_enabled here.

The www template (which is put in /etc/nginx/sites_enabled) and the main nginx.conf ( which ends up in /etc/nginx) both contain rate limiting dependent sections.

nginx.conf

In the main Nginx config we:

  • Change the return code for rate limited requests from the default 503 Service Unavailable to 429 Too Many Requests

  • Create a new custom variable to signify if the request is a POST or not $post_remote_addr

  • Create 2 zones/ buckets for requests

    • One which catches everything that has a non-blank $binary_remote_addr (everything)

    • One which catches everything that has a non-blank custom $post_remote_addr (only POST requests)

See also

https://www.nginx.com/blog/rate-limiting-nginx/

www

In www we:

  • Statically serve too_many_requests.html on port 10823

  • State that we want to apply our rate limiting buckets to incoming requests to the server on port 80 (the application)

    • Set burst allowances for these requests

  • Define a ‘location’ where every request is proxied to localhost:10823 as a GET

  • Tell Nginx that requests that trigger a 429 should be passed to this location

Burst

Nginx rate limiting works as a ‘leaky bucket algorithm’ or ‘FIFO’ pipeline.

As of May 2021, the rate limit is 18 requests a second with 20 burst.

This means that:

  • the fastest a user can be served 2 requests is 0.06 (1/18) seconds apart.

  • if requests arrive quicker than that they are put in a queue.

  • if the queue size exceeds 20 (for that IP) they are served 429s

See also

https://www.nginx.com/blog/rate-limiting-nginx/#bursts

Deactivating rate limiting

You might want to turn off rate limiting for load/ stress testing or for debugging purposes. You can achieve this in 2 ways but both involve changing the DM_RATE_LIMITING_ENABLED variable to something that is not "enabled" and restaging the router.

The first, and preferred method, would be to update the variable in the appropriate manifest variable file.

https://github.com/Crown-Commercial-Service/ccs-digitalmarketplace-aws/blob/main/vars/preview.yml#L10 https://github.com/Crown-Commercial-Service/ccs-digitalmarketplace-aws/blob/main/vars/staging.yml#L10 https://github.com/Crown-Commercial-Service/ccs-digitalmarketplace-aws/blob/main/vars/production.yml#L10

The second (which will be overwritten to enabled if a release is triggered on the router) is to use the Cloud Foundry Command Line Interface to update the variable and restage the router:

cf target -s preview
cf env router DM_RATE_LIMITING_ENABLED "disabled"
cf restage router