As your app's characteristics change, you need to ensure you're using the optimal number and types of dynos. You also have to monitor Heroku's load balancing and memory use to verify that your app is scaling to best effect.
The two primary reasons developers choose to host their apps on Heroku are the service's performance and scalability right out of the box. This is particularly true when processing web dynos, which are any dynos that run the 'web' process type and receive HTTP and HTTPS traffic.
The random selection algorithm that handles HTTP request load balancing accommodates multiple simultaneous connections. It also responds to time outs automatically, as the Heroku Dev Center explains in the article How Heroku Works.
The Dev Center article on scaling dyno formations steps through use of the 'ps:scale' command to set the concurrency level of web and worker process types by adjusting the number dynos running each type. The number can be either an absolute number or an increment of total dynos running. Scaling to '0' stops that particular process type from running.
The 'ps:scale' command adjusts the number and type of dynos running: for a single process type (first screen), for both types (second), as an increment (third), and to stop the process type (fourth). Source: Heroku Dev Center
To maximize memory and CPU use, scale the dynos vertically. For example, scale up to two 2X dynos, down to one 2X dyno, and back to two 1X dynos, as shown in the screens below.
Scale dynos vertically: to two 2X dynos (first screen), to one 2X dyno (second), and to two 1X dynos (third). Source: Heroku Dev Center
Digging deeper into Heroku's routing mesh, memory optimization
The ability to scale dynos without having to deal with load balancing is one of Heroku's most useful tricks. The load balancer passes incoming requests to routers that locate the app's web dynos and forward each HTTP request to one dyno. Each router's per-app request queue holds the overflow once the router exceeds its limit on active requests, which for Cedar apps is 50 requests per dyno.
The routers buffer the header section of all the requests they receive. When a dyno encounters a slow client, it has the header before the body is received. You can use information in the header to determine how long you want to wait before closing the connection at the dyno by dropping the request, as explained in a Heroku Dev Center article.
The three steps Heroku recommends for optimizing memory are to use a concurrent web server, apply instrumentation to determine how well the app is handling the load, and monitor app performance, tweaking as necessary as conditions change. The particulars of the tweaking depend on the language, web framework, and the application itself.
For example, a web server for a Ruby app may need to fork multiple copies of the app to use all available RAM, while a Java app is served adequately by a parameter to the JVM to allocate a larger heap. Heroku Labs' log-runtime-metrics feature provides a clear view into the load and memory use of running dynos, including memory use, swap use, and load average. This and other information is inserted into the app's log stream.
The log-runtime-metrics function adds load and memory use information to the app's log stream. Source: Heroku Dev Center
Monitoring the performance of your apps, databases, IT operations, and business services is simplified and centralized in the Happy Apps application-management service. Happy Apps lets you set up rules so you are alerted via SMS and email whenever incidents or specific events occur. You can group and monitor multiple apps, databases, web servers, and app servers. In addition to an overall status, you can view the status of each individual group member.
Happy Apps is the only app-management service to support SSH and agent-based connectivity to all your apps on public, private, and hybrid clouds. The service provides dependency maps for determining the impact your IT systems will have on other apps. All checks performed on your apps are collected in easy-to-read reports that can be analyzed to identify repeating patterns and performance glitches over time. Visit the Happy Apps site to sign up for a free trial.