What is Thread Pooling?

Thread pooling is the process of creating a collection of threads during the initialization of a multithreaded application, and then reusing those threads for new tasks as and when required, instead of creating new threads. The number of threads for the process is usually fixed depending on the amount of memory available, and the needs of the application. However, it might be possible to increase the number of available threads. Each thread in the pool is given a task and, once that task has completed, the thread returns to the pool and waits for the next assignment.

The Need for Thread Pooling

Thread pooling is essential in multithreaded applications for the following reasons.

• Thread pooling improves the response time of an application as threads are already available in the thread pool waiting for their next assignment and do not need to be created from scratch

• Thread pooling saves the CLR from the overhead of creating an entirely new thread for every short-lived task and reclaiming its resources once it dies

• Thread pooling optimizes the thread time slices according to the current process running in the system

• Thread pooling enables us to start several tasks without having to set the properties for each thread

• Thread pooling enables us to pass state information as an object to the procedure arguments of the task that is being executed

• Thread pooling can be employed to fix the maximum number of threads for processing a particular request

The Concept of Thread Pooling

One of the major problems affecting the responsiveness of a multithreaded application is the time involved in spawning threads for each task.

For example, a web server is a multithreaded application that can service several client requests simultaneously. Let's suppose that ten clients are accessing the web server at the same time:

• If the server operates a one thread per client policy, it will spawn ten new threads to service these clients, which entails the overhead of first creating those threads and then of managing them throughout their lifetime. It's also possible that the machine will run out of resources at some point.

• Alternatively, if the server uses a pool of threads to satisfy those requests, then it will save the time involved in the spawning of those threads each time a request from a client comes in. It can manage the number of threads created, and can reject client requests if it is too busy to handle them. This is exactly the concept behind thread pooling.

The .NET CLR maintains a pool of threads for servicing requests. If our application requests a new thread from the pool, the CLR will try to fetch it from the pool. If the pool is empty, it will spawn a new thread and give it to us. When our code using the thread terminates, the thread is reclaimed by .NET and returned to the pool. The number of threads in the thread pool is limited by the amount of memory available.

To recap then, the factors affecting the threading design of a multithreaded application are:

• The responsiveness of the application

• The allocation of thread management resources

• Resource sharing

• Thread synchronization

Responsiveness of the application and resource sharing are addressed by this chapter on thread pooling. The remaining factors have been covered in the previous chapters of this book.

The CLR and Threads

The CLR was designed with the aim of creating a managed code environment offering various services such as compilation, garbage collection, memory management, and, as we'll see, thread pooling to applications targeted at the .NET platform.

Indeed, there is a remarkable difference between how Win32 and the .NET Framework define a process that hosts the threads that our applications use. In a traditional multithreaded Win32 application, each process is made up of collections of threads. Each thread in turn consists of Thread Local Storage (TLS) and Call Stacks for providing time slices in the case of machines that have a single CPU. Single processor machines allot time slices for each thread to execute based on the thread priority. When the time slice for a particular thread is exhausted, it is suspended and some other thread is allowed to perform its task. In the case of the .NET Framework, each Win32 process can be sub-divided logically into what are known as Application Domains that host the threads along with the TLS and call stack. It's worthwhile to note that communication between application domains is handled by a concept called Remoting in the .NET Framework.

Having gained a basic understanding on concepts of thread pooling and the .NET process, let's dig into how the CLR provides us with thread pooling functionality for .NET applications.

The Role of the CLR in Thread Pooling

The CLR forms the heart and soul of the .NET Framework offering several services to managed applications, thread pooling being one of them. For each task queued in the thread pool (work items), the CLR assigns a thread from the pool (a worker thread) and then releases the thread back to the pool once the task is done.

Thread pools are always implemented by the CLR using a multithreaded apartment (MTA) model by employing high performance queues and dispatchers through preemptive multitasking. This is a process in which CPU time is split into several time slices. In each time slice, a particular thread executes while other threads wait. Once the time slice is exhausted, other threads are allowed to use the CPU based on the highest priority of the remaining threads. The client requests are queued in the task queue and each item in this queue is dispatched to the first available thread in the thread pool.

Once the thread completes its assigned task, it returns to the pool and waits for the next assignment from the CLR. The thread pool can be fixed or of dynamic size. In the former case, the number of threads doesn't change during the lifetime of the pool. Normally, this type of pool is used when we are sure of the amount of resources available to our application, so that a fixed number of threads can be created at the time of pool initialization. This would be the case when we are developing solutions for an intranet or even in applications where we can tightly define the system requirements of the target platform. Dynamic pool sizes are employed when we don't know the amount of resources available, as in the case of a web server that will not know the number of client requests it will be asked to handle simultaneously.

Caveats to Thread Pooling

There is no doubt that thread pooling offers us a lot of advantages when building multithreaded applications, but there are some situations where we should avoid its use. The following list indicates the drawbacks and situations where we should avoid using thread pooling:

• The CLR assigns the threads from the thread pool to the tasks and releases them to the pool once the task is completed. There is no direct way to cancel a task once it has been added to the queue.

• Thread pooling is an effective solution for situations where tasks are short lived, as in the case of a web server satisfying the client requests for a particular file. A thread pool should not be used for extensive or long tasks.

• Thread pooling is a technique to employ threads in a cost-efficient manner, where cost efficiency is defined in terms of quantity and startup overhead. Care should be exercised to determine the utilization of threads in the pool. The size of the thread pool should be fixed accordingly.

• All the threads in the thread pool are in multithreaded apartments. If we want to place our threads in single-thread apartments then a thread pool is not the way to go.

• If we need to identify the thread and perform various operations, such as starting it, suspending it, and aborting it, then thread pooling is not the way of doing it.

• Also, it is not possible to set priorities for tasks employing thread pooling.

• There can be only one thread pool associated with any given Application Domain.

• If the task assigned to a thread in the thread pool becomes locked, then the thread is never released back to the pool for reuse. These kinds of situations can be avoided by employing effective programmatic skills.

The Size of the Thread Pool

The .NET Framework provides the ThreadPool class located in the System.Threading namespace for using thread pools in our applications. The number of tasks that can be queued into a thread pool is limited by the amount of memory in your machine. Likewise, the number of threads that can be active in a process is limited by the number of CPUs in your machine. That is because, as we already know, each processor can only actively execute one thread at a time. By default, each thread in the thread pool uses the default task and runs at default priority in a multithreaded apartment. The word default seems to be used rather vaguely here. That is no accident. Each system can have default priorities set differently. If, at any time, one of the threads is idle then the thread pool will induce worker threads to keep all processors busy. If all the threads in the pool are busy and work is pending in the queue then it will spawn new threads to complete the pending work. However, the number of threads created can't exceed the maximum number specified. By default, 25 thread pool threads can be created per processor. However, this number can be changed by editing the CorSetMaxThreads member defined in mscoree.h file. In the case of additional thread requirements, the requests are queued until some thread finishes its assigned task and returns to the pool. The .NET Framework uses thread pools for asynchronous calls, establishing socket connections, and registered wait operations.