How to reduce CPU Utilization (Windows Software Development) ?

The topic of CPU utilization is pretty challenging. I researched this for the benefits of the project and my team.

This applies to Windows platform.

Definition of CPU Utilization ?

Percentage of elapsed time that all of the threads of this process used the processor to execute instructions. An instruction is the basic unit of execution in a computer, a thread is the object that executes instructions, and a process is the object created when a program is run. Code executed to handle some hardware interrupts and trap conditions is included in this count.

Refer to this URL under 'PercentProcessorTime'


The screen capture above shows total CPU usage is 30%. This is the total 'PercentProcessorTime' of both processes and services.

Thus, CPU utilization is actually a measure of duration.


Multithreading Works In Queue

Understand that Multithreading works in queue. If you start a thread process using Threadstart or default Threadpool, everything will eventually being managed by the main process queue (which queued processes will be subjected to time-slicing). Some people called it the message pump or message loop, others call it main loop.

Number of Processor

Understand that the number of active thread (foreground thread) at a single time must not be more than the number of processor.

This implies that even though we are talking about Multithreading, but it doesn't mean that you can create as many threads as you like or when requires. For instance, if you want to generate a report of 30 days, you can't actually create 30 threads at a single time to process the data into reports and hope that you can calculate one month's report in a split second.

The fact is that if you have a duo core processor, you should only have two active threads (foreground threads) at a single time. And if you create more than two active threads, you will create bottlenecks.

Number of active threads (foreground threads) must not be bigger than the number of processor. Background threads(like those executing threads being executed via default Threadpool.) are those with lower priority than foreground threads.

Refer to this URL.

Default Threadpool

This explains why default Threadpool (system.threading.threadpool) is only suitable for short processes.

--> Because if even though Threadpool starts process requests in a queue, it doesn't have intelligent to evaluate bottlenecks in the main queue before launching the next queued process request. In other words, if you have started 50 threads (process requests) via Threadpool, these 50 requests will be started one after another immediately. It will not take the first request in the queue, start a thread to process it and then wait for the CPU utilization to be proper before starting the next request in the queue.

Thus, this will create a bottleneck because you should only have 2 threads running at a single time. And when there is bottlenecks, CPU utilization will go up (refer to the definition of CPU utilization).

But then why there is this default Threadpool ? It is useful when these 50 processes are short processes, means that if each of these 50 processes can finish within a very short time, then bottleneck can be avoided.

But when these 50 processes require longer time to finish what should you do ?

Custom Threadpool

The solution is to create custom Threadpool.

A custom Threadpool can be implemented in many creative ways, the idea is create thread using 'Threadstart' and implement a mechanism to limit the number of active threads equal to or not more than the number of processor available. (i.e Duo Core = 2)

Benchmark

What percentage level of CPU utilization is considered acceptable ? Based on Photoshop Image Merge automation process, single threaded environment (VB6), and ISO-recorder and even Adobe's LogTransport2.exe where ....


Adobe Photoshop performing image merge.


VB6 environment (single thread environment)


ISO recorder creating ISO image.


Adobe's LogTransport2.exe collecting data

25% to 50% is considered good enough.

Referring to this URL, 75% is considered to be having bottlenecks (or long queues).

Thus, if your CPU utilization ever reaches 75%, you need to optimize it.

CPU Utilization Vs Bottleneck ?

Therefore, if your CPU utilization is high it means there are bottlenecks. By reducing bottlenecks, you will reduce CPU utilization.

How to reduce bottlenecks ?:
  • Implement custom threadpool.
  • In terms of database, use ADO.NET instead of ADODB(especially for insert operation).
  • Reduce object and property deferencing.
  • Improve algorithm such as reducing the number of loops
  • A good way of quick fixing (without improving the algorithm or processes) of high CPU utilization is to slow down the processing.
Sample Code:

Refer to sample code (VB.NET Visual Studio 2008 SP 1).

This code will simulate side by side, on how two set of codes (optimized and not optimized) both which will perform the same objective, affects the CPU utilization.

Please refer to 'Document\UserSetupGuides.doc' in the zip file for details.

This sample successfully simulated that..

With Non optimized code:
  • Run using custom Threadpool, max CPU utilization = < 40%
  • Run using default Threadpool, max CPU utilization = < 70%
with Optimized code:
  • Run using custom Threadpool,max CPU utilization = < 20%
  • Run using default Threadpool,max CPU utilization = < 30%
Stay tune on next topic on how to discover bottlenecks using profiler.

Comments