Why are elementwise additions much faster in separate loops than in a combined loop?

Why are elementwise additions much faster in separate loops than in a combined loop? 💨🔀🔄

Have you ever wondered why elementwise additions are much faster in separate loops than in a combined loop? 🤔 In this blog post, we will dive into this question and explore the reasons behind this performance difference. We will provide you with easy solutions and actionable insights to help you optimize your code. 💪💻

The Context 📜

Let's start by looking at the context around this question. Suppose we have the following core loop in our numerical code:

const int n = 100000;

for (int j = 0; j < n; j++) {
    a1[j] += b1[j];
    c1[j] += d1[j];
}

This loop is executed 10,000 times via another outer loop. To speed up the code, the author changed it to separate loops:

for (int j = 0; j < n; j++) {
    a1[j] += b1[j];
}

for (int j = 0; j < n; j++) {
    c1[j] += d1[j];
}

Surprisingly, the double-loop example is significantly faster than the first example. The first example takes 5.5 seconds, while the double-loop example takes only 1.9 seconds when compiled with optimization and SSE2 enabled. 🚀

Disassembly Comparison 👩‍💻💻

To understand why the separate loops are faster, let's take a closer look at the disassembly code for each loop.

First Loop Disassembly Code:

movsd       xmm0,mmword ptr [edx+18h]
addsd       xmm0,mmword ptr [ecx+20h]
movsd       mmword ptr [ecx+20h],xmm0
movsd       xmm0,mmword ptr [esi+10h]
addsd       xmm0,mmword ptr [eax+30h]
movsd       mmword ptr [eax+30h],xmm0
movsd       xmm0,mmword ptr [edx+20h]
addsd       xmm0,mmword ptr [ecx+28h]
movsd       mmword ptr [ecx+28h],xmm0
movsd       xmm0,mmword ptr [esi+18h]
addsd       xmm0,mmword ptr [eax+38h]

Double Loop Disassembly Code:

addsd       xmm0,mmword ptr [eax+28h]
movsd       mmword ptr [eax+28h],xmm0
movsd       xmm0,mmword ptr [ecx+20h]
addsd       xmm0,mmword ptr [eax+30h]
movsd       mmword ptr [eax+30h],xmm0
movsd       xmm0,mmword ptr [ecx+28h]
addsd       xmm0,mmword ptr [eax+38h]
movsd       mmword ptr [eax+38h],xmm0
movsd       xmm0,mmword ptr [ecx+30h]
addsd       xmm0,mmword ptr [eax+40h]
movsd       mmword ptr [eax+40h],xmm0

From the disassembly code, we can see that each loop of the double-loop example produces fewer instructions compared to the combined loop. This reduction in instructions contributes to the faster execution of the double-loop example.

Cache Behavior and CPU Architecture 🚀💾

It's important to note that the behavior of the loops depends heavily on the sizes of the arrays (n) and the CPU cache. The performance improvement observed in the double-loop example may vary depending on these factors.

To gain more insight into the different cache behaviors, the author posed the following questions:

1. Could you provide some solid insight into the details that lead to the different cache behaviors as illustrated by the five regions on the following graph?

   To answer this question, it would be beneficial to analyze the cache behavior for different array sizes and observe the impact on the performance of the separate and combined loops.

2. It might also be interesting to point out the differences between CPU/cache architectures, by providing a similar graph for these CPUs.

   Understanding the differences in CPU/cache architectures can help us gain a broader understanding of the performance characteristics and bottlenecks associated with elementwise additions in separate and combined loops.

Conclusion and Call-to-Action 💡📣

In conclusion, elementwise additions are much faster in separate loops compared to combined loops due to the reduction in instructions and potential cache optimizations. However, the performance improvement may vary depending on the sizes of the arrays and the CPU cache behavior.

To optimize your code and improve performance, consider using separate loops for elementwise additions. But keep in mind that profiling and benchmarking your code on different array sizes and CPU/cache architectures is crucial to understanding the specific performance impacts. 📊

We encourage you to further explore the details of cache behavior and CPU/cache architectures to gain a more in-depth understanding of the performance implications. Share your findings and experiences with us in the comments section below! 🔍📝

Happy optimizing! 🚀💻