Two ~~weeks~~ months ago (you know how it is, things get busy, you procrastinate, and then forget altogether), Thomas Devaraj (PTC Mathcad Business Development Manager) asked Jakov Kucan (Software Development Director) a question about performance regarding PTC Mathcad’s processing of large matrices, especially in comparison to other software packages.

The problem went like this – generate two square matrices, both n-by-n, of random numbers and multiply them. Pretty simple. Now this problem had come from a customer who was using MATLAB, which is known for its matrix handling capabilities (after all, the name is short for MATrix LABoratory). We knew that duplicating this in PTC Mathcad would be somewhat slower, but just how much?

We came up with a simple worksheet to test how this problem would be handled by Mathcad. The first order of business was writing a script to generate a matrix of n-by-n random numbers. MATLAB has a RAND function that does this, but Mathcad’s RND function only generates one number, so we had to use nested FOR loops to fill in the elements of the matrix. Note that we use the TIME function to time how long the generation takes. We do the same thing with the multiplication, again using TIME to time this step. And then we ran the two scripts with varying values of ‘n’ – 50, 100, 200, 500, 1000, 2000, 5000, 75000, 10000.

We repeated the exercise in MATLAB, using TIC/TOC to time how long it took to generate the matrices and how long it took to multiply them. Here are the results.

These are the total times (generation and multiplication) that PTC Mathcad and MATLAB took for different values of ‘n’. We have PTC Mathcad in solid green, and MATLAB in dashed green. It should come off as no surprise the MATLAB handled this problem faster. When ‘n’ was 500 (generating and multiplying two 500 x 500 matrices), MATLAB took 0.021 seconds to PTC Mathcad’s 0.172 seconds. That’s a factor of 8.2. When ‘n’ was 1000, MATLAB took 0.20 seconds to PTC Mathcad’s 0.59 seconds. A factor of 3. When ‘n’ was 10000 (10000 by 10000 matrices), MATLAB took 96 seconds to PTC Mathcad’s 130 seconds. A factor of 1.35. So as the matrices got bigger, the gap shrank.

What explains the shrinking gap? Let’s take a look at the break-down of numbers. We separate the times into time to create (blue) and time to multiply (red).

MATLAB generates the matrices of random numbers much faster – 2.76 seconds vs 40.3 seconds for N=10000. That’s a 14.6 factor difference! If we look at the time to multiply, however, we see that the times are pretty much even, with PTC Mathcad being ever so slightly faster.

Once ‘n’ exceeds 100, the time spent on multiplication (in Mathcad) exceeds the time spent on generation. So this explains why the total times are what they are. The bulk of the time spent (in multiplication) is the same in both PTC Mathcad and MATLAB. If we visualize the results in logarithmic scale, we can more easily see that the Mathcad and MATLAB total times (in green, solid and dashed, respectively) do converge as ‘n’ increases.

I will admit that I was quite surprised of the results. I knew that Mathcad could handle large matrices, I just didn’t expect it to keep up with MATLAB! Of course, this is a very simple problem, and most problems will entail other functions and manipulations, but if we are just comparing linear algebra operations, it seems that PTC Mathcad runs with the best of them!

Chahid Ghaddar (Software Development Engineer) adds:

“Mathcad matrix data structure is probably more complex compared to MATLAB because it has to support mixed units, nested objects, complex values, and other operations. This adds significant overhead, and will sometimes obscure the actual computation cost, which should be comparable to that of MATLAB. The point here is Mathcad has a different value proposition. Comparing head-on with MATLAB is not the best way to demonstrate Mathcad’s full capabilities.” |

So there we have it. PTC Mathcad and MATLAB each have their strengths and different value propositions. But just because MATLAB is known for its matrix manipulation prowess, don’t discount Mathcad too hastily. Or to word it from another perspective, just because Mathcad is easy to use, doesn’t mean it’s necessarily slower than other tools. It could surprise you!

To see the Mathcad worksheet used to time the generation and multiplication of n-by-n matrices, check the PTC Community.

doesn’t Intel MKL library already have build in functions to generate the random matrix without having to use for loops.

MKL might. The value of Mathcad is not just a passthrough to MKL. We don’t expose all of the MKL functionality. We leverage it where it makes the biggest impact. WIth Mathcad you get a _readable_ document with respectable performance. Can you get better performance with specialized code? Sure, but at what cost?

Mathcad can atleast auto vectorize the for loops and atleast give me warning if it failed to autovectorize it.

what are the plans of supporting the new Intel Xeon Phi co-processors?

We are always looking into improving the computational power of Mathcad to 1) provide faster, more robust, and more precise algorithms (e.g. new dft and idft functions in Mathcad Prime 2.0), and 2) take advantage of the modern computer architectures (see http://wp.me/p18Rcy-2BV). However, Mathcad does not target specific processors. The advantages of Mathcad, in our view, outweigh the performance penalty over custom processor targeted code.