HiveBrain v1.2.0
Get Started
← Back to all entries
debugMinor

Is order of matrix multiplication affecting numerical accuracy of the result?

Submitted by: @import:stackexchange-cs··
0
Viewed 0 times
resultorderthemultiplicationnumericalmatrixaffectingaccuracy

Problem

I have to multiply three matrices of floats: A (100x8000), B (8000x27) and C (27x1).

Is there any difference in accuracy between A(BC) and (AB)C?
If yes - how may I determine the more accurate multiplication order?
Speed is not a factor here.

Matrices A and B contain 8000 samples of (respectively) 100 and 27 features.

Solution

Yes, there are differences in accuracy since with machine numbers the usual properties of arithmetics don't hold.

Machine numbers are defined as
$$ F(\beta,t,m,M)= \{ 0 \} \cup \{ x \in \mathbb{R} : x = sign(x)\beta^{p} \sum_{i=1}^{t}d_i\beta^{-i},\ 0 \leq d_i \lt \beta\ ,\ d_1\ne 0\ , -m \le p \le M \} $$ and represent the subset of $\mathbb{R}$ that your machine is able to represent.

All other numbers must be approximated with a number in this subset (usually by truncating the numbers or rounding them).

Let's assume we are in $F(10,2,m,M)$ meaning we are working in base 10 with two digits.

Let $x=0.1110^1$, $y=0.3110^{1}$ and $z=0.25*10^{1}$.

The associative property of multiplication doesn't hold:

$(xy)z = 0.3410^1 z = 0.85*10^1$

and

$x(yz) = x (0.78 10^1) = 0.86*10^1$

other properties that don't hold are:

-
the associative property of the addition

-
distributive properties

  • $x(y/x)$ isn't always equals to $x$



  • if $xy=yz$ then isn't always true that $x=z$

Context

StackExchange Computer Science Q#111630, answer score: 2

Revisions (0)

No revisions yet.