debugMinor
Is order of matrix multiplication affecting numerical accuracy of the result?
Viewed 0 times
resultorderthemultiplicationnumericalmatrixaffectingaccuracy
Problem
I have to multiply three matrices of floats:
Is there any difference in accuracy between
If yes - how may I determine the more accurate multiplication order?
Speed is not a factor here.
Matrices
A (100x8000), B (8000x27) and C (27x1).Is there any difference in accuracy between
A(BC) and (AB)C?If yes - how may I determine the more accurate multiplication order?
Speed is not a factor here.
Matrices
A and B contain 8000 samples of (respectively) 100 and 27 features.Solution
Yes, there are differences in accuracy since with machine numbers the usual properties of arithmetics don't hold.
Machine numbers are defined as
$$ F(\beta,t,m,M)= \{ 0 \} \cup \{ x \in \mathbb{R} : x = sign(x)\beta^{p} \sum_{i=1}^{t}d_i\beta^{-i},\ 0 \leq d_i \lt \beta\ ,\ d_1\ne 0\ , -m \le p \le M \} $$ and represent the subset of $\mathbb{R}$ that your machine is able to represent.
All other numbers must be approximated with a number in this subset (usually by truncating the numbers or rounding them).
Let's assume we are in $F(10,2,m,M)$ meaning we are working in base 10 with two digits.
Let $x=0.1110^1$, $y=0.3110^{1}$ and $z=0.25*10^{1}$.
The associative property of multiplication doesn't hold:
$(xy)z = 0.3410^1 z = 0.85*10^1$
and
$x(yz) = x (0.78 10^1) = 0.86*10^1$
other properties that don't hold are:
-
the associative property of the addition
-
distributive properties
Machine numbers are defined as
$$ F(\beta,t,m,M)= \{ 0 \} \cup \{ x \in \mathbb{R} : x = sign(x)\beta^{p} \sum_{i=1}^{t}d_i\beta^{-i},\ 0 \leq d_i \lt \beta\ ,\ d_1\ne 0\ , -m \le p \le M \} $$ and represent the subset of $\mathbb{R}$ that your machine is able to represent.
All other numbers must be approximated with a number in this subset (usually by truncating the numbers or rounding them).
Let's assume we are in $F(10,2,m,M)$ meaning we are working in base 10 with two digits.
Let $x=0.1110^1$, $y=0.3110^{1}$ and $z=0.25*10^{1}$.
The associative property of multiplication doesn't hold:
$(xy)z = 0.3410^1 z = 0.85*10^1$
and
$x(yz) = x (0.78 10^1) = 0.86*10^1$
other properties that don't hold are:
-
the associative property of the addition
-
distributive properties
- $x(y/x)$ isn't always equals to $x$
- if $xy=yz$ then isn't always true that $x=z$
Context
StackExchange Computer Science Q#111630, answer score: 2
Revisions (0)
No revisions yet.