principleMinor
Big Oh vs Big Theta
Viewed 0 times
thetabigstackoverflow
Problem
I mathematically understand $f(n) \in O(g(n))$ : $f(n)$ does not grow faster than $g(n)$. More formally, $\exists c, n_0$ s.t. $f(n) \leq cg(n) \forall n \geq n_0$.
Similarly, $f(n) \in \Theta(g(n))$ means that $f(n)$ grows approximately as fast as $g(n)$. i.e. $f(n) \in O(g(n)), \Omega(g(n))$.
What I don't get is why people use big Oh for the running time of an algorithm? Shouldn't we be using big Theta. When we say "Running time" of an algorithm, we refer to worst case running time i.e. $T(n) = max \{ ALG(x): |x| = n \}$.
So, ex: the worst case running time of linear search on an input of size $n$ ($n$ elements and a target value) is $\Theta(n)$ and $O(n)$, but $\Theta(n)$ gives more information. So, why do algorithm books use $O(n)$ and not $\Theta(n)$.
Similarly, $f(n) \in \Theta(g(n))$ means that $f(n)$ grows approximately as fast as $g(n)$. i.e. $f(n) \in O(g(n)), \Omega(g(n))$.
What I don't get is why people use big Oh for the running time of an algorithm? Shouldn't we be using big Theta. When we say "Running time" of an algorithm, we refer to worst case running time i.e. $T(n) = max \{ ALG(x): |x| = n \}$.
So, ex: the worst case running time of linear search on an input of size $n$ ($n$ elements and a target value) is $\Theta(n)$ and $O(n)$, but $\Theta(n)$ gives more information. So, why do algorithm books use $O(n)$ and not $\Theta(n)$.
Solution
A (sloppy) upper bound is easier to prove than a tight upper bound, let alone upper and lower bounds.
Some algorithm's runtime can't be given with the same function as upper/lower bound. E.g. simple sorting algorithms are $O(n^2)$, but have lower bound $\Omega(n)$.
Some insist of striving to give performance in asymptotic terms via $\sim$, where $f(n) \sim g(n)$ if
$$\lim_{n \to \infty} \frac{f(n)}{g(n)} = 1$$
(say as an average, or worst case, in term of number of some critical operation, such as comparisons when sorting). I.e., wiggle room, but no (possibly humongous) constants swept under the rug.
Some algorithm's runtime can't be given with the same function as upper/lower bound. E.g. simple sorting algorithms are $O(n^2)$, but have lower bound $\Omega(n)$.
Some insist of striving to give performance in asymptotic terms via $\sim$, where $f(n) \sim g(n)$ if
$$\lim_{n \to \infty} \frac{f(n)}{g(n)} = 1$$
(say as an average, or worst case, in term of number of some critical operation, such as comparisons when sorting). I.e., wiggle room, but no (possibly humongous) constants swept under the rug.
Context
StackExchange Computer Science Q#53575, answer score: 4
Revisions (0)
No revisions yet.