snippetsqlModerate
How to make postgresql SUM to be more accurate on large amount of floating point data?
Viewed 0 times
postgresqlhowpointamountfloatingmakemoreaccuratelargesum
Problem
I'm trying to SUM 8 million floating point (
However, it returns very inaccurate result.
It should return 137,586.77, but it returns 137,303 (283.77 difference)
Is there a way to force a query to be more precise?
REAL) values with simple query like this:SELECT SUM(metric) FROM metrics;However, it returns very inaccurate result.
It should return 137,586.77, but it returns 137,303 (283.77 difference)
Is there a way to force a query to be more precise?
Solution
As per the postgresql documentation,
If you require exact storage and calculations (such as for monetary amounts), use the numeric type instead.
Since You've used real datatype, you can casting the real values to numeric. Then you can get results as you expected.
If you require exact storage and calculations (such as for monetary amounts), use the numeric type instead.
Since You've used real datatype, you can casting the real values to numeric. Then you can get results as you expected.
SELECT SUM(metric::NUMERIC) FROM metrics;Code Snippets
SELECT SUM(metric::NUMERIC) FROM metrics;Context
StackExchange Database Administrators Q#37658, answer score: 10
Revisions (0)
No revisions yet.