Score:0

Large number division in bash returns wrong result

in flag

I'm trying to convert a very large number of bytes into gigabytes.

echo $(( 41003021288998461440 / 1073741824 ))

This returns 3827300985. That is incorrect. The correct answer would be 38187039354. 11 digits versus 10.

Using different 'scale = 30' or piping through bc doesn't change the answer. What am I doing wrong?

As an alternative I tried this:

awk -v var1=41003021288998461440 -v var2=1073741824 'BEGIN { print ( var1 / var2 ) }' OFMT='%25g'

Which returns "3.8187e+10", which seems to be numerically correct, but I then can't figure out how not to get in scientific notation. Printf "%12d" isn't helping because it can't seem to handle the division in a passed parameter.

I suspect fixing the awk scientific notation issue would probably be easier, but I'd still like to know why the long division with echo just returns a completely wrong result. That's very concerning, and since I do calculations in that way frequently, I'd like to know what I need to do to get echo to calculate accurately.

I also know that I fixed the problem once before... but I lost how I did it and can't recall now, sigh.

muru avatar
us flag
In the awk version, `printf "%12f", ( var1 / var2 )` gives me `38187039353.883324`, and `%12d` gives `38187039353`
hr flag
I think your awk `OFMT` assignment fails because (at least in GNU awk) it only takes effect after the `BEGIN` block is executed: either use `-v OFMT='%d'` or add it inside the block `BEGIN { OFMT="%d"; print ( var1 / var2 )}`. See [Assigning Variables on the Command Line](https://www.gnu.org/software/gawk/manual/gawk.html#Assignment-Options).
hr flag
... in older versions of GAWK, you may also need to enable arbitrary precision support explicitly using the `-M` or `--bignum` options
Score:1
us flag

Bash integers are not arbitrary precision:

Evaluation is done in fixed-width integers with no check for overflow, though division by 0 is trapped and flagged as an error.

A likely upper limit in modern systems would be 2^63 for signed integers:

$ echo $(( 2**63 - 1 ))
9223372036854775807
$ echo $(( 2**63 ))
-9223372036854775808
$ echo $(( 2**62 ))
4611686018427387904 

Your number is waaaay too large (~4x) for that. If you want to do random arbitrary precision arithmetic interactively, use Python:

>>> 41003021288998461440 / 1073741824
38187039353.88332
Score:0
in flag

Finally reconstructed how to do make it calculate accurately. And yes, it did involve using 'scale'.

BYTECOUNT=$(echo "scale=0; (( $BYTECOUNT / 1073741824 )) " | bc )

The above worked (with BYTECOUNT previously set to the grossly large initial number). Odd that scale=0 is necessary since I believe that is supposed to be the default, but it does appear to be necessary to make it explicit in order to get the calculation to come out correctly.

Score:0
bd flag

I find it best to use dc for this:

V=$(echo "8 k 41003021288998461440 1073741824 / p" | dc)
echo $V

This sets the precision to 8, divides the next two values and then pops it off the stack.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.