Error in "double = double - double" statement

Re: Error in "double = double - double" statement

"Mensan" <don'tspamme@ho tmail.com> wrote[color=blue]
> I am trying to compute the differnece between two dwords in visual basic and
> keep getting the wrong value being computed. I have the following structure
> defind:
>
> Public Type HighLowQuote
> TradeDate As String
> OpenPrice As Double
> HighPrice As Double
> LowPrice As Double
> ClosePrice As Double
> Volume As Long
> AdjClosePrice As Double
> AmtChange As Double
> End Type[/color]


Have you tried declaring your Price values as Currency instead of Double?

LFS





-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----

Re: Error in &quot;double = double - double&quot; statement


"Mensan" <don'tspamme@ho tmail.com> wrote in message
news:pyJPb.8948 [email protected] xas.rr.com...[color=blue]
> I am trying to compute the differnece between two dwords in visual[/color]
basic and[color=blue]
> keep getting the wrong value being computed. I have the following[/color]
structure[color=blue]
> defind:
>
> Public Type HighLowQuote
> TradeDate As String
> OpenPrice As Double
> HighPrice As Double
> LowPrice As Double
> ClosePrice As Double
> Volume As Long
> AdjClosePrice As Double
> AmtChange As Double
> End Type
>
> I create an instance of this type as 'q'. I then do the following:
> q.highprice = variant
> q.lowprice = variant
>
> at this point the locals variable window shows the values as:
> q.highprice = 4.1
> q.lowprice = 4
>
> I then fall through the following code:
> q.AmtChange = q.highprice - q.lowprice
>
> q.amtchange is then valued at 9.9999999999999 6E-02 in the locals[/color]
window when[color=blue]
> I believe it should be valued at .1.
>
> Why is the value not being set to .1? Is there a change I can make to[/color]
get[color=blue]
> this calculation to work how I expect or is there some other way to[/color]
perform[color=blue]
> these calculations to get the results I am after?
>
> Any help would be appreciated. Thanks in advance.
>
>[/color]

As Larry said, you may want to use the Currency type instead of doubles.

Welcome to the world of floating point numbers. In your debugger, try
examining the value of
?q.highprice
and then
?(q.highprice = 4.1)
and then
?q.highprice - 4

Depending on where the 4.1 came from, you may discover that it isn't
actually 4.1 in the first place. I can't really tell, because all you
said was q.highprice = variant.

Re: Error in &quot;double = double - double&quot; statement

Mensan <don'tspamme@ho tmail.com> schreef in berichtnieuws
pyJPb.8948$7D.8 [email protected] r.com...

Hello Mensan,

[Snip]
[color=blue]
> q.amtchange is then valued at 9.9999999999999 6E-02 in the locals window[/color]
when[color=blue]
> I believe it should be valued at .1.
>
> Why is the value not being set to .1? Is there a change I can make to get
> this calculation to work how I expect or is there some other way to[/color]
perform[color=blue]
> these calculations to get the results I am after?[/color]

But they are allmost the same ! They just differ a meager
0.0000000000000 04. That's not even a promille of a promille !

Even if you would get that ammount in dollars *every second*, you would,
after a year, not even have a cent. That would actually take something in
the vincinity of a mere 100,000 years :-)

9.9999999999999 6E-02. The E-02 means : shift the decimal-point left two
places to get the actual number. This means that the actual number is
0.0999999999999 996 . And I think that's mighty close to 0.1 .


As others allready explained, that difference is a problem you can get when
a system that can only store numbers in powers of 2 tries to store a
fractional number that is based on powers of 10.

Just try to create the number 0.3 if you can only use 1/2 , 1/4 , 1/8 , 1/16
, etc (wich are 2^-1 , 2^-2, 2^-3, 2^-4, etc) which are the numbers the
computer has to use. I think you will see that you can't ...


To get rid of the *effect*, you could use a storage-type that automatically
rounds on the number of decimals you need (as suggested, use the
currency-type)

On computers that have not got that kind of variable-types you either ignore
the rounding-effect by displaying it with a { using "#####.##" } method
(which will do the rounding for you), or by not using decimals (calculate
the numbers in whole numbers (cents) only. Which, by the way, is what the
currency-type in VB might do).


By the way : This (your problem) is the reason why you should *never* try to
compare two doubles (or even two numbers having fractional parts) with each
other for equality (Like : if double1 = double2 then ....).

Regards,
Rudy Wieser