Baanboard.com

Go Back   Baanboard.com > Forum > Baan Quick Support: Functional & Technical > Tools Development

User login

Frontpage Sponsor

Main

Poll
How big is your Baan-DB (just Data AND Indexes)
0 - 200 GB
19%
200 - 500 GB
27%
500 - 800 GB
4%
800 - 1200 GB
7%
1200 - 1500 GB
7%
1500 - 2000 GB
12%
> 2000 GB
25%
Total votes: 85

Baanboard at LinkedIn


Reference Content

Reply
 
Thread Tools Display Modes
  #1  
Old 26th March 2020, 13:44
raviteja.m123@g raviteja.m123@g is offline
Junior Member
 
Join Date: Jul 2015
Posts: 5
raviteja.m123@g is on a distinguished road
Baan: LN10.4 - DB: SQL - OS: WINDOWS
Problem in display of double value
Baan: Infor ERP 10.5
C/S: None/Unknown

Hello,

I am having a field(say m) in the table which is having domain 'tcqsl1' which is having the value of '0.26300000000001',
I have rounded this value to a 4 decimals and assigned to a variable(say n)
'n' will have 0.263
when I am taking the difference of 'm' and 'n' system is displaying as 0. Ideally it should display as '0.00000000000001'.

for all the variables I used the same domain i.e. tcqsl1

Could you please explain me as I am confused with this....

Thank you in Advance....
Reply With Quote
Sponsored Links
  #2  
Old 26th March 2020, 15:13
günther günther is offline
Guru
 
Join Date: Jan 2002
Location: Ehingen, Germany
Posts: 608
günther is on a distinguished road
Baan: IVc4 - DB: Informix - OS: HP-UX
Hi,

doubles have a limitation to 15 significant digits; see this threadhttp://www.baanboard.com/baanboard/a...hp/t-1890.html.

Here is my test:
Quote:
double m, n

m = 0.2630000000001
n = round(m, 4, 1)

message("m-n=%.10f", m-n) | 0
message("m-n=%g", m-n) | 9.99756e-14
message("m-n=%g", round(m-n, 14, 1)) | 1e-13
Regards
Günther
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem with display date gget79 Tools Development 2 20th March 2013 13:43
SO decimal problem cause display in invoice to customer wrong Predator Distribution, Transportation & Warehousing 3 27th April 2012 16:44
Problem with Text Display in Text Editor raikar_raviraj Tools Development 2 16th January 2012 06:14
Display session problem peterfarouk Tools Development 6 28th July 2003 09:32
Herstellkostenberechnung in BaaN 4c Kai's Forum Users Deutsches Forum 4 5th July 1999 00:00


All times are GMT +2. The time now is 12:24.


©2001-2018 - Baanboard.com - Baanforums.com