Baanboard.com

Go Back   Baanboard.com > Forum > Baan Quick Support: Functional & Technical > Tools Development

User login

Frontpage Sponsor

Main

Poll
As a Customer What would do to keep your ERP Implementation intact
Proactively define Business Process-- Take the Project Ownership
50%
Handover everything to System Integrator from drawing BP till implementation of ERP
0%
Hire more inhouse skilled & capable IT Resource to work directly with SI
50%
Rely on SI Architects/Consultants
0%
Total votes: 2

Baanboard at LinkedIn


Reference Content

Reply
 
Thread Tools Display Modes
  #1  
Old 26th March 2020, 13:44
raviteja.m123@g raviteja.m123@g is offline
Junior Member
 
Join Date: Jul 2015
Posts: 7
raviteja.m123@g is on a distinguished road
Baan: LN10.4 - DB: SQL - OS: WINDOWS
Problem in display of double value
Baan: Infor ERP 10.5
C/S: None/Unknown

Hello,

I am having a field(say m) in the table which is having domain 'tcqsl1' which is having the value of '0.26300000000001',
I have rounded this value to a 4 decimals and assigned to a variable(say n)
'n' will have 0.263
when I am taking the difference of 'm' and 'n' system is displaying as 0. Ideally it should display as '0.00000000000001'.

for all the variables I used the same domain i.e. tcqsl1

Could you please explain me as I am confused with this....

Thank you in Advance....
Reply With Quote
  #2  
Old 26th March 2020, 15:13
günther günther is offline
Guru
 
Join Date: Jan 2002
Location: Ehingen, Germany
Posts: 607
günther is on a distinguished road
Baan: IVc4 - DB: Informix - OS: HP-UX
Hi,

doubles have a limitation to 15 significant digits; see this threadhttp://www.baanboard.com/baanboard/a...hp/t-1890.html.

Here is my test:
Quote:
double m, n

m = 0.2630000000001
n = round(m, 4, 1)

message("m-n=%.10f", m-n) | 0
message("m-n=%g", m-n) | 9.99756e-14
message("m-n=%g", round(m-n, 14, 1)) | 1e-13
Regards
Günther
Reply With Quote
  #3  
Old 6th April 2020, 14:17
OmeLuuk's Avatar
OmeLuuk OmeLuuk is offline
Guru
 
Join Date: Mar 2002
Location: The Netherlands
Posts: 1,131
OmeLuuk is on a distinguished road
Baan: Infor LN 6.1 10.2.1 > 10.7 - DB: MS SQL2008 - OS: VM WindowsServer2008R2
And due to the nature of storing doubles you will always have such small differences, nothing to worry about - use the functions table.round() and use the variables according to the domain restriction, in case of tcqsl1 it would be format %AQ25, numeric 15 before decimal, only 4 digits after decimal.
__________________
Also read Luke 24:4-8
Reply With Quote
Sponsored Links
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem with display date gget79 Tools Development 2 20th March 2013 13:43
SO decimal problem cause display in invoice to customer wrong Predator Distribution, Transportation & Warehousing 3 27th April 2012 16:44
Problem with Text Display in Text Editor raikar_raviraj Tools Development 2 16th January 2012 06:14
Display session problem peterfarouk Tools Development 6 28th July 2003 09:32
Herstellkostenberechnung in BaaN 4c Kai's Forum Users Deutsches Forum 4 5th July 1999 00:00


All times are GMT +2. The time now is 13:24.


©2001-2018 - Baanboard.com - Baanforums.com