Baanboard.com

Go Back   Baanboard.com > Forum > Baan SIGs > Performance & Tuning

User login

Frontpage Sponsor

Main

Poll
For ERP LN feature pack upgrade, what method of install are you using?
Installation Wizard into existing VRC
35%
Installation Wizard into new VRC
42%
Manual into existing VRC
3%
Manual into new VRC
19%
Total votes: 31

Baanboard at LinkedIn


Reference Content

Reply
 
Thread Tools Display Modes
  #1  
Old 5th June 2010, 13:52
makiju's Avatar
makiju makiju is offline
Senior Member
 
Join Date: Oct 2001
Location: Finland
Posts: 117
makiju is on a distinguished road
Baan: IVc4, LN6.1 - DB: Oracle, Informix - OS: all
bdbpost performance
Baan: Other/Unknown
DB: Other/Unknown
OS: Other/Unknown
C/S: None/Unknown

I've always wondered why bdbpost always gets slower during import. No matter what db or hardware or baan version...
For best performance every table should be imported with new bdbpost -process basically...
I think it does not matter what options to use either. What do you think? What is the root cause of this?
Reply With Quote
  #2  
Old 7th June 2010, 03:36
dave_23's Avatar
dave_23 dave_23 is offline
Guru
 
Join Date: Oct 2002
Location: Portland, OR
Posts: 1,303
dave_23 will become famous soon enough
Baan: All - DB: Oracle / MS SQL / DB2 - OS: All
So you mean that the first table loads fast, and then subsequent tables load slower?

i've always assumed fragmented memory for the bdbpost process.

Dave
Reply With Quote
  #3  
Old 7th June 2010, 11:18
günther günther is offline
Guru
 
Join Date: Jan 2002
Location: Ehingen, Germany
Posts: 570
günther is on a distinguished road
Baan: IVc4 - DB: Informix - OS: HP-UX
I've never checked that. But I could imagine that it has to do with transactions; in some programming situations I've seen such a behaviour whe there was one big transaction - after using commit.transaction() after 1000 records the performance was better. That would declare why it's faster to load each table by a single bdb.post.

Günther
Reply With Quote
  #4  
Old 7th June 2010, 18:09
dave_23's Avatar
dave_23 dave_23 is offline
Guru
 
Join Date: Oct 2002
Location: Portland, OR
Posts: 1,303
dave_23 will become famous soon enough
Baan: All - DB: Oracle / MS SQL / DB2 - OS: All
bdbpost does regularly commit though. there's even a setting that lets you choose the commit frequency.

I'll take it a step further - within a single large table, the first tx's are fast, and the final tx's are usually very slow.

There's very little in the RDBMS that would account for this, especially since baan doesn't have FK constraints. Inserts into a table should maintain a fairly consistent speed regardless of the table size (especially if you have "rows before indexes" turned on, which you definitely should)

So my guess is basically that once bdbpost hits close to the 2GB limit of a 32bit process (or the max size of your data segment at the OS level) it starts swapping. And the reason it happens on a large number of smaller tables is because the cleanup between table loads isn't as good as it could be.

Dave
Reply With Quote
  #5  
Old 2nd August 2010, 15:06
Dikkie Dik's Avatar
Dikkie Dik Dikkie Dik is offline
Guru
 
Join Date: Sep 2002
Location: Netherlands
Posts: 585
Dikkie Dik is on a distinguished road
Baan: Triton 3.0 and higher - DB: All - OS: All
My 2 cents (I know I am late):
- As mentioned before the inserts are pretty easy if you use the rows-before-indexes option
- Most of the time you will see that the disk and database buffer gets full over time. These buffers can handle a certain load, but most of the time the disks are the bottleneck at a certain time.

to check if I am correct you do a "ps -ef >> ps.log"
then you grep on the bdbpost process, the driver (if you are still on an ancient portingset) and the used database server process and see how many seconds they increased during these 60 seconds. If the total amount (of the 2 or 3 processes) is close to 60 seconds, you only burn CPU cycles, so no disk issue. If the number is less than 50 seconds, you already have a disk delay of more than 15%.

Hope this helps,
Dick
__________________
BTW: this post has been made on my personal view. My employer might not share my point of view.
Reply With Quote
  #6  
Old 6th December 2010, 17:21
makiju's Avatar
makiju makiju is offline
Senior Member
 
Join Date: Oct 2001
Location: Finland
Posts: 117
makiju is on a distinguished road
Baan: IVc4, LN6.1 - DB: Oracle, Informix - OS: all
Just amazed how well is Red Hat performing on PC hardware...also with bdbpost.
Example:
-dump size on disk = 5GB
-Oracle 11.2 via listener (no 32-bit client on server), default parameters
-baan4c4 with recent porting set

Import time with single bdbpost -process...result was 40 minutes!
RAID 1+0 (4 disks)

I didn't remember to monitor process memory consumption trend...need to do it next time.
Reply With Quote
  #7  
Old 6th December 2010, 17:48
Dikkie Dik's Avatar
Dikkie Dik Dikkie Dik is offline
Guru
 
Join Date: Sep 2002
Location: Netherlands
Posts: 585
Dikkie Dik is on a distinguished road
Baan: Triton 3.0 and higher - DB: All - OS: All
Quote:
Originally Posted by makiju View Post
I've always wondered why bdbpost always gets slower during import. No matter what db or hardware or baan version...
For best performance every table should be imported with new bdbpost -process basically...
I think it does not matter what options to use either. What do you think? What is the root cause of this?
The root cause has been found and solved in portingset 8.7a. Hope this helps.

As a workaround you can increase the commit size. Setting this a factor 10 higher helps significant.
__________________
BTW: this post has been made on my personal view. My employer might not share my point of view.

Last edited by Dikkie Dik : 6th December 2010 at 17:49. Reason: Forgot the workaround
Reply With Quote
  #8  
Old 6th December 2010, 18:27
makiju's Avatar
makiju makiju is offline
Senior Member
 
Join Date: Oct 2001
Location: Finland
Posts: 117
makiju is on a distinguished road
Baan: IVc4, LN6.1 - DB: Oracle, Informix - OS: all
What commit size do you mean?
Porting set note does not contain any information about bdbpost -thing...
Reply With Quote
Sponsored Links
  #9  
Old 6th December 2010, 23:06
Dikkie Dik's Avatar
Dikkie Dik Dikkie Dik is offline
Guru
 
Join Date: Sep 2002
Location: Netherlands
Posts: 585
Dikkie Dik is on a distinguished road
Baan: Triton 3.0 and higher - DB: All - OS: All
Quote:
Originally Posted by makiju View Post
What commit size do you mean?
I mean the -r option. By default bdbpost commits per 100, now you can set it to 1000 or 10000.
Quote:
Originally Posted by makiju View Post
Porting set note does not contain any information about bdbpost -thing...
You are right I see it is in 8.7b

Hope this helps,
Dick
__________________
BTW: this post has been made on my personal view. My employer might not share my point of view.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
performance dll Hitesh Shah Code & Utilities 11 2nd March 2009 13:29
Import Data from File NPRao Tools Administration & Installation 5 30th August 2005 09:01
Baan Performance Viplov Operating Systems & Databases 3 3rd August 2005 10:15
Use of performance boosters - errors... r_nagu Tools Administration & Installation 17 17th August 2004 17:24
Performance Indicator itconsultant DEM & Workflow 2 22nd October 2002 05:44


All times are GMT +2. The time now is 02:56.


©2001-2017 - Baanboard.com - Baanforums.com