Author Topic: System load application group fields  (Read 5358 times)

sandeepveldi

  • Guest
System load application group fields
« on: March 10, 2010, 07:34:29 PM »
Hi,
Did any one try to use the new "System Load" application group provided starting v8.4? If so, can you please let me know what exactly is the data stored inside it and what are the fields in it?

Thanks in advance,
Sandeep Veldi

Alessandro Perucchi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: System load application group fields
« Reply #1 on: March 11, 2010, 01:14:29 AM »
Hi,
Did any one try to use the new "System Load" application group provided starting v8.4? If so, can you please let me know what exactly is the data stored inside it and what are the fields in it?

Thanks in advance,
Sandeep Veldi

Hello Sandeep,

I have installed and looked a little, and this is only a statistic of all the Load ID made into OnDemand via arsload. From what I could see, it tracks only the load, but not the unload.

Here is a list of fields of the folder:

Log Document
Document Date
Load Time
Application Group Name
Application Name
Load Id
Total Elapsed Time (secs)
Index Elapsed Time (secs)
Row Count
Total Pages
Doc Comp Type
Input Size
Output Size
Resource Reuse
Resource Id
Res Comp Type
Res Input Size
Res Output Size
Agid
Aid
Load Name
PriNid
Start
Stop
Doc Begin Date
Doc End Date
Index Type
Lockdown
CFS-OD
Input File Name

Hope this gives an idea of what this new application group does!

At least for me, it is a big help, since I don't have to manage a separate database with the LoadIds/APPG!

Cheers,
Alessandro
Alessandro Perucchi

#Install #Migrations #Conversion #Educate #Repair #Upgrade #Migrate #Enhance #Optimize #AIX #Linux #Multiplatforms #DB2 #Windows #Oracle #TSM #Tivoli #Performance #Audits #Customizing #Availability #HA #DR #JavaApi #ContentNavigator #ICN #WEBi #ODWEK #Services #PDF #AFP #XML

waytec

  • Jr. Member
  • **
  • Posts: 18
    • View Profile
Re: System load application group fields
« Reply #2 on: March 11, 2010, 02:13:43 PM »
Sandeep,  I also have installed with 8.4 and find myself almost never using the log.  Although, at times we have to perform unloads and the System Load Log makes it much easier to to obtain the LoadID's. I can see where the log would be great many reporting task, such as growth trend analysis. The problem is that I am so used to querying the System Log I forget about the Sytem Load log.  It is also reported that installing the Load Log slows load perfromance because of the dual log write.  Try it on your test box.
Wayne Henderson
Sr. Server Systems Engineer

sandeepveldi

  • Guest
Re: System load application group fields
« Reply #3 on: March 17, 2010, 11:22:30 AM »
Alessandro & Waytec,
Thanks for your responses. Can you please let me know if any of you are using the system load application group in production environment? If so, did you see any significant impact in the load timings?

Thanks & Regards,
Sandeep Veldi

Alessandro Perucchi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: System load application group fields
« Reply #4 on: March 18, 2010, 12:07:50 AM »
Alessandro & Waytec,
Thanks for your responses. Can you please let me know if any of you are using the system load application group in production environment? If so, did you see any significant impact in the load timings?

Thanks & Regards,
Sandeep Veldi

Hello Sandeep,

not yet in production, so I have no idea about the impact on Load

Cheers,
Alessandro
Alessandro Perucchi

#Install #Migrations #Conversion #Educate #Repair #Upgrade #Migrate #Enhance #Optimize #AIX #Linux #Multiplatforms #DB2 #Windows #Oracle #TSM #Tivoli #Performance #Audits #Customizing #Availability #HA #DR #JavaApi #ContentNavigator #ICN #WEBi #ODWEK #Services #PDF #AFP #XML

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2229
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: System load application group fields
« Reply #5 on: March 18, 2010, 04:21:12 AM »
Hi Sandeep...

The impact will be minimal.  It's no worse than adding load messages to the System Log -- a single record with multiple fields.

If I remember correctly from the last time I was down there, your loads are sufficiently large for the overhead from Load Log to be a problem.

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

sandeepveldi

  • Guest
Re: System load application group fields
« Reply #6 on: March 18, 2010, 09:58:26 PM »
Thanks again for the responses.

Sandeep Veldi

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2229
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: System load application group fields
« Reply #7 on: March 23, 2010, 06:03:38 AM »
Hey Sandeep...

I just re-read my post -- it should read, "your loads are sufficiently large for the overhead from the Load Log *NOT* to be a problem".

Sorry!  :)

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

krcd

  • Guest
Re: System load application group fields
« Reply #8 on: August 17, 2017, 10:12:13 AM »
Hello, I also have the problem of slowness of loads, some solution for this, I have executed all the maintenance tasks of DB2 and ONDEMAND (ARSMAINT, ARSDB) . Note: attached image

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2229
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: System load application group fields
« Reply #9 on: August 24, 2017, 08:40:53 AM »
There's a few problems here.

Your ObjectID contains KAA.  That means you have 5+ billion files in a single Application Group.  IBM CMOD likes economies of scale -- loading thousands of documents at once.  You're also losing the benefits of CMOD's compression algorithms as well -- because they perform best when the input files are enormous.

Second, if you're loading individual documents at a time into an Application Group, and the expiration type is set to 'Load', and you're doing LOTS of concurrent loads, you're going to have contention issues on the IBM CMOD arsload table as each load tries to insert a row into the table.

Third, you appear to be using TSM - so there's a possibility you might be using 'Migrate on Load'.  While this is normally a good idea (for data redundancy), it slows down loading.  You can change the AG config to move the data 'At next cache migration' -- but you *absolutely* must use the IBM CMOD arsmaint utility to migrate data -- preferably several times a day.

Before doing anything else, I'd consider moving historical data (a year at a time) into a new Application Group with a better configuration, then binding them together at the folder level.

Of course, this is just one Application Group -- if your others are similarly used (and misconfigured) then you probably have other App Groups to fix as well before performance improves.

-JD.
« Last Edit: August 24, 2017, 08:44:47 AM by Justin Derrick »
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

krcd

  • Guest
Re: System load application group fields
« Reply #10 on: August 28, 2017, 05:41:52 PM »
@Justin, thanks for your help.

I made a new group of applications to observe the loading behavior, did not flee changes. Load times do not decrease.

I'll try to migrate to TSM...

Other important information is, maintenance operations were carried out Disks, impacting on filesystem sectors that make up the primary mount point, this disk is incomplete, since it was decompressed from a backup and reached 40% approximately.

Any other comments?