Author Topic: Any easy/fancy ways to gather all loads in one year time frame?  (Read 1488 times)

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 572
    • View Profile
Any easy/fancy ways to gather all loads in one year time frame?
« on: February 23, 2021, 05:20:23 PM »
Business folks are asking for a count of all documents loaded in 2020 for a specific folder. Usually not a big deal, only 6 AG's in that folder..

How I've done it in the past is use the system load / system log folder, and put it into excel..and some simple parsing. But these are large application groups, some have 100k+ loads per month. I've learned obviously that the CMOD client doesn't like copying out more than 30-40k at a time.

My Idea? -- I'm thinking possibly also of just hitting up the SA* tables, of which we have two of. Anyone have any more ingenious ways of doing this?
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: Any easy/fancy ways to gather all loads in one year time frame?
« Reply #1 on: February 23, 2021, 06:24:06 PM »
The CMOD System Load (SA*) tables should be your first stop.  There's no AppGroup field in the OnDemand System Log without doing a full-text search, which is terrible for performance.  A few tweaks of SQL should help you get what you need.

Also, loading 100k files per month is terrible.  Load larger files less frequently to reduce the overhead in CMOD.  :)

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 572
    • View Profile
Re: Any easy/fancy ways to gather all loads in one year time frame?
« Reply #2 on: February 23, 2021, 09:16:12 PM »
The CMOD System Load (SA*) tables should be your first stop.  There's no AppGroup field in the OnDemand System Log without doing a full-text search, which is terrible for performance.  A few tweaks of SQL should help you get what you need.

Also, loading 100k files per month is terrible.  Load larger files less frequently to reduce the overhead in CMOD.  :)

-JD.

Yep, i tinkered with the SA* tables tonight for a few minutes after I posted this and got my results.

I agree with you about the loads. There’s literally a month worth of files. Tens of thousand. With one single document per file.
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: Any easy/fancy ways to gather all loads in one year time frame?
« Reply #3 on: February 25, 2021, 07:52:01 AM »
Yup, the overhead for this is tragic.  Probably more overhead in OnDemand's internal database tables than actual index data.

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR