Author Topic: Generic Indexing : Batch Vs Single  (Read 1590 times)

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 576
    • View Profile
Generic Indexing : Batch Vs Single
« on: June 09, 2021, 11:44:21 AM »
I am noticing that our one Line of Business will send tens of thousands of files to CMOD. All with one-two documents, that uses the generic indexer (User Defined PDF / HTML)

Obviously I've followed best practices with AFP files and the like, send down a large batch, let cmod compress'em.

I am actually trying to convince the business to send down PDF's with PPD's, not sure if they can change the HTML files we receive though.

However, until then..Trying to build a case.

Is there a very large benefit in load times when doing a generic indexed file- To do lets say, 1000 docs in a file, compared to 1-5? I am going to assume so, because arsload is only called once, one set of inserts, one logon/logoff, etc.

thanks in advance
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2229
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: Generic Indexing : Batch Vs Single
« Reply #1 on: June 09, 2021, 12:38:40 PM »
The benefits are primarily reducing overhead in process, overhead in the database, and compression performance.

I've dealt with this in the past -- an insurance company was using CMOD as a glorified print server.  Requests would go from the mainframe, be sent to OnDemand and loaded a few at a time, then employees would search CMOD for their document, and print documents locally for customers to sign.  I cut their storage requirements by 80% for that App Group (with better compression), and reduced the size of their DB2 database by half (most of it from the arsload table), and their TSM database by nearly 75% (because they had 1/100th the archived objects to track).

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 576
    • View Profile
Re: Generic Indexing : Batch Vs Single
« Reply #2 on: June 10, 2021, 09:03:42 AM »
thank you for this.

I checked out our app group settings, and i noticed that one of our "heavy hitters" that comes from DocOrigin i believe.. Is using OD77, so its trying to compress a generic index PDF file and giving us ZERO savings, costing us resources.

Tested this in dev, much quicker as far as loading goes.

As far as the arsload table savings, space savings sounds good! I assume that this will also make arsmaint run much quicker as well.
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING