Author Topic: CMOD File Size Limit  (Read 2293 times)

fnb4321

  • Jr. Member
  • **
  • Posts: 59
    • View Profile
CMOD File Size Limit
« on: September 09, 2020, 06:51:16 AM »
  We are going to be receiving some large PPD files to load into OnDemand (version 10.1.0.5)

What is the max file size that could be ingested into CMOD ?  I think I read somewhere that it was 50MB but want to confirm.

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: CMOD File Size Limit
« Reply #1 on: September 09, 2020, 08:47:54 AM »
If these are things like customer statements or letters, you should aim to have each individual document be less than 50MB, just for performance reasons. 

The 50MB limit is more of a warning than an error, simply because retrieving tens or hundreds of megabytes is a slow process, even today. 

I think the limitation of a a single PDF is 2GB (although I suspect newer versions have increased that limitation).

One of the Unicom / IBM folks should probably chime in here and give us the official answer though.  :)

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

fnb4321

  • Jr. Member
  • **
  • Posts: 59
    • View Profile
Re: CMOD File Size Limit
« Reply #2 on: September 09, 2020, 08:59:03 AM »
Thanks Justin.

These will be concatenated PPD files with 1099 statements. 

So - guess what I need to know is max recommended size of a concatenated file

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: CMOD File Size Limit
« Reply #3 on: September 09, 2020, 09:13:01 AM »
I'd stick with 2GB.  And tax forms are usually quite small, so that should hold several thousand forms each.

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Re: CMOD File Size Limit
« Reply #4 on: September 09, 2020, 02:08:13 PM »
I don't have an official answer because I'll admit I don't completely understand what the interplay of resource collection and whether the PDFs use the standard fonts may have.

Is this on z/OS, will this be loading to OAM?

You may run into the OAM MOS (Maximum Object Size) limit which takes an IPL to increase.

Will you be indexing on zLinux?

Ed
#zOS #ODF

fnb4321

  • Jr. Member
  • **
  • Posts: 59
    • View Profile
Re: CMOD File Size Limit
« Reply #5 on: September 10, 2020, 05:59:26 AM »
  This is Linux environment and will be PPD files.

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Re: CMOD File Size Limit
« Reply #6 on: September 10, 2020, 06:05:46 AM »
  This is Linux environment and will be PPD files.

Linux on z?  There is at least one difference between Linux on z and Linux on other platforms.

Ed
#zOS #ODF

fnb4321

  • Jr. Member
  • **
  • Posts: 59
    • View Profile
Re: CMOD File Size Limit
« Reply #7 on: September 11, 2020, 06:30:22 AM »
Linux Red Hat

Mehmet S Yersel

  • Jr. Member
  • **
  • Posts: 56
    • View Profile
    • LinkedIn Profile
Re: CMOD File Size Limit
« Reply #8 on: September 11, 2020, 11:13:12 AM »
PPD stands for PostScript Print Definition file.. I have not processed this type of a file so far in CMOD, and that's why, I looked it up from the internet. I am not clear what the data looks like.... AFP or XML like tagged text... or something else. A lot depends on the data and meta indexes for what you can and cannot do. 

Question: Is the file going to be stored and retrieved as a single document?

 If yes, you are probably out of luck and you will probably hit the 2 GB maximum document limit of CMOD.

 If no, assuming that the file contains smaller units of data (documents) delimited by some tags or meta indexes that will allow you to easily index them, and you may be in luck. As long as none of the individually delimited documents in the file exceeds 2 GB, you should be OK. I have processed files larger than  15 GB successfully and because the documents in the file were less than 2 GB, CMOD processing went fine.   
#zOS #Multiplatforms
#DB2 #OAM
#AFP #RiCOH AFP2PDF #SnowBound
#Finance #Telecom #Airlines
#ICN #IHS #WAS ND #Cert and Key Management
#Migrations #Data Modeling #RACF-2-CMOD Synch
#FileTek AMMO #ABI #RMDS #RADAR

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: CMOD File Size Limit
« Reply #9 on: September 14, 2020, 06:31:50 AM »
PPD in this case means Page Piece Dictionary, a way of indexing PDFs in CMOD.

Check out Bud Paton's amazing presentation:  http://www.odusergroup.org/forums/index.php?topic=1724.0  (For non-registered visitors or ODUG, you need to register to get access to our webinar replays.)

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR