Max File Size question resurfaced

[Nolan]

An application is wanting to archive a 9.5GB file in one shot.   
It is a line data report, using the Z/OS indexer with 1 key.
Compression is OD77
Large Object is on
Number of pages 100

arsload: Server Version <8.5.0.10> Operating System <z/OS> <04.24.00> Database <DB2> <10.01.0005>                                   
arsload: 12/14/15 18:27:11 -- Loading started, --UNKNOWN-- bytes to process                                                         
OnDemand Load Id = >9115-5977-0-49714IAA-1450117631-1450117631<                                                                     
Information about the load that is stored in order to support certain features has exceeded the recommended size.  The current size
is >463903494< bytes and the recommended maximum size is 100MB.  It is recommended that the load be split into a series of smaller loads.                                                                                                                             
The data that was loaded was not a part of a distribution in ODF.                                                                   
Loaded 2767997 rows into the database                                                                                               
Document compression type used - OD77Lite.  Bytes Stored = >1373922242< Rows = >2767997<                                           
arsload: 12/14/15 18:34:41 Loading completed                                                                                       

Any one have suggestions and advice on what is the max file size should be used. 

Thanks

[Justin Derrick]

A few observations:

Each object is about 3k (9.5GB/2.76M documents).  It doesn't make sense to use large object support -- the individual objects are too small.  Also, you said your compression is OD77, and the load message says it's using OD77Lite.  You might be able to save some more space at the cost of a little more CPU (although I knows MIPS = $$$ on z/OS, so maybe that's a calculated move).

I'm not exactly sure why you're getting that message.  It may be related to large object support (which stores a header in the object file describing the 'large object pieces').

It might be best to open a PMR on this and get an official answer, as understanding this at a deeper level would probably require access to or knowledge of the source code.

[Nolan]

Thanks, I will open a PMR on it and share the results.

DASD/TAPE is cheaper than MIPS   

Cheers

[Alessandro Perucchi]

Maximum file size that can be archived in CMOD:

http://www-01.ibm.com/support/docview.wss?uid=swg21170676

[Justin Derrick]

Maximum file size that can be archived in CMOD:

http://www-01.ibm.com/support/docview.wss?uid=swg21170676

Yeah, this looks like a different issue though.  It's complaining about a 400MB object size -- which is weird, because each document should average about 3kb.  The output file size shouldn't be an issue as long as the document/group size is less than 2GB -- in which case, it's likely very, very small.

I suppose there could be one extra-ordinarily large object in the midst of that 9.5GB input file though.

-JD.