ARSLOAD Problem with Documents greater than 100 MB

[jhe2]

We have try to laod a pdf-document greater than 100 MB via ARSLOAD.

The Loading-Process is failed with following message


 ARS4335I Server Version <9.0.0.5>  Operating System <z/OS> <04.25.00>  Database <DB2> <11.01.0005>                                   
 ARS4312I 10/12/17 11:59:10 -- Loading started, --UNKNOWN-- bytes to process                                                         
 ARS1408W Warning:  A document of size >111647893< bytes was processed.  It is not recommended to store documents in OnDemand greater
 than 50MB in size.  Although data may store successfully, it is possible that the data may not be able to be retrieved successfully.
 ARS1144I OnDemand Load Id = >5603-11-0-7FAA-20170421000000-20170421000000-5608<                                                     
 ARS1106E Connection cannot be established for the >ARCHIVE< server                                                                   
 ARS1157E Unable to store the object >7FAAA<.  Object size 111647893                                                                 
 ARS1146I Loaded 0 rows into the database                                                                                             
 ARS4311E 10/12/17 11:59:11 Loading failed                                                                                           
 ARS4318E Processing failed for file >file-name<                                                               
 ARS4320I 10/12/17 11:59:11 -- Unloading started                                                                                     
 ARS1106E Connection cannot be established for the >ARCHIVE< server                                                                   
 ARS1162E Unable to unload data from OnDemand - LoadId(5603-11-0-7FAA-20170421000000-20170421000000-5608) Rows Deleted(0)             
 ARS4322E 10/12/17 11:59:11 -- Unloading of data failed                                                                               
 ARS1106E Connection cannot be established for the >ARCHIVE< server                                                                   
 Unable to log load information                                                                                                       
 Processing has stopped.  The remaining files will NOT be processed.     

can you help me to solve this problem?

thx jhe2                                                             

[jsquizz]

Hi, what else is in the system log around this time frame.

Also, I am sure that others are going to reccomend to get off of 9.0.0.5 and get to the latest fixpack, or even move to 9.5 / 10.1

[Justin Derrick]

Jeff's right.  v9.0 is out of support.  The last patch for v9.0.x was just released last week -- move to that version before anything else.

Links to the latest patches can always be found on the Wiki:  https://cmod.wiki/index.php?title=Main_Page#IBM_CMOD_Fixpacks_.26_Security_Bulletins

-JD.

[Nolan]

The error message is what you should be focusing on.

 ARS1106E Connection cannot be established for the >ARCHIVE< server 

Are you loading from a different server?  Have you loaded other documents from this server to your CMOD successfully?

[Greg Ira]

Are you using OAM for your storage manager?  You may be hitting the 50MB limit for OAM if you haven't increased the ability to load larger files through IEFSSNxx

[jhe2]

We are using OAM for storage manager und we have incraesed the maximum object size to 256 MB

OAM1 Parms: TIME=LOC  MSG=EM  UPD=Y  QB=Y
             MOS= 256  OTIS=N  LOB=N  DP=N


All Documents lower 100 MB we are loading with out problems.

[jhe2]

successful load of a Document with size lower 100 MB and greater 50 MB:


 ARS4335I Server Version <9.0.0.5>  Operating System <z/OS> <04.25.00>  Database <DB2> <11.01.0005>                                   
 ARS4312I 10/26/17 10:21:40 -- Loading started, --UNKNOWN-- bytes to process                                                         
 ARS1408W Warning:  A document of size >53101401< bytes was processed.  It is not recommended to store documents in OnDemand greater t
 han 50MB in size.  Although data may store successfully, it is possible that the data may not be able to be retrieved successfully. 
 ARS1144I OnDemand Load Id = >5603-11-0-16FAA-20170831000000-20170831000000-5608<                                                     
 ARS1146I Loaded 1 rows into the database                                                                                             
 ARS1175I Document compression type used - Disable.  Bytes Stored = >53101529< Rows = >1<                                             
 ARS4310I 10/26/17 10:21:41 Loading completed                                                                                         

[jhe2]

now we get an other message for Documents greater than 100MB -> unable to allocate enough memory:

ARS1109E Unable to allocate enough memory.  File=arsadmp.c, Line=1470
ARS1157E Unable to store the object >2FAAA<.  Object size 111647893 
ARS1146I Loaded 0 rows into the database                             
ARS4311E 10/26/17 11:06:03 Loading failed                           

[jsquizz]

This is strange since it looks like it is a generic index file you are trying to load.

What system is arsload running from? Is it local to the library server or is it on an obj/remote server (..or something..)

Note - No experience with Z/OS, but I have seen issues like this before, mostly in 8.4

[Justin Derrick]

Unable to allocate memory indicates you've exceeded the user's ulimit.  I'm not sure if this is a z/OS setting, or if it behaves like AIX/Linux under USS -- you can see all your limits with "ulimit -a" at a UNIX shell prompt.

-JD.

[jhe2]

arsload running from z/OS
ARS4335I Server Version <9.0.0.5>  Operating System <z/OS> <04.25.00>  Database <DB2> <11.01.0005>   

[Greg Ira]

Get your limits from OMVS.  Run the command ulimit -a from the command line or from batch and show us the output

[jhe2]

here the output from ulimit -a:

core file         8192b     
cpu time          901       
data size         unlimited
file size         88000000b 
stack size        unlimited
file descriptors  64000     
address space     36928k   
memory above bar  2048m     

[Greg Ira]

They definately seems low.   Ours is currently:
core file         8192b
cpu time          unlimited
data size         unlimited
file size         unlimited
stack size        unlimited
file descriptors  64000
address space     1011240k
memory above bar  2048m

[Ed_Arnold]

I think Greg has nailed it.

address space     36928k

36 meg?  You should be running REGION=0M and getting over a gig of private area.

| Make sure you run the ulimit -a under the userid associated with the load.

| Here's sample JCL:

//ULIMIT   JOB (BLAH,REPORT),'ARNOLD',CLASS=A,REGION=0M,
//             MSGCLASS=X,MSGLEVEL=(1,1),USER=ARSSERVR 
//STEP31 EXEC PGM=BPXBATCH,REGION=0M                   
//STDPARM  DD *                                         
SH ulimit -a                                           
//SYSPRINT DD SYSOUT=*                                 
//STDERR   DD SYSOUT=*                                 
//SYSOUT   DD SYSOUT=*                                 
//SYSPRINT DD SYSOUT=*                                 
//STDERR   DD SYSOUT=*                                 
//STDOUT   DD SYSOUT=*                                 

Ed