Storage object error

[pankaj.puranik]

Hi

While loading a file we got the following error. Has this got something to do with the Storage object size?

arsload: 04/20/11 12:46:21 -- Loading started, --UNKNOWN-- bytes to process
The field name 'DHAccountNum' found in the index file does not match any expected fields.  The field will be ignored.
The field name 'DHCusipNum' found in the index file does not match any expected fields.  The field will be ignored.
The field name 'DHDupeNum' found in the index file does not match any expected fields.  The field will be ignored.
OnDemand Load Id = >5185-1-0-32FAA-14975-14975<
An error occurred.  Contact your system administrator and/or consult the System Log.  File=arsadmp.c, Line=1150
Unable to store the object >32FAAB<.  Object size -1567007475
The last row successfully loaded was 2478
Loaded 2478 rows into the database
arsload: 04/20/11 12:47:42 Loading failed
arsload: Processing failed for file >wmden02p<
arsload: 04/20/11 12:47:42 -- Unloading started
OnDemand UnLoad Successful - LoadId(5185-1-0-32FAA-14975-14975) Rows Deleted(2478)
 
04/20/11 12:47:44 -- Unloading of data was successful
arsload: Processing has stopped.  The remaining files will NOT be processed

[Alessandro Perucchi]

Hello Pankaj,

which version of CMOD do you have?
What is the size of the file you are trying to archive?
Do you use TSM? If yes, do you have something in the logs of TSM?
Is there something in the "System Log" which might help in finding this problem?
Do you have enough disk space to save this document?
How many documents to index? 2478 or more?

It might have something to do with Storage Object size... but not necessarily.

Cheers,
Alessandro

[pankaj.puranik]

Unfortunately I don't have access to these logs and I am not with the team that is resolving this.
But I am interested to know how to solve this.
Here is what I could gather.

What is the CMOD version? CMOD 8.4, TSM Server 5.X, Solaris 10
What is the size of this file that was being archived? (the first file that gave the problem was 2.7 GB)
Is TSM being used? If yes, can we get a copy of that for this error? Yes TSM is being used.
Was there some message on the System Log folder for this load? I couldn't get this.
And how many documents in total were there in this report file? 4139 is the document count in the 2.7GB file

[Justin Derrick]

Um, that's a 1.5GB object.  Are you sure you want someone pulling that back in one piece across the network?  You should be using large object support for these types of reports.

[pankaj.puranik]

Does large object support help during the load process also?
I thought it comes into picture only during retrievals.

[pankaj.puranik]

One more piece of information that I got is that the individual documents are not more than 300 KB.
So large object support in this case may not have a drastic impact on retrievals.
But would it help to have large object support if the input file size is more than 1.5 Gig?

[pankaj.puranik]

This is how it was fixed. I got to know that there were 2 issues.
1. Space in the DEV box.
2. The compression algorithm was changed to OD77.

If anyone can throw some light on this it would be great.

I am not sure if this has got something to do with the compressed object size and the storage object size.

[Alessandro Perucchi]

Hello Pankaj,

Well, CMOD needs during the load to have disk space in order to do the compression -> No compression no additional disk space. Compression -> the compression file size in addition to the normal file.

Maybe that was just the drop to make it crash...

Cheers,
Alessandro

[Justin Derrick]

Hi Pankaj.

If your individual documents are 300k, and your object is 1.5G, there is likely a problem with your indexing as well.  Except in very special circumstances, objects should always be in the 10-Megabyte range.

-JD.