CMOD for z/OS 7.1 and using PDF indexer

[Jon.Bauer@bcbsks.com]

Does anyone have any experience using the PDF indexer?  I have some questions about the Adobe type 1 font libraries.

Thanks.

[Ed_Arnold]

Hi Jon - just a note for anyone following this thread - if you're going to be doing heavy-duty PDF indexing you probably will want to do that on a platform other than z/OS and then load to z/OS from there.

Ed

[Greg Ira]

HI Ed,
Does you statement still hold true for V8.5?

[Justin Derrick]

This holds true for all non-windows platforms.

If you have a large volume of PDFs to load, you'll want to integrate CMOD on Windows with the PDF Indexer into your environment.  It's not an answer I like, but it's the best answer at the moment.

-JD.

[Greg Ira]

Thanks for the response.  We're just diving into the world of PDF indexing now and will have to see if that's an option for our user.

[pankaj.puranik]

Jon and Greg

As JD/Ed suggested you should avoid using PDF indexing on non-windows platform.
We are already doing this on AIX and the loading performance is really bad.

I have one example where the input file has 36000 plus accounts and about 75000 pages.
It took over 8 hours for the arspdoci alone to complete.

It took us more than 2 weeks to load approximately 200 GB of data .

Thanks
Pankaj.

[LWagner]

Gentleman:

We have been  in production loading PDF files to z/OS for 22 months.  We are still on CMOD 8.4.0.3 on z/OS, but migrating this facet of OnDemand 8.5 to AIX soon. We are now loading in parallel. 

We index on 8 Windows servers, on CMOD MP 8.4 . This version does not do PDF compression or resource handling well, so we see about 19 to 1 EXPANSION of the files output from the indexer, and for OAM , these files should not exceed 250 Mb in size. So from this, we have our PDF generator limit PDF container size to 200 pages, which means files of up to about 9 Mb. The result is 400-600 files loaded daily, round robin for indexing to the 8 Windows servers, and this completes in 2-3 hours, with a resulting mainframe storage of 25-45 Gb in OAM storage.

The load to AIX, just started this week, at the same time, completes in about 55 minutes on only two Windows servers.  PDF Compression works well under CMOD 8.5, with files reduced by 60 %. So as we move, we will use about 95% less storage.

If I may be of help, fire a question.

Larry Wagner
LA DWP

[LWagner]

The daily load at this utility is about 55,000 accounts, and 160,000 pages.  We recently had a double load of 112,000 bills, and about 400,000 pages.  It took a couple hours extra to complete.

[LWagner]

In the indexer parms, I have this value pointing to an AIX folder.
FONTLIB=/usr/lpp/Acrobat3/Fonts

[Justin Derrick]

The load to AIX, just started this week, at the same time, completes in about 55 minutes on only two Windows servers.  PDF Compression works well under CMOD 8.5, with files reduced by 60 %. So as we move, we will use about 95% less storage.

That's an amazing achievement.  Would you consider doing a webinar for ODUG to talk us through the process?

[LWagner]

Thanks for the compliment.  I'm not so sure its justified, so much as my neck was pushed out there further sooner with tasks to get done on PDF loads to CMOD. I just go with the flow on data given and software I have to work with.

My instance production environment is CMOD 8.4.0.3 on z/OS using DB2 9.1 and OAM. All line reports are indexed on z/OS.  All PDFs are indexed via Windows CMOD 8.4 index servers.  Our bills have a lot of graphics, and from 30 to 44 different font face combinations of size and face from about 15 font files.  The 8.4 PDF Indexer is not good with shared resources, so we limit our container PDF files for bills to a maximum of 400 pages.  Input file size is about 6-10 MB, and the output PDF sent to z/OS is up to about 220 MB, typically an expansion factor of 19.  Weird errors happen when PDF files exceed 250 Mb going to the z/OS instance for storage. CMOD and OAM manage to then compress this file from 10 to 20 %.

The improvement in PDF output size is simply the result of moving to CMOD 8.5.0.5 on AIX, with CMOD 8.5 Windows indexing servers, which has a better PDF Indexer product.  PDFs are Windows files first, and the Indexer from Adobe seems optimized for Windows.  I did try to index a PDF on z/OS. A two thousand page PDF did not complete processing in two hours.  At that point IBM simply siad they recommended a Windows index server for the task, so I tried it.  No doubt indexing on AIX is a similar problem, the program is not optimized for that environment.

Our internal customer base wanted the customer bills uploaded in 4 hours or less. Based on the average bill size and number of bills daily, we were going to have from 400-600 PDFs daily, with about 20 minutes total indexing and upload time for each PDF.  We achieve the desired upload completion goals using 8 CMOD 8.4 Windows index servers.  Files are sent to each of the 8 servers by a ninth ftp server.

[Ed_Arnold]

I was talking with another person who tried indexing and loading PDFs on z/OS.

I think this was on 8.4.1 which is a big improvement over 8.4.0, PDF storing-wise.

Depending on the FILE TYPE of the input, the results can be dramatic.

VB       very slow                                                     
FB       slow                                                     
HFS      extremely fast                                                     
VIO-VB   middlin'                                                     
VIO-FB   twice as fast as middlin' 
TFS      didn't try

Before completely writing off z/OS, try some benchmarks of your own.
 
I didn't do the test and have no further knowledge.  Would like to try this myself, though.

Ed

[Senthil]

Hi

We are using PDF indexer in z/os. We both used initially in  v7.1.2 and later upgraded 8.4..
We don't have any issues though the volume is relatively small..

Senthil