Large Xerox Files + AFP2PDF

[jeffs42885]

We recently started using Infoprint XT to convert Xerox Metacode jobs to AFP. Some of these files can be quite large.

It seems like we are hitting a limit with the files. Infoprint processes the file in the /var/pdxt/jobs directory, and the first suggestion from XT support was to increase VAR. So we increased var to 15GB.

The file that we are are working with is 3.5 GB, and I watch the AFP file get built, and it fails right around 2.1GB It throws an ONCODE 1040 error and also tells us broken pipe. The file system is JFS2 enabled, and all the ulimits are set to unlimited (for both the account running XT and root)

I am just wondering if anyone has seen this before. These are simple mainframe reports that just contain several hundred thousand pages. I have SEV1/2 tickets open with both IBM and Ricoh on this issue and I am just wondering if anyone has encountered this before.

Thanks in advance,

Jeff

[Justin Derrick]

We got hit with a ulimit issue earlier this year -- if your job is launched by crontab (or any other process owned by root) then you need to make sure that the ulimits for root are also set to unlimited.

-JD.

[jeffs42885]

yep, ulimits are all set to unlimited for both the process account that runs the application as well as root

[jeffs42885]

I have since switched positions but, from what my investigation found was that when AFP2PDF was being called (It was a custom IBM version that would take the "indexed" afp file, and craete the ard out ind)

on VERY VERY large AFP files, year end stuff.. If i ran 1 file, i noticed cpu spike, if i noticed 2 same, 3, system cpu would be at 99.5% busy. I am suprised that this did not cause the box to go down.

So I am not sure if it a bug in the AFP2PDF code, but just be careful if you run ODLoad_afp2pdf to transform Indexed AFP files (Files that come out of Ricohs AVE tool) to PDF to load'em into OD..

[Justin Derrick]

For most modern OS's, being CPU bound for hours (days/weeks/months) shouldn't be an issue.  Running out of RAM/Swap is a no-no though.

I suspect AFP2PDF is a Java app ("write once, run anywhere") and Java is well known for being CPU-hungry.

-JD.

[jeffs42885]

Just an FYI- I think that this was also on outdated hardware that were held back due to licensing, so we were limited by cores, newer CPUs, etc. Once they migrate i expect them to see a world of difference.

Oh, and we tried this with a version written in C and a Java version, same issue.