91
MP Server / Re: User Exit call to SOAP web service
« on: November 04, 2015, 09:06:51 AM »
Thank you Alex for your speedy reply, I will have a look at it!
Lars
Lars
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
92
MP Server / User Exit call to SOAP web service
« on: November 04, 2015, 06:59:21 AM »
Hi,
We need to do a lookup in one of the CMOD User Exits, arsuprep to be precise.
I have customized a User Exit before (arsusec), but that was not nearly as complex.
Has anyone had any experience writing User Exits, preferably in a Windows environment (Unix might do) like ours, and could you in that case tell us how we can call a Web Service via a SOAP call?
We are trying different "intermediate steps", such as a Java module or a DLL acting as the "middle man" here.
Any working examples would be appreciated, as well as pointing out caveats/bewares.
Thanks for reading!
We need to do a lookup in one of the CMOD User Exits, arsuprep to be precise.
I have customized a User Exit before (arsusec), but that was not nearly as complex.
Has anyone had any experience writing User Exits, preferably in a Windows environment (Unix might do) like ours, and could you in that case tell us how we can call a Web Service via a SOAP call?
We are trying different "intermediate steps", such as a Java module or a DLL acting as the "middle man" here.
Any working examples would be appreciated, as well as pointing out caveats/bewares.
Thanks for reading!
93
MP Server / Re: Anyone doing PCI with CMOD?
« on: November 04, 2015, 06:52:23 AM »
Hi, I am new here, and althought this topic has been silent for quite a while, I thought I could add some info.
Back in 2011 or 2012, we got the requirement to PCI-DSS certify the CMOD system of a customer.
They have stored each single receipt for each store worldwide, and they keep them for 11 years (by law).
The requirement that we said yes to was to:
*export each document
* "wash it" from cc number data - i.e. we masked the required numbered of digits by replacing them with asterisks (*) //or maybe it was hash signs (#) can't remember //
* reload it back into OnDemand
NOTE: This would be categorized as option two described by AWHS above - "take CMOD out of scope".
On top of that, all POS systems in all stores had their software upgraded to automatically mask CC number data as described above.
We also require that any new data that is to be archive is "certified free from credit card numbers" - and we of course verify that before we start archiving it.
It was a tedious semi-automatic procedure, but sure enough, we processed and "cleaned" millions of report pages and receipts.
The only downside (which was pretty small) were that some stores complained that they could no longer find customer receipts by searching for CC number. So if that is an important requirement, the "aliase" method described above is probably better.
Back in 2011 or 2012, we got the requirement to PCI-DSS certify the CMOD system of a customer.
They have stored each single receipt for each store worldwide, and they keep them for 11 years (by law).
The requirement that we said yes to was to:
*export each document
* "wash it" from cc number data - i.e. we masked the required numbered of digits by replacing them with asterisks (*) //or maybe it was hash signs (#) can't remember //
* reload it back into OnDemand
NOTE: This would be categorized as option two described by AWHS above - "take CMOD out of scope".
On top of that, all POS systems in all stores had their software upgraded to automatically mask CC number data as described above.
We also require that any new data that is to be archive is "certified free from credit card numbers" - and we of course verify that before we start archiving it.
It was a tedious semi-automatic procedure, but sure enough, we processed and "cleaned" millions of report pages and receipts.
The only downside (which was pretty small) were that some stores complained that they could no longer find customer receipts by searching for CC number. So if that is an important requirement, the "aliase" method described above is probably better.
94
Report Indexing / Re: PDF Indexing with PPD (PagePiece Info)
« on: October 09, 2015, 06:01:10 AM »
Hi Justin and thanks for your reply!
I took a look at the video, and yes, that is what we are already doing!
The question in my initial post above is how to avoid getting "New" resource packages every time you send a new instance of a particular PDF file to OnDemand.
As you can see in the image I attached, it says "New" in the "Resource reuse" column for every new file we add. One would think that when sending the same type of documents, with the same font sets, same background, same images etc (i.e. the same resources), CMOD would be able to recognize the resources as "Existing".
Here is an extract from the Load logs for three consecutive loads:
ARS1142I Resource D:\OnDemandDirectories\arstmp\infoflyttut.0.villkutsk160101.paketforalltid.20151002.194133.pdf.res will be added as resource >12146-11-0<. Compression Type(Disable) Original Size(2237100) Compressed Size(2237100)
ARS1142I Resource D:\OnDemandDirectories\arstmp\infoflyttut.0.villkutsk160101.paketforalltid.20151002.200204.pdf.res will be added as resource >12147-11-0<. Compression Type(Disable) Original Size(2237100) Compressed Size(2237100)
ARS1142I Resource D:\OnDemandDirectories\arstmp\infoflyttut.0.villkutsk160101.paketforalltid.20151002.202231.pdf.res will be added as resource >12148-11-0<. Compression Type(Disable) Original Size(2237100) Compressed Size(2237100)
Even though the resource files have the exact same size for all loads, when I run a File Comparison program, they prove to be quite different indeed.
How come?
So the 2nd question in my original post above is quite simply: can PDF resources be re-used, using this indexing method? If so, what do we need to change?
(As for question 1 above, we will continue experimenting with different types of "no compression" and linearization, but if anyone else has any benchmarktimes performance measurements/timings/examples, it would be nice to compare our respective results.)
I took a look at the video, and yes, that is what we are already doing!
The question in my initial post above is how to avoid getting "New" resource packages every time you send a new instance of a particular PDF file to OnDemand.
As you can see in the image I attached, it says "New" in the "Resource reuse" column for every new file we add. One would think that when sending the same type of documents, with the same font sets, same background, same images etc (i.e. the same resources), CMOD would be able to recognize the resources as "Existing".
Here is an extract from the Load logs for three consecutive loads:
ARS1142I Resource D:\OnDemandDirectories\arstmp\infoflyttut.0.villkutsk160101.paketforalltid.20151002.194133.pdf.res will be added as resource >12146-11-0<. Compression Type(Disable) Original Size(2237100) Compressed Size(2237100)
ARS1142I Resource D:\OnDemandDirectories\arstmp\infoflyttut.0.villkutsk160101.paketforalltid.20151002.200204.pdf.res will be added as resource >12147-11-0<. Compression Type(Disable) Original Size(2237100) Compressed Size(2237100)
ARS1142I Resource D:\OnDemandDirectories\arstmp\infoflyttut.0.villkutsk160101.paketforalltid.20151002.202231.pdf.res will be added as resource >12148-11-0<. Compression Type(Disable) Original Size(2237100) Compressed Size(2237100)
Even though the resource files have the exact same size for all loads, when I run a File Comparison program, they prove to be quite different indeed.
How come?
So the 2nd question in my original post above is quite simply: can PDF resources be re-used, using this indexing method? If so, what do we need to change?
(As for question 1 above, we will continue experimenting with different types of "no compression" and linearization, but if anyone else has any benchmark
95
Report Indexing / PDF Indexing with PPD (PagePiece Info)
« on: October 08, 2015, 07:27:37 AM »
Hi,
We have now implemented an Application that actually uses the "newest" way to index PDF files, namely using PagePiece info.
Environment is Windows 2008, DB2 9.7 and CMOD Server 9.0.0.6. TSM is on a separate W2K8 server.
We have successfully delivered several batches/files of 20000 documents to OnDemand, and so far no problems.
(Well, to be fair, it was nothing BUT trouble on CMOD 9.0.0.3, we had to install the 9.0.0.6 fix pack to remedy that.)
As this is our first document type that uses that, I would say we are still beginners, but it works like a charm!
We are currently looking into some performance issues.
1. Indexing takes surprisingly long time - I suspect this is because of that the PDF is still slightly compressed, although the creator has indeed tried to switch compression off. Also, the PDF files are "linearized" after they are created. (The performance is not bad at all, but it takes nearly 4 minutes to index where I was expecting around 0,5-1 minute.)
2. One of the main benefits with using PDF Indexing is the resource collection. So far however, we have been unable to make OnDemand recognize that this is the same resources in each batch - it says "New" on every load.
At first, we had subsets of (customized) character sets/fonts made for every batch, but when we turned that off and sent the full set(s) with every batch, it still did not catch it - every file is deemed to have its own unique set of PDF resources.
Has anyone already solved this type of tuning problem?
I would be happy to hear your info on how to tune this solution to perfection. In return, I will be happy to share our experiences of this!
I will try to attach an image to this post. The lines highlighted in blue have full font sets and "no PDF compression". The lines below that use compression and have subsetted font sets.
We have now implemented an Application that actually uses the "newest" way to index PDF files, namely using PagePiece info.
Environment is Windows 2008, DB2 9.7 and CMOD Server 9.0.0.6. TSM is on a separate W2K8 server.
We have successfully delivered several batches/files of 20000 documents to OnDemand, and so far no problems.
(Well, to be fair, it was nothing BUT trouble on CMOD 9.0.0.3, we had to install the 9.0.0.6 fix pack to remedy that.)
As this is our first document type that uses that, I would say we are still beginners, but it works like a charm!
We are currently looking into some performance issues.
1. Indexing takes surprisingly long time - I suspect this is because of that the PDF is still slightly compressed, although the creator has indeed tried to switch compression off. Also, the PDF files are "linearized" after they are created. (The performance is not bad at all, but it takes nearly 4 minutes to index where I was expecting around 0,5-1 minute.)
2. One of the main benefits with using PDF Indexing is the resource collection. So far however, we have been unable to make OnDemand recognize that this is the same resources in each batch - it says "New" on every load.
At first, we had subsets of (customized) character sets/fonts made for every batch, but when we turned that off and sent the full set(s) with every batch, it still did not catch it - every file is deemed to have its own unique set of PDF resources.
Has anyone already solved this type of tuning problem?
I would be happy to hear your info on how to tune this solution to perfection. In return, I will be happy to share our experiences of this!
I will try to attach an image to this post. The lines highlighted in blue have full font sets and "no PDF compression". The lines below that use compression and have subsetted font sets.