Messages

16

MP Server / CMOD Load Threshold Metrics - MP 9.5 on AIX with DB2 and PDF Indexer

« on: November 20, 2020, 04:36:52 AM »

Has anyone ever seen when loading PDF files with many contiguous statements where you can cross loading thresholds such that you have to keep file sizes below a certain size to get them to load in a reasonable amount of time?

I have an example where a 200 MB PDF file loads in 4.4 hours but when I try to load one that is 340+MB it takes 11.5 hours to load?  Scratching my head   

I originally thought the dropping unnecessary indexes from the AG was the issue, but the problem is clearly larger than just indexes. 

Note: I have other AGs that can load PDF files that are 500MB in minutes and not hours.

17

MP Server / Re: Unified Content and Access integration Layer (UCAIL)

« on: November 13, 2020, 06:13:28 AM »

It's conceptually very much like CMIS but reportedly benchmarks faster.  We've done some custom solutions like this where we extract load metadata (query strings and load ids) so that web service calls can do a "get list" against SQL in lieu of ODWK - VERY fast.  Then our web service ONLY calls ODWEK when the user wants to "get" the document/statement - executes the stored query string via ODWEK.

The big picture is "enterprise search", and my question is, "Does CMOD provide a way to identify a document ID, a GUID, if you will, so that this can somehow be extracted and stored along with other metadata?  Or do we have to go with an approach like I outline above where we extract load ID and store a query string to be executed for "gets". 

And my CMOD context is PDF statements - while many PDF files only had one statement in them, most were large PDF files - hundreds of megs - that contained tens of thousands of contiguous PDF statements all in the same file.

Thanks!

18

MP Server / Unified Content and Access integration Layer (UCAIL)

« on: November 08, 2020, 08:42:36 AM »

So there are vendors in the marketplace that have productized software to create integrated content access and integration layers that abstract your content repositories (FileNet, CMOD, SharePoint, et. al.) metadata properties (including pointers to documents/reports) to an "access and integration layer" typically storing this metadata in NoSQL database.  This can empower web service access to enterprise search and "get" calls at this "access and integration layer" in lieu of writing API interfaces to all the individual back office content stores.  Assume for the sake of conversation that all noted so far is possible, when it comes to CMOD, is there a way to get to a single "Unique ID" for a specific CMOD document or report?  Given that CMOD is all about where something is within an input file, it's much different than a GUID (globally unique ID) that is stored in a content repository like FileNet or SharePoint.  Just wanted to see if anyone had attempted to do this and what they had come up with as their solution. 

19

MP Server / Re: Changing Application Group Field Indexes to Filters

« on: November 08, 2020, 08:24:04 AM »

Great link - thanks for sharing Justin!!!

20

MP Server / Changing Application Group Field Indexes to Filters

« on: October 20, 2020, 08:33:30 AM »

I have a 9.5.0.4 CMOD OnDemand MP solution running on AIX 7.2 and DB2 10.5, and TSM 7.1 that takes FOREVER (many hours) for me to load a 300MB load file of statements.

As a comparison, I have many other AGs to which I can load the same size files in a fraction (less than an hour) of the time.

What I have found is my AG developer created an inordinate amount of indexes (10), and I think this is the problem with load performance.

All the other AGs that load in a fraction of the time include at most 3 indexes.

Onto my question - Can I just update my AG to change the fields that are Indexes to Filters?

How does DB2 handle this when I drop effectively about 7 indexes?

Thanks so much!!

21

MP Server / Re: Changing AD UPN value - Will this break CMOD 9.5 Users Authenticating to AD?

« on: September 11, 2020, 06:53:13 AM »

Sorry Justin - I probably didn't state the change and why it was being made clearly enough - UPN values are being changed in the organization and I'm wanting to know if this might cause problems with my current CMOD LDAP config.

In looking at my ars.cfg file (attached earlier), I'm not seeing that I'm reliant on the UPN value but am instead using sAMAccountName to bind to LDAP/AD. 

I'm just trying to make sure I'm not missing anything.

Thanks.

22

MP Server / Re: Changing AD UPN value - Will this break CMOD 9.5 Users Authenticating to AD?

« on: July 21, 2020, 09:58:07 AM »

Derrick / Rob -

Here is my edit ars.cfg file with the LDAP section shown here below.

From what I see, I'm not seeing any reference to UPN name and only the sAMAccountName being used to connect to LDAP.

Wouldn't this make you think that any change to the UPN value would NOT affect us and is only ancillary to our use of LDAP?

###########################################
# LDAP Parameters (Library Server Only)   #
###########################################
ARS_LDAP_SERVER=ldapservername
ARS_LDAP_PORT=389
ARS_LDAP_BASE_DN=OU=ouname,DC=dcname,DC=pri
ARS_LDAP_BIND_DN=CN=binduser,OU=Generic,OU=Users,OU=ouname,DC=dcname,DC=pri
ARS_LDAP_BIND_DN_PWD=binduserpassword
ARS_LDAP_BIND_ATTRIBUTE=sAMAccountName
ARS_LDAP_MAPPED_ATTRIBUTE=sAMAccountName
ARS_LDAP_ALLOW_ANONYMOUS=FALSE

23

MP Server / Changing AD UPN value - Will this break CMOD 9.5 Users Authenticating to AD?

« on: July 16, 2020, 02:40:15 PM »

We are using CMOD 9.5 MP on AIX and are planning on updating our Active Directory UPN values as part of another initiative.

In AD, as I appreciate it,  there are the following userid attributes:

userPrincipalName (UPN) – Current value = TESTUSER@DOMAINCONTROLLER.pri    New value = TESTUSER@emaildomain.com
sAMAccountName – current value = TESTUSER      no change

We're effectively changing the userPrincipalName from a network domain to an email address.

Does anyone know if CMOD uses UPN values to authenticate users or the sAMAccountName to authenticate users?

Thanks.

24

MP Server / Getting ARS1110E Error when attempting to load PDFs to updated AG

« on: May 11, 2020, 01:03:59 PM »

I'm attempting to copy an existing application group to _old and add a new application group with the previous application group name.

And while I can get all this to work, I can't load the application group without getting the following error: 

ARS1110E The application group >< does not exist or user >ADMIN< does not have permission to access the application group

The interesting part about the error is that when I run the arsload command utility from the command line to load this PDF data that it doesn't seem to recognize my AG - note in message "The Application group >< does not exist..." - and I'm specifying it explicitly in the following load syntax with a format:

/opt/IBM/ondemand/V9.5/bin/arsload -nvf -c /arstmp -b "VALUE" -B IDX.IGN.AG.APP.IGN.IGN /export/home/userid/VALUE.JOBNAME.AGName.Appname.07312016.101518.pdf

Note: if I remove my newly created application group and re-run the load syntax above it loads just fine

25

MP Server / Re: Load Jobs Failing and no System Log Messages??

« on: January 30, 2020, 04:46:53 PM »

So to add some more color here, after further analysis on this, a teammate of mine noted that the Admin account had permissions to the AG in Test but NOT in PROD.

Having previously tried to load the failed load file in Test, I knew it worked there.

So on a hunch, I added the Admin to the permissions group as “Administrator” in PROD, set up just as it was on the AG in Test.

I attempted to reload the statements, and they loaded like a charm.  And this worked on 2 AGs that had just started failing??

My IBM Case Support is one of the best guys at CMOD Support, and he, like I don’t think Admin access is required, and we have several AGs that don’t have Admin in the perms and they have been loading just fine.

Also, these AGs that JUST started failing have been running without fail or change to them for years?!?!?

I at least wanted to share that my solution so far has been the security change as noted.

These are NOW loading like Champs...

26

MP Server / Load Jobs Failing and no System Log Messages??

« on: January 30, 2020, 10:21:18 AM »

Has anyone seen an instance where load jobs that have run for years start failing but that give absolutely zero messages in the system log except for message 81 (AG object query completed)?

While most of our AGs load just fine, all of a sudden we have few starting to fail with this strange phenomenon?!?!

The content is PDF and is going through the PDF Indexer program.

This is occurring on AIX 7.2 running CMOD 9.5.04 MP with DB2 10.5.3 and TSM 7.1.1.

27

MP Server / Re: Library Server Hangs when LARGE PDF statements (> 200 MB) Retrieved

« on: August 14, 2019, 08:42:39 AM »

Our issue is that we have thousands of pages with 10 images (checks) on each page - 3000 such pages in just this example.

I'll talk to the folks that are preparing these to see if there's optimization they can do, but with examples like this, I think the sheer volume of image pages alone - even if optimized - would be too much to store in a resource file and probably not how CMOD was intended to be used, right?

Also, I thought the PDF Indexer only separated the header graphics (logos, etc.) from the statement and put those in resource files, BUT it appears PDF Indexer is also separating the check images and storing those in the resource files too - is that accurate?

Finally, my biggest need is a query to be able to identify all my reports/statements that takes into account the size of the resource files that will accompany them?  If I can run this, I can sort by my problem statements and start addressing them by size perhaps even prioritizing by resource file size.  If the resource file is not too large, I can probably return those in good order - even if many text pages.

Thanks so much for all you do for the ODUG Community!!!!

28

Content Navigator / Re: ICN fronting CMOD Hangs up for LARGE Files - How to Control??

« on: August 12, 2019, 02:51:05 PM »

Content Navigator team told me to upgrade to ICN 3.0.6 and take advantage of the client side Daeja rendering - lightweight and fast.  They also said there is no governor that you can put into place to preclude large statements from being presented.

29

MP Server / Re: Library Server Hangs when LARGE PDF statements (> 200 MB) Retrieved

« on: August 12, 2019, 02:47:37 PM »

Justin -

In this example, I have 883 pages of text and 2,812 of images - 10 images per page.

Do you think the issue is the optimization of the images OR just the sheer volume of them?

Also, is there a good query to identify my large statements so I can determine the disposition of them?

When I have such large resource files like this, just pulling the statement size from my AG table is insufficient.

Thanks.

30

MP Server / Re: Library Server Hangs when LARGE PDF statements (> 200 MB) Retrieved

« on: August 09, 2019, 01:27:42 PM »

Adding a little more information here, when I query DB2 for my AG table, I see the following for an offending file

STMT_DT RESOURCE    DOCSIZE     PAGECOUNT
------- ----------- ----------- -----------
  17928         854    14947245        3696

When I look at the 854 resource, it's 277MB