Messages

91

MP Server / Re: Error with V10.1 - Loading to HCP

« on: November 11, 2021, 03:02:47 PM »

Hmmm - PH37296, you have that on?

https://www.ibm.com/support/pages/node/6454823?mhsrc=ibmsearch_a&mhq=ph37296

And you're still seeing ARS1159E resource error message?

Ed

Yes.

I updated to V10.1.0.8, and Applied the fix.

92

MP Server / Error with V10.1 - Loading to HCP

« on: November 11, 2021, 09:48:28 AM »

We are trying to load directly into HCP using the API's included in CMOD V10.1. We saw this error using V10.1.0.7, and IBM suggested updating to version 10.1.0.8, and applying hotfix -

https://ak-delivery04-mul.dhe.ibm.com/sar/CMA/IMA/09q33/0/PH37296.10.1.0.8.readme.txt

From our system log-

Code: [Select]

2021-11-09 19:47:29.672532 ADMIN   350766224 Error Yes     88 Application Group Failed Load: Name(AG) LoadId(--INDEXING ONLY--) File(SERVER.XXXXXX.XXXXX.XXXXX.XXXXXXXXXXX.XXXXX.ARD)
2021-11-09 19:47:28.943470 ADMIN   350766224 Error No    466 Hitachi Storage Error: S3 curl error, Return Code=503, Reason=0, File=arssmsms.cpp, Line=2000  Srvr->serv non-SSL<-
2021-11-09 19:47:28.931814 ADMIN   350766224 Error No     24 Object >335< in Application Group >LEH< not found in node >uat-ond<  Srvr->server non-SSL<-
Within the 88-

Code: [Select]

2021-11-09 18:47:28.655797: ARS4308I Indexing completed
2021-11-09 18:47:28.656260: ARS4312I Loading started, 264070985 bytes to process
2021-11-09 18:47:28.956838: ARS1107E An error occurred.  Contact your system administrator and/or consult the System Log.  File=arsload.c, Line=9262
2021-11-09 18:47:28.956999: ARS1159E Unable to retrieve the object >335<
2021-11-09 18:47:28.957090: ARS4311E Loading failed
2021-11-09 18:47:29.670819: ARS4318E Processing failed for file >svm2271pdv.NRSC0101.NRSN1.RTLSTMTHEN.2021110803313.082310730.ARD<
Anyone have any ideas about this issue?

Thanks!

93

MP Server / Re: Issue implementing SSL

« on: October 15, 2021, 10:48:24 AM »

Here's another strange issue. When we do the retrieval, it is trying to retrieve the document from the cache defined on the library server - /opt/app/cmod/ARSCACHE - is defined within ars.cache, defined as our cache only - library server storage.

The very last line of this error, is invalid. The document is physically located - and the application group is defined to our cache object server.

Code: [Select]

62572:47394759460608 10/14/2021 10:13:26:981056 INFO arssmcac.c(3645)ArcCACHE_Retrieve:Current state object=/opt/app/cmod/ARSCACHE/retr/BSA/DOC/4FAAA (ArcI32)errno=2

Code: [Select]

62572:47394759460608 10/14/2021 10:13:26:967014 FLOW arscssm.c(541)ArcCSSMP_DataProcessInit:Return arccs return code=0,ARCCS_OKAY
62572:47394759460608 10/14/2021 10:13:26:967025 FLOW arscssm.c(3671)ArcCSSMP_ObjectRetrieve:Enter
62572:47394759460608 10/14/2021 10:13:26:967030 FLOW arscssm.c(3420)ArcCSSMP_ProcessObjName:Enter
62572:47394759460608 10/14/2021 10:13:26:967036 FLOW arscssm.c(3439)ArcCSSMP_ProcessObjName:Return
62572:47394759460608 10/14/2021 10:13:26:967048 FLOW arscssm.c(3179)ArcCSSMP_Retrieve:Enter
62572:47394759460608 10/14/2021 10:13:26:967059 FLOW arssmcac.c(2752)ArcCACHE_Init:Enter
62572:47394759460608 10/14/2021 10:13:26:967072 INFO arssmcac.c(2798)ArcCACHE_Init:Current state startup->cacheNameList[i]=/opt/app/cmod/ARSCACHE
62572:47394759460608 10/14/2021 10:13:26:967079 INFO arssmcac.c(2809)ArcCACHE_Init:Current state (ArcI32)init->status=2
62572:47394759460608 10/14/2021 10:13:26:967085 FLOW arssmcac.c(2817)ArcCACHE_Init:Return rc=0
62572:47394759460608 10/14/2021 10:13:26:967101 FLOW arssmcac.c(3560)ArcCACHE_Retrieve:Enter
62572:47394759460608 10/14/2021 10:13:26:967145 FLOW arssmcac.c(137)ArcCACHEPri_AssembleName:Enter
62572:47394759460608 10/14/2021 10:13:26:967161 FLOW arssmcac.c(101)ArcCACHEPri_GetDocResStr:Enter
62572:47394759460608 10/14/2021 10:13:26:967167 FLOW arssmcac.c(112)ArcCACHEPri_GetDocResStr:Return
62572:47394759460608 10/14/2021 10:13:26:967176 INFO arssmcac.c(153)ArcCACHEPri_AssembleName:Current state assembledObjName=/BSA/DOC/4FAAA
62572:47394759460608 10/14/2021 10:13:26:967180 FLOW arssmcac.c(155)ArcCACHEPri_AssembleName:Return
62572:47394759460608 10/14/2021 10:13:26:981056 INFO arssmcac.c(3645)ArcCACHE_Retrieve:Current state object=/opt/app/cmod/ARSCACHE/retr/BSA/DOC/4FAAA (ArcI32)errno=2

94

OD/WEK & JAVA API / ODWEK - Flow of returning document

« on: October 13, 2021, 07:48:06 AM »

Have a question about how ODWEK works as far as returning documents from our system.

We are using a distributed system with a library server and object server.

My understanding of the flow, is that the ODWEK client sends the Login Requests/Search request to the library server. The Library server then sends the hit list back to the client.

After that, the client sends the retrieve request to the Library server.

Our question is about the request. The data that comes back - Two scenarios- Which one is true - or is there a third.

1) The object server sends the data to the library server, and then the library server returns it to the ODWEK client.

2) The object server sends the data directly to the ODWEK client, bypassing the library server.

95

MP Server / Re: Issue implementing SSL

« on: October 08, 2021, 02:11:41 PM »

If you have separate library and object servers, then you will need unique SSL certificates for each - you can't re-use server certificate from one server to the other.

For troubleshooting IBM CMOD SSL, use openssl s_client with the -showcerts option - that will show you which certificates are being presented by each server.
-JD.

Here's the new issue haha-

Code: [Select]

78759:47665277347584 10/08/2021 08:59:31:101247 ERROR (null)(0)(null):Server message msg timestamp=2021-10-08 08:59:31.090354 msg userid=USERID msg severity=2 msg num=23 msg text=ARS0023E Object >195FAAA< in Application Group >AG< not found in cache, no other storage defined  Srvr->server1.1.1.1 non-SSL<- msg appl=N msg logid=1540228370
78759:47665277347584 10/08/2021 08:59:31:101263 ERROR (null)(0)(null):ARSLOG message msg num=23
78759:47665277347584 10/08/2021 08:59:31:101572 ERROR (null)(0)(null):CONSOLE message msg num=23
78759:47665497184000 10/08/2021 08:59:31:104615 ERROR arscssm.c(3298)ArcCSSMP_Retrieve:Sending message to library server msg timestamp=2021-10-08 08:59:31.066932 msg userid=M09530 msg severity=2 msg num=23 msg text=ARS0023E Object >195FAAA< in Application Group >AG< not found in cache, no other storage defined  Srvr->server<- msg appl=N msg logid=1540228370


96

MP Server / Issue implementing SSL

« on: October 06, 2021, 09:00:31 AM »

Hi All,

We are in the process of getting SSL Up and running, but it looks like we have an issue connecting from the library server to the object server when SSL is enabled.

When ODWEK Calls are made-

Code: [Select]

124002:47247019583232 10/06/2021 08:41:48:330624 ERROR (null)(0)(null):Server message msg timestamp=2021-10-06 08:41:48.325613 msg userid=ADMIN msg severity=2 msg num=142 msg text=ARS0142E Connection cannot be established for the >zlt12669< server  Srvr->LibServer XXX.XXX.XXX.XXX non-SSL<- msg appl=N msg logid=869449959
124002:47247019583232 10/06/2021 08:41:48:330629 ERROR (null)(0)(null):ARSLOG message msg num=142
124002:47247019583232 10/06/2021 08:41:48:330875 ERROR (null)(0)(null):CONSOLE message msg num=142
124002:47247030097664 10/06/2021 08:41:48:333627 ERROR arssrvr.c(4798)ArcSERVP_ProcessRequest:Sending message to library server msg timestamp=2021-10-06 08:41:48.303407 msg userid=M09530 msg severity=2 msg num=142 msg text=ARS0142E Connection cannot be established for the >LIBSERV< server  Srvr->LIBSERV<- msg appl=N msg logid=869449959
These are from the trace file, when I was trying to load/retrieve from the library server -> object server. The other way around (ARSDOC/ARSLOAD OBJ --FROM/INTO --> LIB) works fine.

Code: [Select]

79057:47879365379840 10/06/2021 10:43:56:122707 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED
79057:47879365379840 10/06/2021 10:43:56:203928 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED
79057:47879365379840 10/06/2021 10:43:56:290131 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED
79057:47879365379840 10/06/2021 10:43:56:406313 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED

Anyone have thoughts?

97

MP Server / Re: Error when trying to uninstall PDF Indexer on 9.5

« on: September 27, 2021, 11:38:10 AM »

We figured since the ownership/permissions were archive - that would be fine. but I guess not.

We also tried root, no bueno

98

MP Server / Re: migrating from DB2 to Oracle

« on: September 27, 2021, 08:05:13 AM »

I think that post was by Alessandro.  The only snag is, he was a prolific poster, so going through his history would take some time, and the instructions would be dated as he left the forums a few years ago. 

-JD.

yeah, i sent him an email earlier in the year when this was a hot topic.

It has since came up again with urgency..and the client thinks its 100% straightforward.

My opinion? extract/reload.

99

MP Server / migrating from DB2 to Oracle

« on: September 23, 2021, 08:23:46 AM »

We are looking at moving from our onprem RHEL boxes that run CMOD 10.5 in a db2 environment, to an Azure environment running oracle.

We don't want to do arsdoc/arsload, because we have so much data. So, I guess our approach will be-

Export the db2 tables to oracle.
Move the mounts to oracle
Make sure all the mounts are "lined up"

I once saw a post on here, that had some high level steps, but I cant seem to find it now. does anyone have advice on doing this? or any tips.

Thanks!

100

MP Server / Error when trying to uninstall PDF Indexer on 9.5

« on: September 20, 2021, 10:53:59 AM »

Was tasked last week with uninstalling PDF Indexer on CMOD 9.5..Redhat environment. Seeing this error when i execute the uninstaller. It worked fine on the corresponding library server that we have the PDF Indexer installed on..anyone seen this?

Code: [Select]

[archive@devobj _uninst950pdf]$ ./uninstallodpdf
Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)

Stack Trace:
java.lang.IllegalArgumentException: No product for ID=deb1d888-1f12-11b2-a877-ad2b62eef1e6
        at Flexeraanw.<init>(Unknown Source)
        at Flexeraan4.ai(Unknown Source)
        at Flexeraan4.ab(Unknown Source)
        at Flexeraan5.<init>(Unknown Source)
        at com.zerog.ia.installer.Installer.bp(Unknown Source)
        at com.zerog.ia.installer.Installer.setMetadata(Unknown Source)
        at com.zerog.ia.installer.InstallerMetaData.setInstaller(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.a3(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.ar(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.consoleInstallMain(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.executeApplication(Unknown Source)
        at com.zerog.ia.installer.Main.main(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:600)
        at com.zerog.lax.LAX.launch(Unknown Source)
        at com.zerog.lax.LAX.main(Unknown Source)
This Application has Unexpectedly Quit: Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)

101

MP Server / Re: IBM Cloud object storage issue in loading

« on: September 14, 2021, 10:22:10 AM »

Hello,

I am configuring IBM cloud object storage with AIX CMOD server, i configured the ars.cfg by adding the paramters
ARS_ICOS_CONFIG_FILE=/opt/IBM/ondemand/V10.5/config/ars.icos
ARS_ICOS_CONFIG_DIR=/opt/IBM/ondemand/V10.5/config

And i configured ars.icos.smp in config directory on AIX and specify Server IP and Enable SSL , SSL is enabled due to the customer said that SSL is enabled

And configured the Storage Set in CMOD by creating storage node and specify the vault name and user name and password ,

after i trying to load documents to a specific application the error LOG say :

ARS115E : Unable to store the object

I cannot trace the error as it is general error , so anyone have an idea about this issue and can help me
or is there any missing configuration that I should add related to the configuration or is there any configuration that need to be done due to the enablement of SSL

What does your ars.icos look like? Also, have you tried a system trace? I've seen with working with ICOS, that this can be helpful when using ALL=15 in trace.settings.

102

Report Indexing / Indexing PPD's - ARSPDOCI 9.5.0.10 completed code 4

« on: September 10, 2021, 10:54:03 AM »

We are loading a bunch of PDF files, This particular application group comes from iText. Most files load, however- Seeing intermittent failures. Among these intermittent failures, there's tons of successful loads, into other application groups as well, originating from the same 2 load-balanced servers running arsload/pdf indexer.

Code: [Select]

ARS4302I Indexing started, --UNKNOWN-- bytes to process
ARS4901I INDEXSTARTBY=1
ARS4901I RESTYPE=ALL
ARS4901I INDEXMODE=INTERNAL
ARS4901I PARMDD=/opt/app/workload/arsload/pdfFile_arsload.parm
ARS4901I INPUTDD=/opt/app/workload/arsload/pdfFile_arsload
ARS4901I OUTPUTDD=/opt/app/workload/arsload/pdfFile_arsload.out
ARS4901I INDEXDD=/opt/app/workload/arsload/pdfFile_arsload.ind
ARS4901I RESOBJDD=/opt/app/workload/arsload/pdfFile_arsload.res
ARS4922I ARSPDOCI 9.5.0.10 completed code 4
Here's an example of the loads/failures

2021-09-10 13:25:03.781009
2021-09-10 13:21:36.915310
2021-09-10 13:21:07.477156
2021-09-10 13:15:26.686449
2021-09-10 13:07:38.617789
2021-09-10 13:07:09.022827
2021-09-10 13:03:01.136332
2021-09-10 13:00:57.789100
2021-09-10 12:56:07.609028 FAILURE
2021-09-10 12:52:41.922350
2021-09-10 12:52:20.029268
2021-09-10 12:51:08.090807
2021-09-10 12:42:54.922132
2021-09-10 12:41:08.919243
2021-09-10 12:38:55.197800
2021-09-10 12:33:52.316784
2021-09-10 12:28:55.989293
2021-09-10 12:22:21.608910
2021-09-10 12:21:07.770830
2021-09-10 12:19:20.388717
2021-09-10 12:15:11.860829
2021-09-10 12:13:57.437055
2021-09-10 12:07:53.629861
2021-09-10 12:06:16.805737
2021-09-10 12:04:30.884266
2021-09-10 11:57:10.667217 FAILURE

I can't find anything on this error, but my sneaking suspicion is that this script that runs arsload, is running- before the file is finished generating, or something.

Either that, or something is up with these boxes- http://www.odusergroup.org/forums/index.php/topic,3186.0.html

103

MP Server / Re: db2 cleanup against system log

« on: September 09, 2021, 01:40:24 PM »

I actually think I got this working.

From my dev env:

Code: [Select]

SA2                             ARCHIVE         T     2016-10-27-13.24.20.294169
SL2                             ARCHIVE         T     2016-10-11-16.02.31.098125
SL3                             ARCHIVE         T     2020-02-22-08.40.37.363118
SL4                             ARCHIVE         T     2020-05-22-08.19.27.671807
SL5                             ARCHIVE         T     2021-06-28-10.33.44.943286

Last row From SL2: (Or I guess just use date created for SL3)

Code: [Select]

2020-02-22-14.40.33.802989
2020-02-22-14.40.33.965463
10000000 record(s) selected.

2020-02-22 was 565 days ago. Set app group retention to, 564 days, or 2020-02-22 + 1 day.

Issue arsmaint command:

Code: [Select]

arsmaint -cd -g 'System Log' -h archive -u jeff -p jeff.stash

Code: [Select]

Application Group Segment Expire: Name(System Log) Agid(5006) Seg(SL2)
Check the SL* Tables

Code: [Select]

SA2                             ARCHIVE         T     2016-10-27-13.24.20.294169
SL3                             ARCHIVE         T     2020-02-22-08.40.37.363118
SL4                             ARCHIVE         T     2020-05-22-08.19.27.671807
SL5                             ARCHIVE         T     2021-06-28-10.33.44.943286

Check the system log. Verified that the first entry was from, you guessed it. 02-22 (After the hh:mm:ss of the date in SL2..)

Code: [Select]

[b]2020-02-22 09:40:34.128011[/b]


104

MP Server / Re: db2 cleanup against system log

« on: September 08, 2021, 10:37:40 AM »

Bumping this back to life.

After trying some variations of the arsmaint command in my test environment,  nothing seems to give the expected results.

1) I set the retention of the system log to 365 days.
2) Ran this command:

Code: [Select]

arsmaint -cd -g 'System Log' -h archive -u user -p pw -t 2021-09-08
3) system log looks untouched

Anyone have any suggestions?

105

MP Server / ARSLOAD Issue - PDF Indexer with PPD loading very slow

« on: September 08, 2021, 08:07:13 AM »

Hi All,

We are loading from a remote CMOD Server (RHEL/DB2 V11/Cache):

Code: [Select]

Load Version <9.5.0.10>  Operating System <Linux> <#1 SMP Mon Feb 22 18:03:13 EST 2021.3.10.0-1160.21.1.el7.x86_64>  OS Userid <CMODADM>  Install Location </opt/workload/ibm/ondemand/V9.5/> Data(unlimited KB) Stack(8192 KB) Core(0 512-blocks) Cpu(unlimited seconds) File(unlimited 512-blocks) Nofiles(16384) Threads(0) Processes(4096)
..Into a fresh CMOD V10.5 install. We are noticing slow load times with a series of application groups. These reports are indexed using PPD's. Client is complaining that these batches of files are taking too long to index and load. Our problem is we have like 50 million docs to load and are only doing 100k/day or something like that. Unfortunately, I can't test this in my dev box..Because there's some kind of issue getting me a sample file. Something with the way they are generated.. Whatever that means.

Some numbers from the system load. from the system load (From the existing production system giving me the "issue") I noticed that index time is taking a lot longer than expected, and also than what i've tested with a sample PPD file from IBM. A much larger file, with much more documents, took less time.

Code: [Select]

       

Elapsed             IDX Time                   Docs                 Pages  Comp                    In                             Out
149.6544         122.1313                 1000                 6945 OD77             39392339             18658650
84.2975          60.9946                 1000                 5975 OD77             38167441             18119565
77.5499          50.2290                  463                 2348 OD77             15994287              4661084
106.8029          85.7542                 1000                 6281 OD77             38336344             18174458

Sample PPD Lab from IBM:

Code: [Select]

Application Group Load: Name(PPD) LoadId(5284-3-0-12FAA-20120123000000-20120123000000-5285) File(file_with_ppd.pdf) InputSize(111236081) OutputSize(50530342) Rows(6444) Time(56.6456) Appl(PPD) InputFileSize(163878229)
What are some considerations that I should look at on this remote server, where the PDF indexer is being used? One of the first things I noticed was the ulimits. I compared them to another CMOD system that uses ACIF, and noticed a difference. Unfortunately we are in the middle of a freeze right now, and I cant touch this..I also can't lay down the 10.5 binaries either.

System presenting the slow loading-

Code: [Select]

Data(unlimited KB) Stack(8192 KB) Core(0 512-blocks) Cpu(unlimited seconds) File(unlimited 512-blocks) Nofiles(16384) Threads(0) Processes(4096)
Existing CMOD System using ACIF, zero issues ever loading.

Code: [Select]

Data(unlimited KB) Stack(unlimited KB) Core(0 512-blocks) Cpu(unlimited seconds) File(unlimited 512-blocks) Nofiles(1000000) Threads(0) Processes(unlimited)
What kind of resources should this remote server be spec'ed out to. Wondering if throwing more memory/CPU at it will speed things up. I also noticed that the load time itself seems long for being such a small file, despite the CMOD boxes/PDF Indexer boxes being in  different data centers. Mentioned "possible" network latency.. But The client didnt like that answer.