Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - jsquizz

Pages: [1] 2 3 4 5 6 7
1
OD/WEK & JAVA API / ODWEK - Flow of returning document
« on: October 13, 2021, 07:48:06 AM »
Have a question about how ODWEK works as far as returning documents from our system.

We are using a distributed system with a library server and object server.

My understanding of the flow, is that the ODWEK client sends the Login Requests/Search request to the library server. The Library server then sends the hit list back to the client.

After that, the client sends the retrieve request to the Library server.

Our question is about the request. The data that comes back - Two scenarios- Which one is true - or is there a third.

1) The object server sends the data to the library server, and then the library server returns it to the ODWEK client.

2) The object server sends the data directly to the ODWEK client, bypassing the library server.

2
MP Server / Issue implementing SSL
« on: October 06, 2021, 09:00:31 AM »
Hi All,

We are in the process of getting SSL Up and running, but it looks like we have an issue connecting from the library server to the object server when SSL is enabled.

When ODWEK Calls are made-
Code: [Select]
124002:47247019583232 10/06/2021 08:41:48:330624 ERROR (null)(0)(null):Server message msg timestamp=2021-10-06 08:41:48.325613 msg userid=ADMIN msg severity=2 msg num=142 msg text=ARS0142E Connection cannot be established for the >zlt12669< server  Srvr->LibServer XXX.XXX.XXX.XXX non-SSL<- msg appl=N msg logid=869449959
124002:47247019583232 10/06/2021 08:41:48:330629 ERROR (null)(0)(null):ARSLOG message msg num=142
124002:47247019583232 10/06/2021 08:41:48:330875 ERROR (null)(0)(null):CONSOLE message msg num=142
124002:47247030097664 10/06/2021 08:41:48:333627 ERROR arssrvr.c(4798)ArcSERVP_ProcessRequest:Sending message to library server msg timestamp=2021-10-06 08:41:48.303407 msg userid=M09530 msg severity=2 msg num=142 msg text=ARS0142E Connection cannot be established for the >LIBSERV< server  Srvr->LIBSERV<- msg appl=N msg logid=869449959

These are from the trace file, when I was trying to load/retrieve from the library server -> object server. The other way around (ARSDOC/ARSLOAD OBJ --FROM/INTO --> LIB) works fine.
Code: [Select]
79057:47879365379840 10/06/2021 10:43:56:122707 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED
79057:47879365379840 10/06/2021 10:43:56:203928 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED
79057:47879365379840 10/06/2021 10:43:56:290131 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED
79057:47879365379840 10/06/2021 10:43:56:406313 ERROR arsgskod.c(2828)ArcGSKOD_Connect:socket_init ssl_rc=420 ssl_str=GSK_ERROR_SOCKET_CLOSED

Anyone have thoughts?

3
MP Server / migrating from DB2 to Oracle
« on: September 23, 2021, 08:23:46 AM »
We are looking at moving from our onprem RHEL boxes that run CMOD 10.5 in a db2 environment, to an Azure environment running oracle.

We don't want to do arsdoc/arsload, because we have so much data. So, I guess our approach will be-

Export the db2 tables to oracle.
Move the mounts to oracle
Make sure all the mounts are "lined up"

I once saw a post on here, that had some high level steps, but I cant seem to find it now. does anyone have advice on doing this? or any tips.

Thanks!

4
MP Server / Error when trying to uninstall PDF Indexer on 9.5
« on: September 20, 2021, 10:53:59 AM »
Was tasked last week with uninstalling PDF Indexer on CMOD 9.5..Redhat environment. Seeing this error when i execute the uninstaller. It worked fine on the corresponding library server that we have the PDF Indexer installed on..anyone seen this?

Code: [Select]
[archive@devobj _uninst950pdf]$ ./uninstallodpdf
Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)

Stack Trace:
java.lang.IllegalArgumentException: No product for ID=deb1d888-1f12-11b2-a877-ad2b62eef1e6
        at Flexeraanw.<init>(Unknown Source)
        at Flexeraan4.ai(Unknown Source)
        at Flexeraan4.ab(Unknown Source)
        at Flexeraan5.<init>(Unknown Source)
        at com.zerog.ia.installer.Installer.bp(Unknown Source)
        at com.zerog.ia.installer.Installer.setMetadata(Unknown Source)
        at com.zerog.ia.installer.InstallerMetaData.setInstaller(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.a3(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.ar(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.consoleInstallMain(Unknown Source)
        at com.zerog.ia.installer.LifeCycleManager.executeApplication(Unknown Source)
        at com.zerog.ia.installer.Main.main(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:600)
        at com.zerog.lax.LAX.launch(Unknown Source)
        at com.zerog.lax.LAX.main(Unknown Source)
This Application has Unexpectedly Quit: Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)

5
Report Indexing / Indexing PPD's - ARSPDOCI 9.5.0.10 completed code 4
« on: September 10, 2021, 10:54:03 AM »
We are loading a bunch of PDF files, This particular application group comes from iText. Most files load, however- Seeing intermittent failures. Among these intermittent failures, there's tons of successful loads, into other application groups as well, originating from the same 2 load-balanced servers running arsload/pdf indexer.

Code: [Select]
ARS4302I Indexing started, --UNKNOWN-- bytes to process
ARS4901I INDEXSTARTBY=1
ARS4901I RESTYPE=ALL
ARS4901I INDEXMODE=INTERNAL
ARS4901I PARMDD=/opt/app/workload/arsload/pdfFile_arsload.parm
ARS4901I INPUTDD=/opt/app/workload/arsload/pdfFile_arsload
ARS4901I OUTPUTDD=/opt/app/workload/arsload/pdfFile_arsload.out
ARS4901I INDEXDD=/opt/app/workload/arsload/pdfFile_arsload.ind
ARS4901I RESOBJDD=/opt/app/workload/arsload/pdfFile_arsload.res
ARS4922I ARSPDOCI 9.5.0.10 completed code 4

Here's an example of the loads/failures

2021-09-10 13:25:03.781009
2021-09-10 13:21:36.915310
2021-09-10 13:21:07.477156
2021-09-10 13:15:26.686449
2021-09-10 13:07:38.617789
2021-09-10 13:07:09.022827
2021-09-10 13:03:01.136332
2021-09-10 13:00:57.789100
2021-09-10 12:56:07.609028 FAILURE
2021-09-10 12:52:41.922350
2021-09-10 12:52:20.029268
2021-09-10 12:51:08.090807
2021-09-10 12:42:54.922132
2021-09-10 12:41:08.919243
2021-09-10 12:38:55.197800
2021-09-10 12:33:52.316784
2021-09-10 12:28:55.989293
2021-09-10 12:22:21.608910
2021-09-10 12:21:07.770830
2021-09-10 12:19:20.388717
2021-09-10 12:15:11.860829
2021-09-10 12:13:57.437055
2021-09-10 12:07:53.629861
2021-09-10 12:06:16.805737
2021-09-10 12:04:30.884266
2021-09-10 11:57:10.667217 FAILURE

I can't find anything on this error, but my sneaking suspicion is that this script that runs arsload, is running- before the file is finished generating, or something.

Either that, or something is up with these boxes- http://www.odusergroup.org/forums/index.php/topic,3186.0.html


6
MP Server / ARSLOAD Issue - PDF Indexer with PPD loading very slow
« on: September 08, 2021, 08:07:13 AM »
Hi All,

We are loading from a remote CMOD Server (RHEL/DB2 V11/Cache):

Code: [Select]
Load Version <9.5.0.10>  Operating System <Linux> <#1 SMP Mon Feb 22 18:03:13 EST 2021.3.10.0-1160.21.1.el7.x86_64>  OS Userid <CMODADM>  Install Location </opt/workload/ibm/ondemand/V9.5/> Data(unlimited KB) Stack(8192 KB) Core(0 512-blocks) Cpu(unlimited seconds) File(unlimited 512-blocks) Nofiles(16384) Threads(0) Processes(4096)
..Into a fresh CMOD V10.5 install. We are noticing slow load times with a series of application groups. These reports are indexed using PPD's. Client is complaining that these batches of files are taking too long to index and load. Our problem is we have like 50 million docs to load and are only doing 100k/day or something like that. Unfortunately, I can't test this in my dev box..Because there's some kind of issue getting me a sample file. Something with the way they are generated.. Whatever that means.

Some numbers from the system load. from the system load (From the existing production system giving me the "issue") I noticed that index time is taking a lot longer than expected, and also than what i've tested with a sample PPD file from IBM. A much larger file, with much more documents, took less time.

Code: [Select]
       

Elapsed             IDX Time                   Docs                 Pages  Comp                    In                             Out
149.6544         122.1313                 1000                 6945 OD77             39392339             18658650
84.2975          60.9946                 1000                 5975 OD77             38167441             18119565
77.5499          50.2290                  463                 2348 OD77             15994287              4661084
106.8029          85.7542                 1000                 6281 OD77             38336344             18174458

Sample PPD Lab from IBM:
Code: [Select]
Application Group Load: Name(PPD) LoadId(5284-3-0-12FAA-20120123000000-20120123000000-5285) File(file_with_ppd.pdf) InputSize(111236081) OutputSize(50530342) Rows(6444) Time(56.6456) Appl(PPD) InputFileSize(163878229)
What are some considerations that I should look at on this remote server, where the PDF indexer is being used? One of the first things I noticed was the ulimits. I compared them to another CMOD system that uses ACIF, and noticed a difference. Unfortunately we are in the middle of a freeze right now, and I cant touch this..I also can't lay down the 10.5 binaries either.

System presenting the slow loading-
Code: [Select]
Data(unlimited KB) Stack(8192 KB) Core(0 512-blocks) Cpu(unlimited seconds) File(unlimited 512-blocks) Nofiles(16384) Threads(0) Processes(4096)
Existing CMOD System using ACIF, zero issues ever loading.
Code: [Select]
Data(unlimited KB) Stack(unlimited KB) Core(0 512-blocks) Cpu(unlimited seconds) File(unlimited 512-blocks) Nofiles(1000000) Threads(0) Processes(unlimited)
What kind of resources should this remote server be spec'ed out to. Wondering if throwing more memory/CPU at it will speed things up. I also noticed that the load time itself seems long for being such a small file, despite the CMOD boxes/PDF Indexer boxes being in  different data centers. Mentioned "possible" network latency.. But The client didnt like that answer.

7
OD/WEK & JAVA API / Connection between ODWEK and Library Server.
« on: September 01, 2021, 08:36:32 AM »
Apologize for the ignorance with the topic, I am not an ODWEK person, at all.

Our ODWEK folks came to us with a question- they want to know what protocol is used by ODWek to communicate with the CMOD servers, and most importantly, whether that protocol is secure.

It is my understanding that TCP/IP is used, and that the connection is secure because ODWEK and the library server, will use the IBM Global Security kit to secure that connection.

Is that accurate? Or am I in left field.

Thanks!

8
MP Server / quick daemon question
« on: July 19, 2021, 08:04:05 AM »
I have a requirement where I "Think" a daemon might be our best bet to load a bunch of files.

I am trying to fire it up, not so much working. It has actually been about 5-6 years since ive had to do this. Here's what I have.

Code: [Select]
arsload -nvf -u jeff -p jeff.stash -B IGN.APP.AG.IGN.IGN.IGN.EXT -t 60 -h archive &
The file name is-

Code: [Select]
9898541175.TESTAPP.TESTAG.IGNOREHTHIS.20210712.010447405.ARD
I go to fire it up-

Code: [Select]
[1] +  Done(6)                 arsload -nvf -u jeff -p jeff.stash -B IGN.APP.AG.IGN.IGN.IGN.EXT -t 60 -h archive &
I have a feeling this is a very simple issue. OF course, I could easily script this as well, but I want to test something for a daemon.


9
MP Server / recreated extracted PDF File?
« on: July 16, 2021, 12:43:51 PM »
I am working with a file that was loaded using PPD's, trying to figure out some slowness.

Once the file is loaded, is there any way to extract the file as a whole, as it was loaded originally into CMOD? i am having issues getting it from the folks that send the files.

Basically, is it possible to some how recreate the file using the .out and the .res file?

Thanks in advance

10
Report Indexing / Generic Indexing : Batch Vs Single
« on: June 09, 2021, 11:44:21 AM »
I am noticing that our one Line of Business will send tens of thousands of files to CMOD. All with one-two documents, that uses the generic indexer (User Defined PDF / HTML)

Obviously I've followed best practices with AFP files and the like, send down a large batch, let cmod compress'em.

I am actually trying to convince the business to send down PDF's with PPD's, not sure if they can change the HTML files we receive though.

However, until then..Trying to build a case.

Is there a very large benefit in load times when doing a generic indexed file- To do lets say, 1000 docs in a file, compared to 1-5? I am going to assume so, because arsload is only called once, one set of inserts, one logon/logoff, etc.

thanks in advance

11
MP Server / Upgrade from Redhat 6-7, CMOD 9.5-10.5, Latency Issues
« on: June 08, 2021, 05:00:42 AM »
Little background of our environment, 1Lib Server, 1 Obj Serv, RHEL + DB2, DB2 storage is on SAN Veritas Cluster, CMOD Cache is also on Veritas Cluster

We just upgraded from Redhat 6.8 to Redhat 7.9 Maipo. After a few days we noticed slow response times from our API's calling via ODWEK (Weblogic). These requests would timeout after around 20 seconds, causing impact to our customers. We also see instances where loading is slow, example- 12.8 Seconds to load a file with 11 Docs..500kb size.. using generic indexer (user defined html file).

We opened up a case with IBM, and the first suggestion was to upgrade CMOD to 10.5, so we did that, the CMOD upgrade went perfectly fine - and the issue persisted. We checked all of our DB2 (V11.1fp4) logs, and we don't see anything. CMOD says they cant see anything wrong, and I agree with them. the only thing they are seeing is a pause/delay in the system log..There was a 40-50 second delay where there was absolutely no activity, whereas for the whole day there is usually stuff going on. After that pause, there's some 6second long queries in the 226 records.

We also pulled in DB2 Support, and we are giving them updates/db2mon/db2logs - but i dont feel like we are getting anywhere.

Veritas is looking at our system, along with Redhat support.

We've engaged several of our DBA's, Storage Engineers, and System Admins to address the issues.

One of our SA's suggested rolling back the RHEL kernel to 7.8, due to some major features added in 7.9. We did that overnight, and sure enough when I logged on this morning, no bueno.

The biggest issue, is that the problem is intermittent. Our next step is to run on one library server instead of lib/obj to rule out any kind of network latency or anything like that. I also think it might be a shot moving the DB2 volumes off veritas to local disk, or something.

I am open to any possible feedback, if anyone has encountered issues going from RHEL6-7.


12
MP Server / db2 cleanup against system log
« on: May 27, 2021, 08:08:00 AM »
Just wondering if its a common practice for folks to run arsmaint against the system log.

Ours is fairly large I guess, 1B+ rows. I am not sure in the scheme of things if that is "large" but it's taking up alot of space.

It's difficult for our client to get space which is.. strange.

We're considering doing a cleanup against the system log, unused storing, 84/85/83/82/81..226/65/66 (Need to check with compliance), arsmaint, etc.

Wondering if anyone does this? Is it common? we'd probably make a backup of all the tables, and compress it, then store it in TSM or our sharepoint.

Thanks gang

13
MP Server / arslog vs system trace for troubleshooting.
« on: May 24, 2021, 11:27:05 AM »
We encountered a production issue on Friday where ARSSOCKD would randomly crash, on two occasions.

We turned on trace but i have concerns with using that in a production system with heavy loads/retrievals/queries, and the issue possibly not happening again. I noticed in our QA system that we are using the arslog user exit. I have never personally used

I tested bringing down DB2, and i saw all those errors in the syslog.log.

ARS0013E DB Error: [IBM][CLI Driver] CLI0106E  Connection is closed. SQLSTATE=08003 -- SQLSTATE=08003, SQLCODE=-99999, File=arsseg.c, Line=6269 does n
ot exist

ARS0013E DB Error: [IBM][CLI Driver] SQL1224N  The database manager is not able to accept new requests, has terminated all requests in progress, or ha
s terminated the specified request because of an error or a forced interrupt.  SQLSTATE=55032  -- SQLSTATE=40003, SQLCODE=-1224, File=arsid.c, Line=30
78 does not exist

Are there any performance implications as to using the arslog user exit in a production system, primarily on the performance side of things? How would this be compared to using TRACE with ALL=3?

14
MP Server / Unable to start CMOD V10.5 after upgrade from V9.5
« on: April 27, 2021, 01:45:37 PM »
We just upgraded from CMOD V9.5 to V10.5.0.2 Our environment is redhat, with db2 v11.1.4.5, redhat linux

I ran all the upgrade steps-

arsdb -I archive -evf, select yes
arsdb -I archive -uv
arsdb -I archive -rfv
arsdb -I archive -rfv

I updated all the configuration files. Verified everything looks good, and we are seeing this error. I attempt to start arssockd, and it gets to the "initializing phase" and then dies out.

From the trace file-

Code: [Select]
55663:47042139134656 04/27/2021 15:34:21:755929 ERROR arssys.c(3949)ArcDB_SystemGetConfig:Unable to execute sql db hstmt=65538 db sql str=SELECT keystore_type, keystore_location, keystore_mkl, keystore_mkl_dt, data_ekey FROM archive.ARSSYS FETCH FIRST 1 ROW ONLY OPTIMIZE FOR 1 ROW FOR READ ONLY db sql rc=-1,SQL_ERROR
55663:47042139134656 04/27/2021 15:34:21:861056 ERROR arssys.c(3950)ArcDB_SystemGetConfig:Server message msg timestamp=2021-04-27 15:34:21.757220 msg userid=ARSSOCKD msg severity=2 msg num=13 msg text=ARS0013E DB Error: [IBM][CLI Driver][DB2/LINUXX8664] SQL0206N  "KEYSTORE_TYPE" is not valid in the context where it is used.  SQLSTATE=42703  -- SQLSTATE=42S22, SQLCODE=-206, File=arssys.c, Line=3950 msg appl=N msg logid=0
55663:47042139134656 04/27/2021 15:34:21:861061 ERROR arssys.c(3950)ArcDB_SystemGetConfig:ARSLOG message msg num=13
55663:47042139134656 04/27/2021 15:34:21:861318 ERROR arssys.c(3950)ArcDB_SystemGetConfig:CONSOLE message msg num=13
55663:47042139134656 04/27/2021 15:34:21:861586 ERROR arssrvr.c(3168)ArcSERVP_GetSystemInfo:TTY message msg num=151 msg text=ARS0151E Unable to initialize server database system parameters.  Contact your system administrator.
55663:47042139134656 04/27/2021 15:34:21:861631 ERROR arssrvr.c(3168)ArcSERVP_GetSystemInfo:CONSOLE message msg num=151 msg timestamp=2021-04-27 15:34:21.861591 msg userid=ARSMSG msg severity=2 msg text=ARS0151E Unable to initialize server database system parameters.  Contact your system administrator. msg appl=N msg logid=0
55663:47042139134656 04/27/2021 15:34:21:861647 ERROR arssrvr.c(3168)ArcSERVP_GetSystemInfo:ARSLOG message msg num=151

Found an earlier post, mentions this-

Code: [Select]
EXPORT
DROP
CREATE
IMPORT
 
Run "arsdb <-I instance_name> -vx ARSSYS"
Run "arsdb <-I instance_name> -vd ARSSYS"
Run "arsdb <-I instance_name> -vc ARSSYS"
Run "arsdb <-I instance_name> -vi ARSSYS"

Before I do that, just wondering if anyone on here has seen this error message as well.

15
MP Server / Issue configuring new environment
« on: March 19, 2021, 07:12:18 AM »
I'm working on a POC and for some reason I cant get the install to work. I've done plenty of these so I am scratching my head. I have a feeling its simple.

1) Instance is created as "db2inst1"-

Code: [Select]
[db2inst1@ondemand ~]$ db2ilist
db2inst1
[db2inst1@ondemand ~]$ groups
db2iadm1
[db2inst1@ondemand ~]$

2) "archive" user is part of db2iadm1, attempting to run arsdb -gcv -I archive

Code: [Select]
[archive@ondemand config]$ groups
db2iadm1
[archive@ondemand config]$ arsdb -I archive -gcv
ARS4014E Unable to load >DB2<

3) ars.cfg / ars.ini are both basically untouched

Code: [Select]
[archive@ondemand config]$ more ars.ini
[@SRV@_ARCHIVE]
HOST=
PROTOCOL=2
PORT=0
SRVR_INSTANCE=ARCHIVE
SRVR_INSTANCE_OWNER=archive
SRVR_OD_CFG=/opt/ibm/ondemand/V10.1/config/ars.cfg
SRVR_DB_CFG=/opt/ibm/ondemand/V10.1/config/ars.dbfs
SRVR_SM_CFG=/opt/ibm/ondemand/V10.1/config/ars.cache

Code: [Select]
ARS_ORIGINAL_CODEPAGE=0
DB2INSTANCE=db2inst1
ARS_DB2_DATABASE_PATH=/arsdb
ARS_DB2_PRIMARY_LOGPATH=/arsdb_primarylog
ARS_DB2_ARCHIVE_LOGPATH=/arsdb_archivelog
ARS_DB2_LOGFILE_SIZE=1000
ARS_DB2_LOG_NUMBER=40

I verified that all the directories are created, owned by archive:db2iadm1, set to 775

I think it's something with the archive profile, permissions or something like that. Usually I use all the default settings - archive instance/archive db.. But I want the default db2instance name of db2inst1 because I will be installing other products on here and I would like to keep things organized :) anyone have suggestions? I unfortunately don't have a dba to help me.



Pages: [1] 2 3 4 5 6 7