arslog vs system trace for troubleshooting.

[jsquizz]

We encountered a production issue on Friday where ARSSOCKD would randomly crash, on two occasions.

We turned on trace but i have concerns with using that in a production system with heavy loads/retrievals/queries, and the issue possibly not happening again. I noticed in our QA system that we are using the arslog user exit. I have never personally used

I tested bringing down DB2, and i saw all those errors in the syslog.log.

ARS0013E DB Error: [IBM][CLI Driver] CLI0106E  Connection is closed. SQLSTATE=08003 -- SQLSTATE=08003, SQLCODE=-99999, File=arsseg.c, Line=6269 does n
ot exist

ARS0013E DB Error: [IBM][CLI Driver] SQL1224N  The database manager is not able to accept new requests, has terminated all requests in progress, or ha
s terminated the specified request because of an error or a forced interrupt.  SQLSTATE=55032  -- SQLSTATE=40003, SQLCODE=-1224, File=arsid.c, Line=30
78 does not exist

Are there any performance implications as to using the arslog user exit in a production system, primarily on the performance side of things? How would this be compared to using TRACE with ALL=3?

[Justin Derrick]

So arslog is usually a shell script that contains some really basic things like detecting failed loads or other errors and either writing them to a file, or sending notifications.  There are only performance implications in the largest systems, when you start getting into things like deadlocks in the filesystem because there are too many writes to the same log file.

Your database errors look like you yanked DB2 out from under CMOD.  Make sure your shutdown scripts are doing things in the correct order, and are double-checking their results.  (Don't just ask CMOD to stop -- wait a few seconds, then see if it's actually down.)

-JD.

[jsquizz]

So arslog is usually a shell script that contains some really basic things like detecting failed loads or other errors and either writing them to a file, or sending notifications.  There are only performance implications in the largest systems, when you start getting into things like deadlocks in the filesystem because there are too many writes to the same log file.

Your database errors look like you yanked DB2 out from under CMOD.  Make sure your shutdown scripts are doing things in the correct order, and are double-checking their results.  (Don't just ask CMOD to stop -- wait a few seconds, then see if it's actually down.)

-JD.

Whoops, my bad again.

The errors i am showing, were just me testing something on our QA box.

I purposely did that. I turned tracing on, and I "yanked" db2. I didn't know we were using arslog exit, i just noticed syslog.log file being generated, and sure enough it had some info in it.

We just are trying to "catch" errors with unexpected CMOD as they happen. And trying to pinpoint as to what causes these failures. We think we have the culprit, but I'm just testing a few scenarios. I don't want to do tracing obviously and pray the thing crashes. Management is wondering why CMOD isn't logging why CMOD isnt crashing, and I tried to explain to them why from a technical standpoint.

This post is also extremely helpful, I think arslog with some tweaking might work.

http://www.odusergroup.org/forums/index.php?PHPSESSID=qr6bcru5vrdasncbi7icjqpkl1&topic=2965.msg11897#msg11897

[Justin Derrick]

Yup, the only thing to remember is that when there's a huge flood of traffic (users) then you're going to hit a server-side I/O limit if you're trying to write everything into the same file.

-JD.