Ondemand Instance not starting

[Gobi21]

Hi All,

I am not able to start my Ondemand instance. Release 8.4.0.2 .

No error message shown once the instance was triggered manually. It is getting stopped immediately after it started.

here are the trace settings: I tried to get some clue from trace file. Trace file is not written.


[TRACE]
COMPONENT_LEVEL=FFFFFFFFFFFFFFFFFFFFFF
#TRACE_FILE=ARCHIVE.trace.log
TRACE_FILE=ARCHIVE.trace.log
TRACE_FORMAT=TEXT
APPEND=0
MAX_LOG_SIZE=0

ARS_TMP=/sbcimp/dyn/data/RAR/OD_App1/temp
ARS_PRINT_PATH=/tmp
ARS_TRACE_SETTINGS=/opt/ondemand/config ------ I have added this and tried if i can get the trace.


How i can enable the trace or find the real cause for this issue. Please share your suggestion.

Thanks
Gobi

[Alessandro Perucchi]

Hello Gobi,

wow, it's back to the futur!! well back to the past... 

Ok... then first thing first.

What happened? Did it work before not starting anymore? What are the changes that happened that makes CMOD not working anymore? Something to do with an upgrade of your unix server?

Now to get more trace, you must modify your ars.cfg to be more like that:

ARS_TMP=/sbcimp/dyn/data/RAR/OD_App1/temp
ARS_PRINT_PATH=/tmp
ARS_TRACE_SETTINGS=/opt/ondemand/config/FILENAME

And the file called /opt/ondemand/config/FILENAME can be like that (exactly what you did):

Code: [Select]

[TRACE]
COMPONENT_LEVEL=FFFFFFFFFFFFFFFFFFFFFF
#TRACE_FILE=ARCHIVE.trace.log
TRACE_FILE=ARCHIVE.trace.log
TRACE_FORMAT=TEXT
APPEND=0
MAX_LOG_SIZE=0

One of the main error I get when CMOD cannot start are the following:

- Problem of permissions somewhere -> Check all the path that CMOD needs to access
- Problem with TSM, check that you don't have any error (again permissions) with TSM
- Problem with the DB, you need to ensure that your DB is not corrupted and working, and that everything is setup correctly, that the CMOD instance owner has dba rights on the database

What can help me from time to time, is the following, I change the file /opt/ondemand/bin/arslog to have something like that:

Code: [Select]

#!/bin/ksh

echo "$@" >> /tmp/myOD.log

exit 0

That way I can see if CMOD writes something in the System Log, which could help me to solve the problem.

Another thing that I do, even if it doesn't work always, is to use (Solaris/AIX) the "truss" command when running arssockd. In Linux you must use the command "strace". Then look for any problem with permissions, etc...

Hope that helps a little bit.

[Gobi21]

Hi Alessandro,

Thanks for your assistance. And Yes.. I am still in the past. Will be moved to CMOD 9.5 hopefully by year end.

I have updated the ars.cfg file.

ARS_TMP=/sbcimp/dyn/data/RAR/OD_App1/temp
ARS_PRINT_PATH=/tmp
ARS_TRACE_SETTINGS=/opt/ondemand/config/checkod.txt


I didn't make any changes to Trace.setttings file.
[TRACE]
COMPONENT_LEVEL=FFFFFFFFFFFFFFFFFFFFFF
#TRACE_FILE=ARCHIVE.trace.log
TRACE_FILE=ARCHIVE.trace.log
TRACE_FORMAT=TEXT
APPEND=0
MAX_LOG_SIZE=0


Also updated the arslog file with the script given by you.

The file permissions look OK to me. However, I could see these errors in TSM.
tsm: ARCHIVE>Q ACT

Date/Time                Message
--------------------     ----------------------------------------------------------
07/05/16   04:08:56      ANR2121W ATTENTION: More than 4092 MB of the database has
                          changed and the last database backup was more than 24
                          hours ago. Use the BACKUP DB command to provide for
                          database recovery.
07/05/16   04:08:56      ANR2102I Activity log pruning started: removing entries
                          prior to 07/06/15 00:00:00.
07/05/16   04:08:56      ANR0104E admactlg.c(3408): Error 2 deleting row from table
                          "Activity.Log".
07/05/16   04:08:56      ANR2102I Activity log pruning started: removing entries
                          prior to 07/06/15 00:00:00.
07/05/16   04:08:56      ANR0104E admactlg.c(3408): Error 2 deleting row from table
                          "Activity.Log".
07/05/16   04:08:56      ANR2102I Activity log pruning started: removing entries
                          prior to 07/06/15 00:00:00.
07/05/16   04:08:56      ANR0104E admactlg.c(3408): Error 2 deleting row from table
                          "Activity.Log".


Is this the cause for this issue??.

I have triggered the Ondemand and I could see a zero byte file created under /opt/ondemand/config/ as per the change in ars.cfg.

-rwxrwxrwx   1 odappmgr a19133         0 Jul  5 04:34 checkod.txt*


I didn't see any file at /tmp/

I have logged in to DB and verified already. I am able to login without any issues and fetch the data using normal queries in TOAD.

Ondemand was working fine a month back and I am not sure what change happened to system.

uname -a
SunOS sstmxxxxx.stm.example.com 5.10 Generic_Virtual sun4v sparc sun4v

Thanks again,
Gobi

[Justin Derrick]

Hi.

I've tweaked the previous post to remove identifying information.  I you have questions, please send me a PM.

-JD.

[Alessandro Perucchi]

I don't see anything strange, except that:

I didn't make any changes to Trace.setttings file.
[TRACE]
COMPONENT_LEVEL=FFFFFFFFFFFFFFFFFFFFFF
#TRACE_FILE=ARCHIVE.trace.log
TRACE_FILE=ARCHIVE.trace.log
TRACE_FORMAT=TEXT
APPEND=0
MAX_LOG_SIZE=0

and

ARS_TMP=/sbcimp/dyn/data/RAR/OD_App1/temp
ARS_PRINT_PATH=/tmp
ARS_TRACE_SETTINGS=/opt/ondemand/config/checkod.txt
...
-rwxrwxrwx   1 odappmgr a19133         0 Jul  5 04:34 checkod.txt*

You said that you change the trace.settings, but the configuration (ars.cfg) shows that the trace file is called checkod.txt... that's not consistent.
either you have everything in checkod.txt, and that's ok...
or you change everything in trace.settings, and then you should change ars.cfg to be something like that:

ARS_TRACE_SETTINGS=/opt/ondemand/config/trace.settings

Otherwise that will never work!