Messages

136

MP Server / Arsdoc get — best optimized way to extract and reload

« on: March 03, 2021, 09:21:30 PM »

So I am working on a migration and I’m just looking at some possible ways to extract and reload my data.

My thought - arsdoc get with -L, read from parameter file with a listing of load ID’s. Do this on an app group by app group basis. We only have like 10, but a good amount of data.

We have a process, a weird one- that extracts from a “short term” application group and loads into a “long term” application group. This was handled by a third party years ago, and since I joined I redid it with a few basic shell scripts. My script generates a list of dates of the month prior, and loops through that + an app group name, and does all the magic. This process is very good. I can’t wrap my head around why they are going from one app group to another. I think it’s for performance at the db level or something

I figure there’s other ways, where I can extract a month of data per file, etc. so I guess my three options I’m trying to weigh

1) retrieve an entire load, reload with arsload
2) retrieve an entire day, reload with arsload
3) retrieve an entire month, reload with arsload.

I’m leaning towards one. I also think that would be easier to recover if there’s some kind of issue.

What’s everyone’s thoughts.

137

MP Server / determine oldest doc in an app group?

« on: February 24, 2021, 05:24:30 PM »

I have about 2k app groups, need to figure out the oldest record per application group.

I'm looking through the tables -> arsag caught my eye, more specifically 'last_doc_dt'

I ran a simple query, and it's coming back blank.

Code: [Select]

LAST_DOC_DT
--------------------------
-

  1 record(s) selected.
Is there anything on lets say, arsseg I can also try? I was also thinking I could use arsseg table.

Code: [Select]

START_DT TIMESTAMP The minimum (oldest) date of documents stored in this folder, in database-native timestamp format.

138

MP Server / Re: Any easy/fancy ways to gather all loads in one year time frame?

« on: February 23, 2021, 09:16:12 PM »

The CMOD System Load (SA*) tables should be your first stop.  There's no AppGroup field in the OnDemand System Log without doing a full-text search, which is terrible for performance.  A few tweaks of SQL should help you get what you need.

Also, loading 100k files per month is terrible.  Load larger files less frequently to reduce the overhead in CMOD. 

-JD.

Yep, i tinkered with the SA* tables tonight for a few minutes after I posted this and got my results.

I agree with you about the loads. There’s literally a month worth of files. Tens of thousand. With one single document per file.

139

MP Server / Any easy/fancy ways to gather all loads in one year time frame?

« on: February 23, 2021, 05:20:23 PM »

Business folks are asking for a count of all documents loaded in 2020 for a specific folder. Usually not a big deal, only 6 AG's in that folder..

How I've done it in the past is use the system load / system log folder, and put it into excel..and some simple parsing. But these are large application groups, some have 100k+ loads per month. I've learned obviously that the CMOD client doesn't like copying out more than 30-40k at a time.

My Idea? -- I'm thinking possibly also of just hitting up the SA* tables, of which we have two of. Anyone have any more ingenious ways of doing this?

140

MP Server / Re: Issue with arsdoc query

« on: February 22, 2021, 09:22:25 AM »

quick update on this.

Since I need only the past "66msg" from the last hour.. So at the top of every hour, I am going to run an ARSDOC QUERY command

"WHERE ODDAT_TIME_STAMP >= $VARIABLE.."

$VARIABLE, is derived using basic `date -u` command. This worked perfectly when I figured out the proper flags to use.

141

MP Server / Re: Issue with arsdoc query

« on: February 12, 2021, 04:05:17 PM »

Remember - you're searching the database field directly. with the OnDemand arsdoc -i parameter.  And the time in the CMOD database is stored as UTC, without timezones.  You need to adjust your query to match your local timezone.  There's probably a clever way to do this at the database level, but I don't know how, exactly.

-JD.

got it, thanks. Colleague is a db2 dba, will bug him

142

MP Server / Re: Issue with arsdoc query

« on: February 12, 2021, 01:02:15 PM »

Time zone settings. 

I knew it.

Seriously, I knew it.. lol<-- Or i thought!

Database timezone = - 50000 (EST)
Server timezone = - Fri Feb 12 16:22:27 EST 2021

When i search for : "WHERE msg_num = '87' and ODDAT_time_stamp > '2021-02-12 00:05:05'"

My first hit comes back at 07:05:05 the previous day.

Am I missing something? lol

143

MP Server / Issue with arsdoc query

« on: February 12, 2021, 11:34:14 AM »

CMOD 9.5, DB2 10.1 - Scratching my head on this one, I've done this before but its been a while. I want to just generate a list of 66 messages for a certain time frame. I think this is what i used last time.

arsdoc query -u admin -p stash.file -h archive -f 'System Log' -i "WHERE TIME_STAMP_DT > '2021-02-12-01.00.00.352804' and TIME_STAMP_DT < '2021-02-12-2.43.45.352804' and MSG_NUM='66'"

I want to get all 66 messages, between 2021-02-12-01.00.00.352804 and 2021-02-12-2.43.45.352804

When I run above arsdoc query, I get this.. Notice the dates look funky... Anyone have a thought?

02/11/21 21:30:34,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(38657GAAA) Off(0) Len(40646) CompOff(0) CompLen(34621) Size(40646) Seg(1) Total Segs(1) Flds(),A,1271258566,
02/11/21 21:30:31,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.009) ObjName(42537GAAA) Off(44384) Len(89182) CompOff(0) CompLen(57683) Size(89182) Seg(1) Total Segs(1) Flds(),A,1271258560,
02/11/21 21:30:30,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.002) ObjName(39760GAAA) Off(0) Len(43120) CompOff(0) CompLen(9725) Size(43120) Seg(1) Total Segs(1) Flds(),A,1271258562,
02/11/21 21:30:29,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(39813GAAA) Off(0) Len(49624) CompOff(0) CompLen(11097) Size(49624) Seg(1) Total Segs(1) Flds(),A,1271342777,
02/11/21 21:00:21,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(38657GAAA) Off(0) Len(40646) CompOff(0) CompLen(34621) Size(40646) Seg(1) Total Segs(1) Flds(),A,1271258566,
02/11/21 21:00:19,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.010) ObjName(42537GAAA) Off(44384) Len(89182) CompOff(0) CompLen(57683) Size(89182) Seg(1) Total Segs(1) Flds(),A,1271258560,
02/11/21 21:00:17,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(39760GAAA) Off(0) Len(43120) CompOff(0) CompLen(9725) Size(43120) Seg(1) Total Segs(1) Flds(),A,1271258562,
02/11/21 21:00:16,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.004) ObjName(39813GAAA) Off(0) Len(49624) CompOff(0) CompLen(11097) Size(49624) Seg(1) Total Segs(1) Flds(),A,1271342777,
02/11/21 20:30:19,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(38657GAAA) Off(0) Len(40646) CompOff(0) CompLen(34621) Size(40646) Seg(1) Total Segs(1) Flds(),A,1271258566,
02/11/21 20:30:17,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.008) ObjName(42537GAAA) Off(44384) Len(89182) CompOff(0) CompLen(57683) Size(89182) Seg(1) Total Segs(1) Flds(),A,1271258560,
02/11/21 20:30:15,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(39760GAAA) Off(0) Len(43120) CompOff(0) CompLen(9725) Size(43120) Seg(1) Total Segs(1) Flds(),A,1271258562,
02/11/21 20:30:14,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.004) ObjName(39813GAAA) Off(0) Len(49624) CompOff(0) CompLen(11097) Size(49624) Seg(1) Total Segs(1) Flds(),A,1271342777,
02/11/21 20:00:32,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(38657GAAA) Off(0) Len(40646) CompOff(0) CompLen(34621) Size(40646) Seg(1) Total Segs(1) Flds(),A,1271258566,
02/11/21 20:00:30,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.006) ObjName(42537GAAA) Off(44384) Len(89182) CompOff(0) CompLen(57683) Size(89182) Seg(1) Total Segs(1) Flds(),A,1271258560,
02/11/21 20:00:28,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.003) ObjName(39760GAAA) Off(0) Len(43120) CompOff(0) CompLen(9725) Size(43120) Seg(1) Total Segs(1) Flds(),A,1271258562,
02/11/21 20:00:26,ADMIN,4,66,Application Group Document Get: Name(APPLICATION_AG_S) Agid(5150) ApplName(APPLICATION_APP_S) Aid(5151) NodeName(-CACHE-) Nid(3) Server(server) Time(0.004) ObjName(39813GAAA) Off(0) Len(49624) CompOff(0) CompLen(11097) Size(49624) Seg(1) Total Segs(1) Flds(),A,1271342777,

144

MP Server / Multiple oracle instances configured on the same box

« on: February 10, 2021, 10:45:49 AM »

We'd like to create a POC where we load to an oracle remote server to measure performance.

Wondering if it's possible to point an existing CMOD/oracle system that is working PERFECTLY to this remote instance. Basically, two CMOD instances, one box.

Assuming we'd just have to modify the ars.ini file to point to a seperate set of ars.** files, create the database with arsdb commands, and fire it up?

145

MP Server / Re: Unable to load to oracle tablespaces

« on: February 05, 2021, 07:32:39 AM »

yeah, Can do all that. CMOD is running as "archive" user -> can create table spaces, touch a file in the directory, etc. I will look into group membership though

146

MP Server / Unable to load to oracle tablespaces

« on: February 04, 2021, 06:33:01 PM »

Hi all,

Fresh oracle DB running 19c (i think..) and CMOD 10.5.

Trying to load using tablespaces,

2021-01-29 02:11:00.050814: ARS1107E An error occurred.  Contact your system administrator and/or consult the System Log.  File=arsload.c, Line=6736

2021-01-29 02:11:00.050851: ARS1158E Failed while attempting to load the database

2021-01-29 02:11:00.311342: ARS1146I Loaded 0 rows into the database

2021-01-29 02:11:00.311378: ARS4311E Loading failed

2021-01-29 02:11:00.311398: ARS4320I Unloading started

Within the system log -

Unable to find SMS tablespace filesystems.  Either the defined SMS filesystems are full or there are no SMS filesystems defined.


these are defined within ars.dbfs-


#‌ Filesystem                        Tablespace Type (SMS)

#‌ --------------------------------------------------------------

‌/u02/oracle/oradata/fs02/archive     SMS

that specified directory exists

/u02/oracle/oradata/fs02/archive: directory

We can manually create a tablespace within above filesystem outside of CMOD, but it wont work for us when we load.

Anyone have suggestions?

147

MP Server / Re: Killing a hung arsload process on MP 9.5 on AIX 7.2

« on: January 19, 2021, 07:17:40 AM »

in 10+ years I've never had any issues just doing a kill -9 on a load process.. then i'll do my troubleshooting, and fire it up again

148

MP Server / Tuning - Migration to Oracle

« on: January 08, 2021, 10:32:01 AM »

We are currently using CMOD 9.5/DB2 11. Our new system we just stood up is CMOD 10.5/Oracle 19C.


1) We are curious about tuning the database, one of the key things we are looking for is SGA size for the database. Are there any other tuning considerations we need to take into account?

2) We are also looking for information regarding - db reorg/runstats. I believe currently we do all of this through arsdb/arsmaint commands. Is that supported with oracle? Is there a specific way to do it.

149

Other / Re: Configure google cloud storage bucket similar to S3

« on: January 06, 2021, 08:18:29 PM »

Hi Deep,

past contract wanted to take this route with IBM Cloud Storage, prior to 10.1

CMOD-->Spectrum --> Google

Looks like google storage is supported that way

150

MP Server / Re: arsdb -rtv fails when creating db (10.5 + oracle)

« on: December 30, 2020, 10:55:01 AM »

Fix-

[@SRV@_ARCHIVE] was not set to the right database name

After this-

arsstash -c -s ars.stash -a 9 -u archive

arsdb ran like a champ, arssockd started up!