Messages

31

Documentation / Re: How to delete both index and object

« on: January 15, 2019, 01:33:09 PM »

Not at my knowledge.

If you del the indexes (arsdoc delete) then the documents is still there in the storage node.

Is it good enough?

Does your load contains a mix of duplicate and none duplicate content? If yes, then a arsdoc delete is the only way.
If not, and each load is either all unique or all duplicates, then the unload is the solution.

If you want to reduce the size of disk space in the storage node, then the way to do would be:

- Take a load that contains duplicates
- do "arsdoc delete"
- then "arsdoc get" to get all the documents from the load
- reload the exported data from the previous point
- unload the original load id.

Of course, testing and backup before is necessary.

32

MP Server / Re: Large Migration + Codepage

« on: January 15, 2019, 01:25:14 PM »

Bumping this back to life with some new things.

Currently, in production we are running TSM 5.5 with a few centera nodes setup. We want to go to Spectrum Protect 8.1 on RHEL. I am wondering if this will work..keep in mind i've never done it this way--

export NODENAME filedata=all toserver=NEWSERVER
for each NODE we have defined

Any thoughts or advice on this approach?

Concerning your needs, so I would say CMOD 8.5 -> 10.1 is no problem, even from AIX to Linux. DB2MOVE is your friend here.
I prefer to use db2move export for exporting the data, and db2move load to load the data into the new server. I use db2look in order to create the database layout before launching db2move load. That way you can adapt if needed the tablespace etc...

Now for the TSM part, if you want to go to a newest version, my advice would be to use exactly your advice, so export nodename.

If you want to go to another device (S3, OpenSwift, ...) then the only way at my knowledge to do it as fast as possible would be to do a "dsmc retrieve" from your nodes, and then use "arsadmin store" to store the objects back into CMOD new storage area.
But for TSM, export nodename is the best way. EXCEPT if you want to change the type of node from Creation Date to Event based, and in that case, you must go to the export all objects from TSM to the new node in the new TSM.

33

MP Server / Re: Message 111 - OnDemand is unable to determine the link for the file

« on: January 12, 2019, 02:36:30 AM »

   114   Unable to open file >/arscache/cache1/0/WDA/RES/3<.  The error number is 2  Srvr->TEST non-SSL<-
Modify message
Report to moderator   Logged

When I see this error, it means for me the following:

* The file located in your system at the following location /arscache/cache1/0/WDA/RES/3 does not exist
* The link located in your system at the following location /arscache/cache1/retr/WDA/RES/3 still exists
* Maybe, but the error message doesn't seem to indicate that, but probably the permissions are not correct.

What to do? Well, I am working with one of our old CMOD server, that nobody took care of... and it takes ages depending on the size of the cache, and how many files to solve all the problems.
The thing I would advice strongly is to have a backup of your cache before doing anything on your cache FS.
We had a backup, and we could solve our problem with great luck, otherwise millions of document could have been lost, since the files were not stored in TSM, but only in cache.... *big sweaty experience*

Basically do you use TSM, or only cache? If you have a TSM then you might need to try to restore the file /arscache/cache1/0/WDA/RES/3 which in your TSM should be located in /WDA/RES/3 or maybe /ARCHIVE/WDA/RES/3 .
The other solution would be if you don't know how to use the TSM retrieval of an archive object (dsmc command or some gui interface), then to use the command "arsmain retrieve", that way you can get your file back in a temp directory or directly in /arscache/cache1/0/WDA/RES/3
If you don't have TSM, then you must go to your Backup system to retrieve the missing file.

Maybe, because of my current scary experience I should write a memo on how to clean up/maintain a CMOD cache... because this is not rocket science, BUT this is so important, and lots of people have no idea how it works and how to cure it.

34

MP Server / Re: Having query regarding the restrict the data range in CMOD client /ICN .

« on: January 12, 2019, 02:24:23 AM »

At my knowledge the only "macro" available for query restrictions are defined in this IBM technotes:

http://www-01.ibm.com/support/docview.wss?uid=swg21611495

So this might be a candidate for an enhancement for CMOD.

35

MP Server / Re: How to delete single statements from a CMOD Load File

« on: January 10, 2019, 11:21:19 PM »

There is workaround (just not as simple as -X) instead of by Loadid  , using  -i "WHERE doc_name LIKE '*****FAA%'" and -s ****

Wow!!!! Thank you for the info that's invaluable!!

36

MP Server / Re: Anyone using CMOD in a Docker container?

« on: January 08, 2019, 05:50:45 AM »

I will play in the next few week to dockerize CMOD for an internal project of our.

I will keep you all informed of my finding, and also share my Dockerfile :-D

37

MP Server / Re: Change Segment Field of existing Application Group

« on: January 08, 2019, 05:46:27 AM »

Well if you need to change it, that's means basically to change the stop/start fields from arsseg table according to the new field, the bit field of the field you want to change to the new field that contains the date for the segment field. And the new date field must be EXACTLY the same date field of the original date field. (date (old format) -> date (old format) // date -> date // date/time (old format) -> date/time (old format) // date/time -> date/time)
And potentially all stop/start entries in the ARSLOAD table if you have a retention load type.
In addition to that, all the LoadID in your System Log will be wrong, so the usage of "arsadmin unload" will not work correctly.

So even if it is possible, I will not advice to do that. Not at all. If you REALLY want to change your segment field, my advice would be to create a new application group, with exactly what you need, and begin to archive documents in this new AG. Have your folder mapped to this new AG in addition to the current AG.
The current (old) AG is considered Read-Only, and when no documents are inside anymore (expired), then you can remove it.

Or the other possibility, which is quite expensive would be to do an AG migration to a new AG. This can be done in 2 ways:
1) Export every load ID from the original AG and load in the new AG
2) Create a new AG, and copy the segment tables from the old to the new AG (arstblsp is your friend) and then in TSM (or your storage manager) to copy the content of the files to the new location. For TSM you could rename the filespace for that.

Of course all these short suggestions means that you have a real good knowledge of how CMOD works in order to not break anything and crash your archive.
So if you don't understand how to proceed on my suggestions, then I would suggest:

- either the new AG creation and switch of the load to the new AG, and let the old AG die slowly
- Or export every load ID from the original AG and load it in the new AG

But in any case try to avoid hack CMOD if you don't know what you are doing, because nobody will save you if something goes bad. And IBM Support won't support you.

In any cases, whatever solution you choose, BACKUP BACKUP BACKUP TEST TEST TEST !!! :-D

38

MP Server / Re: How to delete single statements from a CMOD Load File

« on: January 08, 2019, 05:31:40 AM »

Or even simpler than that (without playing with indexes, or whatever offser/length fields)

- Get the LoadID where the document you want to unload is (let say in old format: 123-123FAAA-1-0-12345-12345)
- Remove the index you don't want (arsdoc delete)
- retrieve all the documents from the LoadID from first step (arsdoc get -c -X loadID ...)
- archive the what you have retrieved in the previous point (arsload -X G -g ... -a ... ) You will have duplicates after this step is finished
- unload the whole original LoadID (that you took in the first step) (arsadmin unload -L LoadID -g ... )

Of course each of these steps must be successful in order to go to the next step, otherwise you will have lost documents! So again backup backup backup before trying/testing/... !

And of course after this method the LOAD will be different that the original one, so if you keep a database/file of the loadids... you might be careful with that method.
Basically this method is used by the CMOD module called ERM (Enhanced Retention Management), which gives you also a new LoadID!!

39

MP Server / Re: Will there be an issue of CMOD DB/arscache prod refresh in non-prod?

« on: December 12, 2018, 07:59:53 AM »

Well in windows the shortcut file is... a file... it means you can open it and look at the target directory that it points to. and maybe change it.
Try with one shortcut file, modify it to point to the correct object path. If that works, then do the same with the other shortcut files that are "wrong".

I needed to do that years ago on a Windows server, that saved our live!

40

MP Server / Re: ARSLOAD daemon vs manual

« on: December 12, 2018, 07:55:56 AM »

To be honest, I've never used the daemon mode. I've tried it a long time ago (something like 10-15 years ago), and didn't like it.
I prefer to have the complete control of what arsload does. And after going to IBM, all my customers that were not on z/OS was doing the same as me. The z/OS people loved the daemon mode.

I think it is a question of taste and color. If you like that mode, then go for it.
If you need more control, and you need to do lots of pre-post processing, then the manual mode might be better for you.

In all case, the advice of @Justin is good concerning adding some code in the logging exit (arslog), but I would just advice to make it as simple as possible, and not put too much logic behind, the shorter and fastest, the better.
Today with CMOD the call to arslog is done in an async mode, that way it doesn't slow down CMOD. But nevertheless try to get the info you need with arslog, and then another script/program will get the output of your custom code to do something more beefy!

Alessandro

41

Content Navigator / Re: Does ICN support loading docuents to CMOD with ARSLOAD Like ODWEK 9.5 API does?

« on: December 06, 2018, 06:44:28 AM »

What do you mean by that?
I am not sure to understand the question/requirement.

Sorry to answer so late, but just saw it now!

42

Content Navigator / Re: ICN not connecting to CMOD

« on: December 06, 2018, 06:39:12 AM »

That's nice to see how others do it!
How I do it, is a bit different. And I think this could be an enhancement for the ICN product in order to have after the deployment of ICN in WAS a working CMOD connection... not like today.
Every time I need to tweak WAS in order to make it work. Every single time... with or without having CMOD installed in the default directory.

What I do is the following:

After deployment of ICN in WAS, I add the following settings:

- From WAS Admin Console: Servers -> Server Type -> WebSphere application servers
   - Then click on your server (eg: server1)
   - Then Server Infrastructure -> Java and Process Management -> Process Definition
   - Then Additional Properties -> Java Virtual Environment
   - Finally in the "Generic JVM arguments" I add the following line:

Code: [Select]

- Djava.library.path=/<custom Path or original path>/ibm/ondemand/V10.1/www
 

Now, the next part is ONLY if you install CMOD as a user OR in a non standard path.

- From WAS Admin Console: Servers -> Server Type -> WebSphere application servers
   - Then click on your server (eg: server1)
   - Then Server Infrastructure -> Java and Process Management -> Process Definition
   - Then Additional Properties -> Environment Entries
   - Now you need to add 2 new entries

Name=ARS_INSTALL_SERVER_Vxxx_DIR
Value=<CMOD Install Path> (ie: /home/archive/ibm/ondemand/V10.1)

and

Name=ARS_INSTALL_ODWEK_Vxxx_DIR
Value=<CMOD Install Path> (ie: /home/archive/ibm/ondemand/V10.1)

Where Vxxx is
   V101   for CMOD V10.1
   V95     for CMOD V9.5
   V90     for CMOD V9.0

So if I have CMOD V10.1 it should look something like:

Name=ARS_INSTALL_SERVER_V101_DIR
Value=/home/archive/ibm/ondemand/V10.1

Name=ARS_INSTALL_ODWEK_V101_DIR
Value=/home/archive/ibm/ondemand/V10.1

Save everything is the Admin WAS Console.
 
Then finally restart WAS.



But now I see thanks to you nishen, that I can replace the last step with the ARS_INSTALL_*_V*_DIR with your explanation:

we had to add the odwek_install_dir parameter in the ICN Ondemand Configuration page as a custom property

Thank you !!! :-D

43

Documentation / Re: CMOD 10.1 ODWEK compatibility

« on: November 28, 2018, 07:12:20 AM »

Well to add some insight on ODWEK and CMOD Server.

What Norbert is talking about is only 1-2 method in the WHOLE ODWEK Java APIs, the xmlparse() method.
And yes, this one needs to be ODWEK Version = CMOD Version

Maybe the new API to archive documents in CMOD (which appeared in V9.5) ODServer.*load*(), then maybe, you need like the same version of the APIs

But for the "client" part of the ODWEK Java API, which is basically the 99% of the APIs, the advice is ODWEK Version >= CMOD Server.
I had never any problems with any of my past customers... It was mainly, searching, retrieving. That's it. And if that's the case, then you should not have any problems.
The only problem, which is NOT a real problem is when you want to use old ODWEK with a new server, then the new functionality are not available.
But that's exactly the same as the Windows Client. If it is older than the server version, you can still access it, but not all functionality will be available.

And depending on your setup, and migration path, it might be a must to have for a few days/weeks/month (I hope not years), that ODWEK Version < CMOD Server Version OR ODWEK Version > CMOD Server Version.
The advice I had from the IBM Labs (ODWEK main developer) was to always use the latest ODWEK Java API to develop and reach the CMOD Server even if the version was 1 main version difference.
That advice was several years ago, before V10.1.
Maybe today with the integration with some of the functionality of the server part, then it might not all be always true for all part of the API (like save documents or xml work).

44

General / Re: C2070-985 Withdrawn

« on: November 28, 2018, 06:59:41 AM »

I have the same question as you... especially since I did this C2070-985 last year... and it is already 6+ month not valid anymore...

I hope somebody knows something...

45

OD/WEK & JAVA API / Re: Unload call

« on: November 22, 2018, 10:27:00 AM »

Except the enhancement proposition, the only way to do a "unload" would be to call the "arsadmin unload" command from Java.