Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Alessandro Perucchi

Pages: [1] 2 3 4 5 6 ... 65
1
MP Server / Re: How to Identify FIELD is UPDATEABLE from the Database
« on: April 03, 2019, 04:36:27 AM »
Thanks Justin,
So no way to pick it up from the DB field value? for atleast version 9.5 / 8.5 separately? if no option then I will have the ARSXML export and parsing as Alternative option please.

thanks for your help and support.


Of course you can do it. Nobody will do anything to stop you from doing it.
But it is considered as not supported.

And from one version to another it can change without anybody telling you (release notes) that something has changed, since this is internal not documented information. IBM can change it at any time.

the official way to do that is really to use the XML as Justin rightly suggested.

If you don't need it, because you need it for some "one shot" action, I suggest that you take one field value without the updateable setting, do a select of that field in the DB.
Change the setting with your admin client/xml, and redo the select, and try to find which bit(s) is/are changed.

As an information, if your field is "updateable", then it doesn't mean you can update the field, your user needs also to have the update right.

2
MP Server / ARS_ORIGINAL_CODEPAGE
« on: April 03, 2019, 04:28:45 AM »
Hello everybody,

I had a problem with our system, because of the code page, what is written in the database, what is displayed, etc...
And during all my test/java development in java and what was going on in prod, I found something very disturbing... and I wanted to know if you have the same experience, and what could be a solution...

Here is what I've done (AIX and Linux only)
- DB2 10.5, database created in ISO-8859-15 codepage, like we have.
- Install CMOD V8.4, create the tables/indexes, load some documents
- Upgrade to V8.5 or V9.5, doesn't matter.
- Add the ARS_ORIGINAL_CODEPAGE=923 as requested.

Here is what is strange, if I change the shell "locale" to UTF-8 when running arsdb -u for the upgrade process... then, I am requested to change ARS_ORIGINAL_CODEPAGE to the value 1208.
If I use the normal ISO-8859-15 code page for the shell, then ARS_ORIGINAL_CODEPAGE should be 923.
I play around to use other code (that was installed in my AIX/Linux) and guess what... the value of ARS_ORIGINAL_CODEPAGE is always dependant on the locale of the shell.

This has cause me a shock, and after discovering that, I could finally solve a bug I had for many years, that I could never reproduced. When I have my dev environment with the same shell locale as in production, then I could reproduce the problem and solve it.

This brings in my mind many questions... why does CMOD relies on the shell to determine the codepage to use?
This is something that can change at any time, for any reasons. Some will put UTF-8, some other to C, so other to something else.
Maybe you migrate from one server to another, and the old one was per default in ISO-8859-1 and the new one is per default configured with UTF-8...
Then Boom, you server will not react as expected, you have strange chars in your clients instead of your ...

Am I dreaming and imagining things? If that is the case, it means that we must be really careful on how to start CMOD, that it use the correct codepage every time, and that the combination DB codepage and shell codepage that starts "arssockd/arsobjd" must be in sync at all time.

What are you thought on that??

3
MP Server / Re: CMOD Server Page Size too big AIX
« on: April 03, 2019, 04:09:26 AM »
Hello,

I don't know exactly what could be the problem. My first impression is that you are using a quite old version of CMOD (V9.5.0.4 is 3 years old 2016-02-18).
Have you tried to upgrade to the latest FP? the latest on V9.5 is FP12.

you are talking about "arssockd", so it means that it has nothing to do with your tomcat.
Except if your java app in tomcat is using a lot of connections, and are doing tons of transactions.

Now, you say that you have some custom security exit. Maybe the problem could be related to that part. Maybe your code does not use malloc, or any allocation whatsoever, BUT you are using I suppose some third-party libraries:
- Does the libraries are 100% without memory leak?
- Does the libraries allows concurrent run?
- Do you open and close each connection in a sane and proper way?
- Do you use it only for login, or do you use it also for the other actions in the security exit?

There might be other questions, but from your descriptions, and without updating to the latest FP, this is where my internal alarm is ringing... security exits.

4
z/OS Server / Re: Upgrade and my 'personal' thoughts
« on: March 12, 2019, 03:05:40 PM »
Hi Alessandro
I think you are missing the point
Why should you restore old Ars%-tables? If you cant keep them in synch with your other tables of the loaded data in all your appl.groups?

Who would like to be in a situation where the data you have is not correct?
You could find yourself in a situation where you have data in OAM, indextables where that data is not in the ARS-tables.


I mean...
1) You stop CMOD
2) You do the system table backup + Full Backup of the segment tables
3) You do the upgrade
4) You start CMOD
5) You test if it works
6) It works, then you open for production. Finished.
7) It doesn't work, then stop CMOD
8 ) restore the system tables and possibly the impacted segment tables
9) be sure to use the old SW
10) start CMOD with the old binary version

If you go that way, you won't have any problems. I've done it countless of time with customer where we had some issues.
And depending on the tests, you might need also to restore some segment tables, because of the archived documents that was used during the tests of functionality.

In that case I would advise to do test in a dummy AG, which is not important if the restore makes it not in sync with the restored info from the system tables.

Of course, if after X days you see that it doesn't work... then yeah... both the segment and system table are not in sync anymore and you have a problem. And I agree with you, this is not something that one wants. Not at all.
Downgrading is not a light operation, and during the changes of version with CMOD, there were lot of time that the changes were so profound that a downgrade was simply impossible.
And a restore was the only way. Today the changes are not anymore that deep as in the V7.X and V8.X versions, but still sometimes they are still some modification here and there.

And to be honnest, I know very little product in the market that allow a downgrade without doing some kind of restore.
So I understand your personal thoughts... but I wonder what else could IBM do in that area... because some upgrades are not reversible.

Maybe doing more tests in your test system, to ensure that your prod will be not impacted with some issues?
But that's not IBM work, it is more on your side.

For my part, I am more in the side to do backup and restore. And if after a few days something is really wrong, since I tried to keep all the input data that is archived in CMOD for at least 1 week. If I need to restore something, then I can restore the DB, and reload everything between the moment of the last backup.
Call me paranoid... but I am never expecting a magical solution from vendors in a timely manner. IBM or not IBM.
If I can wait, then ok, but if that is not a possibility, then I have my safety bag.
And of course, then there is a SLA issue, but better that, than keeping with a unsolvable situation that is going worse and worse with time.

What I did many times, was to clone the production database in a test environment, remove the Storage Set/Node dependencies by setting in the local cache, or a test storage which is like Prod. And do my tests there with all the documents type that we load. And since we keep at least 1 week of input files from production, we can test all the possible scenarios.
Especially if we are not sure of the results of the upgrade.
That way we learn if everything is as we expect, and if not, we don't do the upgrade in prod until we have a fix from IBM or internal teams for some custom code.

5
OD/WEK & JAVA API / Re: CMOD Java Query
« on: March 12, 2019, 01:41:03 PM »
Hello Jandl,

Welcome to the forum!

So, with a small code example, it would have been perfect.
I will try to decrypt your question. So you have done a small test in Java with the OD API in order to search for some documents, and you are using the ODCriteria in order to search for your documents.
You are not using some SQL WHERE clause for that.

Is this correct? If yes, then you need to know that the ODCriteria use the Folder Field name to do the search.
If you use the SQL WHERE clause, then you need to use the Application Group field name.

Let say you want to search with the ODCriteria, then when you search with a date field, you must make sure that you are using the correct date format, other you might search for a different date. To know which format to use, you need to get the date format with the method ODCriteria.getDefaultFmt(). It use the same format that setup in CMOD with the %Y, %y, %M, etc...

If you are searching with the SQL where clause, then this document might of interest: https://www-01.ibm.com/support/docview.wss?uid=swg27036188

I hope what I've written is clear, otherwise I will try to be more explicit :-)

Cheers!





6
z/OS Server / Re: Upgrade and my 'personal' thoughts
« on: March 12, 2019, 01:27:34 PM »
Well, the recovery plan is ALWAYS do a backup, and if doesn't work, then restore it. That is not only with CMOD, but with any product you can find.
And a downgrade is not always possible, even with the OnDemand, which is the most perfect product in the whole universe!

As Justin said, you can do a backup / restore easily of the CMOD system tables with the command arsdb. You don't need a full backup of all the tables and segments, only the ARS% tables.


7
OD/WEK & JAVA API / Re: ODWek Java API - recreateHit issues
« on: March 12, 2019, 01:22:54 PM »
Hmmmm I'm reading the new method ODHit.getOpenDocID(...), interesting, so it means the encryption key method is not anymore needed! Thank you! I will go to bed less stupid than this morning! :-D

8
OD/WEK & JAVA API / Re: ODWek Java API - recreateHit issues
« on: March 12, 2019, 01:20:11 PM »
Well if you get the docid with a user, then use another user to recreate the docid, then it will never work in a default setup.
In order to make it work, you must use an encryption key when you get the docid, and then when you use the other user, you use the same key to decrypt the docid you want to recreate.

Look at this method, because this is exactly what you want to use : ODServer.setEncryptionKey(...). Here is what the Javadoc says:

Quote
Document identifiers (docids) are now encrypted. This method can be called after a user logon to specify the encryption key to be used for docid generation. If this method is not called ODWEK will generate the encryption key. In both instances, the encrypted docid is bound to the userid currently logged in when the docid is generated. This ensures that only this user can retrieve the contents of the document using the docid. This userid binding to the docid can be circumvented by calling this method prior to user logon. When used in this fashion it is imperative that the calling application ensure the docid is only used to retrieve the contents of the document by users with the proper permissions.


9
Documentation / Re: Legal hold (ERM)
« on: March 12, 2019, 01:14:02 PM »
We can place individual documents on legal hold.  How can we place a folder or Application Group on legal hold?
As such you cannot.
As Justin said the only way to do it, is to put EVERY document in your AG, or all the AGs references with the folder with an hold, which means either with the command "arsdoc add_hold" or use the Java API of ODWEK.

10
MP Server / Re: Full Text Search
« on: March 11, 2019, 06:27:57 AM »
I'm under the impression that IBM CMOD Full Text Search is based on Apache Lucene.  Is FileNet CSS the same engine under a different shell?

CSS is also using Apache Lucene.
And the tools from both product seems exactly 1:1 the same. That is why I was in the impression that both product are the same.
But I am not 100% sure! That was my question, if someone knows that, and if yes, can we use one instead of the other, etc...

11
MP Server / Full Text Search
« on: March 08, 2019, 03:56:39 AM »
Hello everyone,

I have found today that the CSS for FileNet and the FTS for OnDemand seems to be really the same. And I was wondering if the CMOD can use the CSS and FileNet the FTS?
Has someone tried it? Then would be the question of Licenses, of course, but beside that aspect, I am wondering if OD FTS == FileNet CS or not ????

12
OD/WEK & JAVA API / Re: Folder Field Definitions with Spaces
« on: March 08, 2019, 03:53:11 AM »
with which technology? Java API?

Have you tried with the ag field name instead of the folder field name?

13
MP Server / Re: OD 10.1 Active Active High Availability implementation
« on: March 08, 2019, 03:51:34 AM »
An additional question, must DB2 also be in HADR mode? Or if we have a "simple" DB2 is also ok?
I know this is not what you want for prod, but for testing or doing a quick prototype.

14
MP Server / Re: Quantifying extract & reload
« on: February 28, 2019, 10:15:32 AM »
Hi all,

Wondering if anyone cares to share their insight, tips, and past experience on this.

As some previous posts state, I am looking into extracting data from an AIX system, using TSM/Centera, into a new system using Linux/ICOS. I am trying to put an actual timeframe on how long it is going to take to extract from AIX, and reload into Linux. I have the extract/reload process down mostly, but I am not sure where to start as far as coming up with solid dates.

There is a total of around 7TB of data to extract, totaling 200 Million Documents across 32 application groups. The totals of data in our app groups range from a few hundred MB, to 4.5 TB.

I understand there are SEVERAL variables to consider as far as this entire process goes, but does anyone have any suggestion as to how I can use our logs to actually put some realistic numbers to this.

Thanks in advance.

The only way to know for sure is to take an AG, and look how well it exports, and then how fast it imports.
From my experience, if you have an AG with lots of small documents, with lots of load ids, then it might take an awful long time, compared to an AG with big objects inside.

Since I've nearly never do an "arsdoc get" / "arsload" for such migration, I have very little details to give you.

15
MP Server / Re: Non-Root Install - Anyone doing this?
« on: February 20, 2019, 04:32:00 AM »
I have done non root implementation of CMOD for the last 15 years, since it was possible to do it. And without any problems.
I would say that now in Switzerland no CMOD is using root account anymore, or at least 99.99% of all customers :-D I don't know all of them!

Now the installation with non-root, as you have found out, you cannot change the installation path. It will ALWAYS be $HOME/ibm/ondemand/Vxxxx or $HOME/IBM/ondemand/Vxxx on AIX.
Of course you can trick with the silent installation file, BUT all upgrades, etc... won't work. This is really sad and InstallShield is not really flexible in that area... I would like that IBM ditch that install software, but one can dream!

Nevertheless, what I also found is that if you install in a non standard installation path (I mean the official /opt/ibm/ondemand/Vxxx or /opt/IBM/ondemand/Vxxx), and it doesn't matter if you install with root or non-root ( $HOME/ibm/ondemand/Vxxx or $HOME/IBM/ondemand/Vxxx is NOT standard installation path for OnDemand strangely enough...), then you need to use the variables:

ARS_INSTALL_SERVER_Vxxx_DIR
ARS_INSTALL_ODWEK_Vxxx_DIR

And be careful that ARS_INSTALL_SERVER_Vxxx_DIR = ARS_INSTALL_ODWEK_Vxxx_DIR.

where xxx = 101 for OD V10.1
xxx = 95 for OD V9.5
xxx = 90 for OD V9.0
xxx = 85 for OD V8.5

Otherwise, I have found that some actions like all tools based on ODWEK (ICN) and some internal tools of CMOD, are lost in space... because for some reasons they still look at the /opt/... standard path...

Regards,
Alessandro

We are analyzing everything from a security standpoint. And just getting much push-back about installing a binary in /home/archive(or whatever instance name

It's good to note that it looks in /opt, thank you for that. We were looking at doing it on a different mount such as /prodInstall/ibm/ondemand/.....

Well the $HOME is not necessarly /home/xxx but the home directory of your installation, so it could be /prodInstall with the user "prodinstall" (for example) which has his home directory as /prodInstall
then you can give other users the access to it, like the user "cmod". Set the correct variables, and DO NOT FORGET the permissions (https://www.ibm.com/support/knowledgecenter/en/SSEPCD_10.1.0/com.ibm.ondemand.installmp.doc/accountslin.htm) and that would be :
- $ODInstallDir/bin/arslog
- $ODInstallDir/bin/arsprt
- previously also $ODInstallDir/bin/arsrdprt
- don't forget $ODInstallDir/config/ars.ini

So that the other user can use your prodinstall installation :-D It is basically like the CMOD Documentation where you use root to install the product, and then use a non-root user to run your instance.
but here you install with user X and use a none-root, none-X user to run your instance!

Just my 0.0002$ !

Pages: [1] 2 3 4 5 6 ... 65