Messages

91

MP Server / Re: Find Total compressed Doc size stored in an app group

« on: January 23, 2017, 09:11:38 AM »

Well that's not easy to do, because there are many combination possible. And per load, you could also have different settings for an AG.

But basically, you need to use the following idea for compressed files:

per DOC_NAME, you get the max(comp_off) and to this value add the associated COMP_LEN.

Example:

DOC_NAME: 1FAAA
MAX(COMP_OFF): 21892981
associated COMP_LEN to MAX(COMP_OFF): 29129

Size of 1FAAA in the cache AND in Centera:  21892981 + 29129 = 21922110 bytes

And you do that for every DOC_NAME in your AG, and you got the size you want.

If the files are not compressed, normally, take for your doc_name, the max(doc_off) and add the corresponding doc_len, and you have for that object the size in cache and in Centera.

To know if your doc_name is compressed or not, you need to look at the field comp_type, if I remember correctly, if the value is "N" (none) "D" (disable), then the content is not compressed, otherwise it is compressed.

I let you figure out how to do it in SQL, shell, java, etc... But you have the logic here.

Regards,
Alessandro

92

MP Server / Re: Archiving large Audio/Video files

« on: January 23, 2017, 08:59:36 AM »

Hello Norbert,

if you look at the segment table, you will see 2 fields:

doc_len   (= Document Length)
comp_len (= Compress Document Length)

both are of the type INTEGER.

If you look at the DB2 documentation, you will see that the type INTEGER has a maximum value of 2147483648, meaning that is the max length of your file that you can archive in CMOD.

Here is some technote:

https://www-01.ibm.com/support/docview.wss?uid=swg21170676

Regards,
Alessandro

93

MP Server / Re: OD Admin Tool - User Types - ARSUSER Table - ADMIN Field

« on: January 23, 2017, 08:54:17 AM »

Again the same things happens all the time...

PLEASE DO NOT DO A QUERY IN THE DATABASE IF YOU DON'T KNOW WHAT YOU ARE DOING.

I don't care if you have done it for 15 years, and it was always OK. because THIS IS NOT OK.

If you do that, then you are dependent on the good will of the CMOD dev that don't change the ARS tables. If they had new fields, or change something, then you are in deep sh%t.

Every BIT on a field in CMOD has a meaning. Like what zancanaro shows... it gives the value, but doesn't give the explanation of each bit, which is also useless. And in the next version if they add new type of admin, or whatever new type of user, then this list will be obsolete... and they could also change the code as they wish...

My STRONG advice, and this will be valid even if the CMOD dev will change the whole database structure (and they can... without any warning, since they didn't document in detail what they are doing in the tables).
Will be to use ARSXML to get all the information you want from the users.

You can also do it in java with the method ODServer.xmlParse(...).

Regards,
Alessandro

94

MP Server / Re: Disabled Users Reporting

« on: January 23, 2017, 03:31:11 AM »

I agree 100% with Justin, if you don't know how the CMOD database is structure, I WILL NEVER use such already made, and most probably wrong SQL.
CMOD use bits in its database field in order to define what needs to be done.

Maybe in your case the bit 256, or 8196 was the one setting the disable flag. If it was 8196, then with your query you will miss 8196+1, 8196+2, 8196+4, etc...

So please if you don't know what you are doing, don't query the database .... use arsxml instead, that way you will be future proof

95

MP Server / Re: Decrease the number of arscaches

« on: January 23, 2017, 03:26:13 AM »

Option 1) Never, that would be crazy...

Option 2) that's what I would do.

Option 3) if the customer doesn't want TSM, then don't impose TSM. At least I will not do it if the customer doesn't want that.

Option 4) I will use option 2 in nearly all cases.

96

MP Server / Re: Question on arsload and pdf indexer

« on: January 23, 2017, 03:15:27 AM »

In order to use PDF indexer it must be installed where you want to use it (either object server, library server or even a remote location).
Be careful with you licenses, because you might be limited on how many PDF indexer you might install.

97

MP Server / Re: How can I check document location from tables directly?

« on: January 23, 2017, 03:09:13 AM »

Hello teeraw,

what Justin said is 100% true.

Don't try to search somewhere else you will loose only time for nothing.
The only thing you will find is new things that has nothing to do with your need.
That's good to find new things which are not related :-D

So the thing you have found is an old thing that tells if the table (segment) is the the database, or is TSM.
Because in the old times, you could migrate whole segments from the database (mainly DB2) into TSM in order to save some disk space on the database.
Yes, 15-20 years ago, disk space was really expensive... and that's the reason that you have the "System Migration" folder/application group, in order to track such actions.
To be honest I have never seen or setup such environment...

So back to your question, you need to do as Justin said, look into the cache "retr" directory for every "select distinc doc_name from SEGMENT_TABLE" if it is in the cache, if it is not, then it should be in TSM.
You have NO other way to do it... But you can keep looking at the database, you will learn (again) lots of nice unrelated things :-D
sorry.

Regards,
Alessandro

98

z/OS Server / Re: How to specify storage node using ARSLOAD

« on: January 23, 2017, 02:48:15 AM »

You simply cannot, since the storage is attached to the application group and not the application.
That simply mean the following, if you want to do that, then you need to create an application group PER document type.
Or said in another way:

You create a new application group, when you need to:

• store the documents in a separate storage
• have other kind of indexes
• have a different retention period
• have a separation of documents due to business logic (eg: don't mix countries outputs, don't mix different companies output, ...)

And no, again, you simply cannot tell arsload where you want to store your document. This is define with your application group, that means (again), that if you want to store the document somewhere else, then you must change your application group destination with the option -g of arsload. That's your one and only way to do it.

99

Documentation / Re: CMOD and LDAP for authorization

« on: January 22, 2017, 08:02:44 AM »

Yep, this is a very common question about cmod and ldap...

as Justin said correctly, LDAP for CMOD is used ONLY for password checking, and NOTHING else.

You must have the whole permission stuff store into CMOD. That's it, no way around it.

I said, no way around it, this is not 100% true, but I will explain it later.

So if you want to keep in sync LDAP and CMOD, then you will need to write a tool to sync what you have in LDAP with CMOD, by creating/updating/deleting user in CMOD.
I had lots of customer who had their own way to do the sync between their central permission repository (not always LDAP), and CMOD.

And I know no tools that does that, "out of the box" for you.

You can use the command "arsxml" for creating, updating, deleting the users and their rights, or you can use the ODWEK method called ODServer.xmlParse(...) which use the exact same XML as the command arsxml.


Before I said, that this was not 100% true, that you must have all the permission in CMOD. If you want, but again this is a self development work -> custom code. Then you can use the permission user exit in order to do a query to your ldap to check if your user has the right or not to do an action.
This must be coded in C/C++. And you must be aware that this is time consuming, and you will most probably get bad reaction time with CMOD compared to today.
It doesn't mean this is not possible, but in order to have it right, it will take a lot of effort from your developpers, and at the same time this will be a nice challenge for them :-P

Well I hope I could answer some of your questions.

If not, then please ask again, and as Justin said, if you have a very clear use case, on how to have some rights/permissions stored in LDAP instead of CMOD and still have good performance, then fill a enhancement requests.
Please be aware that the CMOD dev wants to keep the product as fast as possible, and everything that will not produce fast results will be strongly challenged.

Regards,
Alessandro

100

Windows Client / Re: Window admin client Crashes when trying to delete a folder.

« on: January 22, 2017, 07:49:55 AM »

It might not help you... but I have found with time, that sometimes, some actions were allowed in the past with the Admin client, which are not allowed anymore...
creating some inconsistencies at the database level.
And when you try to delete folders/application groups/application/users/... then the admin client will simply crash without giving any clue...

One time I had the problem that some application group field mapping were duplicated in the database, which should NEVER happen... but it did... and creating lots of problems.
Another problem was that for some objects, you has some left over of rights, with no user associated with it... creating a crash everytime you wanted to modify or delete a folder/application group...
Another time, in a field definition of a folder, one bit of the mask field of a folder field was wrong... should be there. Was is a database problem, a disk problem... don't know... but it was simply wrong. By clearing this bit, everything was fine afterwards.

I have no idea what is causing your problem, but maybe there are some inconcistency in your database.
So as Justin is saying, try the latest greatest version of CMOD Admin client (V9.5.0.7 at the time of writting if I am not wrong), and if you still have a problem, my advice would be to open a PMR, because most probably you might have a more deeper problem.

Regards,
Alessandro

101

General / Re: Segmentation error vmState=0x00040000 Error while calling Java in CMOD 8.5

« on: January 19, 2017, 02:14:47 PM »

With that it is IMPOSSIBLE to give you any answers... that's just "garbage"... No offense..

Either you can reproduce the problem with a simple java program, and you can provide the code for it, so we can analyse it...
or you really need to ask an IBM service team to help you debugging it.

So without knowing exactly what you do, and how you do it... that's simply impossible with so little information to answer you...

Regards,
Alessandro

102

General / Re: CMOD -HCP Integration

« on: January 19, 2017, 02:10:49 PM »

Today (CMOD V9.5.0.7) CMOD supports the following storage manager:

- TSM
- Disks (Cache on a filesystem, NFS, ...)
- OpenStack Swift
- Hadoop HDFS

and that's it. If your HCP can be used in any of these way, then yes, If the HCP use some other way to store documents, then no.
I know that the CMOD team is working to have more storage managers... but which one, and when they will be included in CMOD... that's something I have ZERO idea.
Maybe an enhancement could be requested in such special case.

Regards,
Alessandro

103

Content Navigator / Re: Full report browse option in ICN

« on: January 19, 2017, 01:56:00 PM »

I don't believe it does exist.  I think what they demo is using full text search to accomplish the same thing.  The issue is that if you historical data, the full text search is not available until you re archive the data.   I think I asked for a tool to go back an rebuild the required indexes for the full text search but I don't believe it is available.  At least this is what I remember. 

Wrong Wrong Wrong

You don't need to rearchive your data... that would be crazy! You know that some customers have Exabytes of data in CMOD??? and you want them to re-archive their data??? IBM is sometimes crazy, but for that... nope...

What you need to do, is to define which documents needs to be full text searched, and then CMOD will extract and add the document in the Full Text Search server.

You have some function to add a document to the full text searchable, and you need to use that functionality of arsdoc

Add a document for full text search

Code: [Select]

arsdoc fti_add ...
http://www.ibm.com/support/knowledgecenter/SSQHWE_9.5.0/com.ibm.ondemand.administeringzos.doc/doddo016.htm

Release a document from full text search

Code: [Select]

arsdoc ftp_release ...
http://www.ibm.com/support/knowledgecenter/SSQHWE_9.5.0/com.ibm.ondemand.administeringzos.doc/doddo017.htm


If you want to understand the whole process, my advice would be to read that document:

http://www-01.ibm.com/support/docview.wss?uid=swg21606305
https://www-01.ibm.com/events/wwe/grp/grp006.nsf/vLookupPDFs/Full%20Text%20Indexing/$file/Full%20Text%20Indexing.pdf


I hope that helps understanding what is that option of CMOD, and dispell some misconceptions...

Regards,
Alessandro

104

Content Navigator / Re: ICN and CMOD on mixed platforms

« on: January 19, 2017, 01:43:18 PM »

Hello,

I am not sure to understand your problem... such mix of platform is totally common... and there are no need at all to share any file mount with anything...
Windows, AIX, Linux, z/OS... everything is possible in any mix.

Basically you need to have that:

- 1 host (don't care about the OS) with CMOD server (ok... I care... please don't use windows.... :-( ok.. that's personal CMOD and windows... not good)
- 1 host (don't case again about the OS) with ODWEK, WAS and ICN installed.
- 1 Database for ICN (could be on the same host of CMOD, or same host as ICN, or even another host somewhere else and don't care about the OS either)

That's it!
The most difficult is the ICN configuration.
I would strongly strongly suggest to keep the standard path of ODWEK, and not change it.
If you still want to change it, then don't use the useless variable ODWEK_INSTALL_DIR, because with CMOD 9.5 it simply doesn't work ....
You need to have the following 2 variables:
- ARS_INSTALL_SERVER_V95_DIR
- ARS_INSTALL_ODWEK_V95_DIR

and both needs to point to the installation home dir of CMOD V9.5 (in linux /opt/ondemand/V9.5 standard path per default)

You need to ensure the JDBC / Data source is correctly configured in WAS for the database, and that the database is correctly created and filled during the ICN setup phase.

Ans then it should be working without any tricks/tips...

But clearly the most difficult part is the setup of your ICN... once you got it right, then it will simply work.

I know I needed at first to do the first installation at least 3-4 times in order to understand a bit and to make it work correctly in a proper way... but the first time... god... only frustration.

I hope that helps a little bit. Stick with the IBM documentation, read the what you see in the screens for the setup...

Regards,
Alessandro

105

Documentation / Re: CMOD and LDAP for authorization

« on: January 19, 2017, 01:31:50 PM »

Hello,
We will be implementing CMOD and are planning on using LDAP integration for authentication.  However, we are curious if anyone is leveraging LDAP (AD) to handle authorization and if so how?  This doesn't appear to be native functionality of CMOD.

We have explored using ODWEK to build our own component but there doesn't appear to be CRUD APIS to support this.

Well apparently you haven't read at all the documentation of CMOD... since LDAP and AD with authentication is a native functionality of CMOD since at least 10-15 years.

Here are some informations for you on the subject:

http://www-01.ibm.com/support/docview.wss?uid=swg21597246
http://www-01.ibm.com/support/docview.wss?uid=swg21610510
https://www-304.ibm.com/support/docview.wss?uid=swg21446517

Now... from what I can read between the lines with your question, and tell me if I'm wrong... you want to handle the authentication from ODWEK with LDAP.
Well you must understand the following, it is NOT the client that decide what kind of authentication is used, but the server.

When you create a connection with ODWEK, then you use the method ODServer.logon(...) you will need to provide the user and password.
Then ODServer.logon(...) will discuss with the server, and the server will check how to authenticate the user.
If the CMOD server is not define with LDAP, then it will NOT use ldap... as simple as that.
If you have setup the server with LDAP, then it will use LDAP.
But in any cases, it is NOT the task and responsibility of the client to decide which authentication to use, this is the task of the server side.

Now... maybe you want to do SSO (Single Sign On), and then you need to pass the SSO Token (Kerberos, ...) in order to validate that the token is ok.
Then, again this is NOT the task of the client to do it, you need to pass the information to the server with the help of the method ODServer.logon(...) and then because we are speaking about SSO, you will need to write a C program in CMOD to plug in the server (Security User Exit) that will do the work of validation for the client.

So, all of that to say, that before assuming, you must understand how CMOD works, and more importantly, if you want a good answer, you must explain us what is the use case you want to do. And then we will be able to give you some answers, at least we will try, to the best of our ability.

So again, what do you want to do exactly?

Regards,
Alessandro