Changing indexed report to non index report

[DDP021]

Hello,

We have Applications  that are using the old OS390 indexer to index on Account number.  These reports all share an Application Group used by other Applications.   The users are requesting these reports not be indexed.  Just want then loaded to CMOD as one index.  My question is the only option to create new Application and Application groups for these reports with no indexing?  My guess is the answer is yes.  Don't believe we can just modify the Application indexes themselves since the Application Group is using Account Number field as an index.

[Justin Derrick]

The word 'index' appears in your post a little too often, and it's difficult to figure out exactly what you're trying to do...

Are you suggesting storing data in CMOD without extracting document metadata from the files you're loading?

This might sound crazy, but why not just tell them to not use that field in their document searches?

You'd also need to be extraordinarily careful with performance - it just takes a few terrible database queries to obliterate CMOD's ability to serve user's quickly and easy.

Can you give us some more insight into the problem your users are trying to solve?

-JD.

[DDP021]

HAHA...My bad Justin....This is a mainframe report we are loading into CMOD.  They are breaking it on ORG number.  ( I said account number previously).  They need to verify the Account number.  Apparently the Account number can fall under numerous ORG Number.  So now they need to go into each ORG code index and do a search on the account number.  So they want to have it loaded without breaking on anything.  So if the file is 100 pages, they just want the version to open with 100 pages and then be able to open the version and then do a FIND on the Account Number so it searches all 100 pages.  I realize this isn't the best way, performance wise, to do things.  Below is their explanation for wanting the report to be changed:

CDSB0SNR and CDSB0SNZ are the new account reports for the Institutional Bank and Private Wealth, respectively.  My team uses CDSB0SNR on a daily basis to validate the accuracy of data input for new CDS accounts.  When doing that, every account must be reviewed, regardless of organization. 

I hope this helps explain things.  The way we are doing it is to create new ACIF reports without any breaking (ie Org Code)

[Darrell Bryant]

Could you use the full report browse function? You could keep indexing and loading the report as you do now, but give the users Full Report Browse permission in the folder. They will then see a View Full Report button when that folder is open in the client. Select one document from the Document List, click on the View Full Report button. The first document in the report opens, they can then search within the document as normal. They will see a message to continue searching thru all segments of the report. Click on Continue All and the search string is found, no matter how many documents / segments have to be retrieved. Another plus for this technique is that it works with reports that are already loaded.

[Mehmet S Yersel]

In addition to Darrell's suggestion of loading the report in full and enabling Full Report Browse capability, maybe you can go the extra mile and add a Full Text Search field to the folder criteria so that hit list is generated only with the reports that have the keyword being searched.

If users don't have Windows client, full report browse will not be available on ICN. ICN doesn't support Full Report Browse. However, Full Text search folder field in ICN will reduce hit list generated and still be very valuable and I strongly recommend it. It is a better approach than a user individually opening each document (or full report) from the hit list and then performing a search for the keyword(s) inside the documents.

[Justin Derrick]

So, +1 to Darrell for Full Report Browse.  Also, +1 for myersel for search full text.

Two caveats:

In Full Report Browse, if you have large object support turned on, and these reports are many hundreds to thousands of pages long, there will be a bit hit on the CMOD server when you do searches - as it cycles through retrieving them across the network (then searching) each object.

For full text search, all the work happens on the CMOD server.  And while this works great for modestly sized reports, if a user selects several multi-thousand page reports for full text search...  performance will suffer on the server, and the heavy I/O may impact other users.

If this AG and the pool of users is small, this may all be a non-issue.  But for a large AG with large reports, and a large pool of users, this could be painful.

-JD.

[DDP021]

Thanks to all for all the input!!!....Full Report Browse option doesn't work as they use ICN and not client.

The Full text search would work. These reports are large and they would limit their searches across one days version. As indicated, the report was initially setup to index off of ORG code.  But they are more concerned with Account number.  So as its setup today, they had to go into each ORG Code index for that day to verify the Account number wasn't there. 

For this case, the change was already in place to create new application/application group with no indexing which of course only effects things after change. 

We will definitely keep the Full Test Search in mind, and be aware of how it will be used and how large the data is as to not impact performance if used. 

Again, I appreciate all the input from everyone!

Take care

Dave

[Mehmet S Yersel]

...
For full text search, all the work happens on the CMOD server.  And while this works great for modestly sized reports, if a user selects several multi-thousand page reports for full text search...  performance will suffer on the server, and the heavy I/O may impact other users.
...

This exactly has been the downside to the full text search.

However, let's say we have 10 large reports and 1 user. Full Text Search is not configured for the folder. User wants to search for a keyword in all 10 reports and the keyword is only available in 3 of them, but he doesn't know until all 10 reports have been opened/searched. What will the user do? He will bring all 10 reports to the hit list, then he will open each document and then perform a search inside the open document. What just happened here is all documents have not only been retrieved on CMOD server, but also downloaded to the ICN and rendered for display. This brings more overhead on the ICN server in addition to the overhead on the CMOD server. And it takes much longer for user to accomplish what he wants to do. Since people are most expensive resources in most cases, I prefer to cater for the human elemnet and let the computer do the work.

My point is, in the grand scheme of things, if we can calibrate the search range to be minimal; and if users are trained very well for how potent and yet dangerous this search is, then it is still cheaper to do it even for large reports. Also, users must be trained to wait for the Full Text Search results to come back and they should never kill a running search and start a new one. Each time they kill their session and start a new Full Text Search, the previous search continues on the CMOD server and keeps running until done.

Good Luck with this approach and please make sure users are very well aware of what they are doing and what really happens in CMOD behind the scenes when using this feature.

-Mehmet