OnDemand User Group

Support Forums => Report Indexing => Topic started by: DDP021 on May 16, 2021, 05:03:15 PM

Title: Application group question
Post by: DDP021 on May 16, 2021, 05:03:15 PM
Hello,

On the Application group, we defined a field called REPORT TITLE where the Database Value is the name of the Application and the Displayed value is the description of the report which then is shown in the folder definition under REPORT TITLE.   We have an Application group with several applications defined to it.  We found an issue with several POSTING DATE formats on numerous applications where the format was setup as %y/%m/%d but at some point the data was changed to be %d%m%y.   we have reports to store initially on CACHE for 7 days before migrating to So when ingested the posting dates are wrong which caused the reports to remain on CACHE instead of migrating to TSM for defined life of long term storage.  What we need to do is create new  definitions with the correct format.  But we also need to keep the existing applications so already loaded data can be accessed.  What we are looking to do is rename the current application definition to, as an example TEST-123.  We then would ADD a new REPORT TITLE database definition called TEST-NEW as the Database Value with the same displayed value as TEST-123.  We would then create a new Application name base of the original name of TEST-123 and use the TEST-NEW as the Application Group Indentifier.   My question is, would this work without causing issues with preloaded data?  And also, what exactly is that INDENTIFIER used for? I've attached a few screen shots for reference.  TY
Title: Re: Application group question
Post by: Justin Derrick on May 17, 2021, 04:28:19 AM
The identifier differentiates the Applications from each other in the Application Group definition.  It seems like you're using the field correctly, athough having an index for that field might not be really effective.

As for your date-related issues, I can think of a few ways to detect the issue (check the System Load App Group for disparities between the load date and the start/stop dates...) but the fix would be tricky unless you start digging around inside the database with SQL.  Even if you fixed the database values in the database, I'm not sure how it would affect the migration of data out of the cache and into secondary storage...  There's probably too many configuration variables for me to say for sure.

I have a morbid curiosity about how to fix this...    Please keep us posted on your progress!

-JD.
Title: Re: Application group question
Post by: DDP021 on May 17, 2021, 04:43:15 AM
THanks Justin....We were told by or engineering group that they were seeing CACHE being filled up.   We have, on the Application Group for each load to remain on CACHE for 7 days before migrating and because the POSTING DATE format is configured wrong on the Application definition, it is never moving off of CACHE.  We always assumed the migration off of CACHE was based on date the data was loaded into CMOD and not the POSTING DATE that was being used.  As far as use of the Identifier, when we converted off of the old mainframe ONDEMAND to CMOD, IBM handled all the conversion of the existing reports.  They used the REPORT TITLE as the indentifier on the Application group and created it as a FILTER.  The description (ie DISPLAYED VALUE on the Application Group setup was used on the Folder setup.  So we just kept using that when defining new reports.
Title: Re: Application group question
Post by: DDP021 on May 17, 2021, 06:42:33 AM
Justin,

I understand the that the indentifier differentiates the different applications that are defined to the application group. And as stated we input the report description as the DISPLAYED VALUE. 

I BELIEVE what our engineering group was looking to do is UNLOAD all the current reports that were loaded to CMOD with bad posting date formats.  We would then update the applications definitions with the correct Posting Date Format.  He would then reload the data back into CMOD so it would have the correct date format for the Posting Date.

We need to get confirmation from him on how exactly the CACHE migration works.  As indicated, we have it set to 7 days.  He is saying with the bad POSTING DATE format its never migrating OFF of CACHE to TSM.  But it appears to use the Life of Data and Indexes expiration tpe is set to LOAD which we interpret to being based on LOAD DATE, correct?
Title: Re: Application group question
Post by: Darrell Bryant on May 18, 2021, 05:52:37 AM
The Administrator client help text has this to say about the Load expiration type.

"The system deletes an input file at a time from the application group. The latest date value from the input file and the Life of Data and Indexes determines when the data is eligible to be deleted. Load is the default expiration type."

The date the data was loaded is not used to determine expiration, unless there is no segment date defined.

More information can also be found at the IBM Documentation web site (formerly the Knowledge Center).
https://www.ibm.com/docs/en/cmofm/10.5.0?topic=maintenance-removing-index-data
Title: Re: Application group question
Post by: DDP021 on May 18, 2021, 06:03:10 AM
the issue we are having is with the bad Posting Date formats, its never rolling off of CACHE after 7 days.  Apparently it uses the Posting Date as the starting point and after 7 days (what is set on App Grp) it then rolls to TSM for long term storage.
Title: Re: Application group question
Post by: Darrell Bryant on May 19, 2021, 05:03:05 AM
Objects in cache are located in a directory that is named using the Unix date of the segment date of the report. The Unix date is the number of days since January 1, 1970. If the segment date is incorrect, which it is in your case, then the objects are in the wrong directory. Arsmaint uses the directory name to determine if the objects in that directory need to be processed.
Title: Re: Application group question
Post by: Mehmet S Yersel on May 19, 2021, 10:12:07 AM
Have you identified all offending loads from System Log and System Load? If so, maybe you can consider:

1. arsdoc get with the -X option to extract an entire report
2. arsadmin unload -L option to physically delete the entire report
3. arsload to reload with correct application definitions

Maybe you can repeat this process with a script for each identified file to put everything back in synch with the modified upstream data feed and remove impact to your CMOD system.