OnDemand User Group

Support Forums => Documentation => Topic started by: steven on May 18, 2018, 11:11:48 AM

Title: Retention Changes
Post by: steven on May 18, 2018, 11:11:48 AM
Hey guys.  It seems you always come to the rescue when I'm in a quandary.  Here's my dilemma or challenge if you will.  I have many thousands of PDF's loaded with a 3 year retention.  In our organization we have a department that sets the retention for all documents loading into CMOD.  I just found out that this retention needs to be changed to 7 or 10 years.  I know that originally if retention was changed, the documents had to be unloaded then reloaded into CMOD.  We're on version 9.5.0.4 soon to upgrade to 10.1.X using Z/OS / Unix SS.  I guess what I'm asking is there an easier more efficient way to take the documents we have now and move them to a new storage group?  Yep!  Showing my ignorance here but I know you folks will give me the guidance I need.  Thanks in advance.
Title: Re: Retention Changes
Post by: Justin Derrick on May 20, 2018, 06:29:40 AM
The only thing that is tricky for me is the fact that you're on z/OS.

By changing the retention inside the Application Group definition, you change the retention for ALL documents loaded into that AG, starting at the time the change is made.  (Obviously, you can't get documents back that have already been expired.)  The only snag is that if you're using secondary storage (TSM/OAM) that you need to make sure the change is made there as well. 

Here's a scenario I see all the time:

Retention is extended from 3 years to 7 years in the CMOD client.  However, TSM was set up to expire the documents after 3 years, and wasn't updated -- which means that TSM starts deleting anything that was loaded 3 years+1 day ago.  So CMOD keeps the metadata, and queries come back fine -- but anything older than 3 years fails on retrieval.  The copygroups and management classes need to be updated in TSM as well.

There's another caveat as well.  In a lot of sites, Application Groups use a small number of storage sets - often organized by retention period.  If your AG belongs to a 3 year storage set, and you increase the retention of that storage set in TSM to 7 years, then ALL of your documents that were scheduled to be kept for 3 years will now be kept for 7 -- which is arguably even worse.

Depending on how complex your scenario is, you might just want to extract and reload those PDFs into a properly defined Application Group.

-JD.
Title: Re: Retention Changes
Post by: Greg Ira on May 21, 2018, 05:40:15 AM
If you're on z/OS and using OAM there is a way to put a hold on the expiration.  Of course the best way to handle this is to reload into a new application group but we've used the hold method to stop documents from being expired while we work on the solution.
Title: Re: Retention Changes
Post by: steven on May 21, 2018, 08:21:52 AM
I so appreciate both of your responses and both make great sense.  I didn't elaborate like I should have in that we are moving away from OAM and everything is on "hold" and has been for a while (legal HOLD reasons).  Once the ERM solution is completely in place and we can determine what needs to be "held", we will execute ARSMAINT and make it happen. 

Btw, the storage set I'm referring to below for this AG is set at 185 days on Disk and then put on tape for 3 years.

Here's what happened to us.  We originally set up they typical AG, App and Folders to accommodate a project.  We set them up for a 3 year retention originally and documents began to be loaded.  After a while, the programmers were working on another project and without going through the proper channels began loading documents that should have been 10 years retention.  We were unaware of the activity due to our business loading thousands of documents a day or week and other projects and priorities. 

Then, to add insult to injury, the same programmers began loading legacy data that contained large PDF's into the same setup and basically brought CMOD to its knees because it was a huge hit all at once.  This is how we found out what was really happening.

Now the challenge (knowing we're not deleting anything YET:  breaking all this out to where we wouldn't have to create two more AG's /setups with different retentions is no biggee, but unloading and reloading the data is an issue.  I'm not convinced that would be a viable solution.  But I understand it may be the only one besides changing the SG retention in the AG to 10 years.  But that would change all the reports to a 10 year retention is my understanding.

Again, you guys are great!  I appreciate any ideas you might have.  I'm an avid champion for CMOD and tout it's the best.  I've worked in the business a long time, but this product is typically stable as a rock (knock on wood).
Title: Re: Retention Changes
Post by: Justin Derrick on May 21, 2018, 10:35:30 AM
Yeah, your best bet is to move forward with a planned out extract & reload.  What you describe sounds like a mess, and there may be an opportunity to clean it up by breaking apart the poorly-configured App Groups and getting them into new, properly-defined App Groups.  There may also be an opportunity for storage savings if you can use the new PDF indexer to break apart those PDFs the same way CMOD does AFP -- resources and data, stored separately & compressed.

-JD.