Copy TSM storage objects back to Cache ...

[jfry]

Sorry to ask this again but I can't find any detail on this subject.

I have documents held in TSM that are no longer in cache and I would like to reload them to cache. I am aware that their retentions will be reset and stay longer than originally intended ... that is OK. Also ... this is all to be accomplished in the original CMOD 8.4.1 instance so there is no migration or other movement involved.
 
It looks like the only way to do this is with an "arsadmin retrieve -m 2" (along with other parms) to get objects from TSM and an "arsadmin store -m 1" to store it back into cache. Is this correct .. documentation seems to imply one can copy from TSM to cache in one step using only "arsadmin store" but that doesn't seem correct.

Only one other question ... what is the process to determine which objects are already in cache?

Thanks for any assistance,

Jay

[Justin Derrick]

Backfilling a cache isn't easy.  You've already hit on a few of the points.

It's easier and faster to simply increase the cache retention for the AG that's underperforming, and reduce the caching for others.  This will improve performance over time, and requires practically no effort.

-JD.

[jfry]

Hi Justin,

Thanks for your reply but this is not a performance issue ... it is a remove TSM and tape issue.

So does the retrieve and restore work at all for this? Do I have the "-m" correct for retrieve and restore?

Any way to determine objects currently in cache?

Thanks,

Jay

[Justin Derrick]

Not that you asked, but I normally strenuously resist the idea of CMOD servers without TSM -- in my experience, the overwhelming majority of data in CMOD is 'data at rest' and is rarely (if ever) accessed.  This makes it the perfect candidate for tape.  I try to urge folks to do more research and find their most used data, and cache that -- rather than trying to cram everything onto hot, power-hungry, spinning disks.

That having been said...

You're right, that the way to get data out of TSM is "arsadmin retrieve" and the way to get things into cache is 'arsadmin store'.  I think the "implication" you're looking at is for loading a NEW object into cache AND TSM at the same time. 

The way to determine if a file exists in the cache is to check the retr directory.  The TSM filespace ID = Application Group ID Name.  Check here: /arscache/cache/retr/<AGID>/<TYPE>/<OBJ>

Where AGID is the Application Group ID Name (also in arsag under the column agid_name).
and TYPE is DOC (document) or RES (resource data).
and OBJ is the CMOD object name - a number, followed by FAA, followed by a series of letters, and optionally a $ or a 1.

It's a big job -- good luck!

-JD.

[jfry]

Hi Justin,

Thanks for the information. I understand your reply ... but we have turned into a "no-tape" shop which uses dual site replication for DR and less expensive disk also replicated for backups. Our document archives are fairly small "less than a terrabyte" and also replicated.

I didn't think we would get to a time where tape would go away ... but we have. And without the tape ... the TSM was not necessary.

What I am looking at is to take all of our current documents on TSM tape / disk and move it back to cache. I know this is not the designed or normal way to do things ... and I've been CMOD admin for several years.

Thanks again for your assistance.

Jay

[Justin Derrick]

Ah, yes, with less than 1TB, the cost and maintenance of a tape library doesn't make a lot of sense.

In that case, consider migrating the Application Groups using arsdoc.  It'll give you the opportunity to fix "permanent' parameters like Expiration Type if they're incorrectly set.  You simply use arsdoc get to pull the data out of the AG, then build a new AG, and load the data into it with arsload.  If you use the generic indexer options (-g on arsdoc and -X G on arsload) loading should be quick and easy.

-JD.

[lrfurtado]

Justin,

Sorry to revive this old thread ... In case I wanted to migrate all documents from a specifc Application Group from TSM to cache.

When doing the retrieval with " arsadmin retrieve  " I need to specify each each individual file being retrieved correct?.  


How would I go about figuring out ALL the file names to be retrieved?

I was thinking about querying each segment table associated with this specific application group for the DOC_NAME columns.

The problem I am facing is that I can't find the files ending in 1 like  123FAA1 on the DOC_NAME, should I just assume for every DOC_NAME=123FAAA a file called 123FAA1 would exist?

Or is there another way of issuing a arsadmin retrieve command that would retrieve all loads associated with a specific Application Group in a single shot, without specifying filenames ...


What's the difference content wise between the cache files 123FAA1 and 123FAAA for the same Application Group.




Thanks in advance for your help.

[Justin Derrick]

Yes, you'll need to specify each object name individually, and you can get these from DB2 or TSM.  TSM 5.x allows you to do SQL-style select statements on the internal database tables to get this information.

The files that end in a 1 contain the index data for the object -- very important when you're doing data recovery.

Good luck.

[Alessandro Perucchi]

Just a little enhancement of the answer of Justin.

If you need to restore the cache from TSM, then just take the files that describe in the field "DOC_NAME" in your segment tables for your application group.
And restore only those one. (a "select distinct DOC_NAME from segment_table" ) should give you the list of files to restore.

Don't bother to get the files that are in tsm with the ending ...1
These files are only useful when doing a data recovery, otherwise leave them alone :-)

Sincerely yours,
Alessandro

[Justin Derrick]

Some might argue that discarding the *1 files might be a decision you regret later on... 

-JD.

[Alessandro Perucchi]

Some might argue that discarding the *1 files might be a decision you regret later on... 

-JD.

:-) definitely that's why I was referring to cache only!! I will never discard them ever otherwise!! :-D

Sincerely yours,
Alessandro

[johnnoel]

Good Morning,

I am trying to remember how to work backwards to identify a document given the name on the file system. Said a different way... we installed the CMOD 9.0.0.1 and upgraded the database from the previous version, but just copied over a few of the cache files for now.

I can see <retr><OAA><DOC>104FAA and 104FAA1 for example. How do I query the database to know which metadata to search to find this object from my folders?

I remember doing it many moons ago during training, but haven't done it since.

Thanks,
John