OnDemand User Group
Support Forums => Other => Topic started by: jsquizz on January 30, 2019, 05:51:21 PM
-
Hi All,
Looking down the barrel of starting an upgrade from TSM 5.5 to V7.1.1, On AIX. I have read much documentation and talked to a few folks including IBM about the process, and I am aware of some things that might come my way due to the underlying database changes and conversion process.
My question is this, has anyone done this on their system that uses a centera device class? One of our original plans was to use TSM on RHEL on a separate server and to migrate everything over, but per IBM we can't do that with Centera device classes. So it looks like we are going to be upgrading our AIX servers in place.
Once concern that I do have- Is using CMOD / DB2 and TSM 7.1 all on the same box. Does anyone have experience with this kind of upgrade where centera is involved?
-
I had a lot of customers with TSM and Centera. Centera is a beloved box in Switzerland, even if today Centera is "dead" and has been replaced by a new solution from EMC the ECS.
The company where I work now, are looking to migrating their Centera to ECS.
But back to your initial question, an in-place upgrade will work. Without any problems. If you use DB2 for CMOD, then you must be VERY careful because TSM use DB2 has an internal database, and having 2 separate DB2 instance in the same box is possible, but requires some careful planning and configuration.
Especially, per default DB2 wants to use the whole server memory! So if you have 2 DB2 doing that... I let you imagine the troubles! I know there is a setting in DB2 to tell him to use on X % of the server memory. If you do that in both DB2 (TSM and CMOD) then you are in a good place.
The other problem that could arise, is that you must make sure that CMOD doesn't use the wrong DB2 libraries... I had such problems in the past because of that.
Well that was for the in-place upgrade.
Now, to transfer from one TSM to another, like RHEL as suggested, I don't remember that a copy of a node was not possible if it was plugged with Centera.
I will need to ask the people who were involved in that in some of my previous projects or current workplace.
Unfortunately I am in holidays now until next Wednesday, so I won't be able to give you an answer before next week. So if someone has some experience, I hope he will be able to share, otherwise I will try to give something next week.
Regards,
Alessandro
-
I confirmed with IBM that you can't copy device classes / nodes / all that fun stuff from AIX to Linux.
That's why we are upgrading in place, for now. Thanks for the feedback! I have heard issues about resource contention and things like that.
-
Now, to transfer from one TSM to another, like RHEL as suggested, I don't remember that a copy of a node was not possible if it was plugged with Centera.
I will need to ask the people who were involved in that in some of my previous projects or current workplace.
Unfortunately I am in holidays now until next Wednesday, so I won't be able to give you an answer before next week. So if someone has some experience, I hope he will be able to share, otherwise I will try to give something next week.
Per IBM- That is not possible with Centera. We are also storing files in a file device class which is a compliance/DR concern for us. Based on everything that I have researched and some convos with our IBM AVP : We are going to take the approach of using fresh servers and extracting from AIX and reloading into Linux.
We have an entire Linux CMOD architecture setup and currently running that is not being used. It was setup in 2016 and the migration was never started. CMOD and TSM on independent servers. CMOD 9.5, DB2 10.5, TSM 7.1.1. I think I am going to apply the latest fixpacks for CMOD and DB2, and upgrade to Spectrum Protect 8.1 since we want to use ICOS. We don't have any data in this environment currently.
I have some questions about this, mostly around the extract/reload. I have never had to do an extract like this before of an entire system. I know there's several ways I can do it. Here's my initial thoughts-
ARSXML export of my objects
ARSDOC get
ARSLOAD running as a daemon
As far as the extract goes- I can see two ways of doing it but I am sure there are more. We have 55 application groups, 352 applications.
Some quick metrics show me that in the past 5 years we have loaded a total of 156,802 individual files. The total size for the files retrieved was 2.3TB, and we stored a total of 500GB after compression. 117 Million rows were inserted into the database. Total time for all loads from the 87 records is around 350 hours. Each application group has no more than 17 fields defined.
1) arsdoc get -u admin -p password -h archive -g APPGROUP -L LOADID <--Retrieve each load as one file, let ARSLOAD DAEMON reload.
2) arsdoc get -u admin -p password -h archive -g APPGROUP -acgNv -I "where doc_name='1FAAA' -o OUTPUT <--Retrieve each load as a .out/.ind/.res file, let ARSLOAD reload. The benefit I can see doing this is that I will have the index and it will use the generic indexer instead of calling ACIF/PDF indexer, which would save some system resources I would imagine.
3) Regarding the storing process - Anymore, with the implementation of things like ICOS, and even other high speed archives- Is there really any benefit to loading to cache and then migrating to TSM? I can see how this would have been a case in the past with older storage methods like jukeboxes and optical platters, but I would imagine that new archives even things like Centera, ECS, S3, etc..are significantly faster. I am thinking of having our new environment load directly to spectrum protect.
On an average day, we have around 10,000 retrievals via ODWEK.
Based on the 66 records and similar AFP files with similar resources-
Retrieval from Cache - .017 Seconds
Retrieval from TSM - (TSM5.5/AIX6.1/CMOD and TSM on the same box) - Using a FILE device class - 0.023 Seconds
Retrieval from TSM - (TSM5.5/AIX6.1/CMOD and TSM on the same box) - Using a CENTERA device class - 0.427 Seconds
On an average day, we have around 200 loads, about 98% of them are stored as AFP.
Comparing files that have exact application group settings, and exact indexing settings, with similar AFP structure-
Load directly to TSM - FILE device class - 3.8 Seconds
Load directly to TSM - Centera device class - 4.6 Seconds
Load directly to CACHE - 4.9 Seconds
Our system is small compared to what I've encountered in the past.
Big difference - Wondering what kind of performance increases/decreases we can expect with ICOS, I would imagine probable some better performance using the latest and greatest versions of our software since we are currently so behind.
I'd like some feedback on my two thoughts of retrieval to reload the files- and if anyone has a better more efficient way of doing it I am all ears. Also thoughts on loading directly to Spectrum Protect over cache. Thanks everyone!
-
Normally when we change from storage A to storage B in CMOD, I do the following:
1) Create the target storage node in CMOD
2) Get all the object names from the segment table in CMOD (doc_name)
3) Export all the objects in storage A (if in TSM : dsmc retrieve /ODINST/ABC/) or from ondemand via "arsadmin retrieve". (don't forget the index files the doc_name with the 1 at the end)
4) import the objects from 3) via the command arsadmin store into the new node
5) change the pri_nid in the segment tables to match the new storage node.
6) remove the old storage node from OD
7) remove the old storage node
And that's the fastest way to do it, without the hassle to export all data (arsdoc get) and re-archive it via arsload, which works but it is slow...
-
3) Export all the objects in storage A (if in TSM : dsmc retrieve /ODINST/ABC/) or from ondemand via "arsadmin retrieve". (don't forget the index files the doc_name with the 1 at the end)
And that's the fastest way to do it, without the hassle to export all data (arsdoc get) and re-archive it via arsload, which works but it is slow...
I've been looking at the arsdoc approach, retrieving all (.ind/.out/.res) and reloading with arsload.
Do you happen to have a sample syntax for arsadmin retrieve - and arsadmin load? I can't seem to get the syntax down.
-
It's extremely rare that I disagree with Alessandro... but today is that day. :D
I like arsdoc get / arsload because it gives you option to fix a lot of silly mistakes in the App Group config... Expiration types, Segment size, field lengths, update date types, etc. :) There are a lot of ways to improve an AG & App config when you do that -- which you can't do with Alessandro's method.
-JD.
-
I've been tinkering with that method for the past few days and i'm having pretty good luck so far.
i am getting the DOC_NAME from the ARSLOAD table, part of me thinks that is right..But i've researched it and read it might only get me valid loads with "load" as the expiration type..and we have a bunch that are segment.
But part of me thinks I need to query each individual AG data table for the doc name, extract to a list, and then i can do arsdoc get -G and -i "where doc_name='1FAAA'"
-
It's extremely rare that I disagree with Alessandro... but today is that day. :D
I like arsdoc get / arsload because it gives you option to fix a lot of silly mistakes in the App Group config... Expiration types, Segment size, field lengths, update date types, etc. :) There are a lot of ways to improve an AG & App config when you do that -- which you can't do with Alessandro's method.
-JD.
I am probably going to be doing it this way, I've played with this method it seems like it's the easiest.
I have a listing of load-ID's that I am going to use from 87 records. I also have load ID's from AG deletes/unloads/arsmaint, and what I am doing is creating a very very large list of all Load ID's, and if the ID exists more than once (Saying it was loaded and deleted..) I am going to remove that from my list. This should give me a listing of load Id's that I need to use. Then, I am going to do an arsdoc get and use the -X flag.
one cool thing that I was concerned about- We use arsdoc delete frequently to delete duplicate indexes (argh), and in all my testing I've noticed that if we delete an index, when I go to retrieve the original load it is gone too. Neat!
-
It's extremely rare that I disagree with Alessandro... but today is that day. :D
I like arsdoc get / arsload because it gives you option to fix a lot of silly mistakes in the App Group config... Expiration types, Segment size, field lengths, update date types, etc. :) There are a lot of ways to improve an AG & App config when you do that -- which you can't do with Alessandro's method.
I agree in your disagreement!! :-D
I mean, if you want to do a quick 1:1 migration, then my way is the fastest of all. And it works. I've done it countless of times.
Of course, if you want to change quirks in the AG definition, then I would go your road without thinking a second!!! :-D
All depends on what you want to do.
Regards,
Alessandro
-
It's extremely rare that I disagree with Alessandro... but today is that day. :D
I like arsdoc get / arsload because it gives you option to fix a lot of silly mistakes in the App Group config... Expiration types, Segment size, field lengths, update date types, etc. :) There are a lot of ways to improve an AG & App config when you do that -- which you can't do with Alessandro's method.
I agree in your disagreement!! :-D
I mean, if you want to do a quick 1:1 migration, then my way is the fastest of all. And it works. I've done it countless of times.
Of course, if you want to change quirks in the AG definition, then I would go your road without thinking a second!!! :-D
All depends on what you want to do.
Regards,
Alessandro
Oh trust me! There are several changes we could make, but I am taking the JD approach of if it's not broke don't fix it!
My suggestions for changes are going to be implemented in things going forward!
-
Hi,
Perhaps a bit too late for your project now.
I did the TSM migration with Centera between Solaris TSM 5.5 and Linux (RHEL) TSM 7.1 (not in SSAM mode)
The TSM migration was done over the network using this https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.0/com.ibm.itsm.srv.upgrd.doc/t_srv_upgrd_dbmove_net_s4.html (https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.0/com.ibm.itsm.srv.upgrd.doc/t_srv_upgrd_dbmove_net_s4.html) as guide. As this was my first migration I did have a few try and fail attempts, but at the end it worked.
Sure, you would need to take the same PEA file (the key file for your Centera pool). To migrate CMOD DB2 you could use db2move or write a simple script to do export to IXF and load from IXF. It is fast and does the endian conversion.
If you would prefer data export / import the method described by Alessandro is the right one in essence. You could consider running multi process/thread your export/import to gain speed.
Hope this helps,
N.
-
Hi,
Perhaps a bit too late for your project now.
I did the TSM migration with Centera between Solaris TSM 5.5 and Linux (RHEL) TSM 7.1 (not in SSAM mode)
The TSM migration was done over the network using this https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.0/com.ibm.itsm.srv.upgrd.doc/t_srv_upgrd_dbmove_net_s4.html (https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.0/com.ibm.itsm.srv.upgrd.doc/t_srv_upgrd_dbmove_net_s4.html) as guide. As this was my first migration I did have a few try and fail attempts, but at the end it worked.
Sure, you would need to take the same PEA file (the key file for your Centera pool). To migrate CMOD DB2 you could use db2move or write a simple script to do export to IXF and load from IXF. It is fast and does the endian conversion.
If you would prefer data export / import the method described by Alessandro is the right one in essence. You could consider running multi process/thread your export/import to gain speed.
Hope this helps,
N.
Hi Norbert, Just wondering about this post again, what approach did you use to migrate the database.
-
Hi Norbert, Just wondering about this post again, what approach did you use to migrate the database.
What I did (not a 100% supported by IBM perhaps ;D ) speaking of CMOD migration or clonning:
- assuming you have target DB2 instance for CMOD ready with a database created (I have never used the arsdb -c)
- Export DDL from your primary DB2 using: db2look -d ARCHIVE -a -e -l -x -o ARCHIVE.sql
- Edit ARCHIVE.sql and delete / change all what is not related to table spaces related to segment tables, segment Tables and Indices
- Execute ARCHIVE.sql using db2 -t on your target DB2 (pay attention not to overwrite your primary :-) )
- On the primary instance export ARS system tables using arsdb -x
- On the target instance import ARS tables using: arsdb -i Note: don't remember if arsdb -t is required prior arsdb -i, but it doesn't hurt
- On the primary instance list all your tables "select table_name from arsseg" and put it in an SQL script as "export to ABA1.ixf of IXF select * from ABA1;" one table per line
- on primary instance execte SQL using db2 -t -z myexport.log -o- -f myexport.sql
- write your import SQL script similar to export: load from ABA1.ixf of IXF insert into ABA1 nonrecoverable;
- execute on target system using db2 -t -o- -z myimport.log -f myimport.sql
- Assuming you have configured your ars.ini and ars.cfg you should be able to boot up you new target instance, if this is just a cloning, for upgrade I would suggest to run following before booting up the instance: arsdb -e -f -F -v to drop all indices, then arsdb -u -v this will alter all tables as necessary per target CMOD version, then arsdb -r -f -v and arsdb -s -v to recreate indices and recalc stats
- Once the target instance is up&running: arssyscr -l -m -a to re-create System folders
- to rebuild stats on target segment tables: arsmaint -r
- Note: don't forget your caches, specially system log and all AGs where you load to cache first than to storage or where storage is cache only
- run a few testes before releasing to production, load/unload search and retrieve new and old records
For TSM it is similar (somewhat easier for cloning only) , but it has been a while I've done it.
Hope this helps, please bare in mind this is most likely not supported by IBM ;D and I would not blame them
N.