How to fallback from CMOD 9.0 to 8.4.0

[leodejong]

Hi all,

I'm busy with upgrading CMOD for z/OS from 8.4.0 to 9.0.
I have installed it on our sandbox system and seems to work oke.
Now i making plans to move the new release to production and it is good practice, for me to have a tested fallback plan.
So if on the monday after the implementation, thinks are going bad we can go back to the old version.

But in the manuals there is no mention of even the possibility of a fallback.
What about the database changes and the documents which are loaded with 9.0.
I can only think about a complete backup of the whole DB2 subsystem, catalog and tables with ADRDSSU.
But then we have to reload everything which has been loaded during the time we ran 9.0.

Anybody an idea ??
Leo

[Ed_Arnold]

Leo - I think the only way to fallback would be a Disaster Recovery scenario, where everything gets reinstalled back to a point in time because of the table changes.

You would somehow have to save copies of the data to be loaded.  Yes, I know that is a huge task.

FYI, 9.0.0.2 is now available for both the client and for z/OS.

Ed Arnold

[ewirtz]

Hi Leo,

we have developped a generalised fallback solution.

a) backup CMOD database
b) we copy all data (our data come via spool) before loading them in the migrated CMOD

In case of fallback we do:
a) restore CMOD database
b) delete oam objects that are loaded from the migrated system
c) load all copied documents
d) adapt date field in the index if needed.

Everything is tricky. For instance you have to look for the table arsseg and arsag to prevent data chaos. But it works

regards

Egon

[hakan_carlberg]

Hi Leo

I think this is a BIG issue, and I've complained a lot of times on the ODUG-meetings regarding the 'missing piece' of any 'downgrade'-scripts/function,
WITHOUT restoring an old Image Copy.
Which in itself is impossible!!!, because you can't restore an old IC to the ARS-tables... since some of them are dropped/recreated during the process of upgrade

AND !! If you managed to do some restore...What have you done with your System Log If you restored that, then you can't
do any auditing on who searched/viewed anything under the period that you were running the new release.

This, as I see it(I know I'm stubborn!!) , has to be solved by IBM
As it is right now, it's unacceptable

Regards
/H Carlberg

[Alessandro Perucchi]

Hi Leo

I think this is a BIG issue, and I've complained a lot of times on the ODUG-meetings regarding the 'missing piece' of any 'downgrade'-scripts/function,
WITHOUT restoring an old Image Copy.
Which in itself is impossible!!!, because you can't restore an old IC to the ARS-tables... since some of them are dropped/recreated during the process of upgrade

AND !! If you managed to do some restore...What have you done with your System Log If you restored that, then you can't
do any auditing on who searched/viewed anything under the period that you were running the new release.

This, as I see it(I know I'm stubborn!!) , has to be solved by IBM
As it is right now, it's unacceptable

Regards
/H Carlberg

Hello Hakan,

It is nice to say it in some ODUG-meetings, but if you really want to have an impact on the future of CMOD development, then I might suggest you to open a PMR, or contact your IBM representative and tell them about it.
I know also that there are some improvement that can be transmitted through ODUG to the lab, then it would be good to have also in the correct forum, so it will be part of the wanted suggestion of ODUG.

Sincerely yours,
Alessandro

[hakan_carlberg]

Hi Alessandro

You can be sure I did open a PMR in 2010, when upgrading to 8.4.1, but to no help

Regards
/H Carlberg

[Ed_Arnold]

Perhaps this is more an enhancement request and should be added to the "Enhancement Requests" board?

http://www.odusergroup.org/forums/index.php?board=2.0

Ed

[ewirtz]

Hi all,

"I think this is a BIG issue, and I've complained a lot of times on the ODUG-meetings regarding the 'missing piece' of any 'downgrade'-scripts/function WITHOUT restoring an old Image Copy.Which in itself is impossible!!!, because you can't restore an old IC to the ARS-tables... since some of them are dropped/recreated during the process of upgrade"

Of course you have to look for such table changes and prepare some DDL and unloads of individual tables. So it's possible even for a big shop if Application group data tables have not been changed. We used our approach for the migrations 8.4.3 -> 8.4.4 and 8.4.4 -> 8.5.x.x. And it was tested on dev and uat.

"AND !! If you managed to do some restore...What have you done with your System Log  If you restored that, then you can't
do any auditing on who searched/viewed anything under the period that you were running the new release."

You can save the system log in server.sl_save. This should be anough for autiting.

We have supported our approach with a lot of scripting and programming. This will work for nearly all possible migrations. Besides of the table changes we regard Ondemad a black box. But of course IBM could do the work in a better better way because they can use their internal kowledge to optimize a fallback solution. Our management has insisted on a fallback solution So we could not wait for an IBM solution

regards

Egon

[hakan_carlberg]

Hi..

"
Of course you have to look for such table changes and prepare some DDL and unloads of individual tables. So it's possible even for a big shop if Application group data tables have not been changed. We used our approach for the migrations 8.4.3 -> 8.4.4 and 8.4.4 -> 8.5.x.x. And it was tested on dev and uat.
"
Yes, I had som script when I upgraded from 7.1.2 -> 8.4.1  and to 8.5..
where I did rename the tables and did a lot of load from the 'new' table back to the 'old'-format.
But when going to V9 might be a little harder to do a fallback from..
Today there was an document in 'CMOD Workshop Presentations 6 ' WhatsNewInV9.pdf, where i stated on page 14:
<<<<
OnDemand System Tables
- All OD system tables will be updated (during arsdb -u) to use database timestamps
  (values will be populated during this process.....)
>>>>
- So you can't do a fallback from an earlier unloads(because some ag's might have gotten new seg_ids to them and you will miss them nothing new)
- I can't use my approach, because I doubt(I don't have the V9 code yet) that the old format times will be kept in V9 for the system tables

"
You can save the system log in server.sl_save. This should be anough for autiting...
..But of course IBM could do the work in a better better way because they can use their internal kowledge to optimize a fallback solution...
"
Yes, I can save the System log elsewhere, but how long should I keep that data ?!?!
I agree with you that IBM could do the work in a better way, and that is what I hope for because at the moment.......it's unacceptable,
as of now every Customer has to invent their own way

Regards
Hakan

[hakan_carlberg]

Hi
I forgot ..!!

"Raleigh CMOD Workshop Presentations 1 , 2_Raleigh_CMOD.pdf "
IBM states on page 8:
"
CMOD approx. $100+ Million business for IBM"

IBM has the money and the knowledge...fix this !!

/Hakan

[ibmarthin]

We are about to upgrade from 8.4 -> 8.5.
And since this is my first upgrade I get a little concerned reading this thread, when I know that a FallBack plan is a "must" in our service process.

The 8.5 README.TXT says:
<<
DATABASE UPGRADE STEP
Customers will need to have a database backup and recovery plan implemented prior to performing this procedure.
>>
But there is nothing about how this "plan" is recomended.  I first thought of the Image Copy, but I undestand that it will be useless.
So should I be concerned or does upgrade of OD never fail ?

[ewirtz]

Hi,

as discussed before: the only possibility is storing of all documents loaded in the new CMOD Version additionally in filesystem or OAM or ....

This ist the base to be able to goback in general after new documents have been loaded.

Of course a solution provided by IBM is the best solution.

It's possible, if you do it in an individual approach. But of course it's expensive and it depends on the decision of business how to deal with risk and costs.

regards

Egon

[Alessandro Perucchi]

Hello All,

just one thing... I'm reading all the replies and I've one strange feeling... confusion with the Database table upgrades.

There are 2 kind of tables in CMOD:

• CMOD Internal Tables
• CMOD Customer Tables

The CMOD Custome tables are NEVER touch during an upgrade of CMOD, except if the upgrade path says otherwise (but my experience in the last 15 years, show me that they never changed during an upgrade from CMOD itself).
Only the CMOD Internal tables are touched.

If you want to make a rollback scenario, you need to backup/export/... these internal tables (ARS%), and in case you need to go back, then restore/import/... them, and only them.

You might say... yes but with CMOD V9.0, the new date format will change the customer tables. To that I will say IT IS NOT TRUE. They are changed ONLY if YOU want it to be changed. This is an extra manual step that the customer need to decide, and perform.

That said, a simple "arsdb -x" before doing the upgrade is one way to simply save this internal CMOD tables in case of rollback.

Sincerely yours,
Alessandro

[hakan_carlberg]

Hi

I just would like to make some Points:

1) A backup is useless since you're dropp:ing/recreate the tables during the migrate process.
2) an export.... well all the load's that you've done after upgrade.... they're gone

Regards
/H Carlberg

[ewirtz]

Hi allessandro,

it may be that the content of the current 'CMOD Customer tables' are never(?) changed. But it is not guaranteed that this is true for documents that are loaded after the migration. Additionally we have a link between the CMOD internal tables and the customer tables that are important. This link exist p.e in system log, LOB header, arsload and of course some special attributes of the customer tables itself. In the first comment of Ed he has stated this implicitly when he says: 'You would somehow have to save copies of the data to be loaded'

regards Egon