Author Topic: List of Performance Related Fixes plus a Brief Checklist, Upgrading CMOD on z  (Read 11025 times)

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
I was asked for a brief checklist for items in addition to the migration instructions, sharing it here:

  • If migrating to V8.5 or above for the first time, are you comfortable with the ARS_ORIGINAL_CODEPAGE setting you'll be using?
     
  • do you understand why you see this message ARS0220I Server code page is 1200 ?   Note that your ARS_ORIGINAL_CODEPAGE setting is probably not 1200.
     
  • are you using the new ARSSOCKD parms of "-S -I [instance] -v"?

    Reference:

    http://www.odusergroup.org/forums/index.php?topic=762.0

     
  • Is this CMOD instance on the same LPAR or on a different LPAR than your test system?  If on a different LPAR, ensure that your OMVS settings are the same or greater on the new LPAR.   (operator commands   D OMVS,L and D OMVS,O to check)
     
  • If not already in production, have all DB2 PTFs from the following web pages at least received into your DB2 SMP/E:

    http://www.odusergroup.org/forums/index.php?topic=1525.0

    http://www.odusergroup.org/forums/index.php?topic=1318.0
     
  • Have all CMOD PTFs received into your CMOD SMP/E.  Ensure you have received the CMOD PTF for APAR PI42688 which corrects an error in an error message
     
  • the most common error reported to Level 2 after an upgrade from V8.4.1 or before to V8.5 or above is the restriction that the OS/390 Indexer will now report when an index value is not found and no default value is specified.

    The error message you receive when encountering this will be, for example "8 fields submitted by the indexer, 12 expected"

Ed
« Last Edit: December 15, 2015, 08:16:51 AM by Ed_Arnold »
#zOS #ODF

geoffwilde

  • Administrator
  • Sr. Member
  • *****
  • Posts: 253
  • z/os erm icn
    • View Profile
Re: Brief Checklist, Upgrading CMOD on z
« Reply #1 on: June 19, 2015, 11:24:33 AM »
Thanks Ed!
President, OnDemand Users Group
Lead Technician for Content Manager OnDemand @
US Bank
#zSeries

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Checklist of Performance Related Fixes
« Reply #2 on: July 29, 2015, 01:47:57 PM »
      • PI10044 contained two different improvements:                           
                                                                                 
        (a) better storage management.  no action required to exploit.           
                                                                                 
        (b) delays due to lock contention.  The following ACTION must be taken   
            to improve performance:                                             
                                                                                 
            If installing on CMOD 8.5, the SARSINST(ARSUSECX) must be           
            re-assembled and re-linked after the APPLY.).                       
         
      • PI15871 The ARSLOAD executable is larger than necessary.  This results   
        in increased CPU and EXCPs when ARSLOAD is fetched into memory.         
                                                                                 
        Action required:  None.  Just APPLY the PTF   
                                   
      • PI15930 High CPU ARSYSPIN and ARSLOAD is being fetched into storage     
        every time it was ATTACHed.                                             
                                                                                 
        Action required:  None.  Just APPLY the PTF 
                                   
      • PI22584 ODF performance improvement, reduce the number of program       
        fetches for cobol exits.                                                 
                                                                                 
        Action required:  None.  Just APPLY the PTF   
         
      • PI20473 which is a DB2 APAR whose symptoms have been unearthed by CMOD


      • (Added 11/2/2015) PI50742 390 INDEXER PERFORMANCE IMPROVEMENT.                                                 
                                                                                 
        Action required:  None.  Just APPLY the PTF
         
      • (Added 12/8/2015) 9.5.0.3 rollup PTF includes exploitation of hardware compression/decompression

        Action required:  None.  Just APPLY the PTF.  However, it does require a certain level of CPU.

        (Updated 2/25/2016)  9.5.0.4 changes the action required extensively.   You must read the following technote in order to implement h/w compression:

        http://www-01.ibm.com/support/docview.wss?uid=swg27046215
« Last Edit: February 25, 2016, 10:19:44 AM by Ed_Arnold »
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Modifying a post doesn't bump this topic up on the age list.

So this is a bump as I feel it's worth raising the visibility of the List of Performance Related Fixes.

Ed
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Bump for h/w compression update
« Reply #4 on: February 25, 2016, 10:20:54 AM »
Bump for the h/w compression update rolled out in 9.5.0.4.

Ed
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
One of the changes starting in CMOD V9.0.0 is that the ARSANN table now contains LOBs which can possibly waste a lot of space.           
                                                                           
For more consult the following...       

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W7c1e3103820b_4d9e_add6_b5ade3280dcb/page/Inline+LOBs+%28Large+Objects%29                                     
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Universal Table Spaces
« Reply #6 on: November 14, 2016, 12:50:26 PM »
If you haven't already, consider moving to Universal Table Spaces (UTS). See the DB2 Knowledge Center for more information.

http://www.ibm.com/support/knowledgecenter/SSEPEK_10.0.0/intro/src/tpc/db2z_universaltablespaces.html
 
The essential information from the Knowledge Center is:

You can alter existing table spaces to universal table spaces by using the ALTER TABLESPACE statement. If your database contains any simple table spaces, you should alter them to universal table spaces as soon as possible.

Related to this is a recommendation received:

IF you have a high number of applications per application group (we've seen this work particularly well when it's a few hundred)

AND IF you see a high number of DB2 getpages after moving to 9.5 or above

AND IF you see large increase in CPU consumption in the DB2 address space after moving to 9.5 or above

THEN moving the ARSAPP table to a Universal Table Space (UTS) and implementing inline LOBs is recommended.


http://www.ibm.com/support/knowledgecenter/SSEPEK_10.0.0/perf/src/tpc/db2z_improveperformanceoflobdata.html
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
z/OS Parms for DB2 Performance
« Reply #7 on: October 25, 2017, 10:34:11 AM »
At its heart, CMOD is a DB2 application.

There are two operating system parms that I'm aware of to benefit DB2.

I'm not going to go into detail on them as they're easily google-able.

1. USEZOSV1R9RULES(NO)

Set via your DIAGxx.

Can also be implemented dynamically with a SET DIAG=xx command.

To see if you have this enabled use the operator command D DIAG

Quote

D DIAG                       
IGV007I 12.36.01 DIAG DISPLAY
...     
VSM USEZOSV1R9RULES(NO)     
...

2. MEMDSENQMGMT:  ENABLE

Set via your ALLOCxx

Can also be implemented dynamically using SETALLOC SYSTEM,MEMDSENQMGMT=ENABLE

To see if you have it enabled use the operator command D ALLOC,OPTIONS

Quote

D ALLOC,OPTIONS                 
IEFA003I 13.19.40 ALLOC OPTIONS
SPACE           foo                   
UNIT            foo           
TIOT            foo           
SDSN_WAIT       foo       
VOLUME_ENQ      foo         
VOLUME_MNT      foo         
SPEC_WAIT       foo         
ALLC_OFFLN      foo         
CATLG_ERR       foo     
2DGT_EXPDT      foo         
VERIFY_VOL      foo         
SYSTEM          foo
                MEMDSENQMGMT:     ENABLE

Note:  if implementing dynamically I believe you need to bring up DB2 after the parms are set.

I have no benchmarks, sorry.

Ed

#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Is your shop overriding the default LE values shipped with CMOD?
« Reply #8 on: December 05, 2017, 10:54:24 AM »
CMOD has its own defaults for LE storage settings such as HEAP and HEAPPOOLS.

If you feel you must override the default settings be sure to do the LE storage tuning exercise.

Note: the defaults are bigger in V10.1 than V9.5 which are bigger than the defaults in V8.4.1.

Use the LE option RPTOPTS(ON) to determine what your settings are and where they're set.  RPTOPTS(ON) costs nothing in performance, just writes the options in use to SYSOUT.

One way to specify RPTOPTS(ON):

Quote

//ARSSOC95 EXEC PGM=ARSSOCKD,REGION=0M,TIME=NOLIMIT,
//  PARM='RPTOPTS(ON)/-S -I ARCH950 -v'

Ed

03/10/2020 Update:  Do not override CMOD V10 LE options unless you know exactly why.  Testing has shown that with ARSSOCKD now being 64 bit, there should be no reason to override the default LE parms.  If you're overriding LE in V9.5 or below, remove those overrides.

« Last Edit: March 10, 2020, 12:09:15 PM by Ed_Arnold »
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
(Reference my previous post)

And just what are the LE settings set by the CMOD product?


Content Manager OnDemand for z/OS - Initial Recommended LE Settings

http://www-01.ibm.com/support/docview.wss?uid=swg22011284


Note that bigger is usually better.  I would be concerned only if I had overridden the default LE values to less than their defaults, especially for HEAP.


There is no substitute for doing the LE tuning exercise.

Ed

03/10/2020 Update:  Do not override CMOD V10 LE options unless you know exactly why.  Testing has shown that with ARSSOCKD now being 64 bit, there should be no reason to override the default LE parms.  If you're overriding LE in V9.5 or below, remove those overrides.
« Last Edit: March 10, 2020, 12:09:51 PM by Ed_Arnold »
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
What other performance enhancements should a shop be investigating?

1. For running in a SYSPLEX, the auto-tuning of the JES2 checkpoint first available in z/OS V2.2 is recommended:

https://www.ibm.com/support/knowledgecenter/SSLTBW_2.2.0/com.ibm.zos.v2r2.hasa300/has2v5_automatic_approach.htm


2. Ensure your COFVLFxx, at a minimum, has the following specified:

Quote

CLASS NAME(CSVLLA)        /* Class name for Library Lookaside @P2C*/
      EMAJ(LLA)           /* Major name for Library Lookaside @P2C*/
CLASS  NAME(IRRSMAP)      /* OpenMVS_RACF SMAP Table */               
       EMAJ(SMAP)         /* Major name = SMAP       */               
CLASS  NAME(IRRGMAP)      /* OpenMVS_RACF GMAP Table */               
       EMAJ(GMAP)         /* Major name = GMAP       */               
CLASS  NAME(IRRUMAP)      /* OpenMVS_RACF UMAP Table */               
       EMAJ(UMAP)         /* Major name = UMAP       */               
CLASS  NAME(IRRGTS)       /* RACF GTS Table          */               
       EMAJ(GTS)          /* Major name = GTS        */               
CLASS  NAME(IRRACEE)      /* RACF saved ACEEs        */               
       EMAJ(ACEE)         /* Major name = ACEE       */               

Ed
« Last Edit: June 01, 2018, 11:43:29 AM by Ed_Arnold »
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
At z/OS V2.1 a new version or release was made for zFS V1.5 or V5 depending on who you're talking to.

I talked about this here:

www.odusergroup.org/forums/index.php?topic=1710

There's a great testimonial on the web here:

https://groups.google.com/forum/#!topic/bit.listserv.ibm-main/CLBgzhjJPQk

Ed
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
PH02490 / UI59445- V9.5 only, reduces internal ODF lock contention between ODF and load processing by reducing the length of time ARSLOAD is holding the internal ODF lock.
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
V10 Only, Fix To Reduce CPU Usage of ARSLOAD
« Reply #13 on: April 02, 2019, 11:52:12 AM »
PH09636 / UI62202

No action required, just install the PTF.

Ed
#zOS #ODF

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Recommended OAM service
« Reply #14 on: July 10, 2019, 11:07:38 AM »
OA55720 - HIGH CPU AND UNEXPECTEDLY LONG DURATION OF OSMC PROCESSING FOR 
AN OBJECT STORAGE GROUP   
                                     

ERROR DESCRIPTION:                                             
The customer noticed that when they had an Object Storage Group
with a few hundred thousand objects eligible for transition     
processing and the transition processing resulted in assigning a
management class whose transition criteria set the new pending 
action date (ODPENDDT) to the current date or earlier, that the
processing caused high CPU and took hours to complete. When they
modified their ACS routines to instead assign a new managment   
class whose transition criteria set the new pending action date
to the future, the CPU usage decreased dramatically, as well as
the overall processing time.                                   
                                                               
This issue was incorrectly addressed by FIN APAR OA44651 with   
pending action date behavior changed within base V2R2 release. 
OA44651 pending action behavior was then reverted by APAR       
OA55718. Please see OA44651 and OA55718 for additional details.
                                                               
Additional Keywords: OSMC CPU OAM OBJECT                       
                                                               
LOCAL FIX:                                                     
Ensure that the management class being assigned during         
transition processing sets the pending action date to a future 
date, via either the transition criteria or the expiration     
date in that management class.                                 



OA57287 - OSMC PROCESSING SLOWDOWN FOR STORAGE GROUPS WITH LARGE AMOUNT   
OF OBJECTS (APPRX. HALF A BILLION) AFTER OA55720. 
             

ERROR DESCRIPTION:                                               
After the application of APAR OA55720, OSMC processing slows     
down when a particular storage group has a large number of       
objects. Objects are processed successfully, but the processing 
takes significantly longer. The customer noticed the difference 
in processing time was significant for storage groups that had   
approximately half a billion objects.                           
                                                                 
LOCAL FIX:                                                       
Do not start OSMC processing for storage groups with a large     
amount of objects.                                               

#zOS #ODF