Author Topic: zFS - Enhanced Large Directory Support in z/OS 1.13  (Read 2502 times)

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1208
    • View Profile
zFS - Enhanced Large Directory Support in z/OS 1.13
« on: February 21, 2014, 08:02:05 AM »
I thought this was an interesting enhancement.

Contact zFS support if you have any questions, especially as to whether you have a directory which is considered "large" or not.

OA40530

PROBLEM SUMMARY:                                                     
****************************************************************     
* USERS AFFECTED: zFS Release 13                               *     
*                 FMID:HZFS3D0                                 *     
****************************************************************     
* PROBLEM DESCRIPTION: New function is being added.            *     
*                      Directory operations (lookup, create,   *     
*                      remove) on version 4 directories,       *     
*                      which is the current directory format   *     
*                      in zFS on z/OS V1R13, get               *     
*                      progressively slower as the size of     *     
*                      the directory increases. This may be    *     
*                      seen in modify zfs,query,kn output.     *     
*                      Also, directory read operations         *     
*                      (specifically the readdir plus          *     
*                      operation) can be slow for large        *     
*                      directories.  This can be seen in the   *     
*                      modify zfs,query,lfs output as a low    *     
*                      vnode cache hit rate.                   *     
****************************************************************     
* RECOMMENDATION: APPLY PTF                                    *     
****************************************************************     
Version 4 directories are linear with entries that are not           
stored in any order.  Searching a directory to find that an           
 entry does not exist or to find free space to create a new           
entry means the entire directory is sequentially searched.           
The larger the directory, the more this becomes a                     
performance problem.                                                 
                                                                     
When doing directory read operations (specifically the               
readdir plus operation), vnodes are created for each entry           
name found so that their attributes can be easily obtained           
and provided to the caller as output of the operation.  This         
can cause unnecessary reuse of items in the vnode cache,             
and the stealing of vnode extensions.  Stealing the vnode             
extensions will be delayed if user data needs to be written           
to disk (if the vnode represented a file being updated).             
                                                                     
PROBLEM CONCLUSION:                                                   
                                                                     
TEMPORARY FIX:                                                       
                                                                     
                                                                     
COMMENTS:                                                             
The complete solution to this problem is using version 1.5           
aggregates and extended directories in z/OS V2R1.
  In                 
order to help alleviate this large directory performance             
problem for a limited amount of time (z/OS V1R13 and                 
z/OS V2R1 only) for some larger directories before                   
migrating to version 1.5 aggregates, zFS will allow the               
creation of new Large Fast Lookup Cache buffers above                 
the bar (64 bit storage) that will be used to fully cache             
large directories.  Directories which are eligible to use this       
new cache must be small enough to fit in the cache and yet           
large enough to meet minimum size requirements.                       
                                                                     
This is a partial solution and may help in some cases where           
the size of a frequently accessed directory in a zFS                 
aggregate has become too large and its performance is                 
degraded.  This is a global setting and not specific to any           
one zFS aggregate or directory. All zFS directories meeting           
the criteria defined by the setting of the 'flc' parameter will       
be eligible to use this new cache. The user must be aware of         
their file system environment and directory sizes in order           
to choose the proper values for the 'flc' parameters or they         
will not realize the maximum benefit of this new function.           
                                                                     
...                             
                                                                     
This solution may not be acceptable for all installations.           
For example, if there are hundreds of very large directories,         
then there may not be enough storage available in the ZFS             
address space or in paging datasets or auxiliary storage to           
make this a viable option. If the large directories are not           
frequently accessed, the benefits will be minimal. The               
administrator should take all of these factors into account           
before electing to use Large FLC buffers.  If you do elect           
to use the Large FLC buffers, then you should periodically           
monitor your performance and your directory sizes to                 
ensure the Large FLC buffer definition is properly                   
configured for your environment.                                     


(Much more detail in the APAR text)
#zOS #ODF