Author Topic: arsload stores the same AFP Resource many times  (Read 1654 times)

Norbert Novotny

  • Jr. Member
  • **
  • Posts: 46
    • View Profile
arsload stores the same AFP Resource many times
« on: May 20, 2020, 09:22:57 AM »
Hi guys,
not sure I my understanding of treating resources is correct or not. I have AG/AP with ACIF "RESTYPE=ALL" which does separate resource from data. While resource id 2739 is correctly found as matched,
Code: [Select]
2020-05-20 17:11:33.037909: ARS4312I Loading started, 6286 bytes to process
2020-05-20 17:11:33.075577: ARS1140I Resource /tmp/KS.70636113826651.41b25594-78ca-42a4-b051-a084c1c78101.6395.2020.05.20-11.25.39.afpds.res matches the resource >2739-106-0<

and the resource found in cache is from March-9 in TSM the same resource is stored 26'679 (!) times, given that it is still the same name TSM will try to version it. With so many versions, it starts to slow down the loads as long as 222 seconds ... not talking about space used.

The resource compare is set to 50 as per default, but I don't think so this has anything to do as the resource id is still the same and the content is identical.


Am I missing something? ...or is the is kind of "old" bug, we are on 10.1.0.5 and TSM 6.3 with SSAM mode ON.

Thank you,
 N.
Norbert Novotny
Legal archiving - Swisscom AG

Mobile:  +41-On-request

Dev: #SQL, #Perl, #Java, #C

Interests: #CMOD, #Multiplatforms, #DB2, #Oracle, #TSM, #ERM, #Performance

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: arsload stores the same AFP Resource many times
« Reply #1 on: May 20, 2020, 05:16:18 PM »
Do you have more than 50 unique resource bundles? 

It sounds like a bug that if a resource bundle matches an existing one, that it would try to re-write it -- it should simply record the id number of the matching resource, and then use that in the AG table.

I'd wonder if your documents are somehow slightly different.  I've had customers with a 'perfect storm' of situations that led to them having over 80,000 resources in a single Application Group.  We were able to retrieve the resources, and use a hash function to determine that there were only about 2000 unique resources, but because of the low default (50) it was extraordinarily unlikely that the exact same style of document would be loaded twice within that 50-load window.

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

Ed_Arnold

  • Hero Member
  • *****
  • Posts: 1199
    • View Profile
Re: arsload stores the same AFP Resource many times
« Reply #2 on: May 21, 2020, 06:49:09 AM »
> It sounds like a bug that if a resource bundle matches an existing one, that it would try to re-write it...

Not so.  It always re-writes it so that the resource bundle has the newest date on it.

Ed
#zOS #ODF

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: arsload stores the same AFP Resource many times
« Reply #3 on: May 22, 2020, 05:11:54 AM »
Ah....  I get it...  so that the file has the retention extended in TSM.

So this sounds like a TSM configuration issue...  Changing retver to 1 in the storage policy in TSM should fix it.

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

Norbert Novotny

  • Jr. Member
  • **
  • Posts: 46
    • View Profile
Re: arsload stores the same AFP Resource many times
« Reply #4 on: May 24, 2020, 12:07:39 AM »
Hi guys,
Thank you so much for helping.
I was not aware of such mechanism, but sure now it make sense, to prevent resource expiry on TSM when TSM is configured with time opposed to event based expiry. Also, the cache does not need to be re-written/updated as the resource is stored under /0/.

... gr8, now I need to close the PMR with IBM :D

@Ed, our setting is retver=14, but I think our issue is that we have not ran tsm expiry, That keeps 27k copies of the same resource.

Thank you,
 N.

Norbert Novotny
Legal archiving - Swisscom AG

Mobile:  +41-On-request

Dev: #SQL, #Perl, #Java, #C

Interests: #CMOD, #Multiplatforms, #DB2, #Oracle, #TSM, #ERM, #Performance