Performance problems on Power7?

[Justin Derrick]

Hey everyone...

I've got a customer who is moving from their POWER6-architecture systems to shiny new POWER7 servers running AIX.  As part of the migration to new hardware, the testing team is trying to quantify the performance improvement -- and their findings are that CMOD (8.4.1.4) on P7 is actually SLOWER than it was on similarly equipped P6 servers.

Has anyone else noticed performance problems on newer hardware?  Any tips or tricks?  (I'm already pushing them towards upgrading to CMOD 8.5 for the performance bumps it is supposed to provide.)

Thanks in advance...

-JD.

[demaya]

Hey,

I just had some impact when moving from windows to aix, but more in CSSAP. This was a real hard and long going (10 months) pmr...

Later we had a migration (CMOD 8.4.1.5 and CSSAP 8.4) from P6 to P7 and the performance was improved. Since 8.5 the performance is a little bit better as well.

How do they measure their performance issues? Loading, Retrieving?!

Cheers

[Justin Derrick]

Performance here is measured at the 'front end', either the ODWEK CGI client, or their own custom-written portal using the Java API.  We're just not seeing a jump in performance like we'd expect.  One of the analysts here thinks it might be a single thread that's holding up the other threads, another analyst suspects it could be IHS -- but the performance issue is seen both by the API and CGI interfaces.

Either way, we're not seeing the bump in performance we were hoping for.  I just hope it doesn't take as long to fix as yours did.

-JD.

[demaya]

Hi JD,

you may try setting 'tcp_nodelayack' to 1 (on AIX). Setting this to 1 boosted our performance 600%. You may laugh about this small setting but... its true ;-) But the difference is, we measured it via commonstore.

But you should make this before on testsystem, I may not have to tell you... but save is save

Let me know if that helps!

Cheers

[trgarcia]

Please note that OD 8.4.1 on MP reached End of Service on 9/30/2012.  The customer should be encouraged to move forward to 8.5 or 9.0 soon to be on a supported release. 

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&appname=iSource&supplier=897&letternum=ENUS911-174

[Justin Derrick]

Just a quick update on this one -- I know it's a few months old already.

The customer *is* in the process of upgrading, but things move at a glacial pace here.

The problem we were running into was a default configuration parameter in AIX, which was originally meant to defend against Denial-of-Service attacks.  The basic Denial of Service attack consists of sending so many packets to a server that it can't serve any other users.  The defense against this is to introduce a very short delay (200 milliseconds) to individual IP addresses that are engaging in what could be considered 'abusive' behavior -- say like the remote server running ODWEK that's making 40 requests a second, because someone's running performance tests... 

The solution was to turn on an AIX network tuning parameter called 'tcp.nodelay'.  By turning this feature on, it removes the delay, leading to higher throughput between CMOD and ODWEK users.