Z13 CPU Usage Increase

Brett,

Not a simple question.


1) Are all of your DSNZPARM settings the same with the new machine? Bufferpool sizes, pagefix rules?

2) Did IBM recommend any changes to take advantage of the changes in architecture/instruction-sets with the z13? Have you tried processing with any such changes turned on, and with them turned off?

3) Do you have the same number of ZIIP processors in use, and are the redirection rules and outcomes the same?

4) Did you have to rebind anything on the new box, and have you compared access paths?

5) Where is the increase? DB2 address spaces, or application access spaces, or both (I see that you’ve explicitly mentioned batch jobs)? Are there increases in GETPAGE usage and other performance lead-indicators? What about elapsed times for key processes?

Sorry to throw the kitchen sink at you, but all of those are places to look for root causes.

--Phil

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:04 AM
To: [login to unmask email]
Subject: [DB2-L] - Z13

Good Morning –
We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs.
Has anyone else noticed similar issues?
Has anyone had the issues, and found a solution?

Thank you, Brett Sinclair

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

Everything is the same except for:
A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.
Nothing was changed in DB2 except DB2 rebinds (BIND knows the CPU model number on which the BIND occurred).

The only thing that I see is slower response time. More CPU consumption of a few (but very important DB2 batch jobs). No increase in tables record counts, et al.
I also notice less ZIIP usage—which makes no sense to me at all.

I do believe that from the DB2 systems side, we have looked at just about everything.

Brett





The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Sevetson, Phil [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:16 AM
To: '[login to unmask email]'
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Brett,

Not a simple question.


1) Are all of your DSNZPARM settings the same with the new machine? Bufferpool sizes, pagefix rules?

2) Did IBM recommend any changes to take advantage of the changes in architecture/instruction-sets with the z13? Have you tried processing with any such changes turned on, and with them turned off?

3) Do you have the same number of ZIIP processors in use, and are the redirection rules and outcomes the same?

4) Did you have to rebind anything on the new box, and have you compared access paths?

5) Where is the increase? DB2 address spaces, or application access spaces, or both (I see that you’ve explicitly mentioned batch jobs)? Are there increases in GETPAGE usage and other performance lead-indicators? What about elapsed times for key processes?

Sorry to throw the kitchen sink at you, but all of those are places to look for root causes.

--Phil

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:04 AM
To: [login to unmask email]<mailto:[login to unmask email]>
Subject: [DB2-L] - Z13

Good Morning –
We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs.
Has anyone else noticed similar issues?
Has anyone had the issues, and found a solution?

Thank you, Brett Sinclair

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

-----End Original Message-----


Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

Less ZIIP usage may mean more processing pushed back into the general-purpose processors (or never being redirected to ZIIP in the first place), which could all-by-itself cause the problem you’re referring to. IBM’s Adrian Burke gave a presentation on 2016-12-01 at the New England DB2 Users’ Group about ZIIP usage and control. It’s here (http://www.nedb2ug.com/meetings/december-1-2016 ) and might be useful.

I don’t guarantee that it’s your problem, but a drop in ZIIP usage would pretty definitely be matched by an increase in non-ZIIP usage, other things (like workload) being equal.

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:26 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Everything is the same except for:
A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.
Nothing was changed in DB2 except DB2 rebinds (BIND knows the CPU model number on which the BIND occurred).

The only thing that I see is slower response time. More CPU consumption of a few (but very important DB2 batch jobs). No increase in tables record counts, et al.
I also notice less ZIIP usage—which makes no sense to me at all.

I do believe that from the DB2 systems side, we have looked at just about everything.

Brett





The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Sevetson, Phil [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:16 AM
To: '[login to unmask email]'
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Brett,

Not a simple question.


1) Are all of your DSNZPARM settings the same with the new machine? Bufferpool sizes, pagefix rules?

2) Did IBM recommend any changes to take advantage of the changes in architecture/instruction-sets with the z13? Have you tried processing with any such changes turned on, and with them turned off?

3) Do you have the same number of ZIIP processors in use, and are the redirection rules and outcomes the same?

4) Did you have to rebind anything on the new box, and have you compared access paths?

5) Where is the increase? DB2 address spaces, or application access spaces, or both (I see that you’ve explicitly mentioned batch jobs)? Are there increases in GETPAGE usage and other performance lead-indicators? What about elapsed times for key processes?

Sorry to throw the kitchen sink at you, but all of those are places to look for root causes.

--Phil

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:04 AM
To: [login to unmask email]<mailto:[login to unmask email]>
Subject: [DB2-L] - Z13

Good Morning –
We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs.
Has anyone else noticed similar issues?
Has anyone had the issues, and found a solution?

Thank you, Brett Sinclair

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

-----End Original Message-----

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

I have hardware people looking into that as well. But, as of now, know of no reason why ZIIP usage should go down, and only affect a very few batch jobs. Less ZIIP, should mean less ZIIP for every eligible job.






The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Sevetson, Phil [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:32 AM
To: '[login to unmask email]'
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Less ZIIP usage may mean more processing pushed back into the general-purpose processors (or never being redirected to ZIIP in the first place), which could all-by-itself cause the problem you’re referring to. IBM’s Adrian Burke gave a presentation on 2016-12-01 at the New England DB2 Users’ Group about ZIIP usage and control. It’s here (http://www.nedb2ug.com/meetings/december-1-2016 ) and might be useful.

I don’t guarantee that it’s your problem, but a drop in ZIIP usage would pretty definitely be matched by an increase in non-ZIIP usage, other things (like workload) being equal.

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:26 AM
To: [login to unmask email]<mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Everything is the same except for:
A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.
Nothing was changed in DB2 except DB2 rebinds (BIND knows the CPU model number on which the BIND occurred).

The only thing that I see is slower response time. More CPU consumption of a few (but very important DB2 batch jobs). No increase in tables record counts, et al.
I also notice less ZIIP usage—which makes no sense to me at all.

I do believe that from the DB2 systems side, we have looked at just about everything.

Brett





The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Sevetson, Phil [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:16 AM
To: '[login to unmask email]'
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Brett,

Not a simple question.


1) Are all of your DSNZPARM settings the same with the new machine? Bufferpool sizes, pagefix rules?

2) Did IBM recommend any changes to take advantage of the changes in architecture/instruction-sets with the z13? Have you tried processing with any such changes turned on, and with them turned off?

3) Do you have the same number of ZIIP processors in use, and are the redirection rules and outcomes the same?

4) Did you have to rebind anything on the new box, and have you compared access paths?

5) Where is the increase? DB2 address spaces, or application access spaces, or both (I see that you’ve explicitly mentioned batch jobs)? Are there increases in GETPAGE usage and other performance lead-indicators? What about elapsed times for key processes?

Sorry to throw the kitchen sink at you, but all of those are places to look for root causes.

--Phil

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:04 AM
To: [login to unmask email]<mailto:[login to unmask email]>
Subject: [DB2-L] - Z13

Good Morning –
We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs.
Has anyone else noticed similar issues?
Has anyone had the issues, and found a solution?

Thank you, Brett Sinclair

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

-----End Original Message-----

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

-----End Original Message-----


Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

Brett,

You mention that the issue with increased CPU is NOT in your systems upgraded to V11 - I assume therefore that it is in DB2 10 systems. The optimizer in DB2 11 is less sensitive to faster CPU speeds - meaning that DB2 10 (and prior) is more sensitive. Therefore, an access path change associated with a CPU upgrade is more likely to occur in DB2 10. Lower zIIP may be due to an access path change and reduced parallelism, or loss list prefetch or dynamic prefetch in the new access path.

The solution depends on confirming what the problem is. In static SQL - starting with a REBIND SWITCH can get you back to the prior access path and give you time to investigate.

Regards

Terry Purcell

In Reply to Brett Sinclair:

I have hardware people looking into that as well. But, as of now, know of no reason why ZIIP usage should go down, and only affect a very few batch jobs. Less ZIIP, should mean less ZIIP for every eligible job.






The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Sevetson, Phil [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:32 AM
To: '[login to unmask email]'
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Less ZIIP usage may mean more processing pushed back into the general-purpose processors (or never being redirected to ZIIP in the first place), which could all-by-itself cause the problem you’re referring to. IBM’s Adrian Burke gave a presentation on 2016-12-01 at the New England DB2 Users’ Group about ZIIP usage and control. It’s here (http://www.nedb2ug.com/meetings/december-1-2016 ) and might be useful.

I don’t guarantee that it’s your problem, but a drop in ZIIP usage would pretty definitely be matched by an increase in non-ZIIP usage, other things (like workload) being equal.

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:26 AM
To: [login to unmask email]<mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Everything is the same except for:
A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.
Nothing was changed in DB2 except DB2 rebinds (BIND knows the CPU model number on which the BIND occurred).

The only thing that I see is slower response time. More CPU consumption of a few (but very important DB2 batch jobs). No increase in tables record counts, et al.
I also notice less ZIIP usage—which makes no sense to me at all.

I do believe that from the DB2 systems side, we have looked at just about everything.

Brett





The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Sevetson, Phil [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:16 AM
To: '[login to unmask email]'
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Brett,

Not a simple question.


1) Are all of your DSNZPARM settings the same with the new machine? Bufferpool sizes, pagefix rules?

2) Did IBM recommend any changes to take advantage of the changes in architecture/instruction-sets with the z13? Have you tried processing with any such changes turned on, and with them turned off?

3) Do you have the same number of ZIIP processors in use, and are the redirection rules and outcomes the same?

4) Did you have to rebind anything on the new box, and have you compared access paths?

5) Where is the increase? DB2 address spaces, or application access spaces, or both (I see that you’ve explicitly mentioned batch jobs)? Are there increases in GETPAGE usage and other performance lead-indicators? What about elapsed times for key processes?

Sorry to throw the kitchen sink at you, but all of those are places to look for root causes.

--Phil

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:04 AM
To: [login to unmask email]<mailto:[login to unmask email]>
Subject: [DB2-L] - Z13

Good Morning –
We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs.
Has anyone else noticed similar issues?
Has anyone had the issues, and found a solution?

Thank you, Brett Sinclair

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

-----End Original Message-----

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

-----End Original Message-----


Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

An upgrade could mean several things. Assuming you had variance in everything (DB2 and non DB2) this is a classic capacity paradigm. Several items in capacity world can dent into cpu. You (or your capacity team) can compare a representative sample before the upgrade and after the upgrade using zPCR.

If you can localize the variance to only DB2 jobs path regression as described in the note by Terry is something to pursue. Usually path changes aren't wide spread but you appear to describe a potentially wide spread variance which is more than likely at a higher level originating from the new configuration. Do you observe the variance in dbm1 cpu consumption and dist cpu consumption before and after the upgrade. Take a normalized approach as in cpu time per io or cpu time per commit or something with which you can do a normalized compare.  

Some models have been more vulnerable to variance although not specific to db2.
Venkat

 
In Reply to Brett Sinclair:

Everything is the same except for:
A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.
Nothing was changed in DB2 except DB2 rebinds (BIND knows the CPU model number on which the BIND occurred).

The only thing that I see is slower response time. More CPU consumption of a few (but very important DB2 batch jobs). No increase in tables record counts, et al.
I also notice less ZIIP usage—which makes no sense to me at all.

I do believe that from the DB2 systems side, we have looked at just about everything.

Brett





The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Sevetson, Phil [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:16 AM
To: '[login to unmask email]'
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Brett,

Not a simple question.


1) Are all of your DSNZPARM settings the same with the new machine? Bufferpool sizes, pagefix rules?

2) Did IBM recommend any changes to take advantage of the changes in architecture/instruction-sets with the z13? Have you tried processing with any such changes turned on, and with them turned off?

3) Do you have the same number of ZIIP processors in use, and are the redirection rules and outcomes the same?

4) Did you have to rebind anything on the new box, and have you compared access paths?

5) Where is the increase? DB2 address spaces, or application access spaces, or both (I see that you’ve explicitly mentioned batch jobs)? Are there increases in GETPAGE usage and other performance lead-indicators? What about elapsed times for key processes?

Sorry to throw the kitchen sink at you, but all of those are places to look for root causes.

--Phil

From: SINCLAIR Brett [mailto:[login to unmask email]
Sent: Friday, December 16, 2016 9:04 AM
To: [login to unmask email]<mailto:[login to unmask email]>
Subject: [DB2-L] - Z13

Good Morning –
We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs.
Has anyone else noticed similar issues?
Has anyone had the issues, and found a solution?

Thank you, Brett Sinclair

Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

-----End Original Message-----

-----End Original Message-----


Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

I can confirm that we have this problem with z13
I don't know what it is due exactly, but from my z/os sysprog colleagues,
they said that this is a hardware problem the microcode is less efficient.
IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption
bill in consequence.

This problem has been confirmed also with others z13 sites here in France,
as it was one of the theme of the recent "Z/OS Guide Share France".


Duc

Thank you







The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

I can confirm that we have this problem with z13
I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.
IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.
This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".

Duc




-----End Original Message-----


Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.

Ducky - isn't that an odd name for someone who isn't a duck?!

Doesn't our db2 listserv prohibit such anonymous alias names?

On Tue, Dec 20, 2016 at 4:24 AM, ducky <[login to unmask email]> wrote:

> I can confirm that we have this problem with z13
> I don't know what it is due exactly, but from my z/os sysprog colleagues,
> they said that this is a hardware problem the microcode is less efficient.
> IBM answer : Working as designed, wait Z14
> You IBM vendor should be aware of this and negociate your cpu consumption
> bill in consequence.
>
> This problem has been confirmed also with others z13 sites here in France,
> as it was one of the theme of the recent "Z/OS Guide Share France".
>
>
> Duc
>
>
>
>
> -----End Original Message-----
>

Nadir,
James Campbell mentioned that it in fact may be a real name. The OP has not commented, but “Duc Ky” is a believable name for Southeast Asian languages/cultures.
--Phil S.

From: Nadir Doctor [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 7:55 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Ducky - isn't that an odd name for someone who isn't a duck?!

Doesn't our db2 listserv prohibit such anonymous alias names?

On Tue, Dec 20, 2016 at 4:24 AM, ducky <[login to unmask email]<mailto:[login to unmask email]>> wrote:
I can confirm that we have this problem with z13
I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.
IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.
This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".

Duc




-----End Original Message-----

Nadir


His posts aren't anonymous, Duc signs his emails with his name and if you view the posts via the website you will see his full name, Nguyen Duc Tuan, I guess it's due to his email set up that ducky comes up on his posts, there are others that don't have their full name on their posts. Names sometimes appear strange when they're from a culture foreign to your own, I must admit I find your name amusing from an English language point of view.


For those of you on holiday now, happy holidays. If you're a contractor, hopefully they're not too long!


regards


Chris


________________________________
From: Nadir Doctor <[login to unmask email]>
Sent: 20 December 2016 12:54
To: [login to unmask email]
Subject: [DB2-L] - RE: Z13 CPU Usage Increase

Ducky - isn't that an odd name for someone who isn't a duck?!

Doesn't our db2 listserv prohibit such anonymous alias names?

On Tue, Dec 20, 2016 at 4:24 AM, ducky <[login to unmask email]<mailto:[login to unmask email]>> wrote:
I can confirm that we have this problem with z13
I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.
IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".


Duc




-----End Original Message-----

Just an FYI.



IBM has not announced (as far as I know) the z14 processor, so I am suspicious they would say that.



If it is microcode issue, then the engineers should be able to figure out how to patch it. It just may take some time and they are providing comments to delay a customer from getting impatient.



If it were me, I would be very “in the face” of my IBM Account/Sales team for a problem like this.



Lizette





From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email] <mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase



I can confirm that we have this problem with z13

I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.

IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".



Duc

You can be very "in the face of" but the caveat that z13 performance variability depends on workload is mentioned in almost all marketing materials and presentations. Defective microcode may be a loose term someone may have let out.

z14 or znext may have its own variability too.

With cp upgrades some sites experience pain while others experience just the opposite.

 
Venkat

In Reply to Lizette Koehler:

Just an FYI.



IBM has not announced (as far as I know) the z14 processor, so I am suspicious they would say that.



If it is microcode issue, then the engineers should be able to figure out how to patch it. It just may take some time and they are providing comments to delay a customer from getting impatient.



If it were me, I would be very “in the face” of my IBM Account/Sales team for a problem like this.



Lizette





From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email] <mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase



I can confirm that we have this problem with z13

I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.

IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".



Duc

There are 2 earlier posts from the original customer who stated:
1st - "We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs."
2nd - "Everything is the same except for: A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.".

That implied to me that the problem was mostly occurring in DB2 10 subsystems, and again - mostly in batch jobs. Which is why in my earlier response I pointed out that the DB2 10 optimizer is more sensitive to CPU upgrades than DB2 11.

This discussion has morphed into a general problem with regard to z13 performance that was "confirmed" (???) at Guide/Share in France.

If we (IBM) had a pervasive problem with the z13 - then I expect all of our hardware folks to know about it. I have reached out to a couple of Distinguished Engineers in the zSystems HW team - and heard back from one who was very skeptical. He also sought more information from his peers. Due to the holiday period - many people are out - so I only have the one response.

I do however agree that mileage may vary.

But, given the information from the earlier posts - I would first want to rule out an access path change in DB2 10 for the impacted batch jobs.

Regards
Terry Purcell

In Reply to Venkat Srinivasan:

You can be very "in the face of" but the caveat that z13 performance variability depends on workload is mentioned in almost all marketing materials and presentations. Defective microcode may be a loose term someone may have let out.

z14 or znext may have its own variability too.

With cp upgrades some sites experience pain while others experience just the opposite.

 
Venkat

In Reply to Lizette Koehler:

Just an FYI.



IBM has not announced (as far as I know) the z14 processor, so I am suspicious they would say that.



If it is microcode issue, then the engineers should be able to figure out how to patch it. It just may take some time and they are providing comments to delay a customer from getting impatient.



If it were me, I would be very “in the face” of my IBM Account/Sales team for a problem like this.



Lizette





From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email] <mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase



I can confirm that we have this problem with z13

I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.

IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".



Duc

There is a SIIS problem that has always been bad coding practice and compilers have not generated this style of code for decades. I found one of our old assembler routines did indeed do a STC *+5,blah style instruction... not RENT and kills the cache, if in a loop a real disaster for performance...
Just making old assembler RENT, and a small amount of recoding, is enough to find and then eliminate this problem. Another thing is to make sure assembler local storage is more than 255 bytes away from the last instruction. Generous use of ORG ,256 is your friend here!

DB2 is innocent for this spike in cpu!

Roy Boxwell
SOFTWARE ENGINEERING GmbH and SEGUS Inc.
-Product Development-
Heinrichstrasse 83-85
40239 D?sseldorf/Germany
Tel. +49 (0)211 96149-675
Fax +49 (0)211 96149-32
Email: [login to unmask email]<mailto:[login to unmask email]>
http://www.seg.de

Software Engineering GmbH
Amtsgericht D?sseldorf, HRB 37894
Gesch?ftsf?hrung: Gerhard Schubert, Bettina Schubert

On 23 Dec 2016, at 21:55, Terry Purcell <[login to unmask email]<mailto:[login to unmask email]>> wrote:


There are 2 earlier posts from the original customer who stated:
1st - "We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs."
2nd - "Everything is the same except for: A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.".

That implied to me that the problem was mostly occurring in DB2 10 subsystems, and again - mostly in batch jobs. Which is why in my earlier response I pointed out that the DB2 10 optimizer is more sensitive to CPU upgrades than DB2 11.

This discussion has morphed into a general problem with regard to z13 performance that was "confirmed" (???) at Guide/Share in France.

If we (IBM) had a pervasive problem with the z13 - then I expect all of our hardware folks to know about it. I have reached out to a couple of Distinguished Engineers in the zSystems HW team - and heard back from one who was very skeptical. He also sought more information from his peers. Due to the holiday period - many people are out - so I only have the one response.

I do however agree that mileage may vary.

But, given the information from the earlier posts - I would first want to rule out an access path change in DB2 10 for the impacted batch jobs.

Regards
Terry Purcell

In Reply to Venkat Srinivasan:

You can be very "in the face of" but the caveat that z13 performance variability depends on workload is mentioned in almost all marketing materials and presentations. Defective microcode may be a loose term someone may have let out.

z14 or znext may have its own variability too.

With cp upgrades some sites experience pain while others experience just the opposite.


Venkat

In Reply to Lizette Koehler:

Just an FYI.



IBM has not announced (as far as I know) the z14 processor, so I am suspicious they would say that.



If it is microcode issue, then the engineers should be able to figure out how to patch it. It just may take some time and they are providing comments to delay a customer from getting impatient.



If it were me, I would be very "in the face" of my IBM Account/Sales team for a problem like this.



Lizette





From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email]<mailto:[login to unmask email]> <mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase



I can confirm that we have this problem with z13

I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.

IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".



Duc






-----End Original Message-----

In assembler code Making the csect as RSECT will expose the issue early as some service macros can expand using inlined code. EXecute instruction is another prevalent killer and likely widespread. SIIS has always been an issue but the z13 out of order stream running ahead of the executing code may be making it more vulnerable.

Now that you mention about this, can a garden variety cobol compiler code cause these types of vulnerabilities without RENT option? COBOL compiler doesn't seem to be an optimizing compiler like metal C in general. How many sites attempt a recompile in a newer machine setting the right arch level etc in compiler option etc?

Venkat

In Reply to Roy Boxwell:

There is a SIIS problem that has always been bad coding practice and compilers have not generated this style of code for decades. I found one of our old assembler routines did indeed do a STC *+5,blah style instruction... not RENT and kills the cache, if in a loop a real disaster for performance...
Just making old assembler RENT, and a small amount of recoding, is enough to find and then eliminate this problem. Another thing is to make sure assembler local storage is more than 255 bytes away from the last instruction. Generous use of ORG ,256 is your friend here!

DB2 is innocent for this spike in cpu!

Roy Boxwell
SOFTWARE ENGINEERING GmbH and SEGUS Inc.
-Product Development-
Heinrichstrasse 83-85
40239 D?sseldorf/Germany
Tel. +49 (0)211 96149-675
Fax +49 (0)211 96149-32
Email: [login to unmask email]<mailto:[login to unmask email]>
http://www.seg.de

Software Engineering GmbH
Amtsgericht D?sseldorf, HRB 37894
Gesch?ftsf?hrung: Gerhard Schubert, Bettina Schubert

On 23 Dec 2016, at 21:55, Terry Purcell <[login to unmask email]<mailto:[login to unmask email]>> wrote:


There are 2 earlier posts from the original customer who stated:
1st - "We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs."
2nd - "Everything is the same except for: A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.".

That implied to me that the problem was mostly occurring in DB2 10 subsystems, and again - mostly in batch jobs. Which is why in my earlier response I pointed out that the DB2 10 optimizer is more sensitive to CPU upgrades than DB2 11.

This discussion has morphed into a general problem with regard to z13 performance that was "confirmed" (???) at Guide/Share in France.

If we (IBM) had a pervasive problem with the z13 - then I expect all of our hardware folks to know about it. I have reached out to a couple of Distinguished Engineers in the zSystems HW team - and heard back from one who was very skeptical. He also sought more information from his peers. Due to the holiday period - many people are out - so I only have the one response.

I do however agree that mileage may vary.

But, given the information from the earlier posts - I would first want to rule out an access path change in DB2 10 for the impacted batch jobs.

Regards
Terry Purcell

In Reply to Venkat Srinivasan:

You can be very "in the face of" but the caveat that z13 performance variability depends on workload is mentioned in almost all marketing materials and presentations. Defective microcode may be a loose term someone may have let out.

z14 or znext may have its own variability too.

With cp upgrades some sites experience pain while others experience just the opposite.


Venkat

In Reply to Lizette Koehler:

Just an FYI.



IBM has not announced (as far as I know) the z14 processor, so I am suspicious they would say that.



If it is microcode issue, then the engineers should be able to figure out how to patch it. It just may take some time and they are providing comments to delay a customer from getting impatient.



If it were me, I would be very "in the face" of my IBM Account/Sales team for a problem like this.



Lizette





From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email]<mailto:[login to unmask email]> <mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase



I can confirm that we have this problem with z13

I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.

IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".



Duc






-----End Original Message-----

Good question! All my code is RENT but perhaps someone else knows the answer... I would dearly like to believe that COBOL II and higher do not generate SIIS code...
Volunteers???

Roy Boxwell
SOFTWARE ENGINEERING GmbH and SEGUS Inc.
-Product Development-
Heinrichstrasse 83-85
40239 D?sseldorf/Germany
Tel. +49 (0)211 96149-675
Fax +49 (0)211 96149-32
Email: [login to unmask email]<mailto:[login to unmask email]>
http://www.seg.de

Software Engineering GmbH
Amtsgericht D?sseldorf, HRB 37894
Gesch?ftsf?hrung: Gerhard Schubert, Bettina Schubert

On 24 Dec 2016, at 16:05, Venkat Srinivasan <[login to unmask email]<mailto:[login to unmask email]>> wrote:


In assembler code Making the csect as RSECT will expose the issue early as some service macros can expand using inlined code. EXecute instruction is another prevalent killer and likely widespread. SIIS has always been an issue but the z13 out of order stream running ahead of the executing code may be making it more vulnerable.

Now that you mention about this, can a garden variety cobol compiler code cause these types of vulnerabilities without RENT option? COBOL compiler doesn't seem to be an optimizing compiler like metal C in general. How many sites attempt a recompile in a newer machine setting the right arch level etc in compiler option etc?

Venkat

In Reply to Roy Boxwell:

There is a SIIS problem that has always been bad coding practice and compilers have not generated this style of code for decades. I found one of our old assembler routines did indeed do a STC *+5,blah style instruction... not RENT and kills the cache, if in a loop a real disaster for performance...
Just making old assembler RENT, and a small amount of recoding, is enough to find and then eliminate this problem. Another thing is to make sure assembler local storage is more than 255 bytes away from the last instruction. Generous use of ORG ,256 is your friend here!

DB2 is innocent for this spike in cpu!

Roy Boxwell
SOFTWARE ENGINEERING GmbH and SEGUS Inc.
-Product Development-
Heinrichstrasse 83-85
40239 D?sseldorf/Germany
Tel. +49 (0)211 96149-675
Fax +49 (0)211 96149-32
Email: [login to unmask email]<mailto:[login to unmask email]><mailto:[login to unmask email]>
http://www.seg.de

Software Engineering GmbH
Amtsgericht D?sseldorf, HRB 37894
Gesch?ftsf?hrung: Gerhard Schubert, Bettina Schubert

On 23 Dec 2016, at 21:55, Terry Purcell mailto:[login to unmask email]>> wrote:


There are 2 earlier posts from the original customer who stated:
1st - "We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs."
2nd - "Everything is the same except for: A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.".

That implied to me that the problem was mostly occurring in DB2 10 subsystems, and again - mostly in batch jobs. Which is why in my earlier response I pointed out that the DB2 10 optimizer is more sensitive to CPU upgrades than DB2 11.

This discussion has morphed into a general problem with regard to z13 performance that was "confirmed" (???) at Guide/Share in France.

If we (IBM) had a pervasive problem with the z13 - then I expect all of our hardware folks to know about it. I have reached out to a couple of Distinguished Engineers in the zSystems HW team - and heard back from one who was very skeptical. He also sought more information from his peers. Due to the holiday period - many people are out - so I only have the one response.

I do however agree that mileage may vary.

But, given the information from the earlier posts - I would first want to rule out an access path change in DB2 10 for the impacted batch jobs.

Regards
Terry Purcell

In Reply to Venkat Srinivasan:

You can be very "in the face of" but the caveat that z13 performance variability depends on workload is mentioned in almost all marketing materials and presentations. Defective microcode may be a loose term someone may have let out.

z14 or znext may have its own variability too.

With cp upgrades some sites experience pain while others experience just the opposite.


Venkat

In Reply to Lizette Koehler:

Just an FYI.



IBM has not announced (as far as I know) the z14 processor, so I am suspicious they would say that.



If it is microcode issue, then the engineers should be able to figure out how to patch it. It just may take some time and they are providing comments to delay a customer from getting impatient.



If it were me, I would be very "in the face" of my IBM Account/Sales team for a problem like this.



Lizette





From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email]<mailto:[login to unmask email]><mailto:[login to unmask email]> <mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase



I can confirm that we have this problem with z13

I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.

IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".



Duc






-----End Original Message-----

-----End Original Message-----

Oh, and Merry Christmas one and all!

Roy Boxwell
SOFTWARE ENGINEERING GmbH and SEGUS Inc.
-Product Development-
Heinrichstrasse 83-85
40239 D?sseldorf/Germany
Tel. +49 (0)211 96149-675
Fax +49 (0)211 96149-32
Email: [login to unmask email]<mailto:[login to unmask email]>
http://www.seg.de

Software Engineering GmbH
Amtsgericht D?sseldorf, HRB 37894
Gesch?ftsf?hrung: Gerhard Schubert, Bettina Schubert

On 24 Dec 2016, at 16:05, Venkat Srinivasan <[login to unmask email]<mailto:[login to unmask email]>> wrote:


In assembler code Making the csect as RSECT will expose the issue early as some service macros can expand using inlined code. EXecute instruction is another prevalent killer and likely widespread. SIIS has always been an issue but the z13 out of order stream running ahead of the executing code may be making it more vulnerable.

Now that you mention about this, can a garden variety cobol compiler code cause these types of vulnerabilities without RENT option? COBOL compiler doesn't seem to be an optimizing compiler like metal C in general. How many sites attempt a recompile in a newer machine setting the right arch level etc in compiler option etc?

Venkat

In Reply to Roy Boxwell:

There is a SIIS problem that has always been bad coding practice and compilers have not generated this style of code for decades. I found one of our old assembler routines did indeed do a STC *+5,blah style instruction... not RENT and kills the cache, if in a loop a real disaster for performance...
Just making old assembler RENT, and a small amount of recoding, is enough to find and then eliminate this problem. Another thing is to make sure assembler local storage is more than 255 bytes away from the last instruction. Generous use of ORG ,256 is your friend here!

DB2 is innocent for this spike in cpu!

Roy Boxwell
SOFTWARE ENGINEERING GmbH and SEGUS Inc.
-Product Development-
Heinrichstrasse 83-85
40239 D?sseldorf/Germany
Tel. +49 (0)211 96149-675
Fax +49 (0)211 96149-32
Email: [login to unmask email]<mailto:[login to unmask email]><mailto:[login to unmask email]>
http://www.seg.de

Software Engineering GmbH
Amtsgericht D?sseldorf, HRB 37894
Gesch?ftsf?hrung: Gerhard Schubert, Bettina Schubert

On 23 Dec 2016, at 21:55, Terry Purcell mailto:[login to unmask email]>> wrote:


There are 2 earlier posts from the original customer who stated:
1st - "We have noticed more DB2 CPU consumption ever since we moved to a Z13; especially DB2 batch jobs."
2nd - "Everything is the same except for: A few DB2 subsystems have been upgraded to DB2 v11, but not the one most affected in more CPU consumption.".

That implied to me that the problem was mostly occurring in DB2 10 subsystems, and again - mostly in batch jobs. Which is why in my earlier response I pointed out that the DB2 10 optimizer is more sensitive to CPU upgrades than DB2 11.

This discussion has morphed into a general problem with regard to z13 performance that was "confirmed" (???) at Guide/Share in France.

If we (IBM) had a pervasive problem with the z13 - then I expect all of our hardware folks to know about it. I have reached out to a couple of Distinguished Engineers in the zSystems HW team - and heard back from one who was very skeptical. He also sought more information from his peers. Due to the holiday period - many people are out - so I only have the one response.

I do however agree that mileage may vary.

But, given the information from the earlier posts - I would first want to rule out an access path change in DB2 10 for the impacted batch jobs.

Regards
Terry Purcell

In Reply to Venkat Srinivasan:

You can be very "in the face of" but the caveat that z13 performance variability depends on workload is mentioned in almost all marketing materials and presentations. Defective microcode may be a loose term someone may have let out.

z14 or znext may have its own variability too.

With cp upgrades some sites experience pain while others experience just the opposite.


Venkat

In Reply to Lizette Koehler:

Just an FYI.



IBM has not announced (as far as I know) the z14 processor, so I am suspicious they would say that.



If it is microcode issue, then the engineers should be able to figure out how to patch it. It just may take some time and they are providing comments to delay a customer from getting impatient.



If it were me, I would be very "in the face" of my IBM Account/Sales team for a problem like this.



Lizette





From: ducky [mailto:[login to unmask email]
Sent: Tuesday, December 20, 2016 5:25 AM
To: [login to unmask email]<mailto:[login to unmask email]><mailto:[login to unmask email]> <mailto:[login to unmask email]>
Subject: [DB2-L] - RE: Z13 CPU Usage Increase



I can confirm that we have this problem with z13

I don't know what it is due exactly, but from my z/os sysprog colleagues, they said that this is a hardware problem the microcode is less efficient.

IBM answer : Working as designed, wait Z14
You IBM vendor should be aware of this and negociate your cpu consumption bill in consequence.

This problem has been confirmed also with others z13 sites here in France, as it was one of the theme of the recent "Z/OS Guide Share France".



Duc






-----End Original Message-----

-----End Original Message-----

I am asking my z/os team some reference of their exchanges with IBM guys on
the Z13 problem here in our shop.
Hope i can get back with more details.

Regards
Duc


On Sat, Dec 24, 2016 at 11:06 PM, Boxwell, Roy <[login to unmask email]> wrote:

> Oh, and Merry Christmas one and all!
>
> Roy Boxwell
> SOFTWARE ENGINEERING GmbH and SEGUS Inc.
> -Product Development-
> Heinrichstrasse 83-85
> 40239 Düsseldorf/Germany
> Tel. +49 (0)211 96149-675 <+49%20211%2096149675>
> Fax +49 (0)211 96149-32 <+49%20211%209614932>
> Email: [login to unmask email]
> http://www.seg.de
>
> Software Engineering GmbH
> Amtsgericht Düsseldorf, HRB 37894
> Geschäftsführung: Gerhard Schubert, Bettina Schubert
>
> On 24 Dec 2016, at 16:05, Venkat Srinivasan <[login to unmask email]> wrote:
>
> In assembler code Making the csect as RSECT will expose the issue early as
> some service macros can expand using inlined code. EXecute instruction is
> another prevalent killer and likely widespread. SIIS has always been an
> issue but the z13 out of order stream running ahead of the executing code
> may be making it more vulnerable.
>
> Now that you mention about this, can a garden variety cobol compiler code
> cause these types of vulnerabilities without RENT option? COBOL compiler
> doesn't seem to be an optimizing compiler like metal C in general. How many
> sites attempt a recompile in a newer machine setting the right arch level
> etc in compiler option etc?
>
> Venkat
>
> In Reply to Roy Boxwell:
>
> There is a SIIS problem that has always been bad coding practice and
> compilers have not generated this style of code for decades. I found one of
> our old assembler routines did indeed do a STC *+5,blah style
> instruction... not RENT and kills the cache, if in a loop a real disaster
> for performance...
> Just making old assembler RENT, and a small amount of recoding, is enough
> to find and then eliminate this problem. Another thing is to make sure
> assembler local storage is more than 255 bytes away from the last
> instruction. Generous use of ORG ,256 is your friend here!
>
> DB2 is innocent for this spike in cpu!
>
> Roy Boxwell
> SOFTWARE ENGINEERING GmbH and SEGUS Inc.
> -Product Development-
> Heinrichstrasse 83-85
> 40239 D?sseldorf/Germany
> Tel. +49 (0)211 96149-675 <+49%20211%2096149675>
> Fax +49 (0)211 96149-32 <+49%20211%209614932>
> Email: [login to unmask email]<mailto:[login to unmask email]> <[login to unmask email]%3E>
> http://www.seg.de
>
> Software Engineering GmbH
> Amtsgericht D?sseldorf, HRB 37894
> Gesch?ftsf?hrung: Gerhard Schubert, Bettina Schubert
>
> On 23 Dec 2016, at 21:55, Terry Purcell mailto:[login to unmask email]
> <[login to unmask email]>>> wrote:
>
>
> There are 2 earlier posts from the original customer who stated:
> 1st - "We have noticed more DB2 CPU consumption ever since we moved to a
> Z13; especially DB2 batch jobs."
> 2nd - "Everything is the same except for: A few DB2 subsystems have been
> upgraded to DB2 v11, but not the one most affected in more CPU
> consumption.".
>
> That implied to me that the problem was mostly occurring in DB2 10
> subsystems, and again - mostly in batch jobs. Which is why in my earlier
> response I pointed out that the DB2 10 optimizer is more sensitive to CPU
> upgrades than DB2 11.
>
> This discussion has morphed into a general problem with regard to z13
> performance that was "confirmed" (???) at Guide/Share in France.
>
> If we (IBM) had a pervasive problem with the z13 - then I expect all of
> our hardware folks to know about it. I have reached out to a couple of
> Distinguished Engineers in the zSystems HW team - and heard back from one
> who was very skeptical. He also sought more information from his peers. Due
> to the holiday period - many people are out - so I only have the one
> response.
>
> I do however agree that mileage may vary.
>
> But, given the information from the earlier posts - I would first want to
> rule out an access path change in DB2 10 for the impacted batch jobs.
>
> Regards
> Terry Purcell
>
> In Reply to Venkat Srinivasan:
>
> You can be very "in the face of" but the caveat that z13 performance
> variability depends on workload is mentioned in almost all marketing
> materials and presentations. Defective microcode may be a loose term
> someone may have let out.
>
> z14 or znext may have its own variability too.
>
> With cp upgrades some sites experience pain while others experience just
> the opposite.
>
>
> Venkat
>
> In Reply to Lizette Koehler:
>
> Just an FYI.
>
>
>
> IBM has not announced (as far as I know) the z14 processor, so I am
> suspicious they would say that.
>
>
>
> If it is microcode issue, then the engineers should be able to figure out
> how to patch it. It just may take some time and they are providing comments
> to delay a customer from getting impatient.
>
>
>
> If it were me, I would be very "in the face" of my IBM Account/Sales team
> for a problem like this.
>
>
>
> Lizette
>
>
>
>
>
> From: ducky [mailto:[login to unmask email] <[login to unmask email]>
> Sent: Tuesday, December 20, 2016 5:25 AM
> To: [login to unmask email]<mailto:[login to unmask email]>
> <[login to unmask email]%3E> <mailto:[login to unmask email]>
> <[login to unmask email]%3E>
> Subject: [DB2-L] - RE: Z13 CPU Usage Increase
>
>
>
> I can confirm that we have this problem with z13
>
> I don't know what it is due exactly, but from my z/os sysprog colleagues,
> they said that this is a hardware problem the microcode is less efficient.
>
> IBM answer : Working as designed, wait Z14
> You IBM vendor should be aware of this and negociate your cpu consumption
> bill in consequence.
>
> This problem has been confirmed also with others z13 sites here in France,
> as it was one of the theme of the recent "Z/OS Guide Share France".
>
>
>
> Duc
>
>
>
>
>
>
> -----End Original Message-----
>
>
> -----End Original Message-----
>
>
> -----End Original Message-----
>

An update from my discussions with our (IBM's) zSystems hardware team. I can confirm that there is no pervasive performance problems with the z13 based upon customer feedback. There have been a small number of issues (as with any HW or SW release) - all have been identified and understood up to this point. SiiS has been identified at a handful of customers, but there are solutions.

Our HW team recommended that the original customer (Brett Sinclair) open a PMR such that the HW team can diagnose and (hopefully) address the issue. Similarly for any other customer who has an issue that has gone unresolved. Without diagnosis, it is not possible to conclude that any of the issues discussed in this thread are the same problem.

Regards

Terry Purcell

Representing IBM in this response.

Thank you.







The most important thing is to keep the most important thing the most important thing

Brett Sinclair
AXA Technology Services
Telephone: 1-908-204-0144

From: Terry Purcell [mailto:[login to unmask email]
Sent: Wednesday, January 04, 2017 5:40 PM
To: [login to unmask email]
Subject: [DB2-L] - RE: Z13 CPU Usage Increase


An update from my discussions with our (IBM's) zSystems hardware team. I can confirm that there is no pervasive performance problems with the z13 based upon customer feedback. There have been a small number of issues (as with any HW or SW release) - all have been identified and understood up to this point. SiiS has been identified at a handful of customers, but there are solutions.

Our HW team recommended that the original customer (Brett Sinclair) open a PMR such that the HW team can diagnose and (hopefully) address the issue. Similarly for any other customer who has an issue that has gone unresolved. Without diagnosis, it is not possible to conclude that any of the issues discussed in this thread are the same problem.

Regards

Terry Purcell

Representing IBM in this response.

-----End Original Message-----


Ce message est confidentiel; Son contenu ne represente en aucun cas
un engagement de la part de AXA Technology Services (AXA Tech) sous
reserve de tout accord conclu par ecrit entre vous et AXA Technology
Services (AXA Tech).Toute publication, utilisation ou diffusion,meme
partielle, doit etre autorisee prealablement. Si vous n'etes pas
destinataire de ce message, merci d'en avertir immediatement l'expe-
diteur.

This message is confidential; its contents do not constitute a
commitment by AXA Technology Services (AXA Tech) except where provi-
ded for in a written agreement between you and AXA Technology
Services (AXA Tech). Any unauthorised disclosure, use or dissemina-
tion, either whole or partial, is prohibited. If you are not the
intended recipient of the message, please notify the sender imme-
diately.