DB2 z/OS Reduplexing of Group Buffer Pools

Steven Lamb

DB2 z/OS Reduplexing of Group Buffer Pools

In a DB2 DSG used for an online banking system we have experienced transaction delays in the past when all the GBPs have been automatically re-duplexed following an event such as an IPL of one of the Coupling Facility LPARs. This sysplex also runs a few non--production systems as well, using smaller GBPs.

 

The z/OS team's response to this was to leave the non-prod system GBPs as automatically re-duplexed i.e. defined as DUPLEX ENABLED, with the prod GBP structures not automatically duplexed. being defined as DUPLEX ALLOWED.

Staggered manual commands are run to duplex the production GBP structures following CF LPAR outages.

Overnight we had one of the production GBPs deallocated due to it not being used. A few hours later it was allocated again, but only in simplex mode. An alert was raised for this situation about two hours later, but wasn't handled correctly and so things weren't resolved until about 8 hours after the GBP was first re-allocated. The Ops don't want "false alerts" that would be created during a normal CF LPAR outage, which means that the checks for simplex operation only create an alert after 2 hours, rather than immediately.

 

This approach seems intrinsically wrong to me, but they (the z/OS guys) say that this is the only way to not overload the CF structures and cause transaction delays. Does anybody else have the same sort of situation and how do you handle it?

 

Regards,

Steve

Steven Lamb

RE: DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Steven Lamb)

Nobody using GBP duplexing? !

Olle Brostrom

DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Steven Lamb)
I believe most user do duplexing of Db2’s cache structures!

Best Regards

Olle Broström

From: Steven Lamb [mailto:[login to unmask email]
Sent: den 26 september 2017 08:52
To: [login to unmask email]
Subject: [DB2-L] - RE: DB2 z/OS Reduplexing of Group Buffer Pools


Nobody using GBP duplexing? !

-----End Original Message-----

Raymond Bell

DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Steven Lamb)
Yeah, we are, but it’s in the hands of the Sprogs so I’m not involved. Although I do know a site here in the UK that doesn’t believe it’s important. Suffice to say myself and a former colleague had a different opinion to the ‘we’ve always done it that way’ crowd.

Not being up on CF structures, I’d have thought it better to get Prod up, running and resilient as fast as possible (DUPLEX ENABLED) and non-prod can join the back of the queue once your revenue generator is back. Until the CF is duplexed, you have a single point of failure in your Prod infrastructure. That might be enough to persuade your Sprogs to do it the other way around – or just ENABLE the lot.

Here’s hoping Swindon has better weather today than Reigate. But I doubt it. :o(

Cheers,


Raymond

Raymond Bell
DB2 Database Administrator | IT Operations | Technology | RBS

From: Steven Lamb [mailto:[login to unmask email]
Sent: 26 September 2017 07:52
To: [login to unmask email]
Subject: [DB2-L] - RE: DB2 z/OS Reduplexing of Group Buffer Pools


*********************************************
" This message originates from outside our organisation. Consider carefully whether you should click on any links, open any attachments or reply. If in doubt, forward to ~ Phishing"
*********************************************


Nobody using GBP duplexing? !

-----End Original Message-----
The Royal Bank of Scotland plc. Registered in Scotland No 90312. Registered Office: 36 St Andrew Square, Edinburgh EH2 2YB. The Royal Bank of Scotland is authorised by the Prudential Regulation Authority, and regulated by the Financial Conduct Authority and Prudential Regulation Authority. The Royal Bank of Scotland N.V. is authorised and regulated by the De Nederlandsche Bank and has its seat at Amsterdam, the Netherlands, and is registered in the Commercial Register under number 33002587. Registered Office: Gustav Mahlerlaan 350, Amsterdam, The Netherlands. The Royal Bank of Scotland N.V. and The Royal Bank of Scotland plc are authorised to act as agent for each other in certain jurisdictions.

National Westminster Bank Plc. Registered in England No. 929027. Registered Office: 135 Bishopsgate, London EC2M 3UR. National Westminster Bank Plc is authorised by the Prudential Regulation Authority, and regulated by the Financial Conduct Authority and the Prudential Regulation Authority.

The Royal Bank of Scotland plc and National Westminster Bank Plc are authorised to act as agent for each other.

This e-mail message is confidential and for use by the addressee only. If the message is received by anyone other than the addressee, please return the message to the sender by replying to it and then delete the message from your computer. Internet e-mails are not necessarily secure. The Royal Bank of Scotland plc, The Royal Bank of Scotland N.V., National Westminster Bank Plc or any affiliated entity (“RBS” or “us”) does not accept responsibility for changes made to this message after it was sent. RBS may monitor e-mails for business and operational purposes. By replying to this message you give your consent to the monitoring of your e-mail communications with us.

Whilst all reasonable care has been taken to avoid the transmission of viruses, it is the responsibility of the recipient to ensure that the onward transmission, opening or use of this message and any attachments will not adversely affect its systems or data. No responsibility is accepted by RBS in this regard and the recipient should carry out such virus and other checks as it considers appropriate.

Visit our website at www.rbs.com http://www.rbs.com

Chad Walmer

DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Steven Lamb)
We switched to using maintenance mode and the REALLOCATE command to avoid the “flooding” of the CF a year or two back. I’m not sure when the commands were first introduced but we had been doing it manually as well by building the commands in advance and then running each with a slight delay (bit of a pain and various situations caused issues for us over the years.)

High level overview of the steps:


1. Put the CF in maintenance mode:
SETXCF START,MAINTMODE,CFNAME=cfname

2. Allow the system to move the structures one at a time (there’s even a command to “TEST” what it will do if you want to be cautious):

SETXCF START,REALLOCATE[,TEST]

3. POR or IPL the CF LPAR

4. Take the CF out of maintenance mode:

SETXCF START,MAINTMODE,CFNAME=cfname

5. Allow the system to move the structures back to their optimal location:

SETXCF START,REALLOCATE[,TEST]

The recommended procedure for doing CF maintenance can be found here and this is what we based our process on:

https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieaf100/bestprac.htm

Allowing the system to do the work for us took some time to get comfortable with as we typically like to be in control of these types of processes. In this case though, IBM did a good job of automating a fairly common task among z/OS shops and it has worked well for us. Good luck convincing your system programmers as they can be a grumpy lot and don’t like change (I am one so I can say that -- both grumpy and a system programmer ☺ ).

Chad Walmer

From: Steven Lamb [mailto:[login to unmask email]
Sent: Tuesday, September 26, 2017 2:52 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: DB2 z/OS Reduplexing of Group Buffer Pools


Nobody using GBP duplexing? !

-----End Original Message-----

Steven Lamb

RE: DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Chad Walmer)

Thanks for the replies and for the link Chad.

I think "we" (i.e. the sysprogs) use commands such as SETXCF START,RB,DUPLEX,STRNM=BP00_GBP6, rather than REALLOCATE. I don't know what the difference is, so I've got some reading to do :).

 

We were seeing delays of around 16 - 17 seconds when all the structures on the sysplex were set up with DUPLEX ENABLED and I've been making the prod structures quite a bit bigger since then, so things could potentially be worse now if everything went back to DUPLEX ENABLED.

We've had a suggestion from John Campbell that we could lower the DB2 GBP thresholds before taking a CF LPAR down, to reduce the amount of changed data in them (i.e. GBPCHKPT, CLASST and GBPOOLT) and then reverting back to normal once the CF LPAR is back. This should hopefully reduce the time when the GBPs are frozen until duplexing is re-established.

 

Regards,

Steve

 

PS It's not been too bad here in Swindon today - had some sun; somebody must have arranged it for my birthday :)

Steven Lamb

RE: DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Chad Walmer)

Chad - are your GBP structures set up as DUPLEX ENABLED or ALLOWED?

I've lit the fuse now and our sysprogs are getting curious about this new-fangled method of doing things :)

 

Regards,

Steve

Chad Walmer

DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Steven Lamb)
We are using DUPLEX ENABLED for the GBPs. It does take a couple of times running through the process to get comfortable with it.

From: Steven Lamb [mailto:[login to unmask email]
Sent: Wednesday, September 27, 2017 7:26 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: DB2 z/OS Reduplexing of Group Buffer Pools


Chad - are your GBP structures set up as DUPLEX ENABLED or ALLOWED?

I've lit the fuse now and our sysprogs are getting curious about this new-fangled method of doing things :)



Regards,

Steve

-----End Original Message-----

Steven Lamb

RE: DB2 z/OS Reduplexing of Group Buffer Pools
(in response to Chad Walmer)

Thanks Chad - our z/OS sysprogs are looking into this and might be about to change things! :)

 

Regards,

Steve