[DB2-L] EDM POOL FULL condition after migrating to V8 NFM : more data

Georg Peter

[DB2-L] EDM POOL FULL condition after migrating to V8 NFM : more data
Renzo,



the EDM pool is used for caching internal structures used by DB2 programs. This includes DBDs, SKCTs, CTs, SKPTs, and PTs. It also includes the authorization cache for plans and packages, as well as the cache for dynamic SQL mini-plans.



With V8, DB2 breaks the EDM pool into separate pools: one for DBDs, one for the dynamic statement cache, and one for program elements (i.e., CTs, SKCTs, PTs, SKPTs). As a general rule of thumb, shoot for an 80 percent hit rate with the EDM pool; this means only one out every five times should a structure need to be loaded from disk into the EDM pool. Finally, remember that buffer and EDM pool tuning are in-depth subjects that can't be adequately covered in a single page column such as this. So, study those IBM DB2 manuals-and learn by doing.



(Source: http://www.zjournal.com/index.cfm?section=article&aid=392)

HTH.

With kind regards - mit freundlichen Gruessen,
G e o r g H . P e t e r
-------------------------------------------------------------------
Datenzentrale Baden-Wuerttemberg
Development and Product Support (E3)
Krailenshaldenstrasse 44, 70469 Stuttgart, Germany, Europe
e:mail [login to unmask email]
Phone 0049-711-8108-271
PC-Fax 004971189696071
Internet (only in german language):http://www.dzbw.de < http://www.dzbw.de/ >
----------------------------------------------------------------------
This e-mail is environment friendly and was made only from recycled electrons.

-----Ursprüngliche Nachricht-----
Von: DB2 Data Base Discussion List [mailto:[login to unmask email] Im Auftrag von Renzo razzetti
Gesendet: Donnerstag, 8. Januar 2009 08:51
An: [login to unmask email]
Betreff: [DB2-L] EDM POOL FULL condition after migrating to V8 NFM : more data


Hello,

I m adding more data in my question to see whether there is someone in the list can help me.
Following is the statistic from 1 minute interval. The EDM Pool is always around 40% used, but suddenly the PAGES of PT increase almost to 97% of the EDM Pool.
I want to find the root cause of the problem, to avoid happen other time. I'm already checked the number of transaction in CICS and the number of thread in DB2 and they are not increase.

Could be this a bug in the EDM pool DB2 v8 ?

I will appreciate any suggestion to further analysis this problem.

RR




EDM POOL DP21 16:20 16:21 16:22 16:21
16:21 16:22 16:23 16:27
Elapsed time
1 1 1 6
PAGES IN EDM POOL (BELOW) 37500 37500 37500 37500
HELD BY CT 176 290.42 588 234.67
HELD BY PT 4936 13113.24 34380 8780.87
HELD BY SKCT 9 8.17 6 10.17
HELD BY SKPT 11415 8716.93 1700 7322.49
FREE PAGES 20964 15371.24 826 21151.81
% PAGES IN USE 44.1 59.01 97.8 43.6
% NON STEAL. PAGES IN USE 13.63 35.74 93.25 24.04
FAILS DUE TO POOL FULL 0 0 0 66
































































On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <[login to unmask email]> wrote:
>
> hello everybody !
>
> I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing enviorment.
> Not dynamic application are running, not distributed application is running, not storage procedure is running.
>
>
>
> I have received this error in production. 66 times in the 1 second interval. And the application canceled with RC -904 resource unavailable.
>
> 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
> 004 REASON 00C90089
> 004 TYPE 00000600
> 004 NAME EDM POOL SPACE
> ...................................
> 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
> 013 REASON 00C90089
> 013 TYPE 00000600
>
>
> If we look in the statistic report the EDM Pool for 15 interval during 16:15 to 16:30, we can see only the 40% of the pages are in use. It looks the EDM Pool is not full.
> Can any help to understand what is the problem ? How I can find the problem ?
> Thank you.
>
>
> EDM POOL QUANTITY /SECOND /THREAD /COMMIT
> --------------------------- -------- ------- ------- -------
> PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
> HELD BY CT 208.86 N/A N/A N/A
> HELD BY PT 6682.08 N/A N/A N/A
> HELD BY SKCT 9.79 N/A N/A N/A
> HELD BY SKPT 9193.57 N/A N/A N/A
> FREE PAGES 21405.71 N/A N/A N/A
> % PAGES IN USE 42.92 N/A N/A N/A
> % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
> FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>
> PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
> HELD BY DBD 7632.00 N/A N/A N/A
> FREE PAGES 12368.00 N/A N/A N/A
> FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>
> PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
> HELD BY STATEMENTS 18916.21 N/A N/A N/A
> FREE PAGES 1083.79 N/A N/A N/A
> FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>
> DBD REQUESTS 70916.00 83.29 1.01 0.38
> DBD NOT FOUND 0.00 0.00 0.00 0.00
> DBD HIT RATIO (%) 100.00 N/A N/A N/A
> CT REQUESTS 70178.00 82.42 1.00 0.38
> CT NOT FOUND 11.00 0.01 0.00 0.00
> CT HIT RATIO (%) 99.98 N/A N/A N/A
> PT REQUESTS 3385.0K 3975.56 48.22 18.18
> PT NOT FOUND 3592.00 4.22 0.05 0.02
> PT HIT RATIO (%) 99.89 N/A N/A N/A
>
> PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
> PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
> PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>
>



________________________________


IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia < http://idug.org/lsAU >

IDUG.org <http://www.idug.org> was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >


Abonnieren Sie den monatlichen Infobrief der Datenzentrale Baden-Württemberg und erfahren Sie regelmäßig die neuesten Nachrichten über aktuelle Projekte und Entwicklungen. Melden Sie sich an mit diesem Link http://www.datenzentrale.de/Info-Brief
_______________________________________________________________________________

Datenzentrale Baden-Württemberg, Anstalt des öffentlichen Rechts
Krailenshaldenstr. 44, 70469 Stuttgart
Telefon (0711) 8108-0, Telefax (0711) 8108-350
E-Mail [login to unmask email], Internet www.datenzentrale.de
Vorstand: Karl Tramer (Vors.) und Harald Schätzle, Vorsitzender des Verwaltungsrats: Gunter Czisch
USt-Id-Nr. DE147794223
_______________________________________________________________________________



______________________________________________________________________

* IDUG 2009 Melbourne, Australia * 18-20 March * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Adam Baldwin

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Georg Peter)
Hi Renzo - just a quick thought - have you checked what was running in the
interval when you had problems? You say that the number of threads/CICS
transactions was fairly constant but that may well mask the fact that there
were some particulary hefty packages being loaded.

From you first post I see that you're not running Stored Procedures so the
problem of recursive procedure calls can be ruled out.

Are the failures that you're getting definitely package loads? Have you
checked your RID pool stats?

If you've determined that the problem is with package loads, have you looked
at the sizes of the packages being run?

Cheers, Adam

______________________________________________________________________

* IDUG 2009 Melbourne, Australia * 18-20 March * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Robert Catterall

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Adam Baldwin)
Renzo,

Is your DB2 workload (during the "EDM pool full" times) primarily comprised
of CICS-DB2 transactions? If so, are many of the associated DB2 packages
bound with RELEASE(DEALLOCATE)? Are you using any protected CICS-DB2
threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource
definitions)?

Historically, a lot of DB2 for z/OS data sharing users went with
RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock
propagation to the lock structure. In CICS-DB2 environments, tablespace
lock propagation could be further reduced through the use of CICS protected
entry threads. People were interested in reducing tablespace lock
propagation because such requests often resulted in what's called XES
contention - something that increased data sharing overhead. An
unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected
threads were also used - was an increase (sometimes quite substantial) in
the usage of EDM pool space for sections of in-use packages (the "PT" part
of the pool) - this because a package bound with RELEASE(DEALLOCATE) is
considered to be "in-use" until the associated thread is deallocated (as
opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In
some cases, even minor fluctuations in the CICS-DB2 transaction workload can
lead to significant chnages in the rate of CICS-DB2 thread reuse, and that
(when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic
shifts in "in-use" versus "stealable" PT-related EDM pool pages.

The good news with respect to DB2 V8: due to a new global locking protocol,
RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of the type
of global lock contention known as XES contention (I wrote about this in a
blog entry last summer:
http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html).
So, if you have a fair number of CICS-DB2 program packages bound with
RELEASE(DEALLOCATE), and if that bind specification decision was made so as
to reduce data sharing lock contention, consider changing the bind
specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global
lock contention penalty for doing this, thanks to the afrementioned global
locking protocol enhancement (known as "locking protocol 2," as I recall).

This may or may not explain what you are seeing. Something for you to
consider and investigate.

Robert

On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <[login to unmask email]>wrote:

> Hello,
>
> I m adding more data in my question to see whether there is someone in
> the list can help me.
> Following is the statistic from 1 minute interval. The EDM Pool is always
> around 40% used, but suddenly the PAGES of PT increase almost to 97% of the
> EDM Pool.
> I want to find the root cause of the problem, to avoid happen other time.
> I'm already checked the number of transaction in CICS and the number of
> thread in DB2 and they are not increase.
>
> Could be this a bug in the EDM pool DB2 v8 ?
>
> I will appreciate any suggestion to further analysis this problem.
>
> RR
>
>
>
> EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23 16:27 Elapsed
> time
> 1 1 1 6 PAGES IN EDM POOL (BELOW)
> 37500 37500 37500 37500 HELD BY CT 176 290.42 588
> 234.67 HELD BY PT 4936 13113.24 34380 8780.87 HELD
> BY SKCT 9 8.17 6 10.17 HELD BY SKPT 11415
> 8716.93 1700 7322.49 FREE PAGES 20964 15371.24 826
> 21151.81 % PAGES IN USE 44.1 59.01 97.8 43.6 % NON
> STEAL. PAGES IN USE 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL
> 0 0 0 66
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <[login to unmask email]>
> wrote:
> >
> > hello everybody !
> >
> > I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing
> enviorment.
> > Not dynamic application are running, not distributed application is
> running, not storage procedure is running.
> >
> >
> >
> > I have received this error in production. 66 times in the 1 second
> interval. And the application canceled with RC -904 resource unavailable.
> >
> > 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
> > 004 REASON 00C90089
> > 004 TYPE 00000600
> > 004 NAME EDM POOL SPACE
> > ...................................
> > 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
> > 013 REASON 00C90089
> > 013 TYPE 00000600
> >
> >
> > If we look in the statistic report the EDM Pool for 15 interval during
> 16:15 to 16:30, we can see only the 40% of the pages are in use. It looks
> the EDM Pool is not full.
> > Can any help to understand what is the problem ? How I can find the
> problem ?
> > Thank you.
> >
> >
> > EDM POOL QUANTITY /SECOND /THREAD /COMMIT
> > --------------------------- -------- ------- ------- -------
> > PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
> > HELD BY CT 208.86 N/A N/A N/A
> > HELD BY PT 6682.08 N/A N/A N/A
> > HELD BY SKCT 9.79 N/A N/A N/A
> > HELD BY SKPT 9193.57 N/A N/A N/A
> > FREE PAGES 21405.71 N/A N/A N/A
> > % PAGES IN USE 42.92 N/A N/A N/A
> > % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
> > FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
> >
> > PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
> > HELD BY DBD 7632.00 N/A N/A N/A
> > FREE PAGES 12368.00 N/A N/A N/A
> > FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
> >
> > PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
> > HELD BY STATEMENTS 18916.21 N/A N/A N/A
> > FREE PAGES 1083.79 N/A N/A N/A
> > FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
> >
> > DBD REQUESTS 70916.00 83.29 1.01 0.38
> > DBD NOT FOUND 0.00 0.00 0.00 0.00
> > DBD HIT RATIO (%) 100.00 N/A N/A N/A
> > CT REQUESTS 70178.00 82.42 1.00 0.38
> > CT NOT FOUND 11.00 0.01 0.00 0.00
> > CT HIT RATIO (%) 99.98 N/A N/A N/A
> > PT REQUESTS 3385.0K 3975.56 48.22 18.18
> > PT NOT FOUND 3592.00 4.22 0.05 0.02
> > PT HIT RATIO (%) 99.89 N/A N/A N/A
> >
> > PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
> > PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
> > PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
> >
> >
>
>
> ------------------------------
>
> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>
> *IDUG.org* < http://www.idug.org > was recently updated requiring members to
> use a new password. You should have gotten an e-mail with the temporary
> password assigned to your account. Please log in and update your member
> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>



--
Robert Catterall
Catterall Consulting
www.catterallconsulting.com

______________________________________________________________________

* IDUG 2009 Melbourne, Australia * 18-20 March * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Fred Edgar

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Robert Catterall)
I think Robert is probably on to something. Our EDM pool is about the same size as yours,
but the percentage of pages used for things is very different. We also have a tool that lets
us drill in and see what's using those pages which is handy, but you have to catch it while it's happening. And our hit ratios are better in the high 90's for all but the SQL Cache,
and we're working on that one.

Percent of pool pages
DBDs 5.8%
CTs .3%
PTs 3.4%
SKCTs .0%
CACHE .0%
SKPTs 54.5%
SQL CACHE 19.8%
FREE 9.1%

Fred

Roberrt Catterall <[login to unmask email]> wrote:
Renzo,

Is your DB2 workload (during the "EDM pool full" times) primarily comprised of CICS-DB2 transactions? If so, are many of the associated DB2 packages bound with RELEASE(DEALLOCATE)? Are you using any protected CICS-DB2 threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource definitions)?

Historically, a lot of DB2 for z/OS data sharing users went with RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock propagation to the lock structure. In CICS-DB2 environments, tablespace lock propagation could be further reduced through the use of CICS protected entry threads. People were interested in reducing tablespace lock propagation because such requests often resulted in what's called XES contention - something that increased data sharing overhead. An unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected threads were also used - was an increase (sometimes quite substantial) in the usage of EDM pool space for sections of in-use packages (the "PT" part of the pool) - this because a package bound with RELEASE(DEALLOCATE) is considered to be "in-use" until the associated thread is deallocated (as opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In some cases, even minor fluctuations in the CICS-DB2
transaction workload can lead to significant chnages in the rate of CICS-DB2 thread reuse, and that (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic shifts in "in-use" versus "stealable" PT-related EDM pool pages.

The good news with respect to DB2 V8: due to a new global locking protocol, RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of the type of global lock contention known as XES contention (I wrote about this in a blog entry last summer: http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html). So, if you have a fair number of CICS-DB2 program packages bound with RELEASE(DEALLOCATE), and if that bind specification decision was made so as to reduce data sharing lock contention, consider changing the bind specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global lock contention penalty for doing this, thanks to the afrementioned global locking protocol enhancement (known as "locking protocol 2," as I recall).

This may or may not explain what you are seeing. Something for you to consider and investigate.

Robert

On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <[login to unmask email]> wrote:
Hello,

I m adding more data in my question to see whether there is someone in the list can help me.
Following is the statistic from 1 minute interval. The EDM Pool is always around 40% used, but suddenly the PAGES of PT increase almost to 97% of the EDM Pool.
I want to find the root cause of the problem, to avoid happen other time. I'm already checked the number of transaction in CICS and the number of thread in DB2 and they are not increase.

Could be this a bug in the EDM pool DB2 v8 ?

I will appreciate any suggestion to further analysis this problem.

RR



EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23 16:27 Elapsed time
1 1 1 6 PAGES IN EDM POOL (BELOW) 37500 37500 37500 37500 HELD BY CT 176 290.42 588 234.67 HELD BY PT 4936 13113.24 34380 8780.87 HELD BY SKCT 9 8.17 6 10.17 HELD BY SKPT 11415 8716.93 1700 7322.49 FREE PAGES 20964 15371.24 826 21151.81 % PAGES IN USE 44.1 59.01 97.8 43.6 % NON STEAL. PAGES IN USE 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL 0 0 0 66































































On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <[login to unmask email]> wrote:
>
> hello everybody !
>
> I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing enviorment.
> Not dynamic application are running, not distributed application is running, not storage procedure is running.
>
>
>
> I have received this error in production. 66 times in the 1 second interval. And the application canceled with RC -904 resource unavailable.
>
> 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
> 004 REASON 00C90089
> 004 TYPE 00000600
> 004 NAME EDM POOL SPACE
> ...................................
> 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
> 013 REASON 00C90089
> 013 TYPE 00000600
>
>
> If we look in the statistic report the EDM Pool for 15 interval during 16:15 to 16:30, we can see only the 40% of the pages are in use. It looks the EDM Pool is not full.
> Can any help to understand what is the problem ? How I can find the problem ?
> Thank you.
>
>
> EDM POOL QUANTITY /SECOND /THREAD /COMMIT
> --------------------------- -------- ------- ------- -------
> PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
> HELD BY CT 208.86 N/A N/A N/A
> HELD BY PT 6682.08 N/A N/A N/A
> HELD BY SKCT 9.79 N/A N/A N/A
> HELD BY SKPT 9193.57 N/A N/A N/A
> FREE PAGES 21405.71 N/A N/A N/A
> % PAGES IN USE 42.92 N/A N/A N/A
> % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
> FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>
> PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
> HELD BY DBD 7632.00 N/A N/A N/A
> FREE PAGES 12368.00 N/A N/A N/A
> FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>
> PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
> HELD BY STATEMENTS 18916.21 N/A N/A N/A
> FREE PAGES 1083.79 N/A N/A N/A
> FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>
> DBD REQUESTS 70916.00 83.29 1.01 0.38
> DBD NOT FOUND 0.00 0.00 0.00 0.00
> DBD HIT RATIO (%) 100.00 N/A N/A N/A
> CT REQUESTS 70178.00 82.42 1.00 0.38
> CT NOT FOUND 11.00 0.01 0.00 0.00
> CT HIT RATIO (%) 99.98 N/A N/A N/A
> PT REQUESTS 3385.0K 3975.56 48.22 18.18
> PT NOT FOUND 3592.00 4.22 0.05 0.02
> PT HIT RATIO (%) 99.89 N/A N/A N/A
>
> PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
> PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
> PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>
>



---------------------------------
IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia
IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register here.




--
Robert Catterall
Catterall Consulting
www.catterallconsulting.com


---------------------------------
IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia
IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register here.



______________________________________________________________________

* IDUG 2009 Melbourne, Australia * 18-20 March * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Renzo Razzetti

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Fred Edgar)
Hello Robert

Ok, I have researched about yours suggestions. I'm continuing investigate.
Yes, The packages running at that time (peak hour) are bound with
RELEASE(DEALLOCATE) as we are in datasharing .

One possible cause of the problem is described in the IBM Technote
http://www-01.ibm.com/support/docview.wss?uid=swg21294759. In this technote
explain that Storage shortages can occur if too many cursors that return
result sets are open at the same time. To minimize these storage shortages,
subsystem parameters MAX_NUM_CUR controls the maximum number of cursors that
can be open concurrently. When the value from this subsystem parameter is
exceeded while an application is running, the CALL statement receives
SQLCODE -904.


I would like to know where I can confirm (in which Report) whether we hit
the MAX_NUM_CUR opened current in the thread.

I apprecciate very much your help

RR







In this technote explain that Storage shortages can occur if too many
cursors that return result sets are open at the same time. To minimize these
storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum
number of cursors that can be open concurrently.





On Thu, Jan 8, 2009 at 10:46 PM, Robert Catterall <[login to unmask email]>wrote:

> Renzo,
>
> Is your DB2 workload (during the "EDM pool full" times) primarily comprised
> of CICS-DB2 transactions? If so, are many of the associated DB2 packages
> bound with RELEASE(DEALLOCATE)? Are you using any protected CICS-DB2
> threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource
> definitions)?
>
> Historically, a lot of DB2 for z/OS data sharing users went with
> RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock
> propagation to the lock structure. In CICS-DB2 environments, tablespace
> lock propagation could be further reduced through the use of CICS protected
> entry threads. People were interested in reducing tablespace lock
> propagation because such requests often resulted in what's called XES
> contention - something that increased data sharing overhead. An
> unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected
> threads were also used - was an increase (sometimes quite substantial) in
> the usage of EDM pool space for sections of in-use packages (the "PT" part
> of the pool) - this because a package bound with RELEASE(DEALLOCATE) is
> considered to be "in-use" until the associated thread is deallocated (as
> opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In
> some cases, even minor fluctuations in the CICS-DB2 transaction workload can
> lead to significant chnages in the rate of CICS-DB2 thread reuse, and that
> (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic
> shifts in "in-use" versus "stealable" PT-related EDM pool pages.
>
> The good news with respect to DB2 V8: due to a new global locking protocol,
> RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of the type
> of global lock contention known as XES contention (I wrote about this in a
> blog entry last summer:
> http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html).
> So, if you have a fair number of CICS-DB2 program packages bound with
> RELEASE(DEALLOCATE), and if that bind specification decision was made so as
> to reduce data sharing lock contention, consider changing the bind
> specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global
> lock contention penalty for doing this, thanks to the afrementioned global
> locking protocol enhancement (known as "locking protocol 2," as I recall).
>
> This may or may not explain what you are seeing. Something for you to
> consider and investigate.
>
> Robert
>
> On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <[login to unmask email]>wrote:
>
>> Hello,
>>
>> I m adding more data in my question to see whether there is someone in
>> the list can help me.
>> Following is the statistic from 1 minute interval. The EDM Pool is always
>> around 40% used, but suddenly the PAGES of PT increase almost to 97% of the
>> EDM Pool.
>> I want to find the root cause of the problem, to avoid happen other time.
>> I'm already checked the number of transaction in CICS and the number of
>> thread in DB2 and they are not increase.
>>
>> Could be this a bug in the EDM pool DB2 v8 ?
>>
>> I will appreciate any suggestion to further analysis this problem.
>>
>> RR
>>
>>
>>
>> EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23
>> 16:27 Elapsed time
>> 1 1 1 6 PAGES IN EDM POOL (BELOW)
>> 37500 37500 37500 37500 HELD BY CT 176 290.42
>> 588 234.67 HELD BY PT 4936 13113.24 34380 8780.87 HELD
>> BY SKCT 9 8.17 6 10.17 HELD BY SKPT
>> 11415 8716.93 1700 7322.49 FREE PAGES 20964 15371.24
>> 826 21151.81 % PAGES IN USE 44.1 59.01 97.8 43.6 % NON
>> STEAL. PAGES IN USE 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL
>> 0 0 0 66
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <[login to unmask email]>
>> wrote:
>> >
>> > hello everybody !
>> >
>> > I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing
>> enviorment.
>> > Not dynamic application are running, not distributed application is
>> running, not storage procedure is running.
>> >
>> >
>> >
>> > I have received this error in production. 66 times in the 1 second
>> interval. And the application canceled with RC -904 resource unavailable.
>> >
>> > 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
>> > 004 REASON 00C90089
>> > 004 TYPE 00000600
>> > 004 NAME EDM POOL SPACE
>> > ...................................
>> > 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
>> > 013 REASON 00C90089
>> > 013 TYPE 00000600
>> >
>> >
>> > If we look in the statistic report the EDM Pool for 15 interval during
>> 16:15 to 16:30, we can see only the 40% of the pages are in use. It looks
>> the EDM Pool is not full.
>> > Can any help to understand what is the problem ? How I can find the
>> problem ?
>> > Thank you.
>> >
>> >
>> > EDM POOL QUANTITY /SECOND /THREAD /COMMIT
>> > --------------------------- -------- ------- ------- -------
>> > PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
>> > HELD BY CT 208.86 N/A N/A N/A
>> > HELD BY PT 6682.08 N/A N/A N/A
>> > HELD BY SKCT 9.79 N/A N/A N/A
>> > HELD BY SKPT 9193.57 N/A N/A N/A
>> > FREE PAGES 21405.71 N/A N/A N/A
>> > % PAGES IN USE 42.92 N/A N/A N/A
>> > % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
>> > FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>> >
>> > PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
>> > HELD BY DBD 7632.00 N/A N/A N/A
>> > FREE PAGES 12368.00 N/A N/A N/A
>> > FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>> >
>> > PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
>> > HELD BY STATEMENTS 18916.21 N/A N/A N/A
>> > FREE PAGES 1083.79 N/A N/A N/A
>> > FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>> >
>> > DBD REQUESTS 70916.00 83.29 1.01 0.38
>> > DBD NOT FOUND 0.00 0.00 0.00 0.00
>> > DBD HIT RATIO (%) 100.00 N/A N/A N/A
>> > CT REQUESTS 70178.00 82.42 1.00 0.38
>> > CT NOT FOUND 11.00 0.01 0.00 0.00
>> > CT HIT RATIO (%) 99.98 N/A N/A N/A
>> > PT REQUESTS 3385.0K 3975.56 48.22 18.18
>> > PT NOT FOUND 3592.00 4.22 0.05 0.02
>> > PT HIT RATIO (%) 99.89 N/A N/A N/A
>> >
>> > PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
>> > PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
>> > PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>> >
>> >
>>
>>
>> ------------------------------
>>
>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>
>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>> to use a new password. You should have gotten an e-mail with the temporary
>> password assigned to your account. Please log in and update your member
>> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>>
>
>
>
> --
> Robert Catterall
> Catterall Consulting
> www.catterallconsulting.com
>
>
> ------------------------------
>
> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>
> *IDUG.org* < http://www.idug.org > was recently updated requiring members to
> use a new password. You should have gotten an e-mail with the temporary
> password assigned to your account. Please log in and update your member
> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>


______________________________________________________________________

* IDUG 2009 Rome, Italy * 5-9 October * http://IDUG.ORG/Events *
______________________________________________________________________



IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Robert Catterall

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Renzo Razzetti)
My understanding is that MAX_NUM_CUR applies to DB2 for z/OS stored
procedures. Are you using stored procedures? If so, do you have
MAX_NUM_CUR set to its default value (500)?

To reiterate a point I made in my previous response: you indicated that you
have packages bound with RELEASE(DEALLOCATE) "as we are in data sharing."
This implies that RELEASE(DEALLOCATE) *should* be used as a bind option in a
data sharing environment, and as I pointed out the new global locking
protocol introduced with DB2 for z/OS V8 removes the
global-lock-contention-reduction incentive to use RELEASE(DEALLOCATE).
RELEASE(DEALLOCATE) can improve a DB2 program's CPU efficiency, but the
decision as to whether or not to use this bind option is no longer data
sharing-related, as far as I'm concerned.

Robert


On Sun, Jan 11, 2009 at 8:01 PM, Renzo razzetti <[login to unmask email]>wrote:

> Hello Robert
>
> Ok, I have researched about yours suggestions. I'm continuing investigate.
> Yes, The packages running at that time (peak hour) are bound with
> RELEASE(DEALLOCATE) as we are in datasharing .
>
> One possible cause of the problem is described in the IBM Technote
> http://www-01.ibm.com/support/docview.wss?uid=swg21294759. In this
> technote explain that Storage shortages can occur if too many cursors that
> return result sets are open at the same time. To minimize these storage
> shortages, subsystem parameters MAX_NUM_CUR controls the maximum number of
> cursors that can be open concurrently. When the value from this subsystem
> parameter is exceeded while an application is running, the CALL statement
> receives SQLCODE -904.
>
>
> I would like to know where I can confirm (in which Report) whether we hit
> the MAX_NUM_CUR opened current in the thread.
>
> I apprecciate very much your help
>
> RR
>
>
>
>
>
>
>
> In this technote explain that Storage shortages can occur if too many
> cursors that return result sets are open at the same time. To minimize these
> storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum
> number of cursors that can be open concurrently.
>
>
>
>
>
> On Thu, Jan 8, 2009 at 10:46 PM, Robert Catterall <[login to unmask email]>wrote:
>
>> Renzo,
>>
>> Is your DB2 workload (during the "EDM pool full" times) primarily
>> comprised of CICS-DB2 transactions? If so, are many of the associated DB2
>> packages bound with RELEASE(DEALLOCATE)? Are you using any protected
>> CICS-DB2 threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource
>> definitions)?
>>
>> Historically, a lot of DB2 for z/OS data sharing users went with
>> RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock
>> propagation to the lock structure. In CICS-DB2 environments, tablespace
>> lock propagation could be further reduced through the use of CICS protected
>> entry threads. People were interested in reducing tablespace lock
>> propagation because such requests often resulted in what's called XES
>> contention - something that increased data sharing overhead. An
>> unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected
>> threads were also used - was an increase (sometimes quite substantial) in
>> the usage of EDM pool space for sections of in-use packages (the "PT" part
>> of the pool) - this because a package bound with RELEASE(DEALLOCATE) is
>> considered to be "in-use" until the associated thread is deallocated (as
>> opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In
>> some cases, even minor fluctuations in the CICS-DB2 transaction workload can
>> lead to significant chnages in the rate of CICS-DB2 thread reuse, and that
>> (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic
>> shifts in "in-use" versus "stealable" PT-related EDM pool pages.
>>
>> The good news with respect to DB2 V8: due to a new global locking
>> protocol, RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of
>> the type of global lock contention known as XES contention (I wrote about
>> this in a blog entry last summer:
>> http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html).
>> So, if you have a fair number of CICS-DB2 program packages bound with
>> RELEASE(DEALLOCATE), and if that bind specification decision was made so as
>> to reduce data sharing lock contention, consider changing the bind
>> specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global
>> lock contention penalty for doing this, thanks to the afrementioned global
>> locking protocol enhancement (known as "locking protocol 2," as I recall).
>>
>> This may or may not explain what you are seeing. Something for you to
>> consider and investigate.
>>
>> Robert
>>
>> On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <[login to unmask email]>wrote:
>>
>>> Hello,
>>>
>>> I m adding more data in my question to see whether there is someone in
>>> the list can help me.
>>> Following is the statistic from 1 minute interval. The EDM Pool is
>>> always around 40% used, but suddenly the PAGES of PT increase almost to 97%
>>> of the EDM Pool.
>>> I want to find the root cause of the problem, to avoid happen other time.
>>> I'm already checked the number of transaction in CICS and the number of
>>> thread in DB2 and they are not increase.
>>>
>>> Could be this a bug in the EDM pool DB2 v8 ?
>>>
>>> I will appreciate any suggestion to further analysis this problem.
>>>
>>> RR
>>>
>>>
>>>
>>> EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23
>>> 16:27 Elapsed time
>>> 1 1 1 6 PAGES IN EDM POOL (BELOW)
>>> 37500 37500 37500 37500 HELD BY CT 176 290.42
>>> 588 234.67 HELD BY PT 4936 13113.24 34380 8780.87
>>> HELD BY SKCT 9 8.17 6 10.17 HELD BY SKPT
>>> 11415 8716.93 1700 7322.49 FREE PAGES
>>> 20964 15371.24 826 21151.81 % PAGES IN USE
>>> 44.1 59.01 97.8 43.6 % NON STEAL. PAGES IN USE
>>> 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL 0 0 0 66
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <
>>> [login to unmask email]> wrote:
>>> >
>>> > hello everybody !
>>> >
>>> > I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing
>>> enviorment.
>>> > Not dynamic application are running, not distributed application is
>>> running, not storage procedure is running.
>>> >
>>> >
>>> >
>>> > I have received this error in production. 66 times in the 1 second
>>> interval. And the application canceled with RC -904 resource unavailable.
>>> >
>>> > 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
>>> > 004 REASON 00C90089
>>> > 004 TYPE 00000600
>>> > 004 NAME EDM POOL SPACE
>>> > ...................................
>>> > 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
>>> > 013 REASON 00C90089
>>> > 013 TYPE 00000600
>>> >
>>> >
>>> > If we look in the statistic report the EDM Pool for 15 interval during
>>> 16:15 to 16:30, we can see only the 40% of the pages are in use. It looks
>>> the EDM Pool is not full.
>>> > Can any help to understand what is the problem ? How I can find the
>>> problem ?
>>> > Thank you.
>>> >
>>> >
>>> > EDM POOL QUANTITY /SECOND /THREAD /COMMIT
>>> > --------------------------- -------- ------- ------- -------
>>> > PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
>>> > HELD BY CT 208.86 N/A N/A N/A
>>> > HELD BY PT 6682.08 N/A N/A N/A
>>> > HELD BY SKCT 9.79 N/A N/A N/A
>>> > HELD BY SKPT 9193.57 N/A N/A N/A
>>> > FREE PAGES 21405.71 N/A N/A N/A
>>> > % PAGES IN USE 42.92 N/A N/A N/A
>>> > % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
>>> > FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>>> >
>>> > PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
>>> > HELD BY DBD 7632.00 N/A N/A N/A
>>> > FREE PAGES 12368.00 N/A N/A N/A
>>> > FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>>> >
>>> > PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
>>> > HELD BY STATEMENTS 18916.21 N/A N/A N/A
>>> > FREE PAGES 1083.79 N/A N/A N/A
>>> > FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>>> >
>>> > DBD REQUESTS 70916.00 83.29 1.01 0.38
>>> > DBD NOT FOUND 0.00 0.00 0.00 0.00
>>> > DBD HIT RATIO (%) 100.00 N/A N/A N/A
>>> > CT REQUESTS 70178.00 82.42 1.00 0.38
>>> > CT NOT FOUND 11.00 0.01 0.00 0.00
>>> > CT HIT RATIO (%) 99.98 N/A N/A N/A
>>> > PT REQUESTS 3385.0K 3975.56 48.22 18.18
>>> > PT NOT FOUND 3592.00 4.22 0.05 0.02
>>> > PT HIT RATIO (%) 99.89 N/A N/A N/A
>>> >
>>> > PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
>>> > PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
>>> > PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>>> >
>>> >
>>>
>>>
>>> ------------------------------
>>>
>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>
>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>> to use a new password. You should have gotten an e-mail with the temporary
>>> password assigned to your account. Please log in and update your member
>>> profile. If you are not already an IDUG.org member, please register
>>> here. < http://www.idug.org/component/juser/register.html >
>>>
>>
>>
>>
>> --
>> Robert Catterall
>> Catterall Consulting
>> www.catterallconsulting.com
>>
>>
>> ------------------------------
>>
>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>
>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>> to use a new password. You should have gotten an e-mail with the temporary
>> password assigned to your account. Please log in and update your member
>> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>>
>
>
> ------------------------------
>
> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>
> *IDUG.org* < http://www.idug.org > was recently updated requiring members to
> use a new password. You should have gotten an e-mail with the temporary
> password assigned to your account. Please log in and update your member
> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>



--
Robert Catterall
Catterall Consulting
www.catterallconsulting.com


______________________________________________________________________

* IDUG 2009 Rome, Italy * 5-9 October * http://IDUG.ORG/Events *
______________________________________________________________________



IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Mike Vaughan

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Robert Catterall)
I haven't been following this thread all that closely, but did have one possibility to mention -- I believe you mentioned that this occured shortly after migrating to V8, correct? Did you by chance issue a mass rebind as part of your V8 migration? One of the things we discovered after our v8 migration was that the size of packages (as measured by looking at PKSIZE and AVGSIZE on SYSPACKAGE) was substantially larger after a rebind under V8 than they were under V7, so this could be contributing to the situation as well. This may not help you fix the issue, but may at least help explain why it started occuring.

________________________________
From: DB2 Data Base Discussion List [mailto:[login to unmask email] On Behalf Of Robert Catterall
Sent: Monday, January 12, 2009 8:26 AM
To: [login to unmask email]
Subject: Re: [DB2-L] EDM POOL FULL condition after migrating to V8 NFM : more data

My understanding is that MAX_NUM_CUR applies to DB2 for z/OS stored procedures. Are you using stored procedures? If so, do you have MAX_NUM_CUR set to its default value (500)?

To reiterate a point I made in my previous response: you indicated that you have packages bound with RELEASE(DEALLOCATE) "as we are in data sharing." This implies that RELEASE(DEALLOCATE) should be used as a bind option in a data sharing environment, and as I pointed out the new global locking protocol introduced with DB2 for z/OS V8 removes the global-lock-contention-reduction incentive to use RELEASE(DEALLOCATE). RELEASE(DEALLOCATE) can improve a DB2 program's CPU efficiency, but the decision as to whether or not to use this bind option is no longer data sharing-related, as far as I'm concerned.

Robert


On Sun, Jan 11, 2009 at 8:01 PM, Renzo razzetti <[login to unmask email]<mailto:[login to unmask email]>> wrote:
Hello Robert

Ok, I have researched about yours suggestions. I'm continuing investigate.
Yes, The packages running at that time (peak hour) are bound with RELEASE(DEALLOCATE) as we are in datasharing .

One possible cause of the problem is described in the IBM Technote http://www-01.ibm.com/support/docview.wss?uid=swg21294759. In this technote explain that Storage shortages can occur if too many cursors that return result sets are open at the same time. To minimize these storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum number of cursors that can be open concurrently. When the value from this subsystem parameter is exceeded while an application is running, the CALL statement receives SQLCODE -904.


I would like to know where I can confirm (in which Report) whether we hit the MAX_NUM_CUR opened current in the thread.

I apprecciate very much your help

RR



In this technote explain that Storage shortages can occur if too many cursors that return result sets are open at the same time. To minimize these storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum number of cursors that can be open concurrently.


On Thu, Jan 8, 2009 at 10:46 PM, Robert Catterall <[login to unmask email]<mailto:[login to unmask email]>> wrote:
Renzo,

Is your DB2 workload (during the "EDM pool full" times) primarily comprised of CICS-DB2 transactions? If so, are many of the associated DB2 packages bound with RELEASE(DEALLOCATE)? Are you using any protected CICS-DB2 threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource definitions)?

Historically, a lot of DB2 for z/OS data sharing users went with RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock propagation to the lock structure. In CICS-DB2 environments, tablespace lock propagation could be further reduced through the use of CICS protected entry threads. People were interested in reducing tablespace lock propagation because such requests often resulted in what's called XES contention - something that increased data sharing overhead. An unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected threads were also used - was an increase (sometimes quite substantial) in the usage of EDM pool space for sections of in-use packages (the "PT" part of the pool) - this because a package bound with RELEASE(DEALLOCATE) is considered to be "in-use" until the associated thread is deallocated (as opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In some cases, even minor fluctuations in the CICS-DB2 transaction workload can lead to significant chnages in the rate of CICS-DB2 thread reuse, and that (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic shifts in "in-use" versus "stealable" PT-related EDM pool pages.

The good news with respect to DB2 V8: due to a new global locking protocol, RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of the type of global lock contention known as XES contention (I wrote about this in a blog entry last summer: http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html). So, if you have a fair number of CICS-DB2 program packages bound with RELEASE(DEALLOCATE), and if that bind specification decision was made so as to reduce data sharing lock contention, consider changing the bind specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global lock contention penalty for doing this, thanks to the afrementioned global locking protocol enhancement (known as "locking protocol 2," as I recall).

This may or may not explain what you are seeing. Something for you to consider and investigate.

Robert

On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <[login to unmask email]<mailto:[login to unmask email]>> wrote:
Hello,

I m adding more data in my question to see whether there is someone in the list can help me.
Following is the statistic from 1 minute interval. The EDM Pool is always around 40% used, but suddenly the PAGES of PT increase almost to 97% of the EDM Pool.
I want to find the root cause of the problem, to avoid happen other time. I'm already checked the number of transaction in CICS and the number of thread in DB2 and they are not increase.

Could be this a bug in the EDM pool DB2 v8 ?

I will appreciate any suggestion to further analysis this problem.

RR



EDM POOL DP21 16:20 16:21 16:22 16:21
16:21 16:22 16:23 16:27
Elapsed time
1 1 1 6
PAGES IN EDM POOL (BELOW) 37500 37500 37500 37500
HELD BY CT 176 290.42 588 234.67
HELD BY PT 4936 13113.24 34380 8780.87
HELD BY SKCT 9 8.17 6 10.17
HELD BY SKPT 11415 8716.93 1700 7322.49
FREE PAGES 20964 15371.24 826 21151.81
% PAGES IN USE 44.1 59.01 97.8 43.6
% NON STEAL. PAGES IN USE 13.63 35.74 93.25 24.04
FAILS DUE TO POOL FULL 0 0 0 66


On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <[login to unmask email]<mailto:[login to unmask email]>> wrote:
>
> hello everybody !
>
> I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing enviorment.
> Not dynamic application are running, not distributed application is running, not storage procedure is running.
>
>
>
> I have received this error in production. 66 times in the 1 second interval. And the application canceled with RC -904 resource unavailable.
>
> 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
> 004 REASON 00C90089
> 004 TYPE 00000600
> 004 NAME EDM POOL SPACE
> ...................................
> 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
> 013 REASON 00C90089
> 013 TYPE 00000600
>
>
> If we look in the statistic report the EDM Pool for 15 interval during 16:15 to 16:30, we can see only the 40% of the pages are in use. It looks the EDM Pool is not full.
> Can any help to understand what is the problem ? How I can find the problem ?
> Thank you.
>
>
> EDM POOL QUANTITY /SECOND /THREAD /COMMIT
> --------------------------- -------- ------- ------- -------
> PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
> HELD BY CT 208.86 N/A N/A N/A
> HELD BY PT 6682.08 N/A N/A N/A
> HELD BY SKCT 9.79 N/A N/A N/A
> HELD BY SKPT 9193.57 N/A N/A N/A
> FREE PAGES 21405.71 N/A N/A N/A
> % PAGES IN USE 42.92 N/A N/A N/A
> % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
> FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>
> PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
> HELD BY DBD 7632.00 N/A N/A N/A
> FREE PAGES 12368.00 N/A N/A N/A
> FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>
> PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
> HELD BY STATEMENTS 18916.21 N/A N/A N/A
> FREE PAGES 1083.79 N/A N/A N/A
> FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>
> DBD REQUESTS 70916.00 83.29 1.01 0.38
> DBD NOT FOUND 0.00 0.00 0.00 0.00
> DBD HIT RATIO (%) 100.00 N/A N/A N/A
> CT REQUESTS 70178.00 82.42 1.00 0.38
> CT NOT FOUND 11.00 0.01 0.00 0.00
> CT HIT RATIO (%) 99.98 N/A N/A N/A
> PT REQUESTS 3385.0K 3975.56 48.22 18.18
> PT NOT FOUND 3592.00 4.22 0.05 0.02
> PT HIT RATIO (%) 99.89 N/A N/A N/A
>
> PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
> PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
> PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>
>


________________________________

IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia < http://idug.org/lsAU >

IDUG.org <http://www.idug.org> was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register here.< http://www.idug.org/component/juser/register.html >



--
Robert Catterall
Catterall Consulting
www.catterallconsulting.com < http://www.catterallconsulting.com >





-----Message Disclaimer-----

This e-mail message is intended only for the use of the individual or
entity to which it is addressed, and may contain information that is
privileged, confidential and exempt from disclosure under applicable law.
If you are not the intended recipient, any dissemination, distribution or
copying of this communication is strictly prohibited. If you have
received this communication in error, please notify us immediately by
reply email to [login to unmask email] and delete or destroy all copies of
the original message and attachments thereto. Email sent to or from the
Principal Financial Group or any of its member companies may be retained
as required by law or regulation.

Nothing in this message is intended to constitute an Electronic signature
for purposes of the Uniform Electronic Transactions Act (UETA) or the
Electronic Signatures in Global and National Commerce Act ("E-Sign")
unless a specific statement to the contrary is included in this message.

While this communication may be used to promote or market a transaction
or an idea that is discussed in the publication, it is intended to provide
general information about the subject matter covered and is provided with
the understanding that The Principal is not rendering legal, accounting,
or tax advice. It is not a marketed opinion and may not be used to avoid
penalties under the Internal Revenue Code. You should consult with
appropriate counsel or other advisors on all matters pertaining to legal,
tax, or accounting obligations and requirements.


______________________________________________________________________

* IDUG 2009 Rome, Italy * 5-9 October * http://IDUG.ORG/Events *
______________________________________________________________________



IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Renzo Razzetti

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Mike Vaughan)
Robert

We don't have stored procedures yet in production. I misunderstand, I
thought that MAX_NUM_CUR is for max cursors in any kind of thread. Sorry.

I understand your point about the new global locking protocol introduced
with DB2 for z/OS V8 . We will consider it.

Anyway I would like to find the cause of my EDM Pool full condition last
time. I saw in the Accounting Report, in the 2 seconds where we have the
problem, we have 801 OPEN and 721 CLOSE. How can affect the
RELEASE(DEALLOCATE) and the program not close the cursor ?

Thank you very much for your time
RR



On Mon, Jan 12, 2009 at 10:25 PM, Robert Catterall <[login to unmask email]>wrote:

> My understanding is that MAX_NUM_CUR applies to DB2 for z/OS stored
> procedures. Are you using stored procedures? If so, do you have
> MAX_NUM_CUR set to its default value (500)?
>
> To reiterate a point I made in my previous response: you indicated that you
> have packages bound with RELEASE(DEALLOCATE) "as we are in data sharing."
> This implies that RELEASE(DEALLOCATE) *should* be used as a bind option in
> a data sharing environment, and as I pointed out the new global locking
> protocol introduced with DB2 for z/OS V8 removes the
> global-lock-contention-reduction incentive to use RELEASE(DEALLOCATE).
> RELEASE(DEALLOCATE) can improve a DB2 program's CPU efficiency, but the
> decision as to whether or not to use this bind option is no longer data
> sharing-related, as far as I'm concerned.
>
> Robert
>
>
>
> On Sun, Jan 11, 2009 at 8:01 PM, Renzo razzetti <[login to unmask email]>wrote:
>
>> Hello Robert
>>
>> Ok, I have researched about yours suggestions. I'm continuing investigate.
>>
>> Yes, The packages running at that time (peak hour) are bound with
>> RELEASE(DEALLOCATE) as we are in datasharing .
>>
>> One possible cause of the problem is described in the IBM Technote
>> http://www-01.ibm.com/support/docview.wss?uid=swg21294759. In this
>> technote explain that Storage shortages can occur if too many cursors that
>> return result sets are open at the same time. To minimize these storage
>> shortages, subsystem parameters MAX_NUM_CUR controls the maximum number of
>> cursors that can be open concurrently. When the value from this subsystem
>> parameter is exceeded while an application is running, the CALL statement
>> receives SQLCODE -904.
>>
>>
>> I would like to know where I can confirm (in which Report) whether we
>> hit the MAX_NUM_CUR opened current in the thread.
>>
>> I apprecciate very much your help
>>
>> RR
>>
>>
>>
>>
>>
>>
>>
>> In this technote explain that Storage shortages can occur if too many
>> cursors that return result sets are open at the same time. To minimize these
>> storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum
>> number of cursors that can be open concurrently.
>>
>>
>>
>>
>>
>> On Thu, Jan 8, 2009 at 10:46 PM, Robert Catterall <[login to unmask email]>wrote:
>>
>>> Renzo,
>>>
>>> Is your DB2 workload (during the "EDM pool full" times) primarily
>>> comprised of CICS-DB2 transactions? If so, are many of the associated DB2
>>> packages bound with RELEASE(DEALLOCATE)? Are you using any protected
>>> CICS-DB2 threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource
>>> definitions)?
>>>
>>> Historically, a lot of DB2 for z/OS data sharing users went with
>>> RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock
>>> propagation to the lock structure. In CICS-DB2 environments, tablespace
>>> lock propagation could be further reduced through the use of CICS protected
>>> entry threads. People were interested in reducing tablespace lock
>>> propagation because such requests often resulted in what's called XES
>>> contention - something that increased data sharing overhead. An
>>> unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected
>>> threads were also used - was an increase (sometimes quite substantial) in
>>> the usage of EDM pool space for sections of in-use packages (the "PT" part
>>> of the pool) - this because a package bound with RELEASE(DEALLOCATE) is
>>> considered to be "in-use" until the associated thread is deallocated (as
>>> opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In
>>> some cases, even minor fluctuations in the CICS-DB2 transaction workload can
>>> lead to significant chnages in the rate of CICS-DB2 thread reuse, and that
>>> (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic
>>> shifts in "in-use" versus "stealable" PT-related EDM pool pages.
>>>
>>> The good news with respect to DB2 V8: due to a new global locking
>>> protocol, RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of
>>> the type of global lock contention known as XES contention (I wrote about
>>> this in a blog entry last summer:
>>> http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html).
>>> So, if you have a fair number of CICS-DB2 program packages bound with
>>> RELEASE(DEALLOCATE), and if that bind specification decision was made so as
>>> to reduce data sharing lock contention, consider changing the bind
>>> specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global
>>> lock contention penalty for doing this, thanks to the afrementioned global
>>> locking protocol enhancement (known as "locking protocol 2," as I recall).
>>>
>>> This may or may not explain what you are seeing. Something for you to
>>> consider and investigate.
>>>
>>> Robert
>>>
>>> On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <[login to unmask email]
>>> > wrote:
>>>
>>>> Hello,
>>>>
>>>> I m adding more data in my question to see whether there is someone in
>>>> the list can help me.
>>>> Following is the statistic from 1 minute interval. The EDM Pool is
>>>> always around 40% used, but suddenly the PAGES of PT increase almost to 97%
>>>> of the EDM Pool.
>>>> I want to find the root cause of the problem, to avoid happen other
>>>> time. I'm already checked the number of transaction in CICS and the number
>>>> of thread in DB2 and they are not increase.
>>>>
>>>> Could be this a bug in the EDM pool DB2 v8 ?
>>>>
>>>> I will appreciate any suggestion to further analysis this problem.
>>>>
>>>> RR
>>>>
>>>>
>>>>
>>>> EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23
>>>> 16:27 Elapsed time
>>>> 1 1 1 6 PAGES IN EDM POOL (BELOW)
>>>> 37500 37500 37500 37500 HELD BY CT 176 290.42
>>>> 588 234.67 HELD BY PT 4936 13113.24 34380 8780.87
>>>> HELD BY SKCT 9 8.17 6 10.17 HELD BY SKPT
>>>> 11415 8716.93 1700 7322.49 FREE PAGES
>>>> 20964 15371.24 826 21151.81 % PAGES IN USE
>>>> 44.1 59.01 97.8 43.6 % NON STEAL. PAGES IN USE
>>>> 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL 0 0 0 66
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <
>>>> [login to unmask email]> wrote:
>>>> >
>>>> > hello everybody !
>>>> >
>>>> > I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing
>>>> enviorment.
>>>> > Not dynamic application are running, not distributed application is
>>>> running, not storage procedure is running.
>>>> >
>>>> >
>>>> >
>>>> > I have received this error in production. 66 times in the 1 second
>>>> interval. And the application canceled with RC -904 resource unavailable.
>>>> >
>>>> > 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
>>>> > 004 REASON 00C90089
>>>> > 004 TYPE 00000600
>>>> > 004 NAME EDM POOL SPACE
>>>> > ...................................
>>>> > 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
>>>> > 013 REASON 00C90089
>>>> > 013 TYPE 00000600
>>>> >
>>>> >
>>>> > If we look in the statistic report the EDM Pool for 15 interval
>>>> during 16:15 to 16:30, we can see only the 40% of the pages are in use. It
>>>> looks the EDM Pool is not full.
>>>> > Can any help to understand what is the problem ? How I can find the
>>>> problem ?
>>>> > Thank you.
>>>> >
>>>> >
>>>> > EDM POOL QUANTITY /SECOND /THREAD /COMMIT
>>>> > --------------------------- -------- ------- ------- -------
>>>> > PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
>>>> > HELD BY CT 208.86 N/A N/A N/A
>>>> > HELD BY PT 6682.08 N/A N/A N/A
>>>> > HELD BY SKCT 9.79 N/A N/A N/A
>>>> > HELD BY SKPT 9193.57 N/A N/A N/A
>>>> > FREE PAGES 21405.71 N/A N/A N/A
>>>> > % PAGES IN USE 42.92 N/A N/A N/A
>>>> > % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
>>>> > FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>>>> >
>>>> > PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
>>>> > HELD BY DBD 7632.00 N/A N/A N/A
>>>> > FREE PAGES 12368.00 N/A N/A N/A
>>>> > FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>>>> >
>>>> > PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
>>>> > HELD BY STATEMENTS 18916.21 N/A N/A N/A
>>>> > FREE PAGES 1083.79 N/A N/A N/A
>>>> > FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>>>> >
>>>> > DBD REQUESTS 70916.00 83.29 1.01 0.38
>>>> > DBD NOT FOUND 0.00 0.00 0.00 0.00
>>>> > DBD HIT RATIO (%) 100.00 N/A N/A N/A
>>>> > CT REQUESTS 70178.00 82.42 1.00 0.38
>>>> > CT NOT FOUND 11.00 0.01 0.00 0.00
>>>> > CT HIT RATIO (%) 99.98 N/A N/A N/A
>>>> > PT REQUESTS 3385.0K 3975.56 48.22 18.18
>>>> > PT NOT FOUND 3592.00 4.22 0.05 0.02
>>>> > PT HIT RATIO (%) 99.89 N/A N/A N/A
>>>> >
>>>> > PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
>>>> > PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
>>>> > PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>>>> >
>>>> >
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>
>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>>> to use a new password. You should have gotten an e-mail with the temporary
>>>> password assigned to your account. Please log in and update your member
>>>> profile. If you are not already an IDUG.org member, please register
>>>> here. < http://www.idug.org/component/juser/register.html >
>>>>
>>>
>>>
>>>
>>> --
>>> Robert Catterall
>>> Catterall Consulting
>>> www.catterallconsulting.com
>>>
>>>
>>> ------------------------------
>>>
>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>
>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>> to use a new password. You should have gotten an e-mail with the temporary
>>> password assigned to your account. Please log in and update your member
>>> profile. If you are not already an IDUG.org member, please register
>>> here. < http://www.idug.org/component/juser/register.html >
>>>
>>
>>
>> ------------------------------
>>
>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>
>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>> to use a new password. You should have gotten an e-mail with the temporary
>> password assigned to your account. Please log in and update your member
>> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>>
>
>
>
> --
> Robert Catterall
> Catterall Consulting
> www.catterallconsulting.com
>
> ------------------------------
>
> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>
> *IDUG.org* < http://www.idug.org > was recently updated requiring members to
> use a new password. You should have gotten an e-mail with the temporary
> password assigned to your account. Please log in and update your member
> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>

______________________________________________________________________

* IDUG 2009 Denver, CO, USA * May 11-15, 2009 * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Robert Catterall

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Renzo Razzetti)
Renzo,

I'm afraid that I don't quite understand the question, "How can affect the
RELEASE(DEALLOCATE) and the program not close the cursor?" Could you
re-phrase the question? If you're wondering about the effect of
RELEASE(DEALLOCATE) versus RELEASE(COMMIT) on cursor behavior, my
understanding is that there is no difference. Opening and closing of
cursors is not affected by the RELEASE specification of BIND PACKAGE.

As to how you might suddenly get an EDM pool-full situation and then just as
suddenly see the problem disappear, I believe that this could have something
to do with the degree to which your CICS-DB2 threads are re-used. I
described this scenario in a blog entry that I posted just yesterday (
http://www.catterallconsulting.com/2009/01/db2-for-zos-looking-again-at.html).
I'll try to summarize here, and you can refer to the blog entry for more
information.

Generally speaking, the thread used by an "online" CICS transaction (versus
a background transaction, which is kind of batch-like) will terminate when
that transaction completes (protected threads are an exception to this rule,
so you might check to see if you have any protected CICS-DB2 threads defined
in your system). When that happens (i.e., when the CICS-DB2 thread is
terminated at end-of-transaction), RELEASE(DEALLOCATE) is no different than
RELEASE(COMMIT) when it comes to retention of EDM pool space. EDM pool
space retention is affected when CICS-DB2 threads through which packages
bound with RELEASE(DEALLOCATE) are executed are re-used. For a
non-protected thread to be reused, this is what has to happen (assume that
transaction XYZ is using the non-protected threas in question):

1. Before transaction XYZ completes, another instance of transaction XYZ
(or of a different transaction, *if the different transaction is
associated with the same CICS DB2ENTRY resource as transaction XYZ*) is
queued up waiting for a thread because no threads are available (they're all
being used).
2. Thransaction XYZ completes, and the queued transaction reuses the
thread that transaction XYZ had used.

In many cases, people have enough entry and pool threads defined so that
transactions rarely queue up waiting for a thread to become available. If
there's no queuing, the non-protected threads won't be reused. Instead,
they will be deallocated at end-of-tran, and so RELEASE(DEALLOCATE) won't
affect EDM pool space retention. If you're "close to the line" in this
regard, with a thread re-use rate that is small but non-zero, a relatively
small increase in the transaction arrival rate could cause wait-for-thread
queuing (and therefore, thread re-use) to jump, and this - combined with
packages bound with RELEASE(DEALLOCATE) - can cause a spike in EDM pool
space retention. That, in turn, could cause some pool-full failures to
occur. I described some corrective actions in the previously-cited blog
entry.

Hope this helps.

Robert


On Tue, Jan 13, 2009 at 1:13 AM, Renzo razzetti <[login to unmask email]>wrote:

> Robert
>
> We don't have stored procedures yet in production. I misunderstand, I
> thought that MAX_NUM_CUR is for max cursors in any kind of thread. Sorry.
>
> I understand your point about the new global locking protocol introduced
> with DB2 for z/OS V8 . We will consider it.
>
> Anyway I would like to find the cause of my EDM Pool full condition last
> time. I saw in the Accounting Report, in the 2 seconds where we have the
> problem, we have 801 OPEN and 721 CLOSE. How can affect the
> RELEASE(DEALLOCATE) and the program not close the cursor ?
>
> Thank you very much for your time
> RR
>
>
>
>
> On Mon, Jan 12, 2009 at 10:25 PM, Robert Catterall <[login to unmask email]>wrote:
>
>> My understanding is that MAX_NUM_CUR applies to DB2 for z/OS stored
>> procedures. Are you using stored procedures? If so, do you have
>> MAX_NUM_CUR set to its default value (500)?
>>
>> To reiterate a point I made in my previous response: you indicated that
>> you have packages bound with RELEASE(DEALLOCATE) "as we are in data
>> sharing." This implies that RELEASE(DEALLOCATE) *should* be used as a
>> bind option in a data sharing environment, and as I pointed out the new
>> global locking protocol introduced with DB2 for z/OS V8 removes the
>> global-lock-contention-reduction incentive to use RELEASE(DEALLOCATE).
>> RELEASE(DEALLOCATE) can improve a DB2 program's CPU efficiency, but the
>> decision as to whether or not to use this bind option is no longer data
>> sharing-related, as far as I'm concerned.
>>
>> Robert
>>
>>
>>
>> On Sun, Jan 11, 2009 at 8:01 PM, Renzo razzetti <[login to unmask email]
>> > wrote:
>>
>>> Hello Robert
>>>
>>> Ok, I have researched about yours suggestions. I'm continuing
>>> investigate.
>>> Yes, The packages running at that time (peak hour) are bound with
>>> RELEASE(DEALLOCATE) as we are in datasharing .
>>>
>>> One possible cause of the problem is described in the IBM Technote
>>> http://www-01.ibm.com/support/docview.wss?uid=swg21294759. In this
>>> technote explain that Storage shortages can occur if too many cursors that
>>> return result sets are open at the same time. To minimize these storage
>>> shortages, subsystem parameters MAX_NUM_CUR controls the maximum number of
>>> cursors that can be open concurrently. When the value from this subsystem
>>> parameter is exceeded while an application is running, the CALL statement
>>> receives SQLCODE -904.
>>>
>>>
>>> I would like to know where I can confirm (in which Report) whether we
>>> hit the MAX_NUM_CUR opened current in the thread.
>>>
>>> I apprecciate very much your help
>>>
>>> RR
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> In this technote explain that Storage shortages can occur if too many
>>> cursors that return result sets are open at the same time. To minimize these
>>> storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum
>>> number of cursors that can be open concurrently.
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jan 8, 2009 at 10:46 PM, Robert Catterall <[login to unmask email]
>>> > wrote:
>>>
>>>> Renzo,
>>>>
>>>> Is your DB2 workload (during the "EDM pool full" times) primarily
>>>> comprised of CICS-DB2 transactions? If so, are many of the associated DB2
>>>> packages bound with RELEASE(DEALLOCATE)? Are you using any protected
>>>> CICS-DB2 threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource
>>>> definitions)?
>>>>
>>>> Historically, a lot of DB2 for z/OS data sharing users went with
>>>> RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock
>>>> propagation to the lock structure. In CICS-DB2 environments, tablespace
>>>> lock propagation could be further reduced through the use of CICS protected
>>>> entry threads. People were interested in reducing tablespace lock
>>>> propagation because such requests often resulted in what's called XES
>>>> contention - something that increased data sharing overhead. An
>>>> unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected
>>>> threads were also used - was an increase (sometimes quite substantial) in
>>>> the usage of EDM pool space for sections of in-use packages (the "PT" part
>>>> of the pool) - this because a package bound with RELEASE(DEALLOCATE) is
>>>> considered to be "in-use" until the associated thread is deallocated (as
>>>> opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In
>>>> some cases, even minor fluctuations in the CICS-DB2 transaction workload can
>>>> lead to significant chnages in the rate of CICS-DB2 thread reuse, and that
>>>> (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic
>>>> shifts in "in-use" versus "stealable" PT-related EDM pool pages.
>>>>
>>>> The good news with respect to DB2 V8: due to a new global locking
>>>> protocol, RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of
>>>> the type of global lock contention known as XES contention (I wrote about
>>>> this in a blog entry last summer:
>>>> http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html).
>>>> So, if you have a fair number of CICS-DB2 program packages bound with
>>>> RELEASE(DEALLOCATE), and if that bind specification decision was made so as
>>>> to reduce data sharing lock contention, consider changing the bind
>>>> specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global
>>>> lock contention penalty for doing this, thanks to the afrementioned global
>>>> locking protocol enhancement (known as "locking protocol 2," as I recall).
>>>>
>>>> This may or may not explain what you are seeing. Something for you to
>>>> consider and investigate.
>>>>
>>>> Robert
>>>>
>>>> On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <
>>>> [login to unmask email]> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I m adding more data in my question to see whether there is someone
>>>>> in the list can help me.
>>>>> Following is the statistic from 1 minute interval. The EDM Pool is
>>>>> always around 40% used, but suddenly the PAGES of PT increase almost to 97%
>>>>> of the EDM Pool.
>>>>> I want to find the root cause of the problem, to avoid happen other
>>>>> time. I'm already checked the number of transaction in CICS and the number
>>>>> of thread in DB2 and they are not increase.
>>>>>
>>>>> Could be this a bug in the EDM pool DB2 v8 ?
>>>>>
>>>>> I will appreciate any suggestion to further analysis this problem.
>>>>>
>>>>> RR
>>>>>
>>>>>
>>>>>
>>>>> EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23
>>>>> 16:27 Elapsed time
>>>>> 1 1 1 6 PAGES IN EDM POOL (BELOW)
>>>>> 37500 37500 37500 37500 HELD BY CT 176 290.42
>>>>> 588 234.67 HELD BY PT 4936 13113.24 34380 8780.87
>>>>> HELD BY SKCT 9 8.17 6 10.17 HELD BY SKPT
>>>>> 11415 8716.93 1700 7322.49 FREE PAGES
>>>>> 20964 15371.24 826 21151.81 % PAGES IN USE
>>>>> 44.1 59.01 97.8 43.6 % NON STEAL. PAGES IN USE
>>>>> 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL 0 0 0 66
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <
>>>>> [login to unmask email]> wrote:
>>>>> >
>>>>> > hello everybody !
>>>>> >
>>>>> > I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing
>>>>> enviorment.
>>>>> > Not dynamic application are running, not distributed application is
>>>>> running, not storage procedure is running.
>>>>> >
>>>>> >
>>>>> >
>>>>> > I have received this error in production. 66 times in the 1 second
>>>>> interval. And the application canceled with RC -904 resource unavailable.
>>>>> >
>>>>> > 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 004
>>>>> > 004 REASON 00C90089
>>>>> > 004 TYPE 00000600
>>>>> > 004 NAME EDM POOL SPACE
>>>>> > ...................................
>>>>> > 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE 013
>>>>> > 013 REASON 00C90089
>>>>> > 013 TYPE 00000600
>>>>> >
>>>>> >
>>>>> > If we look in the statistic report the EDM Pool for 15 interval
>>>>> during 16:15 to 16:30, we can see only the 40% of the pages are in use. It
>>>>> looks the EDM Pool is not full.
>>>>> > Can any help to understand what is the problem ? How I can find the
>>>>> problem ?
>>>>> > Thank you.
>>>>> >
>>>>> >
>>>>> > EDM POOL QUANTITY /SECOND /THREAD /COMMIT
>>>>> > --------------------------- -------- ------- ------- -------
>>>>> > PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
>>>>> > HELD BY CT 208.86 N/A N/A N/A
>>>>> > HELD BY PT 6682.08 N/A N/A N/A
>>>>> > HELD BY SKCT 9.79 N/A N/A N/A
>>>>> > HELD BY SKPT 9193.57 N/A N/A N/A
>>>>> > FREE PAGES 21405.71 N/A N/A N/A
>>>>> > % PAGES IN USE 42.92 N/A N/A N/A
>>>>> > % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
>>>>> > FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>>>>> >
>>>>> > PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
>>>>> > HELD BY DBD 7632.00 N/A N/A N/A
>>>>> > FREE PAGES 12368.00 N/A N/A N/A
>>>>> > FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>>>>> >
>>>>> > PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
>>>>> > HELD BY STATEMENTS 18916.21 N/A N/A N/A
>>>>> > FREE PAGES 1083.79 N/A N/A N/A
>>>>> > FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>>>>> >
>>>>> > DBD REQUESTS 70916.00 83.29 1.01 0.38
>>>>> > DBD NOT FOUND 0.00 0.00 0.00 0.00
>>>>> > DBD HIT RATIO (%) 100.00 N/A N/A N/A
>>>>> > CT REQUESTS 70178.00 82.42 1.00 0.38
>>>>> > CT NOT FOUND 11.00 0.01 0.00 0.00
>>>>> > CT HIT RATIO (%) 99.98 N/A N/A N/A
>>>>> > PT REQUESTS 3385.0K 3975.56 48.22 18.18
>>>>> > PT NOT FOUND 3592.00 4.22 0.05 0.02
>>>>> > PT HIT RATIO (%) 99.89 N/A N/A N/A
>>>>> >
>>>>> > PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
>>>>> > PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
>>>>> > PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>>
>>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring
>>>>> members to use a new password. You should have gotten an e-mail with the
>>>>> temporary password assigned to your account. Please log in and update your
>>>>> member profile. If you are not already an IDUG.org member, please
>>>>> register here. < http://www.idug.org/component/juser/register.html >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Robert Catterall
>>>> Catterall Consulting
>>>> www.catterallconsulting.com
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>
>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>>> to use a new password. You should have gotten an e-mail with the temporary
>>>> password assigned to your account. Please log in and update your member
>>>> profile. If you are not already an IDUG.org member, please register
>>>> here. < http://www.idug.org/component/juser/register.html >
>>>>
>>>
>>>
>>> ------------------------------
>>>
>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>
>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>> to use a new password. You should have gotten an e-mail with the temporary
>>> password assigned to your account. Please log in and update your member
>>> profile. If you are not already an IDUG.org member, please register
>>> here. < http://www.idug.org/component/juser/register.html >
>>>
>>
>>
>>
>> --
>> Robert Catterall
>> Catterall Consulting
>> www.catterallconsulting.com
>>
>> ------------------------------
>>
>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>
>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>> to use a new password. You should have gotten an e-mail with the temporary
>> password assigned to your account. Please log in and update your member
>> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>>
>
>
> ------------------------------
>
> *IDUG 2009 - Europe * 5-9 October * Rome, Italy* < http://idug.org/lseu >
>
> *IDUG.org* < http://www.idug.org > was recently updated requiring members to
> use a new password. You should have gotten an e-mail with the temporary
> password assigned to your account. Please log in and update your member
> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>



--
Robert Catterall
Catterall Consulting
www.catterallconsulting.com

______________________________________________________________________

* IDUG 2009 Denver, CO, USA * May 11-15, 2009 * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Renzo Razzetti

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Robert Catterall)
Robert

Thank you very much !!! The blog is very clear !!! Excellent !!!! Thank you
again

RR

On Tue, Jan 13, 2009 at 10:23 PM, Robert Catterall <[login to unmask email]>wrote:

> Renzo,
>
> I'm afraid that I don't quite understand the question, "How can affect the
> RELEASE(DEALLOCATE) and the program not close the cursor?" Could you
> re-phrase the question? If you're wondering about the effect of
> RELEASE(DEALLOCATE) versus RELEASE(COMMIT) on cursor behavior, my
> understanding is that there is no difference. Opening and closing of
> cursors is not affected by the RELEASE specification of BIND PACKAGE.
>
> As to how you might suddenly get an EDM pool-full situation and then just
> as suddenly see the problem disappear, I believe that this could have
> something to do with the degree to which your CICS-DB2 threads are re-used.
> I described this scenario in a blog entry that I posted just yesterday (
> http://www.catterallconsulting.com/2009/01/db2-for-zos-looking-again-at.html).
> I'll try to summarize here, and you can refer to the blog entry for more
> information.
>
> Generally speaking, the thread used by an "online" CICS transaction (versus
> a background transaction, which is kind of batch-like) will terminate when
> that transaction completes (protected threads are an exception to this rule,
> so you might check to see if you have any protected CICS-DB2 threads defined
> in your system). When that happens (i.e., when the CICS-DB2 thread is
> terminated at end-of-transaction), RELEASE(DEALLOCATE) is no different than
> RELEASE(COMMIT) when it comes to retention of EDM pool space. EDM pool
> space retention is affected when CICS-DB2 threads through which packages
> bound with RELEASE(DEALLOCATE) are executed are re-used. For a
> non-protected thread to be reused, this is what has to happen (assume that
> transaction XYZ is using the non-protected threas in question):
>
> 1. Before transaction XYZ completes, another instance of transaction
> XYZ (or of a different transaction, *if the different transaction is
> associated with the same CICS DB2ENTRY resource as transaction XYZ*) is
> queued up waiting for a thread because no threads are available (they're all
> being used).
> 2. Thransaction XYZ completes, and the queued transaction reuses the
> thread that transaction XYZ had used.
>
> In many cases, people have enough entry and pool threads defined so that
> transactions rarely queue up waiting for a thread to become available. If
> there's no queuing, the non-protected threads won't be reused. Instead,
> they will be deallocated at end-of-tran, and so RELEASE(DEALLOCATE) won't
> affect EDM pool space retention. If you're "close to the line" in this
> regard, with a thread re-use rate that is small but non-zero, a relatively
> small increase in the transaction arrival rate could cause wait-for-thread
> queuing (and therefore, thread re-use) to jump, and this - combined with
> packages bound with RELEASE(DEALLOCATE) - can cause a spike in EDM pool
> space retention. That, in turn, could cause some pool-full failures to
> occur. I described some corrective actions in the previously-cited blog
> entry.
>
> Hope this helps.
>
> Robert
>
>
> On Tue, Jan 13, 2009 at 1:13 AM, Renzo razzetti <[login to unmask email]>wrote:
>
>> Robert
>>
>> We don't have stored procedures yet in production. I misunderstand, I
>> thought that MAX_NUM_CUR is for max cursors in any kind of thread. Sorry.
>>
>> I understand your point about the new global locking protocol introduced
>> with DB2 for z/OS V8 . We will consider it.
>>
>> Anyway I would like to find the cause of my EDM Pool full condition last
>> time. I saw in the Accounting Report, in the 2 seconds where we have the
>> problem, we have 801 OPEN and 721 CLOSE. How can affect the
>> RELEASE(DEALLOCATE) and the program not close the cursor ?
>>
>> Thank you very much for your time
>> RR
>>
>>
>>
>>
>> On Mon, Jan 12, 2009 at 10:25 PM, Robert Catterall <[login to unmask email]
>> > wrote:
>>
>>> My understanding is that MAX_NUM_CUR applies to DB2 for z/OS stored
>>> procedures. Are you using stored procedures? If so, do you have
>>> MAX_NUM_CUR set to its default value (500)?
>>>
>>> To reiterate a point I made in my previous response: you indicated that
>>> you have packages bound with RELEASE(DEALLOCATE) "as we are in data
>>> sharing." This implies that RELEASE(DEALLOCATE) *should* be used as a
>>> bind option in a data sharing environment, and as I pointed out the new
>>> global locking protocol introduced with DB2 for z/OS V8 removes the
>>> global-lock-contention-reduction incentive to use RELEASE(DEALLOCATE).
>>> RELEASE(DEALLOCATE) can improve a DB2 program's CPU efficiency, but the
>>> decision as to whether or not to use this bind option is no longer data
>>> sharing-related, as far as I'm concerned.
>>>
>>> Robert
>>>
>>>
>>>
>>> On Sun, Jan 11, 2009 at 8:01 PM, Renzo razzetti <
>>> [login to unmask email]> wrote:
>>>
>>>> Hello Robert
>>>>
>>>> Ok, I have researched about yours suggestions. I'm continuing
>>>> investigate.
>>>> Yes, The packages running at that time (peak hour) are bound with
>>>> RELEASE(DEALLOCATE) as we are in datasharing .
>>>>
>>>> One possible cause of the problem is described in the IBM Technote
>>>> http://www-01.ibm.com/support/docview.wss?uid=swg21294759. In this
>>>> technote explain that Storage shortages can occur if too many cursors that
>>>> return result sets are open at the same time. To minimize these storage
>>>> shortages, subsystem parameters MAX_NUM_CUR controls the maximum number of
>>>> cursors that can be open concurrently. When the value from this subsystem
>>>> parameter is exceeded while an application is running, the CALL statement
>>>> receives SQLCODE -904.
>>>>
>>>>
>>>> I would like to know where I can confirm (in which Report) whether we
>>>> hit the MAX_NUM_CUR opened current in the thread.
>>>>
>>>> I apprecciate very much your help
>>>>
>>>> RR
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> In this technote explain that Storage shortages can occur if too many
>>>> cursors that return result sets are open at the same time. To minimize these
>>>> storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum
>>>> number of cursors that can be open concurrently.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jan 8, 2009 at 10:46 PM, Robert Catterall <
>>>> [login to unmask email]> wrote:
>>>>
>>>>> Renzo,
>>>>>
>>>>> Is your DB2 workload (during the "EDM pool full" times) primarily
>>>>> comprised of CICS-DB2 transactions? If so, are many of the associated DB2
>>>>> packages bound with RELEASE(DEALLOCATE)? Are you using any protected
>>>>> CICS-DB2 threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource
>>>>> definitions)?
>>>>>
>>>>> Historically, a lot of DB2 for z/OS data sharing users went with
>>>>> RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock
>>>>> propagation to the lock structure. In CICS-DB2 environments, tablespace
>>>>> lock propagation could be further reduced through the use of CICS protected
>>>>> entry threads. People were interested in reducing tablespace lock
>>>>> propagation because such requests often resulted in what's called XES
>>>>> contention - something that increased data sharing overhead. An
>>>>> unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected
>>>>> threads were also used - was an increase (sometimes quite substantial) in
>>>>> the usage of EDM pool space for sections of in-use packages (the "PT" part
>>>>> of the pool) - this because a package bound with RELEASE(DEALLOCATE) is
>>>>> considered to be "in-use" until the associated thread is deallocated (as
>>>>> opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In
>>>>> some cases, even minor fluctuations in the CICS-DB2 transaction workload can
>>>>> lead to significant chnages in the rate of CICS-DB2 thread reuse, and that
>>>>> (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic
>>>>> shifts in "in-use" versus "stealable" PT-related EDM pool pages.
>>>>>
>>>>> The good news with respect to DB2 V8: due to a new global locking
>>>>> protocol, RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of
>>>>> the type of global lock contention known as XES contention (I wrote about
>>>>> this in a blog entry last summer:
>>>>> http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html).
>>>>> So, if you have a fair number of CICS-DB2 program packages bound with
>>>>> RELEASE(DEALLOCATE), and if that bind specification decision was made so as
>>>>> to reduce data sharing lock contention, consider changing the bind
>>>>> specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global
>>>>> lock contention penalty for doing this, thanks to the afrementioned global
>>>>> locking protocol enhancement (known as "locking protocol 2," as I recall).
>>>>>
>>>>> This may or may not explain what you are seeing. Something for you to
>>>>> consider and investigate.
>>>>>
>>>>> Robert
>>>>>
>>>>> On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <
>>>>> [login to unmask email]> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I m adding more data in my question to see whether there is someone
>>>>>> in the list can help me.
>>>>>> Following is the statistic from 1 minute interval. The EDM Pool is
>>>>>> always around 40% used, but suddenly the PAGES of PT increase almost to 97%
>>>>>> of the EDM Pool.
>>>>>> I want to find the root cause of the problem, to avoid happen other
>>>>>> time. I'm already checked the number of transaction in CICS and the number
>>>>>> of thread in DB2 and they are not increase.
>>>>>>
>>>>>> Could be this a bug in the EDM pool DB2 v8 ?
>>>>>>
>>>>>> I will appreciate any suggestion to further analysis this problem.
>>>>>>
>>>>>> RR
>>>>>>
>>>>>>
>>>>>>
>>>>>> EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23
>>>>>> 16:27 Elapsed time
>>>>>> 1 1 1 6 PAGES IN EDM POOL
>>>>>> (BELOW) 37500 37500 37500 37500 HELD BY CT
>>>>>> 176 290.42 588 234.67 HELD BY PT 4936 13113.24
>>>>>> 34380 8780.87 HELD BY SKCT 9 8.17 6 10.17 HELD
>>>>>> BY SKPT 11415 8716.93 1700 7322.49 FREE PAGES
>>>>>> 20964 15371.24 826 21151.81 % PAGES IN USE
>>>>>> 44.1 59.01 97.8 43.6 % NON STEAL. PAGES IN USE
>>>>>> 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL 0 0 0 66
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <
>>>>>> [login to unmask email]> wrote:
>>>>>> >
>>>>>> > hello everybody !
>>>>>> >
>>>>>> > I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing
>>>>>> enviorment.
>>>>>> > Not dynamic application are running, not distributed application is
>>>>>> running, not storage procedure is running.
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > I have received this error in production. 66 times in the 1 second
>>>>>> interval. And the application canceled with RC -904 resource unavailable.
>>>>>> >
>>>>>> > 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE
>>>>>> 004
>>>>>> > 004 REASON 00C90089
>>>>>> > 004 TYPE 00000600
>>>>>> > 004 NAME EDM POOL SPACE
>>>>>> > ...................................
>>>>>> > 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE
>>>>>> 013
>>>>>> > 013 REASON 00C90089
>>>>>> > 013 TYPE 00000600
>>>>>> >
>>>>>> >
>>>>>> > If we look in the statistic report the EDM Pool for 15 interval
>>>>>> during 16:15 to 16:30, we can see only the 40% of the pages are in use. It
>>>>>> looks the EDM Pool is not full.
>>>>>> > Can any help to understand what is the problem ? How I can find the
>>>>>> problem ?
>>>>>> > Thank you.
>>>>>> >
>>>>>> >
>>>>>> > EDM POOL QUANTITY /SECOND /THREAD /COMMIT
>>>>>> > --------------------------- -------- ------- ------- -------
>>>>>> > PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
>>>>>> > HELD BY CT 208.86 N/A N/A N/A
>>>>>> > HELD BY PT 6682.08 N/A N/A N/A
>>>>>> > HELD BY SKCT 9.79 N/A N/A N/A
>>>>>> > HELD BY SKPT 9193.57 N/A N/A N/A
>>>>>> > FREE PAGES 21405.71 N/A N/A N/A
>>>>>> > % PAGES IN USE 42.92 N/A N/A N/A
>>>>>> > % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
>>>>>> > FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>>>>>> >
>>>>>> > PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
>>>>>> > HELD BY DBD 7632.00 N/A N/A N/A
>>>>>> > FREE PAGES 12368.00 N/A N/A N/A
>>>>>> > FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>>>>>> >
>>>>>> > PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
>>>>>> > HELD BY STATEMENTS 18916.21 N/A N/A N/A
>>>>>> > FREE PAGES 1083.79 N/A N/A N/A
>>>>>> > FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>>>>>> >
>>>>>> > DBD REQUESTS 70916.00 83.29 1.01 0.38
>>>>>> > DBD NOT FOUND 0.00 0.00 0.00 0.00
>>>>>> > DBD HIT RATIO (%) 100.00 N/A N/A N/A
>>>>>> > CT REQUESTS 70178.00 82.42 1.00 0.38
>>>>>> > CT NOT FOUND 11.00 0.01 0.00 0.00
>>>>>> > CT HIT RATIO (%) 99.98 N/A N/A N/A
>>>>>> > PT REQUESTS 3385.0K 3975.56 48.22 18.18
>>>>>> > PT NOT FOUND 3592.00 4.22 0.05 0.02
>>>>>> > PT HIT RATIO (%) 99.89 N/A N/A N/A
>>>>>> >
>>>>>> > PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
>>>>>> > PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
>>>>>> > PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>>>>>> >
>>>>>> >
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>>>
>>>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring
>>>>>> members to use a new password. You should have gotten an e-mail with the
>>>>>> temporary password assigned to your account. Please log in and update your
>>>>>> member profile. If you are not already an IDUG.org member, please
>>>>>> register here. < http://www.idug.org/component/juser/register.html >
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Robert Catterall
>>>>> Catterall Consulting
>>>>> www.catterallconsulting.com
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>>
>>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring
>>>>> members to use a new password. You should have gotten an e-mail with the
>>>>> temporary password assigned to your account. Please log in and update your
>>>>> member profile. If you are not already an IDUG.org member, please
>>>>> register here. < http://www.idug.org/component/juser/register.html >
>>>>>
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>
>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>>> to use a new password. You should have gotten an e-mail with the temporary
>>>> password assigned to your account. Please log in and update your member
>>>> profile. If you are not already an IDUG.org member, please register
>>>> here. < http://www.idug.org/component/juser/register.html >
>>>>
>>>
>>>
>>>
>>> --
>>> Robert Catterall
>>> Catterall Consulting
>>> www.catterallconsulting.com
>>>
>>> ------------------------------
>>>
>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>
>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>> to use a new password. You should have gotten an e-mail with the temporary
>>> password assigned to your account. Please log in and update your member
>>> profile. If you are not already an IDUG.org member, please register
>>> here. < http://www.idug.org/component/juser/register.html >
>>>
>>
>>
>> ------------------------------
>>
>> *IDUG 2009 - Europe * 5-9 October * Rome, Italy* < http://idug.org/lseu >
>>
>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>> to use a new password. You should have gotten an e-mail with the temporary
>> password assigned to your account. Please log in and update your member
>> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>>
>
>
>
> --
> Robert Catterall
> Catterall Consulting
> www.catterallconsulting.com
>
> ------------------------------
>
> *IDUG 2009 - Europe * 5-9 October * Rome, Italy* < http://idug.org/lseu >
>
> *IDUG.org* < http://www.idug.org > was recently updated requiring members to
> use a new password. You should have gotten an e-mail with the temporary
> password assigned to your account. Please log in and update your member
> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>

______________________________________________________________________

* IDUG 2009 Melbourne, Australia * 18-20 March * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html

Robert Catterall

Re: EDM POOL FULL condition after migrating to V8 NFM : more data
(in response to Renzo Razzetti)
Happy to have helped, Renzo.

Robert


On Tue, Jan 13, 2009 at 8:09 PM, Renzo razzetti <[login to unmask email]>wrote:

> Robert
>
> Thank you very much !!! The blog is very clear !!! Excellent !!!! Thank you
> again
>
> RR
>
>
> On Tue, Jan 13, 2009 at 10:23 PM, Robert Catterall <[login to unmask email]>wrote:
>
>> Renzo,
>>
>> I'm afraid that I don't quite understand the question, "How can affect the
>> RELEASE(DEALLOCATE) and the program not close the cursor?" Could you
>> re-phrase the question? If you're wondering about the effect of
>> RELEASE(DEALLOCATE) versus RELEASE(COMMIT) on cursor behavior, my
>> understanding is that there is no difference. Opening and closing of
>> cursors is not affected by the RELEASE specification of BIND PACKAGE.
>>
>> As to how you might suddenly get an EDM pool-full situation and then just
>> as suddenly see the problem disappear, I believe that this could have
>> something to do with the degree to which your CICS-DB2 threads are re-used.
>> I described this scenario in a blog entry that I posted just yesterday (
>> http://www.catterallconsulting.com/2009/01/db2-for-zos-looking-again-at.html).
>> I'll try to summarize here, and you can refer to the blog entry for more
>> information.
>>
>> Generally speaking, the thread used by an "online" CICS transaction
>> (versus a background transaction, which is kind of batch-like) will
>> terminate when that transaction completes (protected threads are an
>> exception to this rule, so you might check to see if you have any protected
>> CICS-DB2 threads defined in your system). When that happens (i.e., when the
>> CICS-DB2 thread is terminated at end-of-transaction), RELEASE(DEALLOCATE) is
>> no different than RELEASE(COMMIT) when it comes to retention of EDM pool
>> space. EDM pool space retention is affected when CICS-DB2 threads through
>> which packages bound with RELEASE(DEALLOCATE) are executed are re-used. For
>> a non-protected thread to be reused, this is what has to happen (assume that
>> transaction XYZ is using the non-protected threas in question):
>>
>> 1. Before transaction XYZ completes, another instance of transaction
>> XYZ (or of a different transaction, *if the different transaction is
>> associated with the same CICS DB2ENTRY resource as transaction XYZ*)
>> is queued up waiting for a thread because no threads are available (they're
>> all being used).
>> 2. Thransaction XYZ completes, and the queued transaction reuses the
>> thread that transaction XYZ had used.
>>
>> In many cases, people have enough entry and pool threads defined so that
>> transactions rarely queue up waiting for a thread to become available. If
>> there's no queuing, the non-protected threads won't be reused. Instead,
>> they will be deallocated at end-of-tran, and so RELEASE(DEALLOCATE) won't
>> affect EDM pool space retention. If you're "close to the line" in this
>> regard, with a thread re-use rate that is small but non-zero, a relatively
>> small increase in the transaction arrival rate could cause wait-for-thread
>> queuing (and therefore, thread re-use) to jump, and this - combined with
>> packages bound with RELEASE(DEALLOCATE) - can cause a spike in EDM pool
>> space retention. That, in turn, could cause some pool-full failures to
>> occur. I described some corrective actions in the previously-cited blog
>> entry.
>>
>> Hope this helps.
>>
>> Robert
>>
>>
>> On Tue, Jan 13, 2009 at 1:13 AM, Renzo razzetti <[login to unmask email]
>> > wrote:
>>
>>> Robert
>>>
>>> We don't have stored procedures yet in production. I misunderstand, I
>>> thought that MAX_NUM_CUR is for max cursors in any kind of thread. Sorry.
>>>
>>> I understand your point about the new global locking protocol introduced
>>> with DB2 for z/OS V8 . We will consider it.
>>>
>>> Anyway I would like to find the cause of my EDM Pool full condition last
>>> time. I saw in the Accounting Report, in the 2 seconds where we have the
>>> problem, we have 801 OPEN and 721 CLOSE. How can affect the
>>> RELEASE(DEALLOCATE) and the program not close the cursor ?
>>>
>>> Thank you very much for your time
>>> RR
>>>
>>>
>>>
>>>
>>> On Mon, Jan 12, 2009 at 10:25 PM, Robert Catterall <
>>> [login to unmask email]> wrote:
>>>
>>>> My understanding is that MAX_NUM_CUR applies to DB2 for z/OS stored
>>>> procedures. Are you using stored procedures? If so, do you have
>>>> MAX_NUM_CUR set to its default value (500)?
>>>>
>>>> To reiterate a point I made in my previous response: you indicated that
>>>> you have packages bound with RELEASE(DEALLOCATE) "as we are in data
>>>> sharing." This implies that RELEASE(DEALLOCATE) *should* be used as a
>>>> bind option in a data sharing environment, and as I pointed out the new
>>>> global locking protocol introduced with DB2 for z/OS V8 removes the
>>>> global-lock-contention-reduction incentive to use RELEASE(DEALLOCATE).
>>>> RELEASE(DEALLOCATE) can improve a DB2 program's CPU efficiency, but the
>>>> decision as to whether or not to use this bind option is no longer data
>>>> sharing-related, as far as I'm concerned.
>>>>
>>>> Robert
>>>>
>>>>
>>>>
>>>> On Sun, Jan 11, 2009 at 8:01 PM, Renzo razzetti <
>>>> [login to unmask email]> wrote:
>>>>
>>>>> Hello Robert
>>>>>
>>>>> Ok, I have researched about yours suggestions. I'm continuing
>>>>> investigate.
>>>>> Yes, The packages running at that time (peak hour) are bound with
>>>>> RELEASE(DEALLOCATE) as we are in datasharing .
>>>>>
>>>>> One possible cause of the problem is described in the IBM Technote
>>>>> http://www-01.ibm.com/support/docview.wss?uid=swg21294759. In this
>>>>> technote explain that Storage shortages can occur if too many cursors that
>>>>> return result sets are open at the same time. To minimize these storage
>>>>> shortages, subsystem parameters MAX_NUM_CUR controls the maximum number of
>>>>> cursors that can be open concurrently. When the value from this subsystem
>>>>> parameter is exceeded while an application is running, the CALL statement
>>>>> receives SQLCODE -904.
>>>>>
>>>>>
>>>>> I would like to know where I can confirm (in which Report) whether we
>>>>> hit the MAX_NUM_CUR opened current in the thread.
>>>>>
>>>>> I apprecciate very much your help
>>>>>
>>>>> RR
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> In this technote explain that Storage shortages can occur if too many
>>>>> cursors that return result sets are open at the same time. To minimize these
>>>>> storage shortages, subsystem parameters MAX_NUM_CUR controls the maximum
>>>>> number of cursors that can be open concurrently.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jan 8, 2009 at 10:46 PM, Robert Catterall <
>>>>> [login to unmask email]> wrote:
>>>>>
>>>>>> Renzo,
>>>>>>
>>>>>> Is your DB2 workload (during the "EDM pool full" times) primarily
>>>>>> comprised of CICS-DB2 transactions? If so, are many of the associated DB2
>>>>>> packages bound with RELEASE(DEALLOCATE)? Are you using any protected
>>>>>> CICS-DB2 threads (specified via PROTECTNUM in CICS RDO DB2ENTRY resource
>>>>>> definitions)?
>>>>>>
>>>>>> Historically, a lot of DB2 for z/OS data sharing users went with
>>>>>> RELEASE(DEALLOCATE) for package bind in order to reduce tablespace lock
>>>>>> propagation to the lock structure. In CICS-DB2 environments, tablespace
>>>>>> lock propagation could be further reduced through the use of CICS protected
>>>>>> entry threads. People were interested in reducing tablespace lock
>>>>>> propagation because such requests often resulted in what's called XES
>>>>>> contention - something that increased data sharing overhead. An
>>>>>> unfortunate side-effect of RELEASE(DEALLOCATE) - especially when protected
>>>>>> threads were also used - was an increase (sometimes quite substantial) in
>>>>>> the usage of EDM pool space for sections of in-use packages (the "PT" part
>>>>>> of the pool) - this because a package bound with RELEASE(DEALLOCATE) is
>>>>>> considered to be "in-use" until the associated thread is deallocated (as
>>>>>> opposed to being released at SYNCPOINT in the case of a CICS-DB2 tran). In
>>>>>> some cases, even minor fluctuations in the CICS-DB2 transaction workload can
>>>>>> lead to significant chnages in the rate of CICS-DB2 thread reuse, and that
>>>>>> (when RELEASE(DEALLOCATE) is the package bind option) can lead to dramatic
>>>>>> shifts in "in-use" versus "stealable" PT-related EDM pool pages.
>>>>>>
>>>>>> The good news with respect to DB2 V8: due to a new global locking
>>>>>> protocol, RELEASE(DEALLOCATE) is no longer needed to reduce the incidence of
>>>>>> the type of global lock contention known as XES contention (I wrote about
>>>>>> this in a blog entry last summer:
>>>>>> http://www.catterallconsulting.com/2008/08/some-interesting-db2-for-zos-data.html).
>>>>>> So, if you have a fair number of CICS-DB2 program packages bound with
>>>>>> RELEASE(DEALLOCATE), and if that bind specification decision was made so as
>>>>>> to reduce data sharing lock contention, consider changing the bind
>>>>>> specification to RELEASE(COMMIT). In a DB2 V8 system you won't pay a global
>>>>>> lock contention penalty for doing this, thanks to the afrementioned global
>>>>>> locking protocol enhancement (known as "locking protocol 2," as I recall).
>>>>>>
>>>>>> This may or may not explain what you are seeing. Something for you to
>>>>>> consider and investigate.
>>>>>>
>>>>>> Robert
>>>>>>
>>>>>> On Thu, Jan 8, 2009 at 2:50 AM, Renzo razzetti <
>>>>>> [login to unmask email]> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I m adding more data in my question to see whether there is someone
>>>>>>> in the list can help me.
>>>>>>> Following is the statistic from 1 minute interval. The EDM Pool is
>>>>>>> always around 40% used, but suddenly the PAGES of PT increase almost to 97%
>>>>>>> of the EDM Pool.
>>>>>>> I want to find the root cause of the problem, to avoid happen other
>>>>>>> time. I'm already checked the number of transaction in CICS and the number
>>>>>>> of thread in DB2 and they are not increase.
>>>>>>>
>>>>>>> Could be this a bug in the EDM pool DB2 v8 ?
>>>>>>>
>>>>>>> I will appreciate any suggestion to further analysis this problem.
>>>>>>>
>>>>>>> RR
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> EDM POOL DP21 16:20 16:21 16:22 16:21 16:21 16:22 16:23
>>>>>>> 16:27 Elapsed time
>>>>>>> 1 1 1 6 PAGES IN EDM POOL
>>>>>>> (BELOW) 37500 37500 37500 37500 HELD BY CT
>>>>>>> 176 290.42 588 234.67 HELD BY PT 4936 13113.24
>>>>>>> 34380 8780.87 HELD BY SKCT 9 8.17 6 10.17 HELD
>>>>>>> BY SKPT 11415 8716.93 1700 7322.49 FREE PAGES
>>>>>>> 20964 15371.24 826 21151.81 % PAGES IN USE
>>>>>>> 44.1 59.01 97.8 43.6 % NON STEAL. PAGES IN USE
>>>>>>> 13.63 35.74 93.25 24.04 FAILS DUE TO POOL FULL 0 0 0 66
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Jan 4, 2009 at 10:43 AM, Renzo razzetti <
>>>>>>> [login to unmask email]> wrote:
>>>>>>> >
>>>>>>> > hello everybody !
>>>>>>> >
>>>>>>> > I'm running DB2 V8 NFM, Cobol LOCAL applications in datasharing
>>>>>>> enviorment.
>>>>>>> > Not dynamic application are running, not distributed application is
>>>>>>> running, not storage procedure is running.
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > I have received this error in production. 66 times in the 1 second
>>>>>>> interval. And the application canceled with RC -904 resource unavailable.
>>>>>>> >
>>>>>>> > 16.22.44 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE
>>>>>>> 004
>>>>>>> > 004 REASON 00C90089
>>>>>>> > 004 TYPE 00000600
>>>>>>> > 004 NAME EDM POOL SPACE
>>>>>>> > ...................................
>>>>>>> > 16.22.45 STC28433 DSNT500I -DP21 DSNGEPLC RESOURCE UNAVAILABLE
>>>>>>> 013
>>>>>>> > 013 REASON 00C90089
>>>>>>> > 013 TYPE 00000600
>>>>>>> >
>>>>>>> >
>>>>>>> > If we look in the statistic report the EDM Pool for 15 interval
>>>>>>> during 16:15 to 16:30, we can see only the 40% of the pages are in use. It
>>>>>>> looks the EDM Pool is not full.
>>>>>>> > Can any help to understand what is the problem ? How I can find the
>>>>>>> problem ?
>>>>>>> > Thank you.
>>>>>>> >
>>>>>>> >
>>>>>>> > EDM POOL QUANTITY /SECOND /THREAD /COMMIT
>>>>>>> > --------------------------- -------- ------- ------- -------
>>>>>>> > PAGES IN EDM POOL (BELOW) 37500.00 N/A N/A N/A
>>>>>>> > HELD BY CT 208.86 N/A N/A N/A
>>>>>>> > HELD BY PT 6682.08 N/A N/A N/A
>>>>>>> > HELD BY SKCT 9.79 N/A N/A N/A
>>>>>>> > HELD BY SKPT 9193.57 N/A N/A N/A
>>>>>>> > FREE PAGES 21405.71 N/A N/A N/A
>>>>>>> > % PAGES IN USE 42.92 N/A N/A N/A
>>>>>>> > % NON STEAL. PAGES IN USE 18.38 N/A N/A N/A
>>>>>>> > FAILS DUE TO POOL FULL 66.00 0.08 0.00 0.00
>>>>>>> >
>>>>>>> > PAGES IN DBD POOL (ABOVE) 20000.00 N/A N/A N/A
>>>>>>> > HELD BY DBD 7632.00 N/A N/A N/A
>>>>>>> > FREE PAGES 12368.00 N/A N/A N/A
>>>>>>> > FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00
>>>>>>> >
>>>>>>> > PAGES IN STMT POOL (ABOVE) 20000.00 N/A N/A N/A
>>>>>>> > HELD BY STATEMENTS 18916.21 N/A N/A N/A
>>>>>>> > FREE PAGES 1083.79 N/A N/A N/A
>>>>>>> > FAILS DUE TO STMT POOL FULL 0.00 0.00 0.00 0.00
>>>>>>> >
>>>>>>> > DBD REQUESTS 70916.00 83.29 1.01 0.38
>>>>>>> > DBD NOT FOUND 0.00 0.00 0.00 0.00
>>>>>>> > DBD HIT RATIO (%) 100.00 N/A N/A N/A
>>>>>>> > CT REQUESTS 70178.00 82.42 1.00 0.38
>>>>>>> > CT NOT FOUND 11.00 0.01 0.00 0.00
>>>>>>> > CT HIT RATIO (%) 99.98 N/A N/A N/A
>>>>>>> > PT REQUESTS 3385.0K 3975.56 48.22 18.18
>>>>>>> > PT NOT FOUND 3592.00 4.22 0.05 0.02
>>>>>>> > PT HIT RATIO (%) 99.89 N/A N/A N/A
>>>>>>> >
>>>>>>> > PKG SEARCH NOT FOUND 4018.8K 4719.88 57.25 21.58
>>>>>>> > PKG SEARCH NOT FOUND INSERT 3040.00 3.57 0.04 0.02
>>>>>>> > PKG SEARCH NOT FOUND DELETE 3041.00 3.57 0.04 0.02
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>>>>
>>>>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring
>>>>>>> members to use a new password. You should have gotten an e-mail with the
>>>>>>> temporary password assigned to your account. Please log in and update your
>>>>>>> member profile. If you are not already an IDUG.org member, please
>>>>>>> register here. < http://www.idug.org/component/juser/register.html >
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Robert Catterall
>>>>>> Catterall Consulting
>>>>>> www.catterallconsulting.com
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>>>
>>>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring
>>>>>> members to use a new password. You should have gotten an e-mail with the
>>>>>> temporary password assigned to your account. Please log in and update your
>>>>>> member profile. If you are not already an IDUG.org member, please
>>>>>> register here. < http://www.idug.org/component/juser/register.html >
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>>
>>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring
>>>>> members to use a new password. You should have gotten an e-mail with the
>>>>> temporary password assigned to your account. Please log in and update your
>>>>> member profile. If you are not already an IDUG.org member, please
>>>>> register here. < http://www.idug.org/component/juser/register.html >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Robert Catterall
>>>> Catterall Consulting
>>>> www.catterallconsulting.com
>>>>
>>>> ------------------------------
>>>>
>>>> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>>>>
>>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>>> to use a new password. You should have gotten an e-mail with the temporary
>>>> password assigned to your account. Please log in and update your member
>>>> profile. If you are not already an IDUG.org member, please register
>>>> here. < http://www.idug.org/component/juser/register.html >
>>>>
>>>
>>>
>>> ------------------------------
>>>
>>> *IDUG 2009 - Europe * 5-9 October * Rome, Italy* < http://idug.org/lseu >
>>>
>>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>>> to use a new password. You should have gotten an e-mail with the temporary
>>> password assigned to your account. Please log in and update your member
>>> profile. If you are not already an IDUG.org member, please register
>>> here. < http://www.idug.org/component/juser/register.html >
>>>
>>
>>
>>
>> --
>> Robert Catterall
>> Catterall Consulting
>> www.catterallconsulting.com
>>
>> ------------------------------
>>
>> *IDUG 2009 - Europe * 5-9 October * Rome, Italy* < http://idug.org/lseu >
>>
>> *IDUG.org* < http://www.idug.org > was recently updated requiring members
>> to use a new password. You should have gotten an e-mail with the temporary
>> password assigned to your account. Please log in and update your member
>> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>>
>
>
> ------------------------------
>
> *IDUG 2009 - Australasia * 18-20 March * Melbourne, Australia* < http://idug.org/lsAU >
>
> *IDUG.org* < http://www.idug.org > was recently updated requiring members to
> use a new password. You should have gotten an e-mail with the temporary
> password assigned to your account. Please log in and update your member
> profile. If you are not already an IDUG.org member, please register here. < http://www.idug.org/component/juser/register.html >
>



--
Robert Catterall
Catterall Consulting
www.catterallconsulting.com

______________________________________________________________________

* IDUG 2009 Melbourne, Australia * 18-20 March * http://IDUG.ORG/Events *
______________________________________________________________________




IDUG.org was recently updated requiring members to use a new password. You should have gotten an e-mail with the temporary password assigned to your account. Please log in and update your member profile. If you are not already an IDUG.org member, please register at http://www.idug.org/component/juser/register.html