Increase in IFCID frequency

Steve Reznikov

Increase in IFCID frequency

Experts,

Our job that processes SMF data and creates daily accounting and statistics reports started running much longer several days ago. Upon detailed investigation, it was determined that the number of IFCID traces has increased tenfold. Any idea what can be causing this? No additional tracing was turned on prior to the increase.

Thanks.

Before                                                   

           INPUT        INPUT      PROCESSED     PROCESSED    

 IFCID     COUNT     PCT OF TOTAL    COUNT     PCT OF TOTAL   

-------  ----------  ------------  ----------  ------------   

    1         1,438      0.01%          1,438      0.01%      

    2         2,847      0.02%          2,847      0.02%      

    3     4,757,914     49.00%      4,757,914     49.00%      

  105        26,150      0.27%         26,150      0.27%      

  106         1,437      0.01%          1,437      0.01%      

  172             9      0.00%              9      0.00%      

                                                              

            INPUT        INPUT      PROCESSED     PROCESSED   

  IFCID     COUNT     PCT OF TOTAL    COUNT     PCT OF TOTAL  

 -------  ----------  ------------  ----------  ------------  

   202         1,438      0.01%          1,438      0.01%     

   225         1,438      0.01%          1,438      0.01%     

   239     4,759,315     49.00%      4,759,315     49.00%     

   258           755      0.00%            755      0.00%     

   313           433      0.00%            433      0.00%     

                                                              

 

After

 

 

            INPUT        INPUT      PROCESSED     PROCESSED 

  IFCID     COUNT     PCT OF TOTAL    COUNT     PCT OF TOTAL

 -------  ----------  ------------  ----------  ------------

     1         1,440      0.00%          1,440      0.00%   

     2         2,880      0.00%          2,880      0.00%   

     3    32,593,665     49.00%     32,593,665     49.00%   

   105        48,960      0.07%         48,960      0.07%   

   106         1,440      0.00%          1,440      0.00%   

   172             8      0.00%              8      0.00%     

          INPUT        INPUT      PROCESSED     PROCESSED 

 IFCID     COUNT     PCT OF TOTAL    COUNT     PCT OF TOTAL

-------  ----------  ------------  ----------  ------------

  196            12      0.00%             12      0.00%   

  202         1,440      0.00%          1,440      0.00%   

  225         1,440      0.00%          1,440      0.00%   

  239    32,598,224     49.00%     32,598,224     49.00%   

  258           294      0.00%            294      0.00%   

  313         2,058      0.00%          2,058      0.00%   

Michael Hannan

RE: Increase in IFCID frequency
(in response to Steve Reznikov)

Steve,

As a wild guess, did something turn on Class 10 Accounting. This provides much more detail at the Package level, making the records longer and likely to use more overflow records. I had not thought it could be a factor of 10 bigger. The overhead to externalise all the extra info to SMF is thought to be quite small according to IBM Red Book on performance, as Class 2 and 7 are significant cost and doing most of the work already..

Was there any change of thread accounting rollup. This idea seems much less likely. Just throw it into the mix.


In Reply to Steve Reznikov:

Experts,

Our job that processes SMF data and creates daily accounting and statistics reports started running much longer several days ago. Upon detailed investigation, it was determined that the number of IFCID traces has increased tenfold. Any idea what can be causing this? No additional tracing was turned on prior to the increase.



Michael Hannan,
DB2 Application Performance Specialist
CPT Global Ltd

Norbert Jenninger

RE: Increase in IFCID frequency
(in response to Steve Reznikov)

Not only the package data (IFCID 239) but also the Accounting plan level records number increased from 4.75 M to 32.6 M number of records which indicates a change in the way how often Db2 is cutting Accounting records.

There exist several parameters which will influence Db2 in cutting accounting records. Among theses these are DB2 ZPARMS ACCUMAC, PTASKROL, or CMTSTAT, and any changes in the CICS Tx pooling.

(For example, ACCUMAC default is 10, if this was changed to None or ‘0’ then you may see 10 times more accounting records externalized by Db2.    

Regards

Norbert Jenninger

Michael Hannan

RE: Increase in IFCID frequency
(in response to Norbert Jenninger)

Yes good point IFCID 3 Accounting Thread Detail increased too, not just Thread Package level data. I like to think of it as Thread level since Plan is just one column of a row identifier, not very important at that. Mystery to me why it is often known as Plan level.

Michael Hannan,
DB2 Application Performance Specialist
CPT Global Ltd

Peter Conlin

Increase in IFCID frequency
(in response to Michael Hannan)
IFCID 196 (TIMEOUT) was absent in the BEFORE, but there were 12 in the AFTER.

Could there have been massive rollbacks?

From: Michael Hannan [mailto:[login to unmask email]
Sent: Wednesday, March 13, 2019 8:03 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: Increase in IFCID frequency


Yes good point IFCID 3 Accounting Thread Detail increased too, not just Thread Package level data. I like to think of it as Thread level since Plan is just one column of a row identifier, not very important at that. Mystery to me why it is often known as Plan level.

Michael Hannan,
DB2 Application Performance Specialist
CPT Global Ltd

-----End Original Message-----

Steve Reznikov

Thanks all. I just want to reiterate that nothing was changed the way we collect trace data. RE: Increase in IFCID frequency
(in response to Michael Hannan)

Thanks all. 

I just want to reiterate that nothing was changed in the way we collect trace data. 

Norbert,

The first thing that I did, I checked the ZPARM setting to make sure that ACCUMACC was still set to 10.

This is very puzzling.

Steve.

Dil Sasidharan

Thanks all. I just want to reiterate that nothing was changed the way we collect trace data. Increas
(in response to Steve Reznikov)
Are these additional records generated by DDF transactions?

> On Mar 13, 2019, at 10:26 AM, Steve Reznikov <[login to unmask email]> wrote:
>
> Thanks all.
>
> I just want to reiterate that nothing was changed in the way we collect trace data.
>
> Norbert,
>
> The first thing that I did, I checked the ZPARM setting to make sure that ACCUMACC was still set to 10.
>
> This is very puzzling.
>
> Steve.
>
>
> Site Links: View post online View mailing list online Start new thread via email Unsubscribe from this mailing list Manage your subscription
>
> This email has been sent to: [login to unmask email]
> ESAi has well-regarded tools for Fast Cloning, Buffer Pool Tuning, Log Analysis, TDM & more.
> BCV4, BCV5, BPA4DB2, ULT4DB2... modern power tools to get the job done faster & easier than ever.
> http://www.ESAIGroup.com/idug
>
>
> Use of this email content is governed by the terms of service at:
> http://www.idug.org/p/cm/ld/fid=2
>

Steve Reznikov

RE: Thanks all. I just want to reiterate that nothing was changed the way we collect trace data. Increas
(in response to Dil Sasidharan)

Bingo!

Yes Dil, we found that all this data was generated by the SERVER traffic and asked our distributed guys to look into it.

Thanks all for your help.

Dil Sasidharan

Thanks all. I just want to reiterate that nothing was changed the way we collect trace data. Increas
(in response to Steve Reznikov)
Hi Steve,

A misconfigured Spark client instance could create thousands of worker threads and could flood your DBATs. You can check SMF 100 and look at the DBAT numbers/queueing etc.

> On Mar 13, 2019, at 12:09 PM, Steve Reznikov <[login to unmask email]> wrote:
>
> Bingo!
>
> Yes Dil, we found that all this data was generated by the SERVER traffic and asked our distributed guys to look into it.
>
> Thanks all for your help.
>
>
> Site Links: View post online View mailing list online Start new thread via email Unsubscribe from this mailing list Manage your subscription
>
> This email has been sent to: [login to unmask email]
> ESAi has well-regarded tools for Fast Cloning, Buffer Pool Tuning, Log Analysis, TDM & more.
> BCV4, BCV5, BPA4DB2, ULT4DB2... modern power tools to get the job done faster & easier than ever.
> http://www.ESAIGroup.com/idug
>
>
> Use of this email content is governed by the terms of service at:
> http://www.idug.org/p/cm/ld/fid=2
>