Db2 print omon report problem

Tsui Yuk Kai

Db2 print omon report problem
Hi all,
Our shop have around 400gb smf 101 records daily. We spend almost 6 Horus
to print report and use huge cpu time to print by using DB2PM reporting
program. Is there any way to trim down the cpu usage? Any shop have some
problem?

Thanks for sharing

Philip Sevetson

Db2 print omon report problem
(in response to Tsui Yuk Kai)
Tommy,

I defer to other SMF reporting experts on this matter, but here’s the quick take.


1) Do you collect for Prod only, or test as well? How many trace classes and IFCIDs do you have activated, and are any of them known to generate large amounts of data (this information is “soft”, and is published in several places)?

2) Review specifically what you’re gathering with your traces. You’re almost certainly taking in too much, lots of stuff which you don’t actually need:

a. Do a summary of your 400GB by trace type/class/IFCID, and see what is taking up most of the space.

b. Backtrack to what the biggest space hogs are, find out why they’re being collected (who thinks they need them).

c. Ask the stakeholder(s) whether they’re getting enough value from them to justify all that CPU time (it’s NOT free, there are allocated costs, and they should be prepared to justify them).

3) How many STATistical classes do you gather, at what intervals?
Statistical intervals usually aren’t the problem unless you have the collection interval set at “every few seconds” instead of “five to ten minutes”. Checking on STAT, and MONitor classes is just due diligence.

Without knowing anything about your circumstances, I will comment that a lot of people with this problem have between one and a dozen IFCIDs collected which are responsible for most of the data, and those can be turned off for most of the time – the “busiest” IFCIDs should only be turned on when there is a clear, identified need for a particular report.

--Phil

From: Tommy Tsui [mailto:[login to unmask email]
Sent: Wednesday, April 11, 2018 8:37 AM
To: [login to unmask email]
Subject: [DB2-L] - Db2 print omon report problem

Hi all,
Our shop have around 400gb smf 101 records daily. We spend almost 6 Horus to print report and use huge cpu time to print by using DB2PM reporting program. Is there any way to trim down the cpu usage? Any shop have some problem?

Thanks for sharing

-----End Original Message-----
**This e-mail, including any attachments, may be confidential, privileged, or otherwise legally protected. It is intended only for the addressee. If you received this e-mail in error or from someone who was not authorized to send it to you, do not disseminate, copy, or otherwise use this e-mail or its attachments. Please notify the sender immediately by reply e-mail and delete the e-mail from your system.**

Michael Hannan

RE: Db2 print omon report problem
(in response to Tsui Yuk Kai)

In Reply to Tsui Yuk Kai:

Hi all,
Our shop have around 400gb smf 101 records daily. We spend almost 6 Horus
to print report and use huge cpu time to print by using DB2PM reporting
program. Is there any way to trim down the cpu usage? Any shop have some
problem?

Processing your SMF is always going to use a lot of CPU, hopefully off peak CPU, that you don't get billed for. You don't say if your DB2PM reports are detail or summary, so I suppose summary style.

Most shops do not print the detail SMF 101 records in full. DB2PM detail reports are lengthy, even if summarized. Instead can summarize the Thread level records and also the Package level records by date and  hour and other keys fields for your critical peak hours of the day, or all hours and decide which hours to keep long term later when peak is known. 

Possibly also summarize by day and capture which hours were the peak ones in the summary, depending on how you billed for your usage.
 
DB2PE (performance Expert) has capabilities to keep detail records, perhaps only for exception cases of long running threads, and also to summarize the data. You can then load summary data into Db2 Tables, selecting which important fields you want to keep the history of for longer term. I usually keep all numeric fields as FLOAT(4) (also known as REAL) since 8 bytes of FLOAT is excessive precision.

Decide which unimportant columns can be omitted from the Summary Key, to ensure less records kept. e.g. in Data Sharing,might not keep the records for same key but different data sharing members. Perhaps keep only the member with highest contribution and the percent in that member. That way if a CICS Tran runs on 4 members, summary can be one record instead of 4 records.

I have sometimes decided to keep history summarised by day, and keep 24 columns in the row showing the portion of Class 2 CPU in each hour of that day. Any zero hours compress well.  This cuts the number of records kept in history by a lot.

Finally can use SQLs or QMF, or whatever to report from your history tables.

It takes significant time to set up your SMF History storage tables, but once you have the good data, it is very beneficial. It can take quite a few hours at night to do the SMF summarization too, but the results are much more usable that a large to massive DB2PM report.

So my only real options for CPU reduction, are choosing limited important columns from available columns in SMF 101, and summarising by time interval, and discarding some key fields not important, from the Grouping.

Not an easy project that you can implement very quickly. The SMF data is too complex and too much variation depending on Connection Types writing that data.

Try to make Grouping fields have consistency between Thread level records and Package level records, so there is a possibility to join them together later.

Michael Hannan,
DB2 Application Performance Specialist
CPT Global Ltd

Avram Friedman

RE: Db2 print omon report problem
(in response to Tsui Yuk Kai)

You dont provide a reason or use for the reports.
Have you considered not producing them?

In Reply to Tsui Yuk Kai:

Hi all,
Our shop have around 400gb smf 101 records daily. We spend almost 6 Horus
to print report and use huge cpu time to print by using DB2PM reporting
program. Is there any way to trim down the cpu usage? Any shop have some
problem?

Thanks for sharing



Avram Friedman
DB2-L hall of fame contributor
DB2-L 'past' administrator

[login to unmask email]

Michael Hannan

RE: Db2 print omon report problem
(in response to Avram Friedman)

In Reply to Avram Friedman:

You dont provide a reason or use for the reports.
Have you considered not producing them?

Some collection of Db2 Acctng data from SMF is needed, if we are going to be able to discover what DB2 processes cost a lot of money, and measure the improvement after tuning.  Of course it is not always a priority with IBM, that customer might be able to reduce the bills for CPU engines and for software, or avoid early migration to a bigger box. LOL

Up to 50% of the effort in Performance Tuning can be the measurement process. 

There are plenty of DB2 customers who do not have a good handle on what is costing them the most in the peak hours, due to lack of SMF capture and summarisation. Same applies to non DB2 as well. DB2 Acctg data is not enough.

Yes there is no point is the results of SMF summarisation will not be used. It is very tough to get meaningful performance tuning results when there is no cost and performance data collection, on a historical basis.

In general, serious heavy usage DB2 shops, need to capture some DB2 Acctng data, to make measurement and tuning possible to focus on the right things that are high cost. One data is sufficiently summarised, it becomes viable to keep the history for significant time.

Michael Hannan,
DB2 Application Performance Specialist
CPT Global Ltd

Edited By:
Michael Hannan[Organization Members] @ Apr 13, 2018 - 05:59 AM (Europe/Berlin)
Michael Hannan[Organization Members] @ Apr 13, 2018 - 06:02 AM (Europe/Berlin)

Norbert Jenninger

RE: Db2 print omon report problem
(in response to Tsui Yuk Kai)

Depending on the variety of the application workload customers have seen remarkable reduction of CPU and elapsed time with large SMF 101 dataset size (> 90 Mio) while using the TKANMOD load lib of OMEGAMON DB2 V540.

Avram Friedman

RE: Db2 print omon report problem
(in response to Michael Hannan)

With due respect, you are not the original poster asking for advice on this CPU concern.
The original poster should speak for them selfs.

Yes accounting data (and more) may be required to resolve performance problems.
Generally speaking procative performance evaluation goes from the general to specific.
Reactive starts at the specific.
In DB2 trace records the general tends to be the statistics records.

Best wishes
Av

Who, among other things, is the father of Omegamon DB2 

In Reply to Michael Hannan:

In Reply to Avram Friedman:

You dont provide a reason or use for the reports.
Have you considered not producing them?

Some collection of Db2 Acctng data from SMF is needed, if we are going to be able to discover what DB2 processes cost a lot of money, and measure the improvement after tuning.  Of course it is not always a priority with IBM, that customer might be able to reduce the bills for CPU engines and for software, or avoid early migration to a bigger box. LOL

Up to 50% of the effort in Performance Tuning can be the measurement process. 

There are plenty of DB2 customers who do not have a good handle on what is costing them the most in the peak hours, due to lack of SMF capture and summarisation. Same applies to non DB2 as well. DB2 Acctg data is not enough.

Yes there is no point is the results of SMF summarisation will not be used. It is very tough to get meaningful performance tuning results when there is no cost and performance data collection, on a historical basis.

In general, serious heavy usage DB2 shops, need to capture some DB2 Acctng data, to make measurement and tuning possible to focus on the right things that are high cost. One data is sufficiently summarised, it becomes viable to keep the history for significant time.

Michael Hannan,
DB2 Application Performance Specialist
CPT Global Ltd



Avram Friedman
DB2-L hall of fame contributor
DB2-L 'past' administrator

[login to unmask email]