Performance Database (perhaps with an [AD] added)

Phil Grainger

Performance Database (perhaps with an [AD] added)
Dan, I agree 100%

Most performance database problems will stem from trying to measure and keep EVERYTHING

Do you really want to know the performance of EVERY individual SQL statement (or thread) - most (all?) of the time the aggregated values will be perfectly acceptable

This is why most performance analysis tools (Query Monitor, Detector Apptune et al) show you averages AND maximums

But why stop aggregating at statement/thread level - why not go to the next step and aggregate on what the SQL is DOING (and not how it's written)?

One of the reasons I'm where I am now is the consolidation possibilities that you get with EZ-DB2. But ignoring products for the moment, I really didn't think it was possible to collect SQL statement level performance data and get anything other than a huge pile of data - and the more you have the LESS sense it makes, not more. SQL consolidation brings your SQL statement executions down to less than 20% of the initial collected total - making analysis and tuning a much more pleasant prospect. This happens really for a number of reasons:

1. A lot of people still code SQL with literals or with varying length IN LISTSs - SQL monitors usually regard these as all being different statements
2. No two developers will code the same statement in the same way - even if it's down to whitespace. What about varying sequences of columns in SELECT lists, varying sequences of tables in FROM clauses?
3. Most businesses don't ask an infinite number of questions of their databases - usually there are only a few different things you need

Don't forget, you are not normally trying to tune a SINGLE SQL statement (not if you are tuning proactively anyway) but are just trying to make all your "stuff" run faster and better

Just my 2c for a Friday

Phil Grainger
Cogito Ltd.
[login to unmask email]
+44 (0) 1298 872 148
+44 (0) 7505 266 768
www.cogito.co.uk

Attend IDUG 2011 - the premiere events for DB2 professionals.
IDUG North America, 2-6 May, Anaheim California
IDUG EMEA, 14-18 November, Prague Czech Republic



-----Original Message-----
From: IDUG DB2-L [mailto:[login to unmask email] On Behalf Of Daniel Luksetich
Sent: 20 January 2011 21:27
To: [login to unmask email]
Subject: Re: [DB2-L] Performance Database

You can condense the thread data before loading. I find a performance DB critical. Especially when the application is capturing performance metrics which you can correlate to the performance DB. I call that a super performance DB, and the information it delivers is awesome.

So many people do fire fighting and never look at the big picture. They miss major tuning opportunities. A performance DB helps get that picture.

Dan

-----Original Message-----
From: IDUG DB2-L [mailto:[login to unmask email] On Behalf Of [login to unmask email]
Sent: Thursday, January 20, 2011 3:11 PM
To: [login to unmask email]
Subject: [SPAM] Re: Performance Database

I cannot argue with the statement either.

Even with MXG there is too much data to look at.
I only use this to look at small pieces of time to investigate problems
with performance or conflicts.

Otherwise I could spend all of my time loading and reporting on the data.

It would also require more DASD that I am willing to expend.

Carol Sutfin
Corporate DBA
Regions Financial Corp.
(205)261-5214
[login to unmask email]



From: Joel Goldstein - Responsive Systems
<[login to unmask email]>
To: [login to unmask email]
Date: 01/20/2011 02:57 PM
Subject: Re: [DB2-L] Performance Database
Sent by: IDUG DB2-L <[login to unmask email]>



I have to second Avram's caveat.
If you have a small to medium size DB2 system, it may be fine.
If you have a large high volume system, the PDB may become one of your
major resource consumers.

Regards,
Joel

Joel Goldstein
Responsive Systems
IBM Gold Consultant
Buffer Pool Tool for DB2, the worldwide industry standard
Performance software that works...... Predicts IO Rate !!
Predicts Group Buffer Pool performance too
www.responsivesystems.com

Buffer Pool Tool for DB2 on www.LinkedIn.com
Watch the 3-Minute Buffer Pool Tool Movie at:
www.responsivesystems.com/Movie1

tel. (732) 972-1261
fax.(732) 972-9416
----- Original Message -----
From: "Avram Friedman" <[login to unmask email]>
Newsgroups: bit.listserv.db2-l
To: <[login to unmask email]>
Sent: Thursday, January 20, 2011 3:00 PM
Subject: Re: [DB2-L] Performance Database

You might want to think twice or even 3 times about building a DB2
performace database using DB2 tables.

The 2 most common activities against a performace database are
INSERT
DELETE (for purge)

Guess what 2 basic SQL DML commands tend to be the most expensive.

DB2 is in general designed to store data with integerety and
recoverability.
It has an associated cost.

You might want to consider Berry Merrils MXG system that will store your
data as a SAS file.
It comes with lots of sample querry functions that are easy to modify.
The down side is SAS tends to be sequencial processing..
Create Daily files, they can be concatanated together if you need longer
history
and they are easy to discard.

Avram Friedman


On Thu, 20 Jan 2011 09:23:03 -0800, jack fernicola <[login to unmask email]>
wrote:

>Hi,
>
>We are?building a?Performance Database and then would like to populate it
with raw DB2 SMF?(100,101,102) records for historical reporting. The?latest
Omegamon/DB2PM (4.2)product has been installed for this purpose.?
>?
>Does?anyone?have any sample JCL/jobs/mapping for extracting the DB2 SMF data
records?only to be use as?input into the LOAD utility jobs?
>?
>Thanks
>Jack????????
>?
>
>?
>?
>?



>
>_____________________________________________________________________
>* IDUG North America * Anaheim, California * May 2-6 2011 *
http://IDUG.ORG/NA *
>* Your only source for independent, unbiased, and trusted DB2
information. *
>_____________________________________________________________________
>http://www.IDUG.org/mentor
>Mentoring should be a rewarding experience for everyone...
>IDUG is offering up to 80% off when you both come to the conference!
>_____________________________________________________________________
>
>If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is
the home of IDUG's Listserv

_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 *
http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2
information. *
_____________________________________________________________________
http://www.IDUG.org/mentor
Mentoring should be a rewarding experience for everyone...
IDUG is offering up to 80% off when you both come to the conference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is
the home of IDUG's Listserv








The IDUG DB2-L Listserv is only part of your membership in IDUG. If you are
not already an IDUG member, please register here.


_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
_____________________________________________________________________
http://www.IDUG.org/mentor
Mentoring should be a rewarding experience for everyone...
IDUG is offering up to 80% off when you both come to the conference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.872 / Virus Database: 271.1.1/3383 - Release Date: 01/19/11 13:34:00

_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
_____________________________________________________________________
http://www.IDUG.org/mentor
Mentoring should be a rewarding experience for everyone...
IDUG is offering up to 80% off when you both come to the conference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

_____________________________________________________________________
* IDUG EMEA * Prague, Czech Republic * 14-18 November 2011 * http://IDUG.ORG/EMEA *
* If you are going to attend only one conference this year, this is it! *
** The most DB2 technical sessions of any conference
** Access IBM experts and developers
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

tim malamphy

Re: Performance Database
(in response to Phil Grainger)
Avram (the Elder)-

Hey, 10 bucks to a grad student in 72 was pretty good money, and SAS was still part of NCU (or NC state?) back when you were helping debug it. You could get 3 or 4 pitchers at the student union, or fill your gas tank twice.
Is MICS (from Marino/Legent) still around in some form? It kept EVERYTHING you could want, but you needed one or two dedicated admins, and a considerable amount of CPU to process SMF and keep it together. MXG seems to be the sole survivor from that era of Performance Databases, and was worth the price, just for the RMF/SMF record formats.

Thank goodness (as opposed to Goodnight, heh, heh,) my DB2/LUW monitor has a BUILT IN performance database (stored in db2) that is hardly any work at all to maintain or use, and can be stashed on a spare windows box where the DASD/CPU police don't even notice it. A hardy shout out to DBI and their Brother Panther monitor for giving me a performance database with built in drill down and trending without any of the headaches.

But with it being in Db2, I do kind of miss PROC UNIVARIATE and all the stats that could easily be generated sequentially with SAS. I guess relational is ok if you really don't understand the math anyway.



--- On Thu, 1/20/11, Avram Friedman <[login to unmask email]> wrote:

> From: Avram Friedman <[login to unmask email]>
> Subject: Re: [DB2-L] Performance Database
> To: [login to unmask email]
> Date: Thursday, January 20, 2011, 3:42 PM
> On Thu, 20 Jan 2011 21:56:27 +0000,
> Ted MacNEIL <[login to unmask email]>
> wrote:
>
> >>The down side is SAS tends to be sequencial
> processing..
> >
> >Have you used SAS?
> >Sequential is as sequential does.
> >
> Yes I have used SAS, my first experience was beta testing
> the original commercial version when it could be obtained
> for $10 to pay for the tape and postage.  This may make
> me one of the oldest SAS users around.  My beta test
> results were if you forget ';' at the end of statements SAS
> tends to loop.  It got fixed in 1972.
>
> Sequential is the industry standard for performace
> data. 
> SMF is sequencial
> Most monitors like TMON DB2, Mainview DB2, Insight DB2,
> Omegamon DB2, and PM before it was combined with Omegamon
> capture trace information and write it to a sequencial short
> term history file to handle interactive requests.  Just
> like SMF when the short term file fills or a switch occurs
> the short term information is written to a longer term
> sequential repository usually not available to the online
> monitor.  The original question was about the optional
> PE step about storing information in a DB2 based performace
> database. Lots of intelectual interest, few if any practial
> applications.
>
> >>Create Daily files, they can be concatanated
> together if you need longer history and they are easy to
> discard.
> >
> >Concatenation is programmatic only.
> >You can not concatenate with JCL.
> >
> yes and no.
> The first time a raw data file is processed a SAS file is
> created.
> This is actually one ofs the more expensive parts of SAS
> processing because fields are converted into one of SASs
> internal formats and a data dictionary is built. Apending
> several identically structured SAS files is programatic but
> fast and easy
>
> >And, with UPDATE/MERGE you don't need daily files.
> >.
> The reason for the daily or short term files is to support
> querries like tell me about instances of PLAN xyz where get
> pages greater than 1000.
> You don't want to processing a years worth of data when you
> are only interested in yesterday.
> You don't want to build a timestamp index as it complicates
> database utilities and is not going to work very well at
> best.
>    Almost any use of timestamp as a search
> condition involves inequalties
>    Can bet big money that there is no recent
> runstats or properly coded runstats
>    Will usally require several indexes and
> each record / row will have to be fetched and examined any
> way
>        In our example above PLAN
> xyz where get pages greater than 1000.
>        I bet get pages is not
> indexed
>        And I could of just as easy
> suggest SQL calls or Elapsed Time or USER ID or CPU time or
> etc etc etc
> Sequencial is as Sequencial is and will be much faster than
> DB2 direct access via several indexs if file sizes are
> managed by using reasonable time ranges.  In classic
> terms father son sequencial processing almost always beats
> random access in batch processing.
> I does suffer from the lack of a kewlness factor.
>
> Some one in this thread suggested a Performace Database
> could be one of a shops biggest applications.
> A peformance database for a medium to large shop using DB2
> as a repository and well indexed will not be one of the
> biggest applications in the shop.  That is too kind, It
> will be the biggest application by a long shot.
>
> Avram Friedman
>
>
> >-
> >Ted MacNEIL
> >[login to unmask email]
> >
> >_____________________________________________________________________
> >* IDUG North America * Anaheim, California * May 2-6
> 2011 *  http://IDUG.ORG/NA *
> >*   Your only source for independent,
> unbiased, and trusted DB2 information.   *
> >_____________________________________________________________________
> >http://www.IDUG.org/mentor
> >Mentoring should be a rewarding experience for
> everyone...
> >IDUG is offering up to 80% off when you both come to
> the conference!
> >_____________________________________________________________________
> >
> >If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of
> IDUG's Listserv
>
> _____________________________________________________________________
> * IDUG North America * Anaheim, California * May 2-6 2011
> *  http://IDUG.ORG/NA *
> *   Your only source for independent,
> unbiased, and trusted DB2 information.   *
> _____________________________________________________________________
> http://www.IDUG.org/mentor
> Mentoring should be a rewarding experience for everyone...
> IDUG is offering up to 80% off when you both come to the
> conference!
> _____________________________________________________________________
>
> If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of
> IDUG's Listserv
>




_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
** The best DB2 technical sessions in the world
** Independent, not-for-profit, User Run - the IDUG difference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

Isaac Yassin

Re: Performance Database
(in response to tim malamphy)
Hi,
The only problem I've found with SAS was when the bean counters got the bill
:-(
They killed so fast even superman wouldn't have a chance...
Isaac

-----Original Message-----
From: IDUG DB2-L [mailto:[login to unmask email] On Behalf Of tim malamphy
Sent: Saturday, January 22, 2011 9:00 AM
To: [login to unmask email]
Subject: Re: [DB2-L] Performance Database

Avram (the Elder)-

Hey, 10 bucks to a grad student in 72 was pretty good money, and SAS was
still part of NCU (or NC state?) back when you were helping debug it. You
could get 3 or 4 pitchers at the student union, or fill your gas tank twice.
Is MICS (from Marino/Legent) still around in some form? It kept EVERYTHING
you could want, but you needed one or two dedicated admins, and a
considerable amount of CPU to process SMF and keep it together. MXG seems
to be the sole survivor from that era of Performance Databases, and was
worth the price, just for the RMF/SMF record formats.

Thank goodness (as opposed to Goodnight, heh, heh,) my DB2/LUW monitor has a
BUILT IN performance database (stored in db2) that is hardly any work at all
to maintain or use, and can be stashed on a spare windows box where the
DASD/CPU police don't even notice it. A hardy shout out to DBI and their
Brother Panther monitor for giving me a performance database with built in
drill down and trending without any of the headaches.

But with it being in Db2, I do kind of miss PROC UNIVARIATE and all the
stats that could easily be generated sequentially with SAS. I guess
relational is ok if you really don't understand the math anyway.



--- On Thu, 1/20/11, Avram Friedman <[login to unmask email]> wrote:

> From: Avram Friedman <[login to unmask email]>
> Subject: Re: [DB2-L] Performance Database
> To: [login to unmask email]
> Date: Thursday, January 20, 2011, 3:42 PM On Thu, 20 Jan 2011 21:56:27
> +0000, Ted MacNEIL <[login to unmask email]>
> wrote:
>
> >>The down side is SAS tends to be sequencial
> processing..
> >
> >Have you used SAS?
> >Sequential is as sequential does.
> >
> Yes I have used SAS, my first experience was beta testing the original
> commercial version when it could be obtained for $10 to pay for the
> tape and postage.  This may make me one of the oldest SAS users
> around.  My beta test results were if you forget ';' at the end of
> statements SAS tends to loop.  It got fixed in 1972.
>
> Sequential is the industry standard for performace data.
> SMF is sequencial
> Most monitors like TMON DB2, Mainview DB2, Insight DB2, Omegamon DB2,
> and PM before it was combined with Omegamon capture trace information
> and write it to a sequencial short term history file to handle
> interactive requests.  Just like SMF when the short term file fills or
> a switch occurs the short term information is written to a longer term
> sequential repository usually not available to the online monitor. 
> The original question was about the optional PE step about storing
> information in a DB2 based performace database. Lots of intelectual
> interest, few if any practial applications.
>
> >>Create Daily files, they can be concatanated
> together if you need longer history and they are easy to discard.
> >
> >Concatenation is programmatic only.
> >You can not concatenate with JCL.
> >
> yes and no.
> The first time a raw data file is processed a SAS file is created.
> This is actually one ofs the more expensive parts of SAS processing
> because fields are converted into one of SASs internal formats and a
> data dictionary is built. Apending several identically structured SAS
> files is programatic but fast and easy
>
> >And, with UPDATE/MERGE you don't need daily files.
> >.
> The reason for the daily or short term files is to support querries
> like tell me about instances of PLAN xyz where get pages greater than
> 1000.
> You don't want to processing a years worth of data when you are only
> interested in yesterday.
> You don't want to build a timestamp index as it complicates database
> utilities and is not going to work very well at best.
>    Almost any use of timestamp as a search condition involves
> inequalties
>    Can bet big money that there is no recent runstats or properly
> coded runstats
>    Will usally require several indexes and each record / row will have
> to be fetched and examined any way
>        In our example above PLAN
> xyz where get pages greater than 1000.
>        I bet get pages is not
> indexed
>        And I could of just as easy
> suggest SQL calls or Elapsed Time or USER ID or CPU time or etc etc
> etc Sequencial is as Sequencial is and will be much faster than
> DB2 direct access via several indexs if file sizes are managed by
> using reasonable time ranges.  In classic terms father son sequencial
> processing almost always beats random access in batch processing.
> I does suffer from the lack of a kewlness factor.
>
> Some one in this thread suggested a Performace Database could be one
> of a shops biggest applications.
> A peformance database for a medium to large shop using DB2 as a
> repository and well indexed will not be one of the biggest
> applications in the shop.  That is too kind, It will be the biggest
> application by a long shot.
>
> Avram Friedman
>
>
> >-
> >Ted MacNEIL
> >[login to unmask email]
> >
> >_____________________________________________________________________
> >* IDUG North America * Anaheim, California * May 2-6
> 2011 *  http://IDUG.ORG/NA *
> >*   Your only source for independent,
> unbiased, and trusted DB2 information.   *
> >_____________________________________________________________________
> >http://www.IDUG.org/mentor
> >Mentoring should be a rewarding experience for
> everyone...
> >IDUG is offering up to 80% off when you both come to
> the conference!
> >_____________________________________________________________________
> >
> >If you need to change settings,
> >http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of
> IDUG's Listserv
>
> _____________________________________________________________________
> * IDUG North America * Anaheim, California * May 2-6 2011
> *  http://IDUG.ORG/NA *
> *   Your only source for independent,
> unbiased, and trusted DB2 information.   *
> _____________________________________________________________________
> http://www.IDUG.org/mentor
> Mentoring should be a rewarding experience for everyone...
> IDUG is offering up to 80% off when you both come to the conference!
> _____________________________________________________________________
>
> If you need to change settings,
> http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv
>




_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 *
http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information.
*
** The best DB2 technical sessions in the world
** Independent, not-for-profit, User Run - the IDUG difference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is
the home of IDUG's Listserv


--
I am using the free version of SPAMfighter.
We are a community of 7 million users fighting spam.
SPAMfighter has removed 241 of my spam emails to date.
Get the free SPAMfighter here: http://www.spamfighter.com/len

The Professional version does not have this message

_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
** The best DB2 technical sessions in the world
** Independent, not-for-profit, User Run - the IDUG difference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

Ted MacNEIL

Re: Performance Database
(in response to Isaac Yassin)
>Is MICS (from Marino/Legent) still around in some form?

Yes.
It's now CA-NEUMICS.

-
Ted MacNEIL
[login to unmask email]

_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
** The best DB2 technical sessions in the world
** Independent, not-for-profit, User Run - the IDUG difference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

Avram Friedman

Re: Performance Database
(in response to Ted MacNEIL)
When I started school at UW-Madison the student union did not serve beer.
One had to go to a local beer joint.
There where beer places and hard stuff places.
The Beer place I went too was called the H,T.
Few people knew what H.T. stood for and those who knew were smart enough to keep it a secret.
After all these years I can at last publicly tell ... It was the Hotsy Totsy.

Yes the tape came from NCU.

A bit about the history of statistical packages and how I got involved.

There were a small number for first generation single function statistical packages.
The best known happened to be the local UW one called WISCTAB
or the Wisconsin Tabulater.

Second generation packages were muti functional and were also ALL university developed.
The best know of these included BioMed (really owned the market) Osirus, SPSS, and SAS
I was a big SPSS supported because it was the only one that produced near cameral ready output on a line printer so It made my life easy.
When I first used SAS it a a PROC FREQuency by no Crosstabs so a run that had2 dimensions produced a outline type report.

SAS may be "NIE" more ... a joke along the same lines as "GOODNIGHT"
NIE more because SPSS was bought our recently by IBM.
So in the just statistical software world SPSS may endup being the market leader.

I am sorry people who only see the world through the eyes of an OS performance person ...
I dont think SAS was ever the market leader.
As I recall the history SAS was made popular in the OS world via some of the first ever additions to the "SHARETAPE"
Sample code for processing SMF data from Standard Oil.
SAS responded to this unexpected advertising by adding support for 360 time stamp formats.

In the early 70s I worked for the state of Wisconsin and for a time did discriptive statistical reporting on 13 diffrent types of licensed health proffesionals. Thats when I got involved with statistical packages.
I left that particular posistion with the state to go to prison. Dosen't matter why I did not do it anyway.
Actually I started the 3rd program in the contry to teach white collar crime or how to program computers.
One of the ways the program got money to operate is the program was certified by the state collage system for credit earning classes. We offered classes to collage students in the prison on using statistical packages to the near by collages. People would get locked up for a few days (not nights) to learn this stuff from courses I wrote produced and delivered.

In the mid to late 70s I was a Sr SE at Amdahl and the data base specialist for the south central regions.
All the 6 data base specialists got together for the first time ever to discuss how to work together as a team.
I suggested that a big part of starting out was a rough agreement on the question of What is a Database.
Most of my co workers said that if it was spelled IMS then it was a databasem other wise it was something else.
I said
New fangled VSAM was a data base because it had intergrated utilities including optional loging and recovery to current
Statistical packages were data bases because the "Data Steps" built a customized reprocessable file that contained a
data dictionary.
The regional manager for the West region that hosted our meeting was so upset that I insisted on discussing what is a data base at the very first meeting of the team sent a message to my regional manager saying he never wanted me in the Chicago area ever again even on vacation. My regional manager defended me.

And yes there is a punch line
My Regional Manager was Big Jack Noonan in Houston Tx I called him Big Jack because he is one of the few people shorter than my self. Jack left Amdahl to become the vice president of software development at Candle. He was walking in for the first time as I was walking out for the last time and we chatted in the driveway. Big Jack left Candle to become the general manager of SPSS taking over the posistion from the author and original director of SPSS Dr Norman Nie (for those who did not get the Nie more part of the joke

I did the posting that earned me the title Avram (the Elder) in reply to a diffrent list memeber suggesting that perhaps I had never used SAS at all. If the suggestion was today I am not a frequent active user of SAS he would of been correct. To suggest I have no experience with SAS is historically way out of line.

Concerning MICS and database
Mario Morino author of MICS is famous for saying "I doubt if any more than 10 companies will be intersted in this product".
Morino thought IMS transaction work could be thought of as very small batch jobs. He designed a modification to IMS type 1 code to produce transaction and application accounting records.
He decided that he did not want to spend the large amout of time to upgrade the user mods to support IMS/VS 1.1.2 so he sold them to a diffrent company that had there on performace data collection and processing system. The system was called CMF and the company was Boole and Babbage.
Many of you know the history from this point
The usermods were rebranded to Control IMS
Control IMS was enhanced to include Control IMS real time.
Control IMS real time was restructured from a MFS exit and
added features for auto operations and TSO interfaces.
This was called Mainview.
Mainview was renamed Mainview IMS because some one thought there could be other Mainviews as well.
Most of Boole and Babbage was sold to BMC
A few parts were sold to other parties ... write me if you want to know what some of those other parts are.
Morino and a diffrerent company merged together to become Legent,
Legent was purchased by CA.


Avram (the elder) Friedman
Amoung other things
The Father of Omegamon DB2
Av sometimes spelled Ab (one of the roots of my name) actually translates to father in english

On Fri, 21 Jan 2011 22:59:57 -0800, tim malamphy <[login to unmask email]> wrote:

>Avram (the Elder)-

Hey, 10 bucks to a grad student in 72 was pretty good money, and SAS was still part of NCU (or NC state?) back when you were helping debug it. You could get 3 or 4 pitchers at the student union, or fill your gas tank twice.
Is MICS (from Marino/Legent) still around in some form? It kept EVERYTHING you could want, but you needed one or two dedicated admins, and a considerable amount of CPU to process SMF and keep it together. MXG seems to be the sole survivor from that era of Performance Databases, and was worth the price, just for the RMF/SMF record formats.

Thank goodness (as opposed to Goodnight, heh, heh,) my DB2/LUW monitor has a BUILT IN performance database (stored in db2) that is hardly any work at all to maintain or use, and can be stashed on a spare windows box where the DASD/CPU police don't even notice it. A hardy shout out to DBI and their Brother Panther monitor for giving me a performance database with built in drill down and trending without any of the headaches.

But with it being in Db2, I do kind of miss PROC UNIVARIATE and all the stats that could easily be generated sequentially with SAS. I guess relational is ok if you really don't understand the math anyway.



--- On Thu, 1/20/11, Avram Friedman <[login to unmask email]> wrote:

> From: Avram Friedman <[login to unmask email]>
> Subject: Re: [DB2-L] Performance Database
> To: [login to unmask email]
> Date: Thursday, January 20, 2011, 3:42 PM
> On Thu, 20 Jan 2011 21:56:27 +0000,
> Ted MacNEIL <[login to unmask email]>
> wrote:
>
> >>The down side is SAS tends to be sequencial
> processing..
> >
> >Have you used SAS?
> >Sequential is as sequential does.
> >
> Yes I have used SAS, my first experience was beta testing
> the original commercial version when it could be obtained
> for $10 to pay for the tape and postage.� This may make
> me one of the oldest SAS users around.� My beta test
> results were if you forget ';' at the end of statements SAS
> tends to loop.� It got fixed in 1972.
>
> Sequential is the industry standard for performace
> data.�
> SMF is sequencial
> Most monitors like TMON DB2, Mainview DB2, Insight DB2,
> Omegamon DB2, and PM before it was combined with Omegamon
> capture trace information and write it to a sequencial short
> term history file to handle interactive requests.� Just
> like SMF when the short term file fills or a switch occurs
> the short term information is written to a longer term
> sequential repository usually not available to the online
> monitor.� The original question was about the optional
> PE step about storing information in a DB2 based performace
> database. Lots of intelectual interest, few if any practial
> applications.
>
> >>Create Daily files, they can be concatanated
> together if you need longer history and they are easy to
> discard.
> >
> >Concatenation is programmatic only.
> >You can not concatenate with JCL.
> >
> yes and no.
> The first time a raw data file is processed a SAS file is
> created.
> This is actually one ofs the more expensive parts of SAS
> processing because fields are converted into one of SASs
> internal formats and a data dictionary is built. Apending
> several identically structured SAS files is programatic but
> fast and easy
>
> >And, with UPDATE/MERGE you don't need daily files.
> >.
> The reason for the daily or short term files is to support
> querries like tell me about instances of PLAN xyz where get
> pages greater than 1000.
> You don't want to processing a years worth of data when you
> are only interested in yesterday.
> You don't want to build a timestamp index as it complicates
> database utilities and is not going to work very well at
> best.
> ���Almost any use of timestamp as a search
> condition involves inequalties
> ���Can bet big money that there is no recent
> runstats or properly coded runstats
> ���Will usally require several indexes and
> each record / row will have to be fetched and examined any
> way
> � � ���In our example above PLAN
> xyz where get pages greater than 1000.
> � � ���I bet get pages is not
> indexed
> � � ���And I could of just as easy
> suggest SQL calls or Elapsed Time or USER ID or CPU time or
> etc etc etc
> Sequencial is as Sequencial is and will be much faster than
> DB2 direct access via several indexs if file sizes are
> managed by using reasonable time ranges.� In classic
> terms father son sequencial processing almost always beats
> random access in batch processing.
> I does suffer from the lack of a kewlness factor.
>
> Some one in this thread suggested a Performace Database
> could be one of a shops biggest applications.
> A peformance database for a medium to large shop using DB2
> as a repository and well indexed will not be one of the
> biggest applications in the shop.� That is too kind, It
> will be the biggest application by a long shot.
>
> Avram Friedman
>
>
> >-
> >Ted MacNEIL
> >[login to unmask email]
> >
> >_____________________________________________________________________
> >* IDUG North America * Anaheim, California * May 2-6
> 2011 *� http://IDUG.ORG/NA *
> >*���Your only source for independent,
> unbiased, and trusted DB2 information.���*
> >_____________________________________________________________________
> >http://www.IDUG.org/mentor
> >Mentoring should be a rewarding experience for
> everyone...
> >IDUG is offering up to 80% off when you both come to
> the conference!
> >_____________________________________________________________________
> >
> >If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of
> IDUG's Listserv
>
> _____________________________________________________________________
> * IDUG North America * Anaheim, California * May 2-6 2011
> *� http://IDUG.ORG/NA *
> *���Your only source for independent,
> unbiased, and trusted DB2 information.���*
> _____________________________________________________________________
> http://www.IDUG.org/mentor
> Mentoring should be a rewarding experience for everyone...
> IDUG is offering up to 80% off when you both come to the
> conference!
> _____________________________________________________________________
>
> If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of
> IDUG's Listserv
>



>
>_____________________________________________________________________
>* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
>* Your only source for independent, unbiased, and trusted DB2 information. *
>** The best DB2 technical sessions in the world
>** Independent, not-for-profit, User Run - the IDUG difference!
>_____________________________________________________________________
>
>If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
** The best DB2 technical sessions in the world
** Independent, not-for-profit, User Run - the IDUG difference!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

tim malamphy

Re: Performance Database
(in response to Avram Friedman)
Cute.
I thought you were kidding until I googled it.

--- On Sat, 1/22/11, Ted MacNEIL <[login to unmask email]> wrote:

> From: Ted MacNEIL <[login to unmask email]>
> Subject: Re: [DB2-L] Performance Database
> To: [login to unmask email]
> Date: Saturday, January 22, 2011, 3:16 AM
> >Is MICS (from Marino/Legent)
> still around in some form?
>
> Yes.
> It's now CA-NEUMICS.
>
> -
> Ted MacNEIL
> [login to unmask email]
>
> _____________________________________________________________________
> * IDUG North America * Anaheim, California * May 2-6 2011
> *  http://IDUG.ORG/NA *
> *   Your only source for independent,
> unbiased, and trusted DB2 information.   *
> **    The best DB2 technical sessions in the
> world
> **    Independent, not-for-profit, User Run - the
> IDUG difference!
> _____________________________________________________________________
>
> If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of
> IDUG's Listserv
>




_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
_____________________________________________________________________
http://www.IDUG.org/mentor
How can you expand your staff or do succession planning in this economy?
Mentoring is a proven, economical, way to train the next generation of DB2 Users!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

Ted MacNEIL

Re: Performance Database
(in response to tim malamphy)
>I thought you were kidding until I googled it.

CA-NEUMICS

CA as in the company in Islandia, New York.
NEU as in NEUral network
MICS as in MVS Information Control System
(An acronym with in an acronym)

Needless to say, support has gone downtown in the past few years.

I worked at one of the Beta Sites for MICS (the Ontario Ministry of Government Services), and another (post-Beta -- the Bank of Nova Scotia), and all was kopacetic.

We survived the Duquesne/Morino merge (aka Legent).

But, the CA acquisition hurt support.
Voice of experience.

(No offence intended -- just a joke/pun)

I wouldn't touch CA-NEUMICS with a 9-foot Austrian.

-
Ted MacNEIL
[login to unmask email]

_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
_____________________________________________________________________
http://www.IDUG.org/mentor
How can you expand your staff or do succession planning in this economy?
Mentoring is a proven, economical, way to train the next generation of DB2 Users!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv

Tony Saul

Re: Performance Database
(in response to Ted MacNEIL)
And if you don't want CA-NEUMICS or SAS-MXG, then there is also Tivoli Decision
Support (which was the old Service Level Reporter (SLR)) that writes all data to
DB2 tables.
 Regards,
Tony



----- Original Message ----
From: tim malamphy <[login to unmask email]>
To: [login to unmask email]
Sent: Tue, 25 January, 2011 8:34:58 AM
Subject: Re: [DB2-L] Performance Database

Cute.
I thought you were kidding until I googled it.

--- On Sat, 1/22/11, Ted MacNEIL <[login to unmask email]> wrote:

> From: Ted MacNEIL <[login to unmask email]>
> Subject: Re: [DB2-L] Performance Database
> To: [login to unmask email]
> Date: Saturday, January 22, 2011, 3:16 AM
> >Is MICS (from Marino/Legent)
> still around in some form?
>
> Yes.
> It's now CA-NEUMICS.
>
> -
> Ted MacNEIL
> [login to unmask email]
>
> _____________________________________________________________________
> * IDUG North America * Anaheim, California * May 2-6 2011
> *  http://IDUG.ORG/NA *
> *   Your only source for independent,
> unbiased, and trusted DB2 information.   *
> **    The best DB2 technical sessions in the
> world
> **    Independent, not-for-profit, User Run - the
> IDUG difference!
> _____________________________________________________________________
>
> If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the
>home of
> IDUG's Listserv
>




_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 *  http://IDUG.ORG/NA
*
*  Your only source for independent, unbiased, and trusted DB2 information.  *
_____________________________________________________________________
http://www.IDUG.org/mentor
How can you expand your staff or do succession planning in this economy?
Mentoring is a proven, economical, way to train the next generation of DB2
Users!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the
home of IDUG's Listserv





_____________________________________________________________________
* IDUG North America * Anaheim, California * May 2-6 2011 * http://IDUG.ORG/NA *
* Your only source for independent, unbiased, and trusted DB2 information. *
_____________________________________________________________________
http://www.IDUG.org/mentor
How can you expand your staff or do succession planning in this economy?
Mentoring is a proven, economical, way to train the next generation of DB2 Users!
_____________________________________________________________________

If you need to change settings, http://www.idug.org/cgi-bin/wa?A0=DB2-L is the home of IDUG's Listserv