Articles & Content
The Fillmore Group was founded by IBM DB2 Gold Consultant Frank C. Fillmore, Jr.
Note: The content of this blog is the opinion and thoughts of the blogger and does not necessarily represent the opinions of IDUG.
Vote for me! Replication Enhancement to Support ASMIBM recently introduced the Data and AI ideas portal for clients where users are invited to request enhancements. And vote on them -- sort of like voting for class president. I submitted a request earlier today for the addition of Oracle ASM support in IBM InfoSphere Data Replication/Change Data Capture. I did this because I was introduced to an Oracle customer currently using Informatica replication who needs to find an alternate replication solution, as Informatica is moving replication to the cloud. IBM's replication solutions are quite strong compared to other vendor options. The list of ICDC supported database sources and targets continues to grow, with Postgres and Kafka being recent additions. I think IBM should continue to enhance ICDC by adding support for ASM so we can introduce more Oracle customers to ICDC. PLEASE! Vote for my enhancement -- I believe it's I-83 at this link. It's currently in "under consideration" status so I may need to update the link when I have confirmed it's available for voting. I'll post some signs up in the hall announcing my enhancement's candidacy. Vote early and vote often!! Thank you for your support!!
IBM Highlights Gold Consultants on IBM.comThanks to the combined efforts of several Gold Consultants, notably Julian Stuhler and Ted Alexander, IBM Gold Program sponsors and advocates Jim Reed and Reid Gerson, and The Fillmore Group's Megan Byrne, a new page highlighting the Gold Consultants program is available on IBM.com. The page provides links to consultant specialty and contact information. The directory is attached here: IBM Gold Consultants Profiles Best of all, the page answers the frequently asked question, "What exactly are the Gold Consultants?" "An elite group of independent consultants with vast experience in IBM data repositories, unified governance, artificial intelligence (AI) and machine learning IBM Gold Consultants bring extensive industry experience and technical expertise to help IBM clients define and implement strong strategies for their data and analytics initiatives using IBM Db2® on all platforms, IBM Informix, IBM InfoSphere®, IBM CICS® and related technologies. This group is recognized as some of the world’s most experienced independent consultants for these products. With in-depth technical knowledge and communication skills, the consultants can fill in skill gaps within a client’s organization and facilitate better performing solutions, optimal database designs and high availability." The new page link is: https://www.ibm.com/analytics/ibm-gold-consultants Check it out -- it looks much better on screen!
Mid-Atlantic Db2 LUW Meeting – Bethesda, May 7thThank you to the IBM Technical Team, Kay Groski and Warren Heising, for coordinating this terrific agenda and group of excellent speakers for the Mid-Atlantic Spring Db2 LUW Users Group Meeting! The is no cost to attend (again thanks Kay and Warren). Registration is required. Register here. Date: Tuesday, May 7, 2019 Time: 9:00am – 2:30pm Location: IBM Bethesda, Room 02-2A304, 6710 Rockledge Drive, Bethesda, MD 20817 Agenda: 09:00-10:30 SESSION 1: Futures (vNext coming soon!) presented by Jon Lind 10:30-10:45 BREAK 10:45-12:00 SESSION 2: IBM Streams presented by Roger Rea 12:00-12:45 Networking Lunch 12:45-2:00 SESSION 3: Demystifying the Db2 Logger presented by Dale McInnis 2:00-2:15 BREAK 2:15-2:30 IDUG Overview
Free Hands-on IDAA Class Announced – July 18 & 19, Columbus, OhioJoin us for this new and improved for 2019 class to be delivered by IBM Gold Consultant Frank Fillmore. Contents include IDAA v7 early experiences, best practices, and v5 to v7 migration tips. Attendees will learn how an IDAA v7 (aka “Sailfish”) can extend System z’s processing capabilities to enable Hybrid Transaction/Analytic Processing (HTAP). For IBM System z customers considering an IDAA purchase, this class provides an overview of the architecture, use cases delivering value, and insight into the administration of the accelerator. Class is free to IBM System z customers considering an IDAA purchase. The Fillmore Group will provide breakfast and lunch each day; attendees are responsible for their own travel and lodging expenses. Registration is required – Register HERE Dates (2 days): Thursday and Friday, July 18 and 19, 2019 Location: IBM, 6060 Britton Parkway, Building 1, Dublin, Ohio, 43016 Time: 9:00am – 4:00pm EDT Cost: Class is *free* to pre-registered attendees Instructor: Frank Fillmore Topics: An introduction to the V7.1 architecture and Integrated Analytics System (IAS).The heterogeneous accelerator environment: what works well today.Assessment tools to support acquisition: Workload Assessment, Client Value Engagement, PoC.Positioning IDAA within your organization with the most valuable use case.Migrating from the Netezza-based IDAA to v7.1.Lab 1: Basic IDAA Activities Data Studio / IBM Db2 Analytics Accelerator Studio GUITable accelerationQuery processing and analysisEnhancing Value to the IDAA with the IBM Tools Ecosystem Assessment: What is the workload that would benefit from being accelerated?Optimization: How do I monitor and optimize a workload to take advantage of the Accelerator?Administration: Can I manage the Accelerator more effectively?Lab 2: Advanced IDAA Exploitation Incremental Update: keeping accelerated data in sync with DB2 for z/OS OLTPAccelerator-only Tables (AoTs): intermediate tables and ETL processing
Conference & Conference & Conference Oh My!!The buildup and push for the Think Conference was manic. The "EXCITEMENT!!" seemed to come from a small group of IBM marketing folks and the focus on attendance numbers (tough to confirm or deny) was odd. Customers and Business Partners focused on the data management component of the Analytics portfolio, and with a limited budget to attend conferences, seem to have shifted from the IOD/Insight/Think Conference to the International Db2 Users Group (IDUG) conferences. IDUG North America is held in the Spring, this year in Charlotte, South Carolina, from June 2 -- 6. The conference focus is education for IBM database technologies solutions. Artificial Intelligence and Machine Learning aren't the dominating topics, and session selections and content are managed by IDUG volunteers. IDUG NA 2019 costs $2165 to attend, with a $300 early bird discount available until April 22nd. Having IDUG to counterbalance Think makes sense to me. Also, the IDUG community is very user friendly. There are several discount options to make attendance affordable, and as has been the case at most technical conferences, presenters receive a free pass. Now -- the wild card. IBM, through their marketing collaboration with IDG, has scheduled another conference, this one called IBM Data and AI Forum, for Miami, Florida, from October 21 -- 24. This event is the former IBM Analytics University, which I believe evolved from the Db2 Technical Conference. It seems to be IBM's attempt to cross pollinate Db2 users with AI, not necessarily a logical partnership in my opinion, as the potential benefits of AI and Machine Learning seem to be in the conceptual sales stage, being pitched to senior management, while the technical teams running data centers aren't setting the analytics vision. The conference pass cost is far lower than IDUG or Think, at $1295 for a full price pass and $1095 with the early bird discount through July 31. The call for speakers is open and I understand selected speakers will be compensated for presenting. It will be interesting to see how the community responds to all these conference offerings. Hmmm...
Baltimore/Washington Db2 Users Group March Education SessionThe Baltimore/Washington Db2 Users Group will be holding our annual education session on Wednesday, March 20, 2019, at the BWI Hilton. Please note the date is the third Wednesday in March, not the second, when meetings are typically held, as several members will be attending SHARE. As a very special 2019 offer, the group will have two session tracks and two terrific presenters. You are invited to sign up for one of the two tracks. The cost per attendee is $100 and includes session materials and a sit down lunch with the entire group. Date: Wednesday, March 20, 2019 Time: 9:00am -- 4:00pm (Registration and continental breakfast at 8:30am) Location: BWI Hilton, 1739 W Nursery Rd, Linthicum Heights, MD 21090 Register: HERE For DBA’s IBM’s Robert Catterall will present, and for developers, Tony Andrews from Themis. Abstracts for their day-long sessions are below. Track 1: Db2 Database Administration Db2 for z/OS DBA’s: News You Can Use This day-long session for DBAs delivers updates on four topics pertinent to database administration. Each unique segment will provide up-to-date information on: Leveraging large z/OS LPAR memory resources for improved Db2 application performance.Key metrics for Db2 subsystem and application performance monitoring and tuning.DDF application architecture and workload management.Modern Db2 for z/OS physical database design.Presenter: Robert Catterall Robert Catterall is an IBM Senior Consulting Db2 for z/OS specialist. He started his IT career with IBM in 1982, and worked throughout the 1990s as a member of the Company's Db2 for z/OS National Technical Support team. From 2000 to 2007, Robert worked as a database technology strategist for CheckFree (now part of Fiserv). After working for three years as an independent Db2 consultant, he rejoined IBM in 2010. Robert is a past President of the International Db2 Users Group (IDUG), and a member of IDUG's Speakers Hall of Fame. He presents frequently at conferences and user group meetings, and blogs about Db2 for z/OS at http://robertsdb2blog.blogspot.com/. Track 2: Db2 Developers New Application Features for Db2 12 Are you still coding and designing SQL from the 90’s? The last several releases of Db2 have brought quality new features that can be used to improve both functionality and performance for today’s complex applications. This event will help catch you up! Morning Segment: • Global Variables: Creation, Usage, Rules, V12 Enhancements • Transparent Archiving: Automatic Archiving of Data, Data Retrieval Options, Global Variables for Archiving • Sequence Objects: Creation, Usage, Sequence vs Identity • Advanced Groupings: Group By / Having Review , Grouping Sets, Rollup, Cube • OLAP Ranking Functions Upgrades: o Rank, Dense_Rank, Row_Number Review o Ranking Moving Sums and Avgs o Rows Between o Range Between o Partition By • SQL Set Operations: Union/Union All Review, Intersect/Intersect All, Except/Except All, Union Select Into Afternoon Session: • SQL Merge: Overview, V12 Enhancements • SQL : Select From Update, Select From Insert • SQL Piece-Wise Delete: V12 Enhancement • SQL Pagination: V12 enhancement • SQL QUERYNO: Review • SQL Procedural Language: Piece Wise Delete, Autonomous Processing, SQL PL support for Triggers • Performance Enhancements: o New Stage 1 Predicates o V10, V11 New Performance Review o V12 Performance Enhancements o Sparse Indexing o V12 Runtime Adaptive Indexing o V12 Fast Traverse Indexing Presenter: Tony Andrews Tony Andrews has more than 25 years’ experience in the development of relational database applications. Most of this time, he has provided development and consulting services to Fortune 500 companies and government agencies. Tony has written literally thousands of queries and programs during his development years, and has also served as a database analyst. For the last 10 years, Tony has been splitting his time between performance and tuning consulting engagements along with training. His main focus is to teach today’s developers the ways of RDBMS application design, development and SQL programming --- always with a special emphasis on improving performance. He is a current IBM Champion, and regular speaker at many regional user groups, IDUG NA, and IDUG EMEA. He is also the author of a book for developers titled ‘DB2 SQL Tuning Tips for z/OS Developers’. *the Baltimore/Washington Db2 Users Group is a non-profit organization funding a portion of this meeting. If registration levels exceed our funds the per attendee cost will need to be raised to offset expenses.
IBM Replication 2019 New Features Webinar #IBMAnalytics #IBMzThanks to my colleagues, Kim May and Ed Lynch, for a terrific IBM InfoSphere Data Replication (IIDR) new features webinar delivered today. The recording of the webinar is found here. These are the presentation materials: Data Replication Update 2019-02-07 -- Final
IBM 2019 Replication Updates – 4 in 45 Minutes – Featuring Kafka!Join replication experts Frank Fillmore and Ed Lynch for this information-packed webinar with news and announcements about IBM’s replication solutions. Whether you are a current IBM replication user, investigating replication for a new deployment, or working with a competitor’s solution, this webinar will help you understand what IBM offers today, and the enhancements that will help you succeed with replication in 2019. At a sprint! In forty-five minutes Ed and Frank will deliver an overview of IBM’s current offerings, the four most important 2019 enhancement announcements, and 3 special pricing offers that can help your organization realize significant savings. We know you want your data to be available when you need it and where you need it, and fast. We can help. Registration is required. Please register HERE. Date: Thursday, 2/7/19 Time: 1:00pm – 2:00pm EST Presenters: Frank Fillmore is the Founder and President of The Fillmore Group, Inc., an IBM Business Partner specializing in replication solutions in the IBM Analytics portfolio. Frank has been a Db2 Gold Consultant since 1998 and is currently an IBM Champion for both Analytics and System z. Frank frequently presents on cutting-edge topics at regional and international Db2 Users Group meetings and is an IBM authorized instructor for delivery of replication courses including Q Replication, ICDC and pureScale. Frank leads a team of expert consultants delivering presales, implementation and enablement support to IBM customers. Frank is currently supporting a pureScale implementation for a US Federal agency. Ed Lynch was with IBM for over 25 years as a leader and innovator in Data Analytics and Information Integration. Leveraging his balanced and diversified knowledge of data and Information Technology with business needs and capabilities, Ed consistently recommends and delivers best of breed technical solutions. A trusted advisor to his customers and colleagues, Ed’s cross-industry knowledge enables him to be an effective go-to resource, particularly for difficult and complex challenges, and his success in many diverse roles at IBM, including development, technical team management, marketing and sales, showcase his flexibility. Ed is currently supporting a complex replication implementation for a large, US-based financial institution.
SQL to Aid #Db2 for z/OS EBCDIC to Db2 for LUW Unicode Conversion #IBMAnalytics #IBMzDr. Sheldon Cooper of "The Big Bang Theory" regularly records a YouTube video called "Fun with Flags" so I'm going to call this blog post "Fun with SQL". Sheldon may be a future Nobel Prize winner, but I've been named an IBM Champion for Analytics and System z for 2019. The problem statement: a large US government agency is contemplating migrating a packaged application from System z to Linux. The current and future database repositories are Db2 for z/OS and Db2 for LUW respectively. As a System z Champion I've presented on the continued relevancy and efficacy of the IBM mainframe, but that's not the focus of this post. One of the challenges of porting data is the conversion of character encoding from EBCDIC (Extended Binary Coded Decimal Interchange Code) which is the legacy encoding protocol used on IBM mainframes and System i to Unicode. Db2 for z/OS supports Unicode, but I'm not going to address anything other than EBCDIC encoding at the source and Unicode at the target (Sorry, ASCII). In brief, EBCDIC represents an 8 bit binary encoding scheme that can represent 256 different characters. For example: the letter "F" (one of my faves) is represented as 1100 0110 in binary and C6 in hexadecimal. For national language support amid the variety of Single Byte Character Sets (SBCS) used around the world -- think the German umlaut (i.e. ä ö ü) -- the 256 byte universe of EBCDIC wasn't nearly enough. The Unicode solution is to use more than 1 byte to represent some characters. For example: the "¢" cent sign takes one byte to represent in EBCDIC, but 2 bytes in Unicode. select length(ebcdic_str('¢')) as EBCDIC, length(unicode_str('¢')) as Unicode from sysibm.sysdummy1;EBCDIC UNICODE 1 2This problem has manifested itself as an error when moving a CHAR(5) column of data from Db2 for z/OS to Db2 for LUW. If, say, the pesky "¢" is present in the string and all 5 characters are significant (i.e. no trailing blanks), the string won't fit in the same column width on the target platform. Most customers find this out the hard way when the Db2 for LUW LOAD command or IMPORT utility posts errors. There are two possible solutions: data cleanup (Do we really need the pesky '¢' in a Comments column?)lengthen the target columns to accommodate Unicode expansionBoth can be time-consuming and expensive and will depend on a variety of factors beyond the scope of this blog post. The purpose here today is to identify the extent of the problem before you actually try to port the data. I've constructed this SQL statement to do just that. select 'SELECT', rtrim(name)||', HEX('||rtrim(name)||') AS HEX, LENGTH(EBCDIC_STR('||rtrim(name)||')) AS L_EBCDIC, LENGTH(UNICODE_STR('||rtrim(name)||')) AS L_UNICODE FROM', rtrim(tbcreator)||'.'||rtrim(tbname), 'WHERE LENGTH(EBCDIC_STR('||rtrim(name)||')) < LENGTH(UNICODE_STR('||rtrim(name)||')) AND LENGTH(UNICODE_STR('||rtrim(name)||')) >', '( SELECT LENGTH FROM SYSIBM.SYSCOLUMNS WHERE TBCREATOR = '''||rtrim(tbcreator)||''' AND TBNAME = '''||rtrim(tbname)||''' AND NAME = '''||rtrim(name)||''' );' from sysibm.syscolumns where tbname like 'ICM%' and encoding_scheme <> 'E' and coltype in ('CHAR', 'VARCHAR');So what are we doing here? The SQL statement above scans the Db2 for z/OS catalog and generates SQL statements like the one below that compares the length of the data in a column represented in EBCDIC with the length of that same data represented in Unicode. It also compares the length of the Unicode representation of the data with the maximum column length. SELECT DEFAULTVALUE, HEX(DEFAULTVALUE) AS HEX, LENGTH(EBCDIC_STR(DEFAULTVALUE)) AS L_EBCDIC, LENGTH(UNICODE_STR(DEFAULTVALUE)) AS L_UNICODE FROM XXXX.ICMSTCOMPATTRS WHERE LENGTH(EBCDIC_STR(DEFAULTVALUE)) < LENGTH(UNICODE_STR(DEFAULTVALUE)) AND LENGTH(UNICODE_STR(DEFAULTVALUE)) > ( SELECT LENGTH FROM SYSIBM.SYSCOLUMNS WHERE TBCREATOR = 'XXXX' AND TBNAME = 'ICMSTCOMPATTRS' AND NAME = 'DEFAULTVALUE' );The columns that are evaluated are chosen by you in the last line of the first SQL statement that generates the list of candidates. This is where knowledge of the data is crucial. If, for example, you have a VARCHAR column of a maximum 500 bytes and there's typically no more than 50 bytes of data, you won't exceed the maximum length of the column even with a slew of special characters and rampant expansion. As such you might only be interested in smaller, fixed length columns. I only chose columns with tables that had a certain name pattern (ICM%) and column types of CHAR and VARCHAR. My IBM Gold Consultant confrères will point out that the only test that matters is wheter the maximum column length will be exceeded. The first predicate just determines if any expansion will occur. But remember: I'm channeling Dr. Sheldon Cooper and err on the side of completeness. Be sure to test this first in a non-Prod environment before turning it loose on Production. Also run it during times of light subsystem utilization. The first, master query runs quickly against the SYSIBM.SYSCOLUMNS table. The resulting scans will produce a lot of I/O. Let me know if this has been helpful. And to all a happy, healthy, and prosperous 2019!
2019 Events – Coming Soon!With 2018 coming to a close I am focused on plans for 2019. Our current plans are being driven by changes in IBM's services delivery model and product enhancements to be available in 2019. Change can be good! In 2018 IBM's Lab Services organization opted to extend their use of Business Partners to support customer engagements. Lab Services has long maintained relationships with skilled partners to augment their staff consultants; in 2018 this utilization was increased. The Fillmore Group staffed several Lab Services engagements and supplied half a dozen consultants to support a variety of engagements including implementations, migrations, healthchecks, and staff augmentation, supporting replication solutions InfoSphere Change Data Capture, pureScale, Kafka, and Q Replication, as well as Db2 and Hadoop. In staffing these engagements we became aware of several interesting customer situations and opportunities we hope will deliver better results to our customers in 2019. Through a chance conversation with a Federal agency looking to identify the best replication tools for their next generation applications, we learned about some enhancements to the replication portfolio solutions expected to be delivered in 2019. To support our IBM colleagues in spreading the word about the upcoming enhancements, and to (no surprise!) encourage customers to contact us if they are in need of support, the 2019 schedule of events below was drafted. Some of these "events" consist primarily of outreach to customers who may be interested. If these apply to you, feel free to contact me for more information. Here's to a busy 2019! 1. CDC Healthcheck Services. Delivery through Lab Services. a. Audience: current CDC customers considering new replication environments and/or looking to better support current replication. b. Tasks/activities: a review of current replication environments, staff enablement, monitoring review, performance tuning and/or planning new replication environments. c. Deliverables: Healthcheck report, enablement recommendations, plan for new replication sources and targets. 2. IBM Replication Roadmap Webinar. a. Audience: current and prospective replication customers. b. Content: a review of current and new replication offerings. c. Deliverables: recommendations/actions needed to initiate CDC purchases. 3. Kafka Webinar. a. Audience: current and prospective customers looking to replication to Kafka. b. Content: a review of Kafka, use cases, and where Kafka is best deployed. c. Deliverables: recommendations/actions to initiate Kafka deployment. 4. Replication Whitepaper. a. Audience: current and prospective customers. b. Content: TBD – replication competitive options, pricing, and/or Db2 for z data focus. c. Deliverable: whitepaper explaining IBM replication options compared with competitor solutions. 5. Kafka Quickstart Services. Delivery through Lab Services. a. Audience: customers implementing replication to Kafka. b. Tasks/activities: Kafka installation, replication set-up, and staff enablement. c. Deliverables: data replicating to Kafka. 6. Kafka Training – Extending KM020G ICDC Class a. Audience: new Kafka customers. b. Tasks/activities: instructor-led modules to demonstrate and explain Kafka implementation and administration. c. Deliverables: 2-days of formal classroom instruction with hands-on exercises. 7. IAS to IAS Replication Quickstart Services. Delivery through Lab Services a. Audience: customers implementing IAS to IAS replication. b. Tasks/activities: Software installation, replication set-up, and staff enablement. c. Deliverables: IAS replication, monitoring in place. 8. Competitive Replication Webinar. a. Audience: Oracle GG and Attunity customers. b. Content: a review of the advantages of IBM replication solutions. c. Deliverables/action items: action plan to PoC IBM replication offerings.