O/T: DB2 z/OS data replication, CDC and Attunity / IBM

Steven Lamb

O/T: DB2 z/OS data replication, CDC and Attunity / IBM

Does anybody have any experiences they're willing / able to share of Attunity's or IBM's products for data replication, with DB2 on z/OS as the source (and Kafka as the target)?

Some topics of concern to us are:

  • The initial set-up time i.e. copying data from the source tables where there are billions of rows involved. Are there any issues with locking in the source tables? We're talking a 24x7 banking environment, so performance is critical.
  • The effects of re-compressing the DB2 data during Reorgs. Does this mean having to re-build the Attunity / IBM  "copy" of the data?
  • Any adverse effects on DB2 logging? 

Regards,

Steve

Kim May

O/T: DB2 z/OS data replication, CDC and Attunity / IBM
(in response to Steven Lamb)
Steve,



The Fillmore Group is presenting a webinar at 1pm Eastern today on the “best of” 2019 replication enhancements, one of which is Db2 for z Remote Capture (which allows you deploy replication off the mainframe, saving the system resources and their associated costs) and another is around IBM’s CDC Kafka target. Info on the webinar is on our blog at www.thefillmoregroup.com/blog http://www.thefillmoregroup.com/blog . Both presenters – Ed Lynch and Frank Fillmore – have worked with a large financial customers to implement and tune replication – I am sure they will welcome your questions – hope you can make it.



Kim



From: Steven Lamb [mailto:[login to unmask email]
Sent: Thursday, February 7, 2019 9:16 AM
To: [login to unmask email]
Subject: [DB2-L] - O/T: DB2 z/OS data replication, CDC and Attunity / IBM



Does anybody have any experiences they're willing / able to share of Attunity's or IBM's products for data replication, with DB2 on z/OS as the source (and Kafka as the target)?

Some topics of concern to us are:

* The initial set-up time i.e. copying data from the source tables where there are billions of rows involved. Are there any issues with locking in the source tables? We're talking a 24x7 banking environment, so performance is critical.
* The effects of re-compressing the DB2 data during Reorgs. Does this mean having to re-build the Attunity / IBM "copy" of the data?
* Any adverse effects on DB2 logging?

Regards,

Steve



-----End Original Message-----

Steven Lamb

RE: O/T: DB2 z/OS data replication, CDC and Attunity / IBM
(in response to Kim May)

Thanks Kim. Unfortunately our "security systems" won't let me connect from work :(

I'll try it from home instead

 

Regards,

Steve

Kim May

O/T: DB2 z/OS data replication, CDC and Attunity / IBM
(in response to Steven Lamb)
Steve,



I understand…and trying to reach the slides on the phone while sitting in the parking is just a bit too much.



We will be recording the session and will be posting it on our blog – and the slides – they will be there.



And much of the material came from IBM so of course if there is something specific you want to know about, ask IBM.



Kim



From: Steven Lamb [mailto:[login to unmask email]
Sent: Thursday, February 7, 2019 10:29 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: O/T: DB2 z/OS data replication, CDC and Attunity / IBM



Thanks Kim. Unfortunately our "security systems" won't let me connect from work :(

I'll try it from home instead



Regards,

Steve



-----End Original Message-----

John Bucaria

O/T: DB2 z/OS data replication, CDC and Attunity / IBM
(in response to Steven Lamb)
Hi Steve,

We’re using Attunity Replicate with Db2 z/OS as the source and MapR Hadoop as the target. We are employing 2 replication scenarios. One set of jobs do daily full-load-only processing. The other set do a one-time full-load followed by continuous (24x7) CDC processing.

Attunity Load processing opens an ODBC connection to Db2. Locking can be controlled by the isolation level of the connection. Run times for the full loads depend on available resources on your z/OS system which will either promote or limit the amount of Db2 parallelism employed for reading the table(s).
CDC processing does not read the Db2 tables so there are no locking considerations during CDC. The Db2 log is read for updates occurring to tables involved in CDC tasks and those updates are passed to the target. We have not seen any adverse effects on Db2 logging.

We’re not using any data compression, so I can’t comment on that.

Best,
John

From: Steven Lamb <[login to unmask email]>
Sent: Thursday, February 07, 2019 9:16 AM
To: [login to unmask email]
Subject: [DB2-L] - O/T: DB2 z/OS data replication, CDC and Attunity / IBM


Does anybody have any experiences they're willing / able to share of Attunity's or IBM's products for data replication, with DB2 on z/OS as the source (and Kafka as the target)?

Some topics of concern to us are:

* The initial set-up time i.e. copying data from the source tables where there are billions of rows involved. Are there any issues with locking in the source tables? We're talking a 24x7 banking environment, so performance is critical.
* The effects of re-compressing the DB2 data during Reorgs. Does this mean having to re-build the Attunity / IBM "copy" of the data?
* Any adverse effects on DB2 logging?

Regards,

Steve

-----End Original Message-----

Steven Lamb

RE: O/T: DB2 z/OS data replication, CDC and Attunity / IBM
(in response to John Bucaria)

Thanks John.

Our situation will be like your second scenario - an initial load followed by continuous CDC processing. Basically, we don't want anything to interfere with the DB2 performance. We have connected systems that don't like delays of more than a couple of seconds and this can impact customers.

If Attunity uses ODBC connections for the load, we'll need to check our set up - number of threads kind of thing.

This is a data-sharing environment and it's financial data, so I assume / hope the products are designed to cope with re-assembling changes in the correct order.

We're still in the early stages (proof of concept), so we've got a way to go yet. 

 

Regards,
Steve

Kim May

O/T: DB2 z/OS data replication, CDC and Attunity / IBM
(in response to Steven Lamb)
Steve,



Are you using the new CDC Remote Capture? That allows you to move the CDC processing off the mainframe to another platform, and keeps the replication from impacting your z performance. It’s a great option if you are concerned about Db2 for z performance –



Kim



From: Steven Lamb [mailto:[login to unmask email]
Sent: Thursday, February 7, 2019 11:20 AM
To: [login to unmask email]
Subject: [DB2-L] - RE: O/T: DB2 z/OS data replication, CDC and Attunity / IBM



Thanks John.

Our situation will be like your second scenario - an initial load followed by continuous CDC processing. Basically, we don't want anything to interfere with the DB2 performance. We have connected systems that don't like delays of more than a couple of seconds and this can impact customers.

If Attunity uses ODBC connections for the load, we'll need to check our set up - number of threads kind of thing.

This is a data-sharing environment and it's financial data, so I assume / hope the products are designed to cope with re-assembling changes in the correct order.

We're still in the early stages (proof of concept), so we've got a way to go yet.



Regards,
Steve



-----End Original Message-----

Steven Lamb

RE: O/T: DB2 z/OS data replication, CDC and Attunity / IBM
(in response to Kim May)

Hi Kim,

 

At the moment we're still in the preliminary stages of the project and we're evaluating the available products; we haven't actually bought anything yet.

 

Regards,
Steve