Hi everyone, I am back after taking a blogging break in 2018. It was nice seeing everyone during the year at data management and Db2 conferences, user group presentations, and various other functions. During 2018, I continued to enjoy the dynamic problem solving of challenging consulting assignments. Thank you to all my clients.
It has been an interesting year helping with the architecture of health care systems, security audits, and developing new Docker, cloud and Kubernetes infrastructures that exposed interesting technology issues. Security audits, system and application performance tuning, architecture, strategy, and design reviews have never been so important to maintain and improve performance and response time to handle all the new uses of data. Analytics Db2 IDAA and other ML and AI technology advancements continue to show massive performance advantages over Hadoop and other NoSQL DBMSs.
So with several years’ worth of data management predictions recorded you can easily see my track record by clicking the following links.
2014 found here (http://davebeulke.com/five-data-management-trends-2014/),
2015 found here (http://davebeulke.com/five-data-management-trends-2015/),
2016 found here (http://davebeulke.com/five-data-management-trends-for-2016/),
2017 found here (http://davebeulke.com/five-data-management-trends-and-predictions-for-2017/)
2018 found here (http://davebeulke.com/five-data-management-predictions-for-2018/),
You can see that I don’t always go for the five easiest typical data management trends or play it too safe. So, here are my Five Data Management Predictions for 2019.
- Cloud becomes the “must-have” new paradigm
for developing, building, and addressing scalable, big data
business application requirements. Platform-as-a-Service
(PaaS) as the dominant cloud service model (over IaaS) solves all
those pesky performance and operational problems with which no one
wants to deal. Using Docker containers and Kubernetes management,
allows the user to experience cloud ease of use first hand. PaaS
and all its components helps business to quickly develop, test,
implement and scale for their new business opportunities.
For example, health care, banks, telcos, government agencies, and other companies have tens of millions of mobile phone users who can order products and services or perform an action based on a tweet. Perception is reality—if it is not cloud, it is old, legacy, and no one wants to deal with it even though without legacy systems many businesses would come to a halt. Besides, the cloud has all the new cool open source technology and buzzword services every developer wants on their resume.
A great way to understand and hear more about the new “Kubernetes and the Path to Serverless” and an integration of a FORTRAN legacy systems into an AWS Lambda Cloud implementation is to listen to Kelsey Hightower’s keynote at KubeCon 2018 found here (https://www.youtube.com/watch?v=oNa3xK2GFKY). The new cloud native application technology is made to look so easy.
- Security and hacking continue to be a major
issues. This year was another banner year for the bad
guys, nation states, your competition, or unthinking guys who
allowed data misuse. I was personally hit by several of the hacks
again this year. The first was the fallout from Facebook’s
platform/data misuse by foreigners trying to influence election
results. Also Twitter, Instagram, and others allowed their
platforms and users access to more data or to more fake data than
they thought possible. Next were the hacks of credit card and other
information at Macy’s, Under Armour, Uber, and others. I
guess this is why my email continues to get all the spam, free
offer, and BitCoin extortion emails from everyone in the world.
Surprise! I guess turning on two factor authentication, and adding browser blockers using anonymous browsing mode on isn’t effective after all. Blockchain will continue to be implemented at the margins, but until its performance issues are solved, 2019 will only see marginal adoption. Maybe 5G will help blockchain adoption?
- Buzzword alert: every company and everyone will have a
digital transformation (DX) strategy. If your company
doesn’t it will. Management wants to transform the enterprise
into a digital driven enterprise. “Refocus, reorganize,
rationalize, and implement strategic IT data management procedures
to enable digital transformation incorporating new management,
governance, and security approaches in a highly connected
competitive environment.” This sentence came out of a recent
management meeting. There were no IT people in the room to press
for any realistic measurements, project specifics or any specifics
Being a consultant for many years and working through these types of company management trends, directives, and especially blue sky bold statements, I am reminded of twenty years ago when every company was transforming to the “new paradigm of the new internet online strategy”. Yes it was the right direction and winners and losers were defined by which companies did managed the new change effectively. Some companies are still getting online now; some never made it online and went bankrupt. Some made it online, but had the wrong online strategy and destroyed their business. The point is that yes, it may be correct to go after digital transformation and utilize AI, ML, and IoT but it must be done with a beneficial business case. The digital transformation will be difficult and perilous unless specific realistic plans with business benefits are developed, justified, and implemented cost effectively. Digital transformation and new technology are great, but business is about making a profit.
- It will be very difficult for management to be patient
enough to wait for the development/implementation of AI and ML
projects. For example, everyone must realize that the
existing marketing data warehousing system took several years to
develop and many more for marketing to produce profitable marketing
plans. The system may utilize 10 characteristics with 10 behaviors
being fit into 50 different marketing models. Now take those
current data warehousing marketing techniques and super charge them
using AI for increased sales and profit. AI will want to
continually improve the system for every one of your 4 million
customers. The original AI project was committed to produce only 20
million marketing profitable models. Developing the additional
personalized marketing AI models and validating them so that they
produce a profit may take more time than
AI and ML have been baking in academia for decades. Currently the upper management and the business press communities are thinking that their IT personnel can produce something AI or ML quickly. The reality is that new AI and ML project success will be probably approach the success of data warehousing projects which was only about 20 percent. Producing AI and ML systems will probably be less successful than that because of their complexity, but the payoff for the successful companies will be huge and change the competitive industry. Since AI and ML systems will have huge impacts and expectations, the IT hype and bust cycle will be more dramatic within companies, more costly and more devastating to companies that get it wrong than any other recent IT trend. Even though there are huge risks management by magazine and CIOs who want to make a fast impact will go for broke and try to leverage the new AI and ML into every business processes.
- The IT technology dystopia is here and will only
accelerate as the gap between technology, methodologies,
application languages, and overall skills continues to
widen. Recent experiences with cloud infrastructures show
that when it works, the processing, performance, and success can be
amazing. When there is a problem, these infrastructures can be very
difficult to debug and fix because of the number of interfaces,
corporations involved and diverse new technology. One bad script or
bureaucracy can ruin everyone’s day.
The dystopia continues as the new Docker, cloud, Swarm methodologies , and their associated application languages take over, and the new, recently graduated, employees don’t want to work on that old legacy stuff. Vanishing are is the detailed knowledge of the underlying code base, data factors, legacy programming language, and the software skills to debug it all. Massive Assembler, COBOL, FORTRAN, C#, and other code bases still run the world economies and all of their support people are retiring. Colleges and the professors provide computer science degrees without teaching almost any programming skills. Digital transformation will only happen if these legacy systems that keep the company revenues going are transformed so that the new workforce wants to leverage them with the new technologies.
My next webinar is Best Design and Performance Practices for Analytics, AI, and ML. The webinar talks to my recent experiences with these various types of systems.
Implementing analytic applications is easy with a variety of cloud, application appliance, and hybrid database alternatives. Designing application analytical systems requires comprehensive data management and usage analysis to design and buildcreate the best ideal performance analytical application that to produces meaningful insights.
Having designed, deployed, and worked with successful five-, eight-, twenty-two-, and forty-two billion row analytical database systems, this presentation will discuss the application design patterns that provide sustainable performance for fast analytic answers while using big data.
Through this presentation, you will learn the application analytics architectures, the superior design patterns, and application techniques to maximize performance for insightful analytic answers.
The webinar is brought to you through the DB2 Night Show Webinar. Click here to get to that webinar. that can be found here (https://www.dbisoftware.com/db2nightshow/).
I will also be speaking at the IBM THINK conference in San Francisco in February. Please sign up for my session 3186A here (https://myibm.ibm.com/events/think/all-sessions/session/3186A ) on the IBM THINK conference website.
Best Architectures and
Performance Practices for Analytics
Wednesday, 3:30 PM – 4:10 PM | Session ID: 3186A
AMC at Metreon | Theatre 4
Dave Beulke is a business strategist, systems architect and performance expert specializing in big data, data warehouses, and high-performance internet solutions. He is an IBM Gold Consultant, Information Champion, President of DAMA-NCR, former President of International DB2 User Group (IDUG), and frequent speaker at national and international conferences. His strategies, architectures, and performance tuning techniques enhance analytics, security, and performance so organizations can better leverage their information assets and save millions of dollars in time to market, CPU, development, and overall costs. Follow his blog at davebeulke.com or on Twitter here (@DBeulke) or connect through LinkedIn here (https://www.linkedin.com/in/davebeulke).
President – DAMA-NCR
IBM Gold Consultant
Inaugural IBM DB2 Information Champion
email: [login to unmask email]
Dave Beulke & Associates
a division of Pragmatic Solutions, Inc
Syspedia - Find, understand and integrate your data faster!