From DataOps to Database Pros
Another year, another THWACKcamp. Our resident database expert, Kevin Kline, met with some of the brightest lights in the DB world for two fascinating webcasts. Both are available for viewing right now.
DataOps: An Overview
In “Shift-Left: A Better Approach to DataOps” Kevin meets with Kevin M. Sparenberg to discuss some of the issues a typical database administrator will face on the job. We’re all familiar with the term Development Operations (DevOps). In this webcast, Kevin asks us to consider a related concept; Data Operations (DataOps).
DataOps combines an integrated, process-oriented perspective on data that combines automation and aspects of agile practices. The end goal of DataOps is to improve quality, speed, and collaboration.
In software development, shift left-refers to the practice of moving testing, quality assurance, and performance evaluation early in the development process. Adopting a DataOps approach to database maintenance means carefully examining your internal processes and shifting to the left important aspects of your applications and databases, like performance and security, which would normally occur much later in the development process.
While DevOps is centered on product development, DataOps focuses on keeping your business running by shortening the cycle time for data integration, analytics, and alignment with organizational goals. DataOps is about helping stakeholders make data-driven decisions. The goal is to cut barriers between data managers and data consumers.
Core concepts:
- Establish progress and performance measurements at every stage of the data flow- including benchmarks of data-flow cycle times.
- Define rules with an abstracted semantic layer. This way everyone ‘speaks the same language’ and agrees what data (and metadata) is and is not.
- Validate with continuous-improvement human feedback loops so that end-users can trust the data.
- Automate as many stages of the data flow as possible, including business intelligence, data science, artificial intelligence, and analytics.
- Collect benchmarked performance information.
- Find and reduce bottlenecks through database optimization.
- Establish governance and discipline, with a focus on two-way data control, data ownership, and transparency.
- Design scalable processes for growth and extensibility using data flow models that calculate for volume growth and a variety of data.
Databases: A Primer for Pros
Meanwhile, in “What to Do if Someone Drops a Database in Your Lap and Runs,” Kevin speaks with the talented (and creative) Homer McEwin, also known as ‘KillaDBA’. If you've ever had an application owner suddenly declare you’re now the data person responsible for their database, this session is for you.
Core concepts:
- Tools: Better tooling gives you and the application owner the insight needed to make smarter, more informed decisions.
- Security and compliance: Don’t make this an afterthought or you will pay the price. Hackers abound.
- Monitoring: When you don’t know what normal is, you won’t know what abnormal is. At a minimum, use built-in alerting and monitoring.
- Setup, configuration, and patch management: Keep cumulative updates (CU) and operating system (OS) patches up to date.
- Plan: Preparation today saves work tomorrow. Plan your downtime. Watch consumption trends.