Build A Proactive Data Lifecycle
Data governance for HIPAA & HITRUST organizations
Cloud Data Migrations
We will focus on avoiding your next license renewal of on-premise database technologies like Oracle and Teradata by rapidly migrating your infrastructure to scalable, usage-based billing in the cloud.
AWS RDS PostgreSQL and MySQL, or Azure SQL Managed Instance and Azure MySQL Database are recommended for always-on workloads in the cloud.
For analytic workloads, Snowflake is our first choice for true elastic performance and usage-based billing, but for batch operations AWS Athena+Glue or Azure Data Factory+Data Catalog on top of an object-store based data lake backed by S3 or ADLS2 can be a huge first step.
Your firewall is not the answer to the challenges of managing a secure data lifecycle in HIPAA/HITRUST, GDPR or CCPA environments. Using a systemic approach with clearly defined milestones we can help you close the gap, establish a certified de-identified data set (CDDS) and enforce auditable role-based access controls to PHI and PII.
Vendors like Alation and Informatica offer sophisticated data catalogs but there’s no “easy button” for a proactive approach to data governance where privacy is the default setting.
Your data is the most strategic and durable asset you own, it deserves user-centric, full lifecycle protection.
From a Jira or Rally tune-up, to mentoring your team’s journey to continuous delivery in Azure DevOps, our technically-minded agile coaches will pave the way toward your data-driven future.
Successfully moving from a production support mentality to “DataOps” requires more than a training class. Software tools must be upgraded, and business stakeholders must be willing to change process in order to simplify and accelerate time-to-delivery.
The technical challenges can easily be met, but the social complexity of this type of work can make an outside voice invaluable.
Ways Organizations are Using Data
To Drive Process
To Drive Strategy
To Monitor Performance
To Manage Risk
Source: Forbes, 2018
Why Work With Pipeline?
Our consulting hours are always pegged to milestones in the Statement Of Work. No more paying for a full-time consultant while waiting for IT to setup her account. We don’t bill unless business value is delivered.
What Can You Do?
- Develop a prototype
- Gain outside expertise
- Keep a project moving between hires
- Jump-start your Agile journey
- Reinforce a team
- On-demand data modeling with ERWin, ER/Studio, SQLDBM, Oracle SQL Developer, TOAD
Data Warehouse Consulting
Snowflake, Redshift, Synapse,
Data Lifecycle Management
AWS S3, Glue & DMS; Azure DLG2, Azure Data Flow, SSIS, Informatica
Agile DataOps Enablement
Jira, Rally, Azure DevOps, Office 365, PowerBI, Tableau, Looker
- Cloud data warehouse POC and pilots with Snowflake, Redshift (and coming soon Azure Synapse!)
- Establishing enterprise Data Quality or Data Governance practices
- Oracle and Teradata cloud migrations (and coming soon SQL Server!)
- Jira, Azure DevOps or Rally data-driven portfolio management for Agile data teams
- SAFe and DAD scaled agile consulting for small-to-medium size organizations
- ETL / ELT orchestration with Airflow and Data Build Tool (dbt)
- Establishing secure, governed Azure and AWS data lakes
- Rock solid change-data-capture pipelines from MySQL, PostgreSQL, SQL Server to warehouse with AWS DMS, Matillion or Fivetran
Our Favorite Articles
- Reading List: Enterprise Data ArchitectureAdmitting we have a problem is the first step toward fixing it… A decade old essay on Doing It Wrong to depress you, then one on Agile Data Testing (that might be even older!) to lift the spirits. Virtualize theContinue reading “Reading List: Enterprise Data Architecture”
- The Five Laws of Data WarehousingMy sincere apologies to Issac Asimov. Software is automation. Data warehouses are software. Your data warehouse should be a robot, programmed and tuned to provide the right data to the right people at the right time, while constantly checking itself for errorsContinue reading “The Five Laws of Data Warehousing”
- DevOps KPIs You Will Be Asked For, EventuallyHow many of these have you spotted in the wild? And which obvious ones did I miss? Operational Metrics Work in progress, by resource, team, project, and sprint Capitalizable time ratios for developers and teams (e.g. capex vs. opex) CycleContinue reading “DevOps KPIs You Will Be Asked For, Eventually”