What should a data analytics workflow look like?

A value-creating analytics workflow goes beyond ad hoc spreadsheet analysis. It must meet several core requirements:

• Data scale: Analytical systems must handle large, variable volumes of data without manual adjustments.
• Collaboration scale: The workflow should support small teams or thousands of contributors equally well—without coordination bottlenecks.
• Accessibility: Different user personas (engineers, analysts, decision-makers) should participate as peers, not in silos.
• Velocity: The workflow should facilitate rapid analysis without introducing excessive overhead or governance friction.
• Correctness: Results must be trustworthy, with built-in mechanisms for validation and error detection.
• Auditability: All transformations, analyses, and outputs must be reproducible, with tracked changes.
• Governance: Data access and usage must be controlled to comply with internal rules and external regulations.
• Criticality: The system should support both explorative work and mission-critical production needs, without requiring refactoring.
• Reliability an resilience: The system should remain robust in the face of failures, minimizing both downtime and the impact of errors.

Example: A retail company has seasonal sales spikes. During holiday campaigns, their data platform must seamlessly scale data pipelines and support concurrent collaboration between category managers, financial analysts, and data engineers—all while ensuring that promotional reports are correct, auditable, and accessible only to authorized personnel.

The stakeholders: hats, not badges

Every organization leveraging analytics has individuals playing three roles:

1. The engineer

Designs and maintains reusable data assets (pipelines, models, metrics) that enable the broader analytics function.

2. The analyst

Explores data, derives insights, and develops recommendations that inform business choices.

3. The decision-maker

Acts on the data-driven recommendations and is responsible for directing business action based on insights.

Note: Individuals can and should shift between these 'hats' as needed. In a mature workflow, this flexibility allows for reduced friction and fosters innovation. For example, an analyst should be empowered to adjust model transformations if needed, without waiting in a ticket queue for engineering.

Example: At a SaaS startup, a product manager (decisionmaker) collaborates with a growth analyst (analyst) to evaluate a new feature's impact. When they discover that the relevant event data isn't yet available, the analyst applies their engineering skills to adjust the event pipeline, unblocking the analysis in hours instead of weeks.

The Analytics Development Lifecycle (ADLC)

The Analytics Development Lifecycle (ADLC) draws directly from the software development lifecycle, recognizing that analytics systems are ultimately software systems. The ADLC applies to all analytical assets—pipelines, models, dashboards, and more—by guiding them through eight cyclical stages:

1. Plan

Changes to analytics systems start with planning. This includes identifying business requirements, consulting stakeholders, anticipating downstream impacts, and planning for long-term maintenance. The goal is to clarify why a change is needed and how it will be safely and sustainably implemented.

Example: A marketing team wants to understand conversion rates by new customer acquisition channels. The analyst gathers requirements, checks if existing models capture the necessary data, and consults channel owners on business definitions.

2. Develop

Actual creation or modification of analytical assets happens here. This should be anchored in code—whether SQL, Python, or another open language—to maximize reproducibility, reviewability, and independence from any single tool.

Best practices in this stage include following style guides, documenting changes, prioritizing readability, and ensuring code is generic and reusable where possible.

Example: The analyst writes new transformations to classify historical leads by updated channel definitions. They use existing macros and test helpers to avoid duplicating code present elsewhere.

3. Test

All changes must be thoroughly tested before release. This involves three broad types:

• Unit Tests: Confirm logical correctness of new code.
• Data Tests: Ensure the code behaves properly with actual data.
• Integration Tests: Verify that changes don't break dependencies or complementary systems.

Automated testing is mandatory—manual checks are not scalable or reliable.

Example: Before merging, the analyst writes unit tests that check customer channel categorization logic, and data tests to validate that all records have a valid channel. Integration tests confirm that existing dashboard queries referencing this field continue to function.

4. Deploy

Moving changes into production should be automated and pain-free. Deployments must be triggered from source control, happen without causing user interruptions, and include automated rollback in case of failure.

Example: Once all tests pass and code is peer-reviewed, merging to the main branch triggers an automated deployment. Smoke tests post-deployment quickly catch unexpected edge cases, and any failures auto-revert to the previous version.

5. Operate and observe

Once in production, the system must be continuously monitored. This includes system health, data quality, and user-impacting issues. Mature systems detect and resolve issues quickly—ideally before users notice.

Key metrics (uptime, throughput, error rates) must be defined and measured. Incident response processes and on-call rotations ensure rapid remediation.

Example: Post-launch, data completeness metrics are monitored; if a channel is found with missing assignment, an alert triggers so engineering and analytics teams can resolve it quickly.

6. Discover

Users must be able to easily find existing data assets—tables, dashboards, reports—via simple interfaces, without informal bottlenecks or reliance on internal networks.

Example: A new analyst joins the team and, using the platform's universal search bar, locates all assets related to "customer acquisition," including source models, transformation code, and trusted dashboards.

7. Analyze

The real business value of analytics is created here, through exploration and interpretation of data. Mature systems allow users to conduct exploratory analysis in-place, and as insights mature, fold findings back into the broader system via the Plan and Develop stages.

Feedback and questions from users should spark continuous improvement cycles.

Example: The analyst uses the discovered data assets to identify a surprising drop in conversion for a newly launched channel. They share interim results, gather feedback from marketing, and formalize findings into a new dashboard, which then undergoes its own cycle of testing and deployment.

The loop: continuous improvement

The ADLC is intentionally cyclical. Any insight or change at the Discover & Analyze phase should motivate a new Plan phase to improve or modify existing analytical assets. This feedback flow is crucial—mature analytics systems improve over time, informed by both system monitoring and user experience.

Hypothetical end-to-end scenario

Imagine a financial services company seeking to reduce fraudulent account signups:

1. Plan: Compliance, risk, and data teams define requirements for a new fraud detection model. They involve customer support (a key stakeholder) to align on business needs and downstream notification requirements.
2. Develop: Data engineers extend existing user event pipelines, while analysts prototype new feature engineering logic within reusable models and scripts.
3. Test: All components are unit tested, data validated on a staging environment, and integration tests confirm that fraud alerts don't erroneously trigger notifications to legitimate users.
4. Deploy: Following code review and merge, the full set of changes is deployed automatically, with no downtime.
5. Operate & Observe: Engineers set up real-time monitoring for false positives/negatives. An incident is quickly detected when a legitimate signup is flagged, leading to a rollback and targeted fix.
6. Discover & Analyze: Analysts explore fraud patterns in new data assets, sharing findings with compliance and operations, and suggesting further refinements.
7. The loop restarts as these ideas go back to planning for the next iteration.

Requirements for the discover and analyze phase

A mature workflow must guarantee:

• Frictionless discovery of assets and data lineage
• In-place operations—no downloading or emailing files
• Built-in feedback mechanisms for every analytical artifact
• Transparent processes for requesting/approving access
• Provenance and change tracking for all data and assets
• Flexible environment selection (dev, staging, prod)
• Minimal exposure to technical implementation details for end users

Conclusion

A mature analytics workflow is not a luxury—it's a necessity for any organization that seeks to build trust in data, act quickly on insights, and scale analytics impact across the business. The Analytics Development Lifecycle (ADLC) provides a practical, software-inspired framework that integrates best practices at every layer of the stack, among all participants.

Fully realizing this vision will require both iterations in process and continued evolution of analytical tooling. By embracing clear workflow stages—Plan, Develop, Test, Deploy, Operate, Observe, Discover, and Analyze—and breaking down organizational silos, companies can achieve analytics that are not only fast and flexible, but also governed, reliable, and value-generating.

ADLC is the new benchmark for modern analytics. Organizations that embrace this approach will have a decisive edge—not just in technological capability, but in business adaptability and innovation.

Frequently asked questions about analytics workflows

What is an analytics workflow?

An analytics workflow is a structured approach to processing and analyzing data that goes beyond ad hoc spreadsheet analysis. A mature analytics workflow must handle large volumes of data, support collaboration at scale, remain accessible to different user types (engineers, analysts, decision-makers), deliver insights quickly, ensure correctness and auditability, maintain proper governance, support both exploratory and mission-critical needs, and provide reliability even when facing system failures.

What is an analysis workflow?

An analysis workflow is the structured process that guides how data is transformed into actionable insights. In mature organizations, this follows the Analytics Development Lifecycle (ADLC), which applies software development principles to analytics. The workflow involves planning based on business requirements, developing analytical assets using code, testing for correctness, deploying changes safely, operating and monitoring systems, discovering existing assets, and analyzing data to create business value.

What are the steps of workflow?

While traditional workflows may have 5 steps, the Analytics Development Lifecycle (ADLC) presents a more comprehensive 8-stage approach:

1. Plan - Identify requirements and consult stakeholders
2. Develop - Create or modify analytical assets using code
3. Test - Verify correctness through unit, data, and integration testing
4. Deploy - Move changes to production through automated processes
5. Operate & Observe - Monitor system health and data quality
6. Discover - Find existing data assets through simple interfaces
7. Analyze - Explore and interpret data to create business value
8. Loop back to planning for continuous improvement

What is the actual workflow of data analysis?

The actual workflow of data analysis follows a cyclical pattern where insights drive improvements. Starting with planning, analysts define requirements with stakeholders. They then develop solutions using code, thoroughly test for correctness, and deploy changes to production environments. Once live, systems are monitored for performance and data quality.

Users discover relevant data assets through search interfaces, then analyze this data to extract business insights. These insights often identify new opportunities or improvements, beginning the cycle again with planning for the next iteration.

The most effective workflows support three key stakeholder roles—engineers who build data assets, analysts who derive insights, and decision-makers who act on recommendations—while allowing individuals to shift between these roles as needed to reduce friction and foster innovation.