Growth Marketing and Conversion Optimization

01Purpose and scope

• This manual supports the course Growth Marketing and Conversion Optimization. It describes how the work is actually done in a product or marketing team that is trying to grow revenue, sign-ups, or repeat use through measured changes rather than large one-time campaigns.
• The scope covers the full path a person takes with a business: first exposure through paid, organic, referral, or partner channels, the first meaningful action such as a sign-up, trial, or add-to-cart, the activation point where the person gets value, and the later steps that bring them back. Conversion optimization is the narrower practice of improving the share of visitors who complete a specific step. Growth marketing is the wider practice of finding which steps limit the business and choosing where to spend effort to move them.
• The manual is written for practitioners who run experiments, analysts who support them, and managers who set priorities and budgets. It does not replace the judgment of a qualified legal, privacy, or accessibility professional, and it does not authorize data collection or testing on any population. Where a rule depends on jurisdiction, such as consent requirements for tracking or disclosure rules for pricing tests, the current authoritative source for that jurisdiction governs.

02Core principles

• Start from a constraint, not a feature idea. A useful growth effort begins with a measured bottleneck, such as a drop between the pricing page and checkout that is larger than the drop at other steps. Ideas for new features or banner colors come after the bottleneck is clear, not before.
• Treat every change as a hypothesis with a stated expected effect, a measurement plan, and a decision rule written before the test starts. A hypothesis should name the audience, the change, the metric, the expected direction, and the size of effect that would justify shipping the change. Without a written decision rule, results get interpreted after the fact to match whatever the team already wanted.
• Protect the business from harm before chasing lift. A test that raises sign-ups while lowering refund rates or support contacts has not helped. Guardrail metrics, such as cancellation rate, average order value, page speed, and complaint volume, should be defined for every test and reviewed alongside the primary metric.
• Prefer changes that are cheap to reverse and easy to explain. Copy edits, ordering of form fields, and default selections can usually be tested in days. Pricing changes, onboarding redesigns, and changes to how data is collected carry higher stakes and need longer review.
• Accept that most experiments fail. A realistic program may see a minority of tests produce a clear win. The value comes from the learning recorded in each test, which narrows the next round of ideas.

03Planning and preparation

• Begin with a funnel map. List each stage a user moves through, from first visit to the business outcome, and attach a count or rate to each stage for the last several weeks of data. Note where data is missing or unreliable. Many teams find that a tag that fires on only some browsers or that double counts returning visitors distorts the funnel more than any design problem does.
• Build an opportunity backlog from three sources: the funnel map, customer research such as support tickets, interviews, and session recordings reviewed with consent and privacy controls in place, and the ideas submitted by the rest of the organization. Score each opportunity on expected impact, confidence in the evidence, and effort. A simple scoring sheet with agreed definitions is enough. The scores are a conversation starter, not a formula that replaces judgment.
• Estimate sample size before committing to a test. The required sample depends on the baseline rate, the smallest change worth detecting, the significance level, and the desired power. Low-traffic pages often cannot support a two-way comparison within a reasonable time, and the honest choice then is a larger change, a qualitative study, or a different page. Running a test that cannot reach a conclusion wastes the traffic it uses.
• Set up the testing environment before the test is designed. This includes an experiment tool or in-house assignment logic that splits traffic at random, a consistent way to assign returning users to the same variant, event tracking verified in a staging environment, and a shared place to log hypotheses and results. Confirm that the control experience is the current production experience and that the variant loads without breaking page layout or analytics.
• Agree on roles before launch. One person owns the hypothesis and decision. An analyst owns the measurement plan and the final readout. An engineer or developer owns implementation. A legal or privacy reviewer approves any change that affects personal data, pricing displayed to customers, or claims made in copy.

04Recommended workflow

1. 1
   : Pull the funnel data for the period you plan to study and confirm that counts reconcile with a second source, such as billing or the order system. Record the baseline conversion rate for each step and the segment definitions you will use.
2. 2
   : Review the backlog with the team and select one or two bottleneck stages to work on this cycle. Write a hypothesis for each candidate in the form: for this audience, changing this element should move this metric in this direction because of this observed behavior.
3. 3
   : Define the primary metric, the guardrail metrics, the minimum detectable effect, the sample size, and the planned run length. Write the decision rule. For example, the variant ships if the primary metric improves at the agreed confidence level and no guardrail falls beyond its agreed limit.
4. 4
   : Build the variant with the smallest change that tests the hypothesis. Review the design with the people who will support customers on that page, since they often know which confusions are common. Verify tracking in staging with test accounts and check the page on mobile and desktop, plus any assistive technology the product is expected to support.
5. 5
   : Launch with a traffic split that matches the design, and monitor only for technical failures during the run. Do not look at the primary metric repeatedly and stop early when it looks favorable. If you need interim monitoring, decide in advance how it will be handled, for example by using a sequential method designed for that purpose.
6. 6
   : At the planned end, analyze the result against the pre-set decision rule. Check that the traffic split was close to the planned ratio, that the variants received comparable traffic mix across devices and channels, and that no outage or promotion affected one arm more than the other. Report the effect size with its uncertainty, not only whether it was significant.
7. 7
   : Record the outcome in the experiment log, including what was learned even if the variant lost. Ship, roll back, or extend the test according to the decision rule. If the result suggests a follow-up, add it to the backlog with the evidence attached.

05Quality checks, risk controls and compliance

• Check the data before reading the result. Confirm that the assignment logic was not overridden by a cookie or a manual redirect, that event counts are not duplicated, and that bot traffic has been filtered by the same rules in both arms. A surprising win is often a tracking bug, so treat any large effect on a small change with suspicion until it is verified.
• Watch for sample ratio mismatch. If a 50/50 split produces a 53/47 split of users, something in assignment, redirects, or tracking is probably wrong, and the result should not be trusted until the cause is found. This check is cheap and catches many problems.
• Review consumer-protection and pricing risk. Showing different prices to different customers, implying scarcity that is not real, or making hidden charges part of a test can create legal and reputational exposure in many jurisdictions. Countdown timers and stock messages must reflect actual conditions. Anything touching price, eligibility, or terms needs approval from the people responsible for those policies.
• Handle personal data under the rules that apply to your location and your users. Tracking pixels, session replay, and cross-site identifiers may require consent or disclosure, and the required mechanism differs by jurisdiction. Limit collection to what the test needs, avoid capturing form field contents, and confirm retention periods with the privacy function. The current authoritative privacy regulator or legal counsel for each market is the right source for these decisions.
• Test accessibility. A variant that improves clicks for most users but breaks keyboard navigation or screen reader labels creates a real problem for some customers and may conflict with obligations that apply to your organization. Include basic accessibility checks in the quality review for every variant.
• This guidance does not replace qualified professional judgment, approved internal procedures, or required authorization. Regulated sectors such as health, financial services, and credit have specific rules on advertising claims and offers, and changes in those areas need sign-off from people trained for them.

06Collaboration and documentation

• Keep one shared experiment log that every test uses. Each entry should include the hypothesis, the owner, the start and end dates, the audience definition, the variants with screenshots, the primary and guardrail metrics, the decision rule, the result with effect size and interval, and the decision taken. A log that is consistently filled in is more valuable than a polished deck written once a quarter.
• Hold a short weekly review where people present new hypotheses, running tests, and completed results. Keep it to a fixed agenda: what finished, what decision was made, what is launching next, and what is blocked. Invite engineering, design, customer support, and finance when their input changes the decision. Support staff often know which complaints a test will affect.
• Make sure the people who will act on a result understand the trade-off before the test begins. A variant that increases trial sign-ups but attracts lower-value customers may be good for one team and harmful for another. Agree in advance how such conflicts are resolved and who has authority over them.
• Document the reasons for shipping, not just the change itself. Six months later, a future team member needs to know why a particular form order exists or why a price is displayed a certain way. A short paragraph on the evidence and the constraint is usually enough.
• Share negative results. A test that found no effect on a cherished idea prevents the same idea from being retested by someone else. Keep failed experiments in the log with the same care as winners.

07Measures and continuous improvement

• Track program-level measures as well as individual test results. Useful measures include the number of tests completed per month, the share of tests with a pre-registered decision rule, the share with a valid sample ratio, the proportion of shipped changes that held their effect after a follow-up check, and the time from hypothesis to decision. These describe how well the process works, which matters more than any single win.
• Track business measures that the test program is meant to move, such as activated users per thousand visitors, conversion from trial to paid, or repeat purchase rate at a fixed interval. Compare them against a holdout group where possible. Gains that appear in test results but not in business totals usually signal a measurement issue or a shift in traffic mix.
• Review the backlog scoring periodically. If the team consistently overestimates impact for certain types of change, such as button wording, adjust the expected values in the scoring sheet using the completed results. Over time, this calibrates the team's judgment against evidence.
• Plan for diminishing returns. Early tests on a neglected funnel often produce large gains, but later tests on the same page usually produce smaller ones. At that point, the more valuable work may be in pricing, onboarding, retention, or the product itself, which may require different skills and closer work with engineering.
• Revisit tooling and tracking every few cycles. Event definitions drift as products change, and a funnel that was accurate last year may now include a step that no longer exists. Schedule a periodic audit of the tracking plan, and document changes in the same place as the experiment log.

08Common mistakes and how to prevent them

• Stopping a test when it looks like a winner. Early peeks inflate false positives. Prevent this by fixing the run length or using a sequential method chosen in advance, and by writing the stopping rule into the test plan before launch.
• Testing too many things at once. A variant that changes headline, image, price display, and button color together can win, but the team will not know which element caused the change. Prevent this by testing one clear idea per variant, or by using a deliberate multivariate design with enough traffic to support it.
• Optimizing a metric that is easy to move but not valuable. Clicks on a call to action are simpler to count than paid retention, but moving clicks can attract low-quality users. Prevent this by making the primary metric as close as possible to business value and always reviewing guardrails.
• Ignoring segments that behave differently. An overall result can hide a strong loss for returning customers or mobile users in a particular region. Prevent this by pre-specifying the few segments that matter and reviewing them with caution, since multiple segment comparisons also inflate false positives.
• Treating a single test as settled. Effects can fade as novelty wears off, or change when a seasonal or promotional period ends. Prevent this by running a follow-up check on shipped changes at a later date and noting conditions that may have affected the result.
• Letting the program become a feature factory. Teams under pressure to show activity may run many small tests that carry little learning. Prevent this by tying each test to a named bottleneck and reviewing whether the backlog still targets the most important constraint.

09Practical checklist

• Confirm the funnel baseline reconciles with a second data source before choosing a target.
• Write the hypothesis, primary metric, guardrails, minimum detectable effect, and decision rule before building the variant.
• Calculate sample size and confirm the page has enough traffic to reach a conclusion in a reasonable time.
• Verify tracking and assignment in staging and check the variant on mobile, desktop, and keyboard navigation.
• Obtain privacy, legal, or pricing review for any change touching personal data, prices, offers, or regulated claims.
• Launch the test without changing the decision rule and avoid acting on interim results.
• Check for sample ratio mismatch and tracking errors before reading the primary metric.
• Report effect size with uncertainty and record the decision and learning in the shared log.
• Schedule a follow-up check on shipped changes to confirm the effect held.
• Review program measures monthly, including completion rate, decision quality, and business outcomes.

★Learning sources

• Micro Degree: Growth Marketing and Conversion Optimization
• Micro Degree: Marketing Strategy Development
• Micro Degree: Programmatic Advertising Management
• Micro Degree: Lead Generation Operations