Note: The job is a remote job and is open to candidates in USA. Reflex Media, Inc. is the parent company of Seeking.com, the world’s largest premium dating platform, currently undergoing a brand transformation. They are seeking a Growth Experimentation Manager to own the experimentation program, design and execute tests, and translate findings into actionable marketing decisions.
Responsibilities
- Own the full experimentation program end-to-end as a hands-on operator
- Design the test, build the audience, ship the variant, read the result, and write the recommendation before the next standup
- Set the testing roadmap, run rigorous experiments across paid media, lifecycle, landing pages, onboarding, and creative, and translate findings into concrete channel and creative decisions
- Build, maintain, and prioritize a comprehensive testing roadmap across paid media, email/lifecycle, SEO, landing pages, onboarding, paywall, and creative
- Develop a hypothesis backlog informed by quantitative funnel analysis, qualitative user insights, competitive research, and channel team input
- Define and enforce a structured prioritization framework (ICE, PIE, or a custom variant) to stack-rank tests by potential impact, confidence, and ease of execution
- Establish testing velocity benchmarks and ensure the team runs the optimal number of concurrent, non-interfering experiments at all times
- Maintain an always-current view of what is being tested, what has been learned, and what is next, and communicate it weekly to leadership and channel owners
- Design A/B, multivariate, holdout, and geo-based experiments with proper control groups, adequate sample sizes, and pre-defined success metrics
- Partner with Data & Analytics and Engineering to ensure correct instrumentation, event tracking, and attribution before any test launches
- Define primary, secondary, and guardrail metrics for every experiment to capture both intended lift and unintended side effects
- Apply the right statistical method for the test type, traffic volume, and decision urgency, and know when to call a test early vs. wait for significance
- Identify and mitigate threats to validity: novelty effects, seasonality, sample ratio mismatch, network effects, and interaction effects between simultaneous tests
- Conduct deep post-experiment analysis going beyond top-line win/loss to understand segment-level effects, interaction effects, and downstream impact on LTV
- Distinguish between statistical significance and business significance
- Build and maintain a results repository capturing test parameters, outcomes, learnings, and confidence levels, making institutional knowledge searchable and actionable
- Present findings to cross-functional stakeholders in a clear, non-technical narrative that connects test outcomes to business strategy
- For every completed experiment, deliver a structured recommendation by a committed date: ship it, iterate on it, or kill it, with supporting rationale and proposed next steps
- Translate winning test results into channel-specific updates: campaign and targeting changes for Paid Media; flow logic and segmentation for Lifecycle/CRM; landing page and onboarding updates with Product; creative direction briefs for the Creative team; SEO and UX recommendations from engagement tests
- Flag when a result should trigger a broader strategic shift vs. a tactical tweak, and escalate those moments proactively
- Build iterative test sequences where each experiment compounds prior learnings, rather than running isolated one-off tests
- Connect the dots across channels. Surface patterns from paid media tests that should inform lifecycle messaging, and vice versa
- Build shared creative and messaging frameworks derived from test learnings that every channel owner can apply
- Partner with the Lifecycle Marketing & CRM Manager, the SEO Manager, and the Performance Media Manager to ensure experimentation coverage across organic, paid, and owned funnels
- Serve as the primary liaison between Marketing and Data & Analytics for testing, translating business questions into test designs and analytical outputs back into marketing action
- Evaluate, implement, and manage experimentation tooling: A/B testing platforms, feature flagging, geo-experiment tools, and analytics integrations
- Define and document testing standards, naming conventions, QA checklists, and launch/kill criteria so experiments are run consistently across the team
- Champion a test-and-learn culture. Run workshops, share weekly wins and learnings, and build org-wide confidence in data-driven decision making
- Identify gaps in tracking, attribution, or event reliability that limit clean experimentation, and advocate for fixes with Engineering and Data
Skills
- 3 to 5+ years of hands-on experimentation, CRO, growth marketing, or analytical marketing experience, preferably at a consumer subscription, marketplace, mobile-first, or dating/social platform. The right wiring matters more than the years
- Ready to run on day one. You can audit an existing test program, identify what is misdesigned, and ship a better-instrumented experiment in week one
- Deep understanding of experimental design: control/treatment setup, sample sizing, statistical significance, power calculations, novelty/seasonality/SRM threats, and how to mitigate them
- Hands-on, in-the-tool expertise with at least one experimentation or A/B testing platform (Optimizely, VWO, Statsig, LaunchDarkly, Eppo, Convert, AB Tasty, or comparable). You build the tests yourself
- Strong analytical fluency. Comfortable in SQL and directly in the warehouse, plus a product analytics tool (Amplitude, Mixpanel, Looker, Heap, or similar) to pull your own analysis and pressure-test Data team outputs
- Proven track record of running experiments across at least two of: paid media, email/lifecycle, landing pages, onboarding, paywall, or in-product marketing, with numbers you can speak to
- Outstanding communication and storytelling. You can write a crisp experiment brief and present nuanced results to a non-technical audience with equal confidence
- Experience with holdout group design and geo-based experiments, and a clear point of view on when each is the right tool
- AI fluency. You already use ChatGPT, Claude, Cursor, Cowork, or similar in your daily workflow to draft test plans, generate variant copy and creative, analyze results, write SQL, and replace work you used to do by hand. You can describe the AI workflows you have built for yourself. If someone has to explain to you why AI matters, this role is not for you
- Operator's discipline. You ship on time, your QA is tight, nothing leaks between channels, and you document every result
- Experience at a consumer subscription, marketplace, or dating/social platform where engagement and retention are primary growth levers
- Familiarity with Bayesian experimentation methods and a clear view on when to apply them vs. frequentist approaches
- Background in behavioral economics or consumer psychology. Understanding why people behave the way they do makes for sharper hypotheses
- Experience building or contributing to a company-wide experimentation program from the ground up, including governance, tooling selection, and team training
- Exposure to multi-armed bandit testing, contextual bandits, or adaptive experimentation
- Experience designing experiments around AI-driven personalization or recommendation systems
- Bachelor's degree in Statistics, Economics, Mathematics, Computer Science, Marketing, or a related field, or an equivalent track record of shipping rigorous experiments
Benefits
- Full-time, exempt, fully remote within the US. Strong preference for candidates open to relocating to Las Vegas, NV.
- Health, dental, vision, 401(k), and a standard benefits package.
- Direct exposure to the Director, CMO, and Co-CEOs from day one.
- Clear growth path to Senior Growth Experimentation Lead within 12 to 18 months, based on outcomes.
Company Overview