Building an experiment repository: a knowledge base of what works

Your team runs experiments every month. A/B tests on your landing page, new ad variants, a reworked email flow. But ask yourself one question: could you find out today, within five minutes, what the test from eight months ago delivered? At most B2B companies the answer is no. The results are scattered across slides, Slack messages and the memory of someone who has since left. Valuable knowledge disappears that way, and you unknowingly run the same tests again.

An experiment repository solves that. It is a searchable knowledge base of everything you have tested, what worked and what did not. In this article you will read why such a library makes the difference, what belongs in it and how to build one without it turning into an administrative burden.

What is an experiment repository?

An experiment repository is a central, searchable collection of all your growth experiments and their results. Not separate reports, but one place where every experiment is recorded in the same way: the hypothesis, what you changed, for whom, what came out of it and what you learn from it.

The difference with an ordinary archive sits in the word searchable. A folder full of PDFs is an archive. A repository in which you can filter by page, funnel stage, channel or theme is a knowledge base. You never consult the first one; you use the second one every week.

Think of a question like: “What do we already know about price display on our quote page?” In a good repository you type that in and immediately see the three tests you once ran around it, with their outcome. Without a repository, you start from scratch.

This belongs in a broader system. Experimenting is not a standalone tactic but part of a growth marketing agency approach in which you make SEO, CRO, content and paid work together as one growth engine. The repository is the memory of that engine. If you want to understand the bigger picture, first read our pillar on what growth marketing actually involves.

Why insights disappear without a repository

Without a fixed place to record results, knowledge leaks away in three ways.

People leave. The colleague who ran the tests takes the context with them. New team members start blank and repeat mistakes you thought you had left behind long ago.

Results turn into anecdotes. “That green button worked better, I think.” Without recorded data, every insight becomes a memory that distorts over time. Decisions then lean on gut feeling instead of on evidence.

Tests get run twice. You test something, forget it, and six months later someone proposes exactly the same change. You waste traffic, time and budget on a question you had already answered.

The painful part is that this loss is invisible. You never directly notice that you have lost an insight. You only feel, over the years, that your team learns more slowly than it should. A repository makes learning cumulative: every experiment builds on the previous one instead of existing beside it.

What you record per experiment

The value of your repository stands or falls with consistency. Record every experiment in the same way and you will be able to filter through it later and spot patterns. Keep the format short enough that nobody skips it. Per experiment you record at least this:

• Hypothesis: what you expected and why. For example: “If we put social proof above the fold, the number of demo requests goes up, because visitors doubt our credibility.”
• What you changed: the concrete adjustment, ideally with a screenshot or link.
• Audience and channel: for whom and where the test ran.
• Duration and volume: how long the test ran and on how much traffic or how many contacts.
• Result: what happened to your key metric, plus whether the difference was meaningful.
• Conclusion: what you learn from it and what the next step is.

Those last two are the most important. Raw numbers without interpretation are hard to reuse. The conclusion is what your future self really wants to read back.

A good format also enforces discipline upfront. Anyone who has to write the hypothesis down in advance thinks more sharply about what they actually want to learn. How you get to such a structured test flow, you can read in our article on the growth experiment process from hypothesis to conclusion.

Tagging is what makes it searchable

Recording is half the work. The other half is structuring it so you can filter later. That is what tags are for. A handy set of tags to start with:

• Page or element: homepage, quote page, checkout, email flow.
• Funnel stage: awareness, consideration, conversion, retention.
• Channel: organic, paid, email, direct.
• Theme: price display, social proof, forms, copy.
• Outcome: winner, loser, no difference.

With these tags your repository becomes an instrument instead of a storage place. You can answer questions like: “Which form changes have ever worked?” or “What do we know about conversion in the consideration stage?” That is exactly the kind of reusable insight that separate reports never give you.

Be strict about your tagging conventions. One field that half the team leaves empty or fills in differently weakens every filter. Agree on which tags are mandatory and keep the list short enough that filling it in does not become a hurdle.

Negative results are worth their weight in gold

The biggest thinking error is keeping only your winners. An experiment that delivered nothing feels like a failure you would rather forget. But that loss is an answer: you now know that this hypothesis does not hold for your audience.

If you do not keep your negative results, someone will propose exactly that change again in six months, full of conviction. You have no evidence to contradict them, so you run the test again. The same months, the same traffic, the same outcome.

An honest repository keeps winners and losers with equal care. That also changes the culture. A failed experiment is no longer a wasted month but a recorded insight. It takes away the fear of testing bold things, and it is exactly those bold tests that produce the biggest leaps.

How to start without over-engineering

You do not need an expensive tool to get started. In fact, starting with software your test volume does not justify is the fastest way to let the project bleed out. Start simple and let the repository grow with your pace.

Step 1: pick one place. A shared spreadsheet or a board in your project tool is enough in the beginning. All that matters is that everyone can access it and that it is the same source.

Step 2: lock down a template. Create one template with the fields above. That way every experiment looks the same and filtering stays possible.

Step 3: fill it in retroactively. Add the last handful of experiments you still remember. That way the repository feels immediately useful instead of empty.

Step 4: make it a habit. Agree that an experiment is only “done” once it is in the repository. No record, no closure. That single agreement keeps the system alive.

Only when you run dozens of experiments per quarter and multiple teams are reading along does it pay off to look at a dedicated tool. Until then, simplicity wins. A repository that gets used beats a perfectly configured one that nobody opens. Wondering whether you should manage your experiments yourself or get support instead? Then read our piece on setting up or outsourcing an experimentation programme.

From archive to growth accelerator

An experiment repository sounds like admin, but in reality it is a lever. It turns separate tests into a growing knowledge system in which every experiment builds on the previous one. Your team learns faster, does not repeat expensive mistakes and makes decisions based on evidence instead of memory.

The difference between companies that experiment and companies that really accelerate sits exactly here: not in the number of tests, but in how well they remember what those tests delivered.

Do you want to build a testing culture in which insights become reusable and growth becomes predictable? Get in touch with us and we will look together at how to make experiments a lasting part of your growth engine.