Hiring a CRO Agency: The Questions to Ask

Conversion work is unusually easy to sell and unusually hard to verify. Nobody sees a test that was never conclusive, and nobody audits a report saying a variant “trended positive”. So the buyer ends up judging the pitch deck, which is the one thing every agency is good at.

The published advice is not much help either. Search the topic and you will find lists of sixteen or twenty-three questions, which is about twenty more than a first call allows, and none of which tell you what a good answer sounds like. A long list also hides the fact that the questions are not equally important. One of them decides whether the engagement can work at all.

So this article does the opposite. Six questions, ordered so each answer is harder to fake than the last. Hiring a CRO agency is a decision about method, and traffic decides the method before anyone in the room gets a say.

That question is about traffic. IZI Digital Marketing sells conversion and analytics work, so we have an obvious interest in you buying some. We are still starting with the arithmetic that disqualifies many buyers from the standard version of this service, because a programme sold to the wrong site fails slowly and expensively for both sides.

Before the questions themselves, it helps to hear how practitioners describe the engagement they are selling. The discussion below covers the ground most first calls skip.

PART 1 · DIAGNOSE

What You Are Buying When Hiring a CRO Agency

IN BRIEFYou are buying a method for deciding what to change, not a set of changes. Two agencies can ship the same button colour and only one of them can tell you afterwards whether it helped. A serious analytics and CRO engagement sells you the second thing.

Conversion rate optimisation covers at least four different products that all get sold under one name. Knowing which one you are being quoted for is the difference between a fair price and a confusing one.

• Research and diagnosis. Session recordings, funnel analysis, form drop-off, user testing and a ranked written list of where enquiries leak. This part survives whatever gets built afterwards.
• Design and copy changes. New layouts, rewritten offers, simplified forms. Valuable, but this is design work with a conversion brief attached, not experimentation.
• Experimentation. Controlled tests establishing whether a change caused the difference. The expensive one, with a traffic prerequisite most Malaysian sites do not meet.
• Measurement infrastructure. Event tracking, key events, dashboards. Without it nothing above can be judged, which is why it usually goes first.

When someone quotes you for “CRO”, they are quoting for some blend of those four. The blend is the deal, and it is almost never spelled out in the proposal.

There is a reason they get bundled. Research alone is a hard sell, because a document is not a result. Bundling lets an agency quote a monthly figure without committing to which component gets the hours. So when you are hiring a CRO agency, the first job is not negotiation. It is unbundling.

BENCHMARK BRIEFING 1 OF 4

How Long a Test Takes at Malaysian SME Traffic

IN BRIEFAt a two per cent conversion rate, detecting a twenty per cent relative lift takes roughly 39,000 visitors across both variants. A site with 5,000 visitors a month needs about eight months for one such test. That single figure decides whether optimisation is worth funding at your traffic level.

The model below uses the parameters Adobe recommends as standard, 95 per cent confidence and 80 per cent power, at a two per cent baseline conversion rate.

Two consequences follow, and both matter when hiring a CRO agency. The smaller the site, the larger the lift has to be before anyone can prove it happened. And adding variants does not speed this up: Adobe notes that with five experiences, the chance of at least one false positive reaches 18.5 per cent even at 95 per cent confidence.

PART 2 · DIAGNOSE

The Traffic Question Most Pitches Avoid

IN BRIEFLow traffic does not make conversion work pointless. It makes split testing the wrong tool. The honest alternatives are research, rebuild and before-and-after measurement, all of which depend on an analytics setup that measures enquiries correctly in the first place.

Most Malaysian SME sites sit in the top two rows of that table. A dental clinic in Puchong, a B2B supplier in Shah Alam, a professional services firm in Bangsar — 3,000 to 8,000 monthly visitors is healthy for these businesses, and a twenty per cent lift is worth real money but takes most of a year to prove by testing.

An agency that promises a testing programme to that site will do one of three things. It will stop tests early, which Adobe lists among the most common pitfalls in testing and a reliable way to manufacture false positives. It will pile on variants to fill the calendar, multiplying the false positives along with the comparisons. Or it will quietly stop testing and ship design changes while the invoice still says experimentation.

None of that is necessarily dishonest. It is what happens when a service designed for high-traffic e-commerce gets sold to a business with a contact form and 4,000 monthly visitors. Low traffic has its own honest playbook, and it is not a lesser one.

• Qualitative research over statistical proof. Ten recorded sessions and five customer phone calls will tell you more about a low-traffic site than a test that will not conclude until next year.
• Whole-page rebuilds, not isolated tweaks. If you cannot detect a twenty per cent change, do not spend the year chasing one. Rebuild the page around the problem the research found and take the bigger swing.
• Before-and-after measurement, honestly caveated. Compare like periods, note the seasonality, and say plainly that the comparison shows correlation rather than proof. That sentence is the mark of a serious partner.
• Leading indicators that move faster than sales. Form starts, form completions, scroll depth to the offer, WhatsApp taps. These accumulate volume quicker than enquiries and give you a read within weeks.

BENCHMARK BRIEFING 2 OF 4

What CRO Engagements Cost in the Malaysian Market

IN BRIEFFour engagement shapes dominate the local market, and their first-year totals differ by more than ten times. The cheapest is never the audit that sits in a drawer. It is the audit whose findings get built.

Ranges are indicative of local quotes, not a quotation from anyone. Bars show relative first-year commitment.

PART 3 · DESIGN

Six Questions to Ask on the First Call

IN BRIEFSix questions, asked in this order, will separate a research-led firm from a template one inside forty minutes. Use the same six on every agency on your CRO shortlist so the answers stay comparable.

How to vet a CRO agency in six questions

Ask them in sequence: each narrows what the next answer can plausibly be.

1. At my traffic, what counts as evidence? The answer should cite your actual visitor and conversion numbers within a minute. If it does not, nobody has opened your analytics, and what follows is a template.
2. What will you look at before proposing anything? Strong answers name inputs: session recordings, form field drop-off, search terms, sales call notes, the enquiry inbox. Weak answers name outputs, usually a redesign decided before the call.
3. Which single number are we moving, and how is it measured today? This exposes measurement gaps fast. If your key events are wrong, month one is analytics repair, and the proposal should say so.
4. What happens when a test loses, and who decides what ships? Losing variants are the normal case. An agency with no decision rule for a flat result will present flat results as wins.
5. Who implements, and how do changes reach my site? Client-side scripts in a testing tool are quick but can flicker and slow pages. Developer-built changes are cleaner but need your development queue. Either is fine; not knowing which you are buying is not.
6. What do I own when this ends? Research documents, test records, tracking configuration and account ownership should all be yours. Get the handover format in writing before you sign.

When hiring a CRO agency, the tell is how quickly each answer moves from your business to their process. A firm that has looked at your site keeps referring back to it.

BENCHMARK BRIEFING 3 OF 4

What a Credible First Twelve Months Looks Like

IN BRIEFQuarter one produces almost no conversion gain, and a proposal that promises otherwise is describing a different business. The useful question is what you should be shown at each quarter, so a stalling engagement becomes visible in month four rather than month eleven.

The timeline assumes a site around 20,000 monthly visitors, where limited testing becomes viable.

PART 4 · DEPLOY

Which Engagement Shape Should You Sign?

IN BRIEFTraffic decides the method, and the method decides the contract. Most Malaysian SMEs should sign an audit plus a build sprint rather than a testing retainer, and should revisit the retainer question only once traffic clears roughly 15,000 a month.

Whichever you sign, put three things in the contract rather than the kick-off email. Name the research inputs required before any recommendation, so a redesign cannot arrive in week two with no diagnosis behind it. Fix the reporting format and date, so a quiet quarter is visible immediately. Specify what transfers to you at the end, by name.

One smaller clause is worth the argument too: tools and analytics properties get created under your accounts, with the agency added as a user. Recovering a testing tool from a former supplier is a miserable month.

BENCHMARK BRIEFING 4 OF 4

Reading the Answers You Get Back

IN BRIEFEach of the six questions has a recognisable strong answer and a recognisable evasion. The same reading applies when you vet an SEO agency, where the evasions rhyme almost exactly.

PART 5 · DRIVE

Judging the Engagement at Ninety Days

IN BRIEFNinety days is too early to judge results and exactly right to judge method. Four checks tell you whether the engagement is working, and none of them require the conversion rate to have moved yet. Run them against the measurement baseline agreed at the start.

Most clients who regret hiring a CRO agency could have seen it coming in month three. They waited for results instead of inspecting the process.

• A written diagnosis you disagree with in places. A diagnosis you nod along to entirely is usually generic. Specific findings provoke argument, because they touch decisions someone made on purpose.
• A decision that changed because of evidence. Something on the backlog should have moved up or been dropped. A backlog nobody has reordered in three months is a list, not a programme.
• Reporting that talks about enquiries. If month three still runs on sessions, bounce rate and click-through, the measurement work was skipped and nothing downstream can be trusted.
• Something actually shipped. Research that never becomes a live change is the most common failure here, and it is nearly always a queue problem rather than a skill problem.

FAQ

Frequently Asked Questions

1. What should I ask a CRO agency before signing?

Start with what counts as evidence at your traffic level, because that answer decides whether the rest of the proposal is achievable. Follow it with their research inputs, the single metric being moved, their rule for losing tests, who implements changes, and what transfers to you at the end.

2. How much does a CRO agency cost in Malaysia?

A one-off diagnostic audit typically sits in the low thousands of ringgit, while a continuous testing retainer runs into the tens of thousands across a year. The spread is scope, not quality. Match the shape to your traffic first, then compare prices only between agencies quoting the same shape.

3. How long before a CRO agency shows results?

Expect the first quarter to produce a diagnosis rather than a conversion gain. Real movement usually appears from month four, once the first changes have been live long enough to read. A proposal promising measurable lift in six weeks is either counting a metric that is not enquiries, or reading a test before it finished.

4. Can a CRO agency help if my site has low traffic?

Yes, but not through split testing. Below roughly 15,000 monthly visitors, the useful work is qualitative research, rebuilding the pages where enquiries visibly leak, and measuring before and after with seasonality acknowledged. Buy that rather than a testing retainer that cannot reach a conclusive result.

5. Should I hire a CRO agency or a web designer?

A designer changes how the page looks; a conversion specialist decides what to change and how you will know it worked. If you already know what is broken, hire the designer and save the difference. If nobody knows where enquiries are lost, the diagnosis is the thing worth paying for.

THE VERDICT

The Decision in One Paragraph

Hiring a CRO agency comes down to matching method to traffic, then holding the agency to that method. Work out which row of Briefing 1 you sit in, sign the engagement shape that fits it, and ask the six questions of every firm on your list so the answers stay comparable. The first question does most of the filtering: an agency that quotes a testing programme without asking your monthly visitor count has told you what you need to know before you spend a ringgit.

Conversion problems are rarely mysterious once someone looks properly. The Baymard Institute’s aggregation of fifty studies puts the average documented cart abandonment rate at 70.22 per cent, and the leading reasons it records are unexpected extra costs, forced account creation and checkouts that run too long. Those are design decisions, not consumer whims.

Carry one last framing into the call. Hiring a CRO agency is not a bet on a supplier being clever. It is a bet on a method surviving the quarter when two tests lost and the third was inconclusive. The six questions all circle the same thing: will this firm still tell me the truth when its own recommendation did not work?