Lead scoring with your own data: which signals actually predict a close

Lead scoring exists for one very concrete purpose: deciding who your sales team calls on Monday at nine, when there are forty new contacts and time for fifteen calls. It is not there to "understand the customer better" or to fill a section of the monthly report. If the score does not change the order of the call list, it does not exist — even if it is built and there is a nice number from 1 to 100 on every CRM record.

The usual problem is not technical. Almost any company with a reasonably populated CRM has enough data for a useful first model. The problem is that the score gets built on the wrong signals, trained on a history that carries every bias of the sales team, and delivered as a bare number with no explanation — so reps ignore it within three weeks. This guide covers those three things: which signals really matter, how not to inherit the past, and how to get the score into the daily life of sales.

What is lead scoring and what does it solve?

Lead scoring means assigning each contact an estimated probability of becoming a customer (or a qualified opportunity, depending on where you draw the line) based on what you know about them and what they have done. That probability orders the sales work queue.

There are two families and they are worth keeping apart:

• Rule-based scoring. Someone decides that being a CFO adds 20 points, opening three emails adds 10 and a free email address subtracts 15. It is transparent, it can be built in an afternoon, and it is surprisingly competitive at low volumes.
• Model-based scoring. An algorithm learns from your history which combinations of signals ended in a sale. It wins when there is volume (hundreds of closes, not ten) and when interactions between variables matter: a signal that is good in one segment and bad in another.

The practical difference: rules reflect what you believe works; a model reflects what has happened. Neither reflects what should happen, and that distinction is the expensive one.

A score does not tell you who is going to buy. It tells you who deserves the next hour. It is a time-allocation tool, not a crystal ball.

Which signals actually predict a close?

This is where working scores part ways with decorative ones. The general rule: recent intent signals beat profile signals, and your own behavioural data beats anything you bought.

Signals that tend to carry real predictive weight:

• Visits to decision pages. Pricing, comparisons, contact page, case studies. A blog visit is worth little; three pricing visits in five days are worth a lot.
• How fast the lead responds. How long they take to reply to the first email or pick up the phone. One of the most consistent signals there is, and almost nobody records it.
• Fit with the customers who renew. Industry, headcount, whether they run the system you integrate with. Note: fit with who renews, not with who signs.
• Who is asking. Role and decision level. An operations manager asking about a specific process weighs more than a CEO requesting "general information".
• How specific the request is. A form that says "we handle 12,000 invoices a year as PDFs and key them in by hand" is a different animal from "I'd like information".
• Source of the contact. Referral, branded search, generic campaign, content download. Source explains an enormous share of the variance and is usually the first field you already have clean.

Signals that nearly always disappoint: email opens (noisy since mail clients started prefetching them), total visit counts with no page detail, third-party scores whose formula you cannot see, and any purchased demographic data you have not been able to verify.

And one to handle with tongs: lead age. It often comes out as highly predictive, but what it frequently measures is that your team already worked that lead, not that the lead was good. It is the first place leakage shows up.

Historical bias: why the model repeats the past

This is the most expensive failure and the least visible. Your history is not a neutral sample of leads: it is a sample of leads your team chose to spend time on. If for three years nobody called companies under 20 employees, your history will say those companies do not buy. The model will learn that and bury them at the bottom of the queue forever. The loop closes: the prophecy fulfils itself.

What to do about it, without turning it into a research project:

• Separate "never contacted" from "contacted and lost". They are different outcomes, and dumping both into "did not buy" is exactly what creates the bias. If your CRM cannot tell them apart, that is your first fix — before any model.
• Hold back a random share of the queue. Between 5% and 10% of leads get worked without looking at the score. That is your control group and the only uncontaminated data you will ever have. Without it, a year from now you will not be able to tell whether the model is right or merely confirming itself.
• Audit which variables actually mean "the rep already decided". Number of logged calls, whether an opportunity exists, whether there is an estimated close date. Those predict beautifully and are useless: you do not have them at the moment you need to decide. That is leakage, and it makes a model look excellent in testing and mediocre in production.
• Look at performance by segment, not just overall. A model with a good headline number may be doing well in the majority segment and terribly in the small one — which may be the one with the best margin.

The honest test: train on data up to a date, validate on the months after it. If you validate by shuffling periods at random, you are letting the model see the future and the results mean nothing.

How do you build a lead score step by step?

| Step | What you do | Sign you can move on | |---|---|---| | 1. Define the outcome | Decide what counts as success: qualified opportunity, proposal sent or signed contract | Sales and marketing give the same definition without arguing | | 2. Signal inventory | List which fields exist, which are genuinely filled in, and when each is available | No field more than 40% empty makes the list | | 3. Rule-based baseline | Build a simple rule score and measure it | You have a conversion rate per band to compare against | | 4. Model | Train on history, validating forward in time | The model clearly beats the rules in the validation period | | 5. Integration | Push the score into the CRM with its top three reasons | The rep sees it without opening another screen | | 6. Measure and retrain | Control group, monthly review, quarterly retrain | You know the random group's conversion versus the prioritised one |

Step 3 gets skipped almost every time and that is a mistake. Without a rules baseline you cannot answer the question leadership will ask: "and how much better is this than what we were doing?" On top of that, in more cases than people expect, well-built rules hold their own and save you the whole model. We go deeper into that comparison in applied predictive analytics.

On step 2: when a signal becomes available matters as much as the signal itself. If "industry" is filled in by the rep after the first call, you cannot use it to decide who to call. It sounds obvious and it is the most repeated mistake.

How do you get sales to actually use it?

A bare number from 1 to 100 gets ignored. What does get used:

• Bands, not decimals. Three or four categories (A, B, C, discarded) each with an action attached. "87 points" tells nobody anything; "A: call today" does.
• The three reasons behind the score. "Visited pricing twice this week · Industry fits · 80-employee company". That turns the score into a call script and is the difference between adoption and abandonment.
• On the screen where they already work. In the CRM lead view, ordering the list. Not in a separate report or a weekly email. If it requires switching tools, it will not happen. We wrote about this in integrating ERP and CRM.
• A channel to disagree. A "this doesn't fit" button with a reason. It does two things: the rep feels the system listens, and you accumulate the cases where the model fails, which is gold for the next retrain.
• An explicit agreement about the Cs. If nobody calls them, say so. If they go into an automated sequence, build it. The grey zone ("we'll see") is where trust is lost.

One warning: do not publish the score as a leaderboard across reps. The moment scoring is used to evaluate people instead of to allocate work, the input data starts getting gamed and the system dies within two months. It is the same pattern we see in change management.

How do you measure whether the score works?

Two metrics, and neither of them is model accuracy:

• Conversion by band. What share of the As become customers versus the Bs and Cs. If band A does not convert clearly better than band B, the score is not ordering anything. This is the metric leadership understands.
• Lift against the control group. Conversion of model-prioritised leads versus the share worked at random. It is the only honest measure of the value added.

And a third number worth watching: the cost of false negatives. The leads the model sent to the bottom that then bought from someone else. Hard to measure, but if you sell high-ticket, a single false negative per quarter can outweigh all the efficiency you gained. When that is the case, the cut-off threshold should be generous: better to call too many.

A sensible cadence: monthly review of conversion by band, quarterly retrain or whenever something in the business changes (new product, new market, price change). A model trained two years ago on a different catalogue is predicting a world that no longer exists. To set up the tracking without inventing metrics, see choosing KPIs people actually use.

When do you not need a scoring model?

Three situations where building a model is throwing money away:

• Few closes per year. Below roughly a hundred annual closes there is nothing stable to train on. Use rules and spend the effort cleaning the CRM.
• The team calls everyone anyway. If there is spare sales capacity for the whole lead queue, prioritising adds nothing — there is nothing to ration. The problem then is demand generation, not scoring.
• The input data is a mess. If half your leads have no industry, no size and no reliable source, the model will learn noise. Data quality first, model second. Always in that order.

Worth saying plainly too: a good score does not fix a bad sales process. If the real problem is that leads get contacted four days later, ordering them better will not save you. Speed to first contact usually has more impact than any model, and costs far less.

Frequently asked questions

How much data do you need for AI lead scoring?

At minimum, a few hundred known outcomes (won and lost) with their signals recorded as of the moment the lead arrived. With less than that, a well-designed rule score performs just as well or better and takes days instead of weeks to build.

Does it involve personal data, and what does GDPR say?

Scoring business contacts does process personal data, so it needs a lawful basis, information for the data subject and a documented decision about profiling. As long as a human still decides who gets called and the score only orders the queue, the fit is reasonable; if the system discards automatically, the bar rises.

How often should the model be retrained?

Quarterly as a default, and whenever something material changes: catalogue, pricing, market or acquisition channel. What really triggers a retrain is not the calendar — it is a drop in band A conversion.

Can scoring replace rep qualification?

No, and framing it that way is the fast route to abandonment. The score decides the order of the queue; qualification is still a conversation that confirms budget, urgency and decision maker. The model is right on average, the rep is right on the specific case.

If you are weighing up a lead score and are not sure whether your CRM can carry the weight, start by looking at which fields are genuinely filled in and how many comparable closes you have: that alone tells you whether it is a model or a set of rules. That is exactly what we review in the audit, and if you would rather talk it through for half an hour before deciding anything, let's talk.