How to measure DM response time: median, p90 and the out-of-hours split

When a team says “we reply in four minutes on average”, one of three things is usually true: they never calculated it, they excluded messages that arrived overnight, or a handful of very fast replies pulled the mean up. Measuring response time is harder than it looks — and done right, it is the most honest number you have about whether automation is earning its keep.

The short answer: define first response time as the gap between the moment a person's message arrives and the first meaningful reply your business sends; use the median and p90 instead of the mean; report working hours and out of hours separately; and do not count the automated greeting as a reply — track it as its own metric.

What exactly are we measuring? Three different “response times”

Three separate things go by the same name, and confusing them causes half the arguments.

If your panel shows a single “response time” number, it is probably measuring the first — and that number does not mean you are fast, only that automation is running.

Picking the right starting moment

The clock should start when the person's message reaches you. Obvious in principle, but three traps in practice:

• Conversation or message? If someone sends five messages in a row, the clock does not restart five times. The start is the first unanswered message in that conversation.
• A closed conversation reopening. The same person writing three weeks later starts a new measurement; it is not added to the old conversation.
• Conversations you started. If you opened it with a template message, there is no “response time” to speak of. Keep those out of the calculation, or your number improves artificially.

Why not the mean?

Response times are almost always right-skewed: most messages are answered in minutes, a few hours later. The mean is dragged around by that tail.

An example: nine of ten messages answered in 2 minutes, one in 10 hours. The mean comes out at 62 minutes — which describes neither the nine nor the one. The median is 2 minutes, and that is the typical experience.

So report two numbers together:

• Median (p50): what a typical customer waits. Set your target here.
• p90: the time within which 90% of messages are answered. This is where you see the worst experience.

A team with a low median and a very high p90 thinks it is fast while forgetting one customer in every ten. That is where the real improvement lives.

Splitting out of hours

Replying at 9am to a message that arrived at 2am produces a seven-hour response time. That number measures your opening hours, not your team. Mixing the two makes two mistakes at once: it makes the team look bad unfairly, and it hides the hours in which you really are slow.

The practical method: tag every message as “in hours” or “out of hours” by arrival time, and report the two groups separately. For the out-of-hours group, ask a different question: “how many messages have piled up by the first hour of the morning, and how many of them still turn into a sale?”

This is also where you can see what automation does overnight. A greeting does not lower response time — but it tells the waiting person when to expect an answer and keeps the conversation from drifting away. To measure that, look at how many out-of-hours conversations continue the next day.

A one-month measurement plan

1. Week 1 — raw data. Change nothing and record: arrival time, first automated message time, first human reply time, channel, outcome.
2. Week 2 — baseline. Compute the median, p90 and the in-hours/out-of-hours split. This is your baseline; do not compare it with anything else.
3. Week 3 — one change. Change exactly one thing: turn on the out-of-hours greeting, or move the three most common questions into automation.
4. Week 4 — measure again. Recompute the same numbers. Is the difference meaningful, or is it inside normal weekly noise?

Changing three things at once means that at the end of week four you will not know which one worked.

Setting a target

Most of the “industry average” numbers floating around have no published study behind them. The most cited measurement of the relationship between response time and conversion was published in Harvard Business Review: James B. Oldroyd and Kristina McElheran sent test enquiries to 2,241 companies and found that firms replying within an hour were roughly seven times more likely to qualify the lead than firms replying later. The study ran on B2B web forms in the US, so it does not transfer one-to-one to DMs — but the direction is clear.

A practical target structure:

• In hours: median under 5 minutes, p90 under 30 minutes.
• Out of hours: automated greeting under 60 seconds, and keep the return time you promised.
• Compare these numbers with your own last month, not with someone else's average.

For channel-size data specific to Turkey, see the Turkey messaging statistics page — every figure there is given with its source.

Four things that break the measurement

• Counting the automated greeting as a reply. Your number drops to seconds and stops meaning anything. Keep it separate.
• Including test conversations. Internal trials improve the average artificially.
• Counting spam and irrelevant messages. Unanswered spam inflates p90.
• Merging channels. Expectations differ between Instagram and WhatsApp; one combined number hides which channel is slow.

Frequently asked questions

Should the automated greeting count towards response time?

No — measure it separately. A greeting tells the customer their message arrived; it does not answer their question. Merging the two produces a response time in seconds while hiding customers who actually waited hours. The right setup tracks two metrics side by side: automated first contact and first human reply.

How do I calculate the median and p90?

Sort a month of response times from smallest to largest. The middle value is the median (with an even number of records, the average of the middle two). The p90 is the value at 90% of the list: with 200 records, the 180th item in the sorted list. In a spreadsheet, MEDIAN and PERCENTILE will do both.

Should I set a target for out-of-hours messages?

Not a response-time target. Set two different ones instead. First, how quickly the automated greeting goes out — it should be seconds. Second, how quickly the queue that built up overnight clears once you open. The second number also tells you how many people you need on the first shift.

How many months of data do I need for a meaningful comparison?

If your volume is low (under 100 conversations a month), a single month is noisy — compare two- or three-month windows. With a few hundred conversations a month, monthly comparison is fine. Either way, flag campaign periods separately: response time during a sale week cannot be compared with a normal week.

Do I need a separate tool for this?

Not at the start. An exported message list and a spreadsheet are enough for the first couple of months. What really matters is writing down which moment you count as the start, so that you keep measuring the same thing even after you change tools.

Where to start

If you do one thing this week: export the last 30 days of messages, split them into in-hours and out-of-hours, and compute the median of each group. You will probably end up with two numbers you have never seen before. That is your baseline.

From there, automation funnel metrics shows you where people drop out of a flow, and the pre-launch checklist is worth a pass before you build your first one. To see these numbers ready-made in your own panel, try Merkuva free for 14 days.