222: “In Support, I Believe…” with Ryan Klausner

Charlotte Ward: 0:13

Hello and welcome to episode two hundred and twenty-one of the Customer Support Leaders Podcast. I'm Charlotte Ward. Today we listen to some support beliefs. My name is Charlotte Ward. I am a customer support leader, the host of this very podcast, and in support I believe that metrics should measure but not rule. In customer support, metrics are something of an obsession at all levels. They are a necessary tool for understanding and informing strategy if you're a support leader. But I believe fundamentally that most organizations fail to put them to good use. They're often used to measure the effectiveness of staff. Are your agents responding fast enough? Are they closing out cases quickly enough? Do they did they close enough cases last month? What about that CSAT score or God forbid NPS? Is that above an acceptable minimum either for an agent or for the organization? And if the answer to any of these questions is no, it's not on target, it's not high enough, it's not quick enough, it's not good enough. What does that single data point really tell you? What does that really mean? Focusing solely on the metric or the metrics is where the process is really in danger of breaking down. We have to be really careful with how we handle and read metrics in support. They can be our friend or our foe. They can measure our effectiveness, they can identify points for improvement, they can guide us towards paths of increasing success. But they can trip us up. We can focus too much on them. We can lose ourselves in the data. And goodness me, we can fall foul of Goodhart's law. I know if you've listened to this podcast before, you'll be familiar with my obsession with the law that was laid out in the seventies by a British economist. The law that says any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. Rephrased by Marilyn Strath and later, some twenty odd years later, as when a measure becomes a target, it ceases to be a good measure. And actually, when a measure becomes a target, it's even more dangerous if the data points are low, it's even more dangerous if we focus on one thing and not a set of metrics and how they relate to each other and the wider story. In focusing on one measure, one metric, it is almost certain to cease to be a good measure of our effectiveness. It will very quickly become a foe. Take CSAT, for instance, all those surveys sent anywhere between an hour or a week after a support ticket was closed. Well, they're of questionable use. Done right, they can provide you with valuable, quantitative, and actionable information. Done badly, they can provide you with irrelevant or even misleading data that gives you no insights into any changes you need to make. For instance, as a customer, when a survey lands in my inbox, it's not always clear what they're asking. Even as a support professional, I can recognize MPS as the classic how likely am I to recommend question? But as a layperson, the question might be a little confusing. Most people don't realize the MPS question is seeking information about the organization, not the service you just received. And where does that confusion take a human person to make a best guess at what they ought to respond or not respond at all? Questionable data. And the fact is that again, even as a customer, that numbers don't always fit. Have you ever had that nagging feeling that you wish the scale was out of ten instead of out of five? Or out of a hundred instead of out of ten? I have. Maybe I'm a little exact, but the inexactitude of some scales frustrates me, and so I undermark. I'd rather give an organization nine out of ten than ten out of ten. If there was something tiny but irrit irritating in the experience, out of a hundred they might well have got a ninety-eight. There is just so much subjectivity and nuance in data that we fail to see if we obsess about the numbers alone and not about the story. A year and a half into this pandemic, we're all used to seeing those graphs, we're all used to the questions the journalists pose to leaders around the world. What does this graph really mean? What does this data point mean today? What does it mean compared to yesterday? And actually, in a world where everyone is an amateur data scientist, it's the bigger story that's really more important. The the questions that journalist after journalist parade out over Zoom to Downing Street or the White House or any other government or institution or scientific body around the world are always the same. Over the last year, how many times have we heard are we flattening the curve? Is this number less than yesterday? And the fact is that I think that the same questions are asked because there is a problem between the connection of the presentation of the data and the narrative that the data gives with the numbers themselves. They feel disconnected. The speech overlaid over these graphs somehow only works to obfuscate the real facts and really hide the wider story. And intentional or not, I don't believe most people are aware of the issues with single data points. And this for me is the problem with data in support. In support we need data. We have to operationalise around data. We have to want to fight a better fight and a better informed fight tomorrow. We want to serve our customers better and more efficiently and in a more informed way tomorrow. But for that to happen, we ain't we need to worry less about the definition of every single tiny data point and look at the bigger meaning of the metrics over time. We definitely need to ask the right questions of our data. And fundamentally, this is what I believe. In support, metrics should measure and not rule. That's it for today. Go to