Recomendados
Más contenido relacionado
Destacado
Destacado (20)
Último
Último (20)
Good honest (social performance) measurement
- 1. GOOD HONEST MEASUREMENT The rocky road to imperfection.
- 2. WHOAREYOU?
- 7. WHY ?
- 9. value for $
- 11. Measurement keeps you ACCOUNTABLE
- 14. 80% Is that good?
- 19. PROCESS matters.
- 20. Why? What? How? When? Who? 5 Ws
- 21. WHY do you exist?
- 22. Mobilizingbusinesses to help abolish slavery.
- 23. WHAT do you measure?
- 24. Rate of human trafficking in the Greater Mekong Sub-region.
- 25. HOW WHEN do you measure?
- 26. Third party sentinel surveillance once a year at key locations.
- 27. WHO said it was you?
- 28. Matched control Logical attribution
- 30. go do some GOOD sam.ng@shift360.ch
Notas del editor
- We’ve heard great talks about the mechanics of measurement. This ain’t one of those. In this talk, I focus on the principles of measurement.You will notice I’ve called this the rock road to imperfection because I believe we need to realise we’re far from getting the balance right.If we start with that understanding, and we have a relationship that accepts this, we’re at least pointed in the right direction.
- I come from NZ. I am not Cambodian even though I look Cambodian. I’ve been here for just 5 months, and I love it here. I am also very new to the social enterprise scene. I consider this an advantage.
- Like the rest of you, I’m here to do some good & find ways to contribute to a better world, a better Cambodia. I happen to come from a background where I started and run a number of successful businesses.I’m passionate about tech, design and most of all business as a way of addressing biggest issues of our time.The vehicle I’m using to do good at the moment is shift360. Shift360 is a org looks to address inequalities by bringing different voices to the table – government, civil society, faith, and business. Hence the 360.We try to involve the best groups to tackle key problems. Problem we care most about is healthy sustainable employment without economic marginalization – ie The poor getting poorer.We are looking at ways to support early stage social biz before they become attractive to mainstream investment with an “angels program”. For purposes of this presentation, consider us representatives of impact investors.
- My agenda has three things to talk about.Firstly why we measure things as a clue to understand what we measure and how.Secondly attitudesthat we have, that we need to have. For businesses to be healthy and productive, we need these attitudes. Lastly, good processes that make measurement successful. Process that is broad enough to apply to any organisation in any field.
- I’d like to illustrate with an example from one of my own businesses. I’m on the board and every quarter we get reports to tell us health of the business.When we think measurement we think of this. Graphs, charts, dots, trend lines. These are all important. It takes a lot of work to collect all this data and to report on it. I feel quite bad because now that I’m on the board, I don’t really analyse this data much. There are pages and pages of this. Luckily software now generates this data for us.For me, what I want to know, is a) what behaviour does this drive? 2) what’s the most important data?We found we had far too much data and it caused a bit of analysis paralysis. What we did after this was to distill all this data into 5 of the most important points. I also think the most important thing is the attitude behind collecting and reporting on this data, and the resulting behaviour change that this data drives.
- So we reduced all that data down to this scorecard. This is what I look at. To me, this is a lot better.There’s elegance in this scorecard. This tells us if we were winning or not across 5 different scales.There’s danger in this chart which I’ll talk about later.My point is we can very quickly overcomplicate measurement and make it unusable. Not only that, we create huge burden on data collection that to be honest, stakeholders can sometimes ignore, or at best glance over.So we’ve been talking a lot about private sector so I thought I’d show you a bit of what businesses actually do.
- A recapon why we measure.In private sector, it is easy we measure to make money. Social business it’s harder because, yes we make money but also we want to “do good”It is harder because it may be inherently immeasurable to measure “goodness” BUTI think we have a tendency to overdo it and like my previous example – over complicate it, when much better substitutes may exist.
- Here’s why I think we want to measure things. No one wants to feel like a loser (Lance on right). Everyone wants to feel like a winner (Lance on left).We all want to win. It feels good. We respond to incentives and positive emotions are often the best incentives. Now maybe you’re offended that I just reduced “doing good” down to something as selfish as winning. Only those selfish private business types care about winning. Let’s not ignore the ego. Money makes the world go around, but ego is what powers money. To use the analogy above, knowing which race we’re in helps us develop our entire business strategy.
- The other obvious reason to measure is to know we are getting value for money. In biz this is our ultimate – and daily test.Social businesses are subject to the same test – a more rigorous test. Shall I put money in this venture, or that? Which gives me better value? It’s like buying a new TV – which one gives me the best bang for buck? Investors have a limited pot of money also.Even if it’s incredibly unfair to compare, people will still want to compare. The same can be said for you – that you want to lead or work for an organisation that gives you value for your time and energy. If we choose not to measure properly we are saying we don’t value our money or time or energy.
- There are two types of thinking that measurement can create. I feel like I’ve lived in both camps before.Camp 1 is the one I’d liketo think I’m in – an aspirational attitude. Measurement can inspire this kind of response.It gets me out of bed in the morning and inspires me to improve. You’re like an atheletewith a personal best. You need to know how to be better only if you know how good you currently are. This is healthy measurement. It’s a healthy motivation.
- Here’s the other response that measurement drives. Accountability. It’s a good word but it’s also word that can signal (intentionally or not!) - “I don’t think you’re doing a very good job. Or I don’t trust you.” Or, “I want to check up on you.”It’s not always like that, motives are often pure, but we can sometimes see measurement in this negative light.Measurement is to keep me in line. It’s a compliance exercise I must do. It’s a bit like having to the gym being written into your job description. It’s painful but you have to do it. Unlike the gym, you might not even believe it’s good for you.Which camp are you in? If you’re more on one side than the other then everything you see will be coloured by this perspective.If you’re in this camp, it’s going to be harder to have a conversation with you than if you were in the former.
- So let’s look at attitudes essential to effective measurement of social performance.The first and most important is honesty.I love honest people. I’m scared of honest people. Honest people are rare.Honest doesn’t mean blunt. It means a quiet humility & self confidence. It means bravery. Why this philosophical rant? If we lack honesty, we’re all just playing games with each other – a total waste of time and resources.The most important person to be honest with is yourself. Is what you’re doing worth doing? Are you really measuring the right things or the most convenient things? Are you really doing your best? Are you working hard to do good or to look good?As an investor, the more honest we think you are with yourself, and with us, the more likely we will just take whatever metric you give us.
- One thing that I like about this sector is that really, we’re all playing for the same team. Unlike the picture I had before of one person on the podium, I think winning for all of us looks more like this. It’s a team sport that we’re playing.I think this is where honesty gets personal. If we are honest with ourselves and each other, if we can see each other as partners rather than competitors, we can do a lot. The investor – investee relationship is probably where this analogy is most relevant. We really are playing for the same team so there’s no reason we should be dishonest.
- Here’s an example of why honesty is important. When we’re given a stat like this – it’s just a number.Do you know if 80% is good? Of course you’ll ask – 80% of what? Context matters. Context is only good if you’re honest.We can use statistics to make us look good. It’s easy. Measurement devoid of honesty is totally counter productive and a waste of time.
- Back to this example. We’re at 110% YTD profit. Is that good?Yeah it looks great. But what’s the context for that number?If we’re playing on the same team and we have a good relationship then answer you give me might be different to the answer that the metric is inferring.And yes of course there are ways of keeping you honest, keeping you accountable but frankly that takes too much precious energy.Statistics and measurements can do anything you like them to do. They’re only useful if the right attitudes are in place. Measurement and accountability is a distant second for a strong honest relationship.In this case, this is not a great result. We’ve got big expenses coming up and with a different time frame this is just a modest result.
- So the second attitude that you need is to be adaptable. This has been said before. It’s more important to be adaptable than to be right.Don’t let great be the enemy of good.Just do something rather than the perfect thing.As an entrepreneur, you need to be confident, to be self assured – but most of all adaptable. You will pick the wrong metric, you will measure the wrong thing and honesty will tell you when you are wrong. Get used to it. We’re here to solve complex problems. It’s going to happen. You’re also dealing with people, and people are irrational. Donors and even investors are irrational.
- Here’s an example of why you need to be adaptable, like a chameleon. Here’s some excellent research on why and how people give. There are 6 different behavioral types and guess what – they all use different metrics to define winning looks like. That’s not what they might tell you, but underneath they have different levers. What you measure and report will need to adapt to your stakeholders and which one of these quadrants they fall in.
- Last attitude is determination. Don’t you love that steely look of determination? You must not give up easily. Things that are worth doing are hard.Measurement is worth doing. Measurement can be hard.This picture really says it all. You need to look like that.
- We’ve talked about the attitudes you need, but what are the processes you can use?Process is like the recipe. We’ve got some essential ingredients in the attitudes that we’ve already discussed. Now because I’m new to this field, I’m just going to pick a really simple logical process. Lots of smart people have done lots of thinking there. You can spend hours lost on Google looking up social measurement frameworks. Looking at what both Komal from Asia IIX, it looks like they pretty much follow this process also.
- And here’s the process. It’s called the 5W’s. Actually there’s a where but I’ve left that out.This is a common tool to think through a problem and cover your bases. Comes from journalism.We ask these 5 questions of:Why (you exist)What (to measure)How and When (to measure)Who (says it was you?!)Often end up only focusing on the What. But as you can see that’s just one of the Ws. Now a little warning ahead of time. Because this is so simple a framework there’s no reason why you can’t come up with your own measurement framework in the next 10 minutes. So get your pens out. We’ve got some work to do.
- Why questions are always the hardest.First reaction is rattle of long list of reason.It’s incredibly hard to keep this very short and simple. I’m going to use an example from a social enterprise I’m involved in.
- So without knowing anything about the mekong club, I hope this explains to you why we exist.It’s annoying because it doesn’t fully describe everything we do but it’s close because it tells you the most important thing that we do.Notice verb (mobilizing), target (business), and outcome (abolish slavery).Do you know what you do? Can you describe it in a sentence like this? I think it’s very important. I’ll even give you one minute of precious presentation time to formulate this and write it down. Ready?Pick a verb, a target and an outcome. It doesn’t need to be perfect but if you’re brave, I want you to write it down then hold it up, or tell your neighbour. Do it now. You’ve got one minute.
- Next is the what. This is the meat in the sandwich. Like the one sentence you’ve written down, we now need to write down just one thing. If you could just measure one thing and one thing alone – suppose you meet Bill Gates and he asks you what is the one thing you measure – what would you tell him?It has to be the thing that matters the most. It has to be as simple as possible.The clearer your mission the easier this is.Ok so you’re ready, you’re already thinking about it?
- Here’s an example from the MC. We measure labour trafficking rates. Again it’s imperfect as there’s so much context that is missing but this is our kingpin. It speaks to our outcome of abolishing slavery.Okay so now go – write down what that one metric is that you’d measure. Don’t let perfect be the enemy of good.
- Now we’re getting to the how and then when. This is the mechanics of measurement. This is where everything’s different for most people. Perhaps like MFIs there’s a standard.The key here is to do it properly. Understand sampling techniques. You want a baseline. You want the right intervals. Ask the right questions. You really just want to think ahead basically. Make friends with a statistician. Use technology.
- For the Mekong Club it’s something called sentinel survelliance. This is a form of surveying that track the prevalence, severity, trends and changes in human trafficking patterns and flows, both internal and cross-border. Done over time and on key migration paths and border crossings. Luckily for us there’s a tool that we can already use rather than to make one up.I won’t get you to do this for your own business because it’ll take a lot longer than a minute and I’m already running out of time.
- Last but most definitely not least is the Who question. One critical consideration here is to be aware that correlation does not imply causation.Suppose SS showed that trafficking rates were going down, we can’t pat ourselves on the back yet. There might be a correlation between what we’re doing and the reduced rate of trafficking but it might not mean we were responsible.This is tough. This is where we got to be really honest and really determined. Because this is hard, and sometimes prohibitively expensive.Biggest criticism of impact investment and social biz.What we are trying to understand is the counterfactual.What would have happened without you? How can you show that? You already know what you’ve done and if you can find out what would happen without you then the net of that is your impact.==="Correlation does not imply causation. though correlation is necessary for linear causation.In the late 1940s, a nationwide study conducted over several years found a high correlation between the incidence rate of new cases of polio among children in a community, and per capita ice cream consumption in the community. (Equivalently, a simple regression model, using ice cream consumption to predict the rate of occurrence of new polio cases, had a high coefficient of determination.) Fortunately for those of us who like ice cream, a re-examination of the data showed that the high values of both variables occurred in communities where the study collected data in the summertime, and the low values of both occurred in communities where the data was collected during the winter. Polio – which we now know to be a communicable viral infection – spreads more easily when children gather in heterogeneous groups in relatively unsanitary conditions, i.e., it spreads more easily during summer vacation than when the children are in school. The high correlation in no way provided evidence that ice cream consumption causes or promotes polio epidemics.
- So to be honest, we haven’t figured this one out for MC. We’ve got a couple of options, one is to find a similarly matched control elsewhere in the MC. This is close to a randomised control trial but no where near the rigor. So for instance understand the trends in a comparable location with comparable people and situations. Really hard.Another is logical attribution, which really is to say – “look, given these circumstances, it’s really hard to say that it wasn’t us”. The outcome is coupled with the intervention. Time and time again we cant’ find another reason why it’s happening. And we can align that to our strategy to say we did x, and the results suggest y is related to x.
- Okay to close. Once a smart person said to me, it’s all about ownership and not obligation.I’ve got a photo here of a treadmill. Going to the gym can be like obligation. I should. I must.This will only take you so far. That’s why the treadmill is dusty and in parts.But if you own it. If you really want it, you’ll figure it out. Sooner or later – where there’s a will, there’s a way.
- It’s been great to meet so many of you in the last two days. I admire and respect what you’re all doing. We are all playing for the same team so let’s keep this up – more talking and sharing.Now go do some more good.Any of you who want to talk more – here’s my email address.