I'm one of those touchy feelie types who thinks we need to make sure we look at more than straight metrics. Sure, we have to get "established" door-count equivalent data for our managers (or manager's managers), but what about some of the deeper stuff?
The hard part is finding the balance between all the metrics that gives the "hard data" folks want and the still looks into what I'd call the stuff of real value (ie, what is the reach? how do people feel about the org/site? Are they compelled to act? How do you you measure ROI on a video that goes viral?).
Because the balance point for that value is different for every org (think of feedback elements as being sliders that you can adjust more or less), the "formula" and criteria for success is going to be different for every org, too.
Maybe for one org, you need to weigh heavily the value of having people tweet an idea that benefits your mission? What about if they talk about it in a bar? How do you measure that and what does it look like?
I've seen estimates are that only 20 percent of what happens via twitter goes through the twitter site. So even from a pure metrics angle, you'd be missing something.
Another angle... you may not be able to totally measure ROI for any of these things to the satisfaction of the people in charge. But what about showing them the cost of NOT doing the program or social media campaign or whatever? Are they willing to be absent from the conversation? Are they prepared to lose out on that value just because you can't put a hard number on it?
Sorry this is mostly questions, but sometimes that's the problem with the interesting topics!
BTW, I've really been getting into podcasts from SXSW lately. Here are a few that are at least tangentially related to assessment:
- Stats Guru Nate Silver being interviewed by Stephen Baker:
http://sxsw.com/node/1666 - Social Media Non-profit ROI Poetry Slam:
http://sxsw.com/node/1510 (silly format, interesting info)
- Future of Social Networks:
http://sxsw.com/node/1500 (social networks will be like air/water --everywhere -- what impact will this have on business models?
- Old Man Nielsen vs. new Market Research
http://sxsw.com/node/1660 (mostly user-research/testing)-
