Often one looks to a blog to provide constructive criticism on a particular topic, perhaps in response to a contribution that someone else has made to the discourse on that topic.  But once in a while something comes along that’s as uncritiquable (is that even a word?) as a Ferrari.  What’s to complain about when it’s that good.  No cup holder?  Well the same is true of an excellent article that the Public Relations Journal has just published,  Penned by measurement mavens David Michaelson and Sandra Macleod both of Echo Research, it’s called “The Application of ‘Best Practices’ in Public Relations Measurement and Evaluation Systems.”  While there’s nothing earth shatteringly new here, it is encouraging to see best practices laid out so clearly and succinctly.  And while this may be old hat to some in the measurement world (at least in theory if not always in practice), it is a MUST READ for the general practitioner and practitioners-to-be. 

If it ain’t broke, don’t fix it.  So, sans critique, I offer up a summary of the nine best practices the authors propose below (I’d provide a link to the article were I not fearful of copywright laws):

In two main categories:  1)  methods and proceedures and 2) quality and substance of findings. 

Methods & Proceedures:

1.  Setting clear and well-defined research objectives

2.  Applying rigorous research design that meets the highest standards of research methods and ensures reliable research results

3.  Providing details supporting documentation with full transparency. 

Quality & Substance of Research Findings

4.  Designing the research to demonstrate the effectiveness of public relations activities

5.  Linking PR outputs to outcomes

6.  Using findings to aid in the development of better communications programs

7.  Demonstrating an impact on business outcomes

8.  Being cost effective

9.  Having applicability to a broad range of PR activities

Representatives from Microsoft and Cymfony are addressing the IPR measurement conference delegation with a presentation entitled: The Next Generation of Enterprise Measurement for Enterprise.  (To be clear they’re talking only about media measurement).  You’d expect a company like Microsoft to have a data rich / savvy / hungry culture.  And they are, clearly.  So when the issue of measuring (editorial and social media coverage anyway…oh, and buzz, too, though I’m not fond of that term) came up, there was a desire to arrive at a single number that represents the quality of the coverage and to track that number over time.  I appreciate the context–they are the most written about organization on the planet–and I appreciate the need to solve what Microsoft calls measurement anarchy–they had multiple editorial measurement systems in place.  However, I worry about the danger in reducing something as complex as PR and scads of media content analysis data to one number.  As a colleague of mine points out, a single score for media coverage (however complex the methodology used to generate it) is a bit like getting a 4.7 from the Russian ice dancing judge.  Wonderful, but what does that mean?  The alternative?  More data and more analysis, and using the ice dancing analogy, akin to sitting down with your coach and reviewing a video of your performance and discussing it in great detail.  The whole point of measurement and the data it yields is to provide extra depth and insight to communications.  Now, certainly not all organizations are created equal and not all organiztions will want to (or should) measure alike, so whatever works for you and whatever floats your boat, I suppose.  But a little information is a dangerous thing. 

In fairness to Microsoft and Cymfony, we haven’t and aren’t likely to see behind the methodological curtain.  Still, what’s not clear, and this is my hope for Microsoft, is whether or not this single number can be reversed or deconstructed to go back to more rich data that may lie behind it for more in-depth analysis.  Also, what's not yet happening, Microsoft and Cymfony admit, are attempts to link this score to other market, brand, employee and reputation research.

I thought, perhaps, some PRoponent / PRomulgator readers might be interested to hear that Doug Kells, former Director of Research at Cormex, (a leading Canadian media content analysis and measurement firm) is now a free agent and looking for a measurement gig.   

I have a great deal of respect for Doug.  We first crossed paths in a client-vendor context.  Five years back I was at Bell Canada and Doug was the lead analyst on our account.  We’d later work together for a time at Cormex. 

Doug’s background / skill set is fairly rare, generally, but even within the measurement crowd:

* Four years as an analyst at Bowdens (was later bought by the Observer Group-turned-CISION)

*Nine years at Cormex Research where he played a sort of hybrid role that has seen him sharpen both media content analysis skills but, perhaps more importantly, all the methodological, number crunchy, SPSS syntactical and even IT stuff that one needs in a measurement firm to take simple clips from the measurement by thud factor approach to providing methodologically sound and rigorous analysis.  There aren’t many of those in Canada. 

Doug will make a valuable contribution to the North American measurement scene, regardless of where he ends up.  I wish him luck and I ask that if anyone on the reading end of this might like to see Doug’s resume and/or talk with him, please let me know.  I’d be happy to forward his CV and he can be reached here.      

For those that haven't already been made aware, the 'other' measurement event is coming up in mid November.  It's sponsored by Cision and put on in partnership with IABC. 

If any of you are like me, travel and events budgets can mean making tough choices about which (and how many) conferences to attend.  Why do I characterize this as the 'other' event?  While this event looks interesting--some great speakers and some key topics--it looks to be the Chicago (Second City) to the IPR Measurement Summit's New York.  That doesn't mean it's not valuable, (Chicago IS a wonderful, world-class city afterall) it just means, IMHO, that to choose one might mean choosing in favour of the IPR Measurement Summit.  And, the audiences are, generally, different.  Huge over-generalization, but, I suspect that the IPR Measurement Summit is more about the measurementarati talking to itself (which is important) and the Cision - IABC event is targeted more at the communications practitioner writ large. 

I was thankful and honoured when PR Conversations invited this Measurement PRoponent / PRomulgator to opine on the current state of the art (or science?) of measurement in Canada. While it’s a topic that could easily spawn a healthy hundred-plus-page graduate thesis, the post is a reasonably succinct, high-level, non-exhaustive horizon scan of the measurement scene in Canada, broken up into bite-sized morsels to cover the topics of: industry associations; Canadian vendors; U.S vendor spillover; the client context and the spectrum of “outs”; standards, patterns ‘n packages; the agency sphere; homegrown, thought leadership on measurement writ large; Canadian measurement events and conferences; education; awards; the wrap; and the ask.  Here's the post and here's hoping it spurrs a number of similar states of the art-type posts and comments from other parts of the world. 

It seems event organizers always relegate measurement to the end of the agenda.  Though it's sometimes tricky to speak to an audience who's zipping up their bags and rolling out the door to catch a flight, at least it's encouraging to see so many conferences that are not exclusively focused on measurement including a segment on the topic.   

Case in point: The Canadian Institute's Government Communications event to be held in Ottawa on November 26 and 27, during which I'll be steering the SS Measurement for a three and a half hour deep dive.  So if anyone out thee wants to come aboard, book your ticket on the soon-to-work www.canadianinstitute.com/govcom site. 

I'll carve the time into two sections:

1.  The framework or strategy of measurement

2.  Specific tools & tactics

Paul Argenti, professor at the Tuck School of Business Research, co-founder and chief advisor for Communications Consulting Worldwide (Fleishman's research and measurement consulting arm) and leading measurement thinker, was commissioned by Factiva to publish a white paper entitled Best Practices in Media Measurement

It's an interesting read.  Though it's very well written, presents a compelling case and, rightly, places PR measurement in a broader business measurement imperative and context, the main thesis of the paper is by no means radical or new:  that "establishing causation" between communications efforts and tangible business outgrowths is critical.  The paper presents establishing causation as the future of communications measurement.  I'd argue that we're already there and that we have been for years.  Those in the communications industry would be hard pressed to have missed umpteen articles in various trade pubs, conference speeches, workshops and webinars pointing to marketing mix modelling used at Miller Brewing and Proctor and Gamble, among others, I'm sure.  Though leading academics like Don Stacks would be, perhaps rightly, skeptical, the advances in technology and statistical methods (in one form or another) that Argenti refers to in the paper have been around, as Mark Weiner points out, for some time. 

That said, the idea that technology is both making this type of measurement easier and more imperative does have merit.   

 

There's a tendency out there to want to (over)simplify measurement solutions into patterns and packages such that a client of a certain size with a certain need could be slotted into a bronze, silver, gold or platinum measurement package. 

The easy answer ain't always the right one in measurement any more than it is in building one communications plan to suit all client needs (not to mention their reporting structure, operating environment, management style, communications department culture, PR practitioners' roles, their research orientations and the prevailing model of PR resident in the organization).   

I suspect I'm preaching to the converted, but, in my view, each measurement program MUST be built to meet specific needs. 

Adhere to an indstry standard?  Heck no.  There isn't one and nor should there ever be.  

Regard industry best practices (such as those that come out of the Institute for PR's Measurement Commission)? 

Absolutely. 

I feel like this is a bit of an uphill battle sometimes, but it is something that we must continue to educate key stakeholders on.