Is "Measurement" Social Media's Bailout?

12/16/09

Permalink 07:22:10 am, by jfiore English (CA)
Categories: Opinion

Is "Measurement" Social Media's Bailout?

This idea that social media tangibles need to be teased out using formulas familiar to business is nothing more than a knee-jerk reaction that does us all a disservice. We need to accept the fact that determining influence in social media cannot

be justifiably accurate in a one-size-fits all way. I'm going to draw on a few examples to highlight these notions.

On Mice and Measurement

Olivier Blanchard's recent post teased out some important themes and named names - you can read it and all 300+ comments here. The controversial aspects of the post stem from Olivier's view on ISMA and their approach to marketing their training. It ended-up stirring controversy in a number of categories which included a debate on certification. The interesting part is that the comments (all 300+) included agreement, detraction (namely with the approach and tone Olivier used in both his post and interaction with detractors) and a wide range of more eclectic voices who felt strongly that there were moral, ethical and academic implications that needed to be drawn into the debate.

This was the part that was most interesting to me because if the original intent of the post was its public service element, it seemed to lose its essence during Olivier's handling of the debates more feverish moments. More importantly, I'm not certain that we could draw out any conclusions that people who have relied on ISMA's training in the past or those considering it would place a lot of value, if any, on this discussion merely because it is so difficult to keep track of the multiple layers, themes, topics, repetitiveness and pausing of the more cogent thoughts of the conversation.

It is in cases like this where a focus on measurement can become a futile exercise of the blind leading the blind, and where measurement is nowhere near important as actually studying the external perception of this debate. This means aggregating every blogroll instance, tweet and mention to form an opinion on which bits resonated most with online audiences. More importantly, that if we use this incident to gauge audience perception toward ISMA, statistics will be nothing more than a footnote, and the focus becomes discerning whether the mentions are carrying the posts original intent, or whether its overshadowed by a mirroring of the same muddled and complicated tug-of-war match.

Purposing Social Media in Buckets

What would make the most sense in situations such as this is to compile all the mentions related to Olivier's post and come-up with a total count. Let us call this the bucket. The main purpose here is to determine how much of the noise can be directly related to ISMA, and how to separate it from discussion that is focused more on Olivier's handling or approach.

After we've turned down the noise, it would make sense to draw-up some questions that need answering. If I needed to impart my own opinion on this particular matter, it would come down to the comments and handling all the time. Comments aren't only a proxy for engagement, they are a window into revealing the impact and the way the influence of a conversation sets the tone for way the message gets repackaged in online environments. ISMA seemed to get the "comments are a proxy for engagement" bit as both people who were named chimed-in and made their opinions known.

Their comments were timely, one was more graceful than the other, and the takeaway for me was the Olivier did more harm to his advocacy efforts by taking every opportunity to belittle them. With all due respect to the people participating in that conversation, public service is never about the person, chest thumping, or the need to wave the "I'm Right" sign, and always about the awareness element.

Put the Rulers Away and Ask Questions

The "purpose" bucket prepares us most with the question answering. It also gives us some depth of understanding into what's at the heart of the matter. No Alexa, Technorati or Google rank will provide that insight for you any better than the written words used by folks engaged in the conversation. Next we start by asking some very basic questions. Are these companies being characterized in a manner that is ground-breaking? Is this a new development, or is it something that the companies tried to sweep under the rug? Has the information shocked even those who thought they knew their companies inside-out? Is this a product/service or employee issue?

What kind of questions to ask depends on each clients unique case. This said, we could simply ask what the client hopes to gain from social media involvement and interaction. If it's dollars, then this is where responsibility sets in, and the need to explain there is no handout. Social media will not bail you out of the mess you're already in, whether that be on a financial or reputational scale. The folks who are making money in social media are doing so in an exploratory manner, and the ones succeeding in establishing a positive online reputation and presence are doing so through hard work and a real commitment to engaging and building relationships. These are intangibles where a positive outcome may be identified, but it cannot always be measured or quantified.

Social Influence - Are We There Yet?

What these two starting points (buckets and question asking) accomplish is the kind of understanding that allows us to move to the "now what" stage. This might provide the insights required for devising measurable outcomes, but most importantly, it should begin to answer questions. It isn't always about science or about incidents that can be checked off on a survey or questionnaire.

We also need take a hard look at the standards being espoused and advanced in the current social media monitoring space. Measurement is not about being the thousandth customer using the leading tools or instruments, or about the tendency to follow the products or tools endorsed by most experts. Sometimes it means being the first person to think independent of popular opinion, and through the noise and clutter of espoused "best practices" with clarity and deliver on the promises you make to your client.

The most important point to tie-in here is that our understanding of influence, and our ways of determining the influence of people and incidents where brands are being mentioned is still in a raw form. A lot about the way we place emphasis on the words people use or the image they personify is often absent of the variables we use to determine authenticity in the physical world. More importantly is how kinship in social media continues to sustain outdated social modalities, and along with it is the creeping in of hierarchical, elitist, sexist, racist, ageist thought and opinion which confines and forces us to behave or transform ourselves to fit in. No example proves this better than James Chartrand revealing that he is a she, and the sad commentary that continues to unfold with regard to a bloggers need to mask her identity to overcome the hurdles presented by gender bias in a male dominated world.

It is stories like James C's that frames our understanding of social influence in a way that is reminiscent of the long car drive with an inquisitive toddler and their penchant for asking "are we there yet?" The answer is an obvious and resounding "not yet", and we do have some way to go still, especially with regard for the need to eject the rules and obstacles that can come in the way of establishing our online identities without prejudice. This has obvious connections to our approaches towards establishing the standards by which we measure social media and its application in the business world, and this will always be contingent on widespread acceptance from the crowds rather than commodified and false universality. Until we shake this perception that social media is the "be all" and "end all" to patch-up or mend business woes, we continue to draw in the element we are all trying so hard to shake out.

What say you? As always, your thoughts are most welcome.

Post on Twitter


Site Meter

Comments, Pingbacks:

Comment from: autom [Visitor] Email · http://autom.x.iabc.com
nicely articulated Joseph. you've expressed some thoughts here which confirm some of my own musings.

so in an effort to elaborate (and hopefully not regurgitate) here are some coins bouncing your way:

first, while i see your intent in referencing the 'bail out' notion, i would point out that this premise likely speaks to a certain segment of the broader online social community. i'm not discrediting this notion at all as some are likely of this mindset; however, for social media to be perceived and widely accepted as a sound, viable tool for business, it clearly needs a quantifiable framework, much like how psychology was once regarded as a pseudo-science until of course statistics-based analysis led to plausible theories that somehow proved to explain, predict and assess certain modes of behaviour. but what about Jungian archetypes..how'd he measure that? hmm..

"determining influence in social media cannot be justifiably accurate in a one-size-fits-all way" - agreed. in fact, while influence measurements out there are available in several forms (e.g., measuring targeted initiatives such as product promotion vs measuring clout in thought leadership among peers or within a given industry), what escapes me is *how* each form of measure correlates accurately with each other to provide a realistic assessment of progress/success. if metric X is using a different unit of measure than metric Y to assess the same thing, how does one determine which body of evidence is more reliable? i think perhaps there needs to be a more visible and marked movement to standardize metrics so we are singing the wonders (or horrors) of social media from the same song sheet..perhaps this movement has already started/is starting to happen(?)

measuring the value of a conversation - this is perhaps where the influence meter gets most nebulous. everyone has an opinion on any given subject, be they certified (or certifiable) experts or passers-by voicing visceral reactions. the ruling mantra in social media to evoke and encourage dialogue is but a 'teaser' as you pointed out. so how are we to interpret a 300+ number of comments as being a positive or negative, useful or useless, lead-generating or reputation-breaking metric? well, let's see..how about we go old school and evaluate all comments as if they were answers to an open-ended question in a standard survey. this requires assigning values to each comment, coding each entry and parsing the results in the form of a market research report. why not? this is a tried and true method..why can't we use this approach (perhaps some recently developed apps already do?) oh but wait..what if we've come across a good post and there are only 1 or 2 comments? is that alone a telling indicator of the content originator's social influence? what if the content was not actually meant to be measured?

bit a rambling-fest..sorry..my key takeaway is that the social measurement crowd needs to align themselves which each other and aim for standardization, in the same spirit as other groups like the open source community. what do you think?




PermalinkPermalink 12/16/09 @ 10:15
Comment from: jfiore [Member] Email
Excellent points all the way through Autom!

I particularly feel that jotting down rudimentary models (your example of a comment log) is the way to start. As a point on how our firms assigns tonality, we do factor in the handling of a post (based on positive/negative/neutral commentary) to arrive at an overall score. Some of the more advanced frameworks for determining influence may actually begin to compile more analysis on the people who have participated. Did they post anonymously? Does the link in their sig or their comment reference a discussion on the topic or incident?

The alignment and convergence of models is perhaps the most challenging. A part of me believes this can be accomplished in a way that can be meaningful to more than the people involved with advancing it. Though I also believe that there is significant meaning derived from treating incidents on a case study basis and using the experiences in dealing with different situations as a useful reference point without the slavish adherence to let the ruler dictate.
PermalinkPermalink 12/16/09 @ 10:35
Comment from: Mark W Schaefer [Visitor] Email · http://www.businessesGROW.com/blog
A lot of information here but the Blanchard example is a good illustration of the measurement conundrum. Sure you can put sentiments into "buckets" but a real-live reader probably is not going to wade through 250 comments but may still come up with an overall of idea of winners and losers.

How do they do this? I think a statistical analysis of reader behavior will probably be completed at some piont. My guess is that the original blog and first five comments may get read throughly enought to form an impression. So in a sentiment analysis they should get extra weight. I think the sentiment of any "big name" commenters anywhere in the post would get a little extra credit as well as maybe the last few questions ... just my guess of how a real analysis might be made more meaningful.

Then you have to look at all the spin-off blogs (and comments) of course. This whole thing is a mess isn't it? : )

Glad people smarter than me are figuring this out.
PermalinkPermalink 12/16/09 @ 22:39
Comment from: jfiore [Member] Email
Mark, you bring some excellent thoughts to the discussion, and your ideas on analysis advancements are very much in alignment with mine. Namely this idea of a computational aspect to speed-read through incidents, and highlight or flag posts which have a skewed or disproportionate sentiment scoring that gets determined by some form of cursory language assessment. The usefulness of this kind of machine assistance is further complimented by this idea of discovering all spin-offs or references in blog-land, message boards, Twitter, Facebook, etc.

This aggregated tally or score-keeping could mean off-loading the discovery portion (and using the example of Olivier's post, this could be a significant time-saver) to allow the people part to take over (i.e. question asking, strategy, engagement, etc.).

I've always believed that machines are only as smart as the people using them, so any advancement in the pursuit of measurement must strike that balance of taking the machine portion as far as it can go to let the smarter (people) portion figure things out.
PermalinkPermalink 12/17/09 @ 07:33

Leave a comment:

Your email address will not be displayed on this site.
Your URL will be displayed.

Allowed XHTML tags: <p, ul, ol, li, dl, dt, dd, address, blockquote, ins, del, span, bdo, br, em, strong, dfn, code, samp, kdb, var, cite, abbr, acronym, q, sub, sup, tt, i, b, big, small>
(Line breaks become <br />)
(Set cookies for name, email and url)
(Allow users to contact you through a message form (your email will NOT be displayed.))

RepuTrack™

RepuTrack™ monitors online media from Web sites, blogs, message boards, forums, chatrooms, microblogs, social networking sites, video and images worldwide.

To schedule a free online-demonstration of RepuTrack™, click here.

RepuTrack™ - Online Reputation Monitoring

July 2010
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Search

Categories

Misc

XML Feeds

What is RSS?

Who's Online?

  • Guest Users: 4