Understanding Research – Political Polls and Their Context

Date: October 26, 2012 | IQS Research | News | Comments Off on Understanding Research – Political Polls and Their Context



Yesterday, the President of IQS Research Shawn Herbig spent an hour on the radio discussing some of the intricacies involved in the research and polling process.  Given the current election season, one thing we know for certain is that there is no shortage of polling results being released.

So that begs the question, how do we know which polls are right and which are not?  Is each new poll released on a daily basis reflecting real changes in how we think about the candidates?  Is polling and research indicative of emotions or behaviors, or both?  These are some the things Herbig tackled yesterday.

We posted a discussion late last year about how it may be a good idea to look at what are called polls of  polls, which take into consideration the summation of research done on a particular topic (in this case, political polling).  This will help to “weed out” fluff polls that may not be very accurate, and to place a heavier emphasis on the trend rather than specific points in time.

But beyond this, understanding the the  methodology behind polls is useful when deciding whether or not those results are reliable.  A few things to note:

1. What is the sample size? – Political polls in particular are attempting to gauge what an entire country of over 200 million registered voters think about an election.  A sample size needs to be 385 to be representative of a population of 200 million.  But oftentimes you see polls with around 1,000 respondents.  Oversampling allows researchers to make cuts in the data (say, what women think , or what what African Americans think) and still maintain a comfortable confidence level in the results.

2. How was the sample collected? – Polls on the internet, or ones that are done on media websites, aren’t too trustworthy.  They attract a particular group of respondents, thus skewing the results one way or another.  Scientific research maintains that a sample must be collected randomly in order for those results to be Representative in a population.  In other words, each person selected for a political poll, for instance, must have an equal chance to be selected as any other person in the population.

3.  Understand the context of poll/research – When the poll was taken is crucial in understanding what it is telling us.  For instance, there was a lot of polling done after each one of the presidential debates.  Not only did researchers ask who won the debate, but they also asked who those being polled were going to vote for.  After the first debate (which we could argue went in Romney’s favor), most polls showed the lead Obama had going into the debate had vanished.  Several polls showed Romney with a sizable lead.  But was this a statistical push due to the recent debate and the emotion surrounding it? Or was this increase real?

Recent polls show a leveling between the two candidates now that the debates are over, and a more objective look at the candidates can be achieved.  However, it is nearly impossible to eliminate emotion in responses, especially in a context as controversial a politics.

4. Interpreting Results – Interpretation ties in nicely with understanding the context of the research that you are viewing.  But there is a task for each of us as we interpret, and that is to leave behind our preconceived notions about the results.  This is very hard to do, as it is a natural human instinct to believe what justifies our own reasoning.  This is know as Confirmation Bias, and it can impact the way we accept or discount the research.

Taking all this into account can help us to sift through the commotion and find the value of the research being produced.  This isn’t just for political polling, but can be used for all research that you encounter.  Being good consumers of research can take a lot of effort, but it is the only way to gain a more realistic view of the world around you.

view all

Keeping Those Goals Top-of-Mind

Date: August 23, 2012 | Joshua Holt | News | Comments Off on Keeping Those Goals Top-of-Mind

We recently released a special report that discussed the lack of preparedness many students face as they enter the collegiate experience.  It’s been getting some nice attention at sites like US News & World Report, Education Week, and Pearson Education.  A news affiliate of NPR also ran a story on the subject.

Surely, the topic of college readiness is a very pertinent one in today’s discourse, as enrollment rates are continuing to increase, yet drop outs are also on the rise.  This relates to a matter that we all face, whether we are enrolling in college or not – and that is goal setting.

A recent study that will appear in the Journal of Organizational Behavior makes a claim that sheds some light on why many of us fail to accomplish the things we set out to do.  This can relate to a student who has to drop out of college his or her first year because they cannot handle the pressures of the college atmosphere.  Or it can relate to the common businessperson as he or she attempts to develop and accomplish goals for the future.  The researchers argue that “people in contemporary economies seem to know that they should ‘think long term,’ when in fact they base their choices and behaviors primarily…on short-term considerations.”

The meaning of this statement is simple enough to grasp – we are hypocritical when it comes to our goal setting.  We all realize that long term planning is necessary, but we don’t take the necessary steps to get there.  Instead, we perform as though we are accomplishing many small tasks for the near term future.  While many correctly make the claim that goals are best handled when we develop “action steps” to accomplish them, we nevertheless lose our way in many cases toward the ultimate end.  So maybe this is not so much hypocritical, but rather we just don’t have a good knowledge on how to effectively set goals.

Students don’t succeed, in short, because they are not preparing for the long haul of college education – they don’t know how to.  They take it “day by day,” and despite what many will tell you, this is certainly not an effective way to plan for the future.  In the business world, it is important to create efficient and sustainable action plans that work toward a single end.

Of course, these researchers have some good tips on how to stay motivated toward those long term goals – 8 of them to be precise.  They were developed from in-depth interviews of professionals across a wide array of fields, which the researchers then condensed into a common pattern of eight key sources that benefit the psychological underpinnings of how we go about pursing our goals:

1. Allegory – it benefits us to compare our own goals to great accomplishments of  the past, like landing on the moon or being the first to fly across the Atlantic.  This provides some grandeur and takes the “ordinary” out of it.

2. Futurity – think of the future benefits accomplishing the goal will have.  How will it impact your own grandchildren one day?

3. Self – Set the goal in such a way that invokes your own personal identity and reputation.  How will it affect you personally?

4. Singularity – Maybe this goal is unique.  Even if it is not, it helps to frame it that way so that you can have some novelty to associate with it.  Perhaps no one in the world has set out to do this.  It may not be true, but it sure does make it more exciting and helps to build that motivation.

5. Knowledge – What kind of new knowledge will be created from accomplishing this goal?  Be of the mindset that any knowledge that’s created is good.

6. The Work – Embrace the challenges of the goal.  Kind of like the mantra, “No pain, no gain,” keep the end in mind and know that the hard work put into the goal will pay dividends in the end.

7. Embeddedness – Prove your skeptics wrong.  Use any negativity that you have encountered, such as naysayers, to further motivate you.  This will help gain social legitimacy for the goal in your own eyes.  All the hard work is “not for nothing.”

8. Progress – Consistently measure the progress you have made toward your goal.  Thinking in a way that emphasizes forward movement in the direction of the long term goal can help.  This is where a series of short-term goals to help accomplish the final end can be beneficial, so long as they are relevant to the long-term goal.

As you may notice, these various motivators tend to start off “big-picture” and narrow to more specific measurements of success.  Keep this in mind as you begin your journey to long-term goal pursuits.

view all

Why Pay Is ALWAYS Too Low And Why It Doesn’t Matter

Date: July 31, 2012 | Shawn Herbig | News | Comments Off on Why Pay Is ALWAYS Too Low And Why It Doesn’t Matter

Your employee engagement surveys have been completed, and the research shows that your employees are dissatisfied with their compensation levels. Should you be worried?

The good news: You’re not alone. Employees are never satisfied with what they’re paid.

The better news: It (typically) doesn’t matter.

If your company compensates its workforce at a rate that’s competitive within both your industry and region, research consistently shows that increasing compensation does little to improve employee satisfaction or performance.

While responses vary from position to position (for example, finance personnel tend to be more realistic when gauging relative compensation levels than line workers), employees usually think they should be paid more than they actually are. Thus, the majority respond negatively when surveyed about their salaries or hourly wages.

Obviously, if an employer chooses to pay below market rates, it risks increased turnover. Good workers will leave, while poor workers stay on. Plus, the talented job candidates needed to spur company growth will likely accept offers from competitors who pay better.

There are companies who choose to pay at the low end of industry pay ranges. These firms typically invest very little in employee training which offsets the cost of high turnover rates. The long-term effectiveness of this approach is debatable.

The goal is to accurately identify industry pay ranges, and set compensation levels within that range. The recommended approach is to research trade association data, instead of web-based data which often doesn’t consider regional and positional factors (such as the cost-of-living and shortages of qualified candidates), or to work with a research company like IQS Research, which often performs this type of industry research for its clients.

An exception to this approach is if your company demands a higher level of commitment from its salaried employees. For example, if your staff is expected to work a minimum of 60 hours per week while staff in comparable firms work 45 hours, salaries should be adjusted upward to reflect the increased workload.

The bottom line is that your compensation packages should be commensurate with the performance you require of each position, and competitive within your industry and region. In short, it should be at a level that attracts and retains the talent your firm needs (no less and no more). But that still doesn’t mean that your employees will give you high marks when they complete your employee engagement survey.

view all

What Can Research Really Tell You?

Date: July 26, 2012 | Shawn Herbig | News | Comments Off on What Can Research Really Tell You?

Many people don’t believe that marketing research can accurately predict everything about everybody. That’s not only true, it’s very rational.

There’s no disputing the fact that research can’t determine every detail about each individual’s behavior. But, it’s also true that 80% of human behavior is very predictable (including the fact that humans aren’t always rational).

For example, it’s impossible to predict whether or not you’ll be run over a bus as you leave the coffee shop. But, the likelihood of you getting into your car instead of your bike (after you dodge that bus), turning right at the next intersection, and then exceeding the speed limit, is extremely predictable.

That’s because well-crafted research identifies how you (and people like you) think when you make decisions. And, this series of thought patterns reveal with great accuracy how you’ll respond to the various choices you confront in your daily life.

So, is it research or Sherlock Holmes-like deductive reasoning that provides insight into your individual behavior? And, should you be uncomfortable with the precision with which these predictions can be made?

The good news is that “big brother” does not have nefarious intentions for you, or any individual. You can remain happily anonymous as profiles are developed on the shopping patterns of people who purchase the same kinds of merchandise you do.

Marketing research enables retailers to compile data on the preferred shopping experiences of specific demographic groups, then send targeted mailers to those groups which include coupons on the items they desire.

These retailers aren’t interested in who you are — they’re interested in what you want. This data enables them to provide you and your fellow consumers with timely information on how the retailer’s offerings meet your group’s needs.

Granted, the more information they gather on the purchases you make, the more accurate your individual profile becomes (and the more accurate the “Holmesian deductions” about your preferences become).

But, the end result is that the experience you receive from these marketing savvy companies become increasingly reflective of experience you prefer. And, thus, your experience is optimized.

Research helps companies serve you better, and helps you become a more effective and efficient consumer.

It’s elementary, my dear Watson.

view all

Apathy vs Hate In City Research

Date: July 12, 2012 | Shawn Herbig | News | Comments Off on Apathy vs Hate In City Research

Let’s suppose you’re a city manager responsible for serving residents and companies in your beloved hometown. These folks are the lifeblood of any community, and it’s your job to keep them happy.

Yet, all they hear about is how terrible your city is. Certain groups loudly complain about underfunding of your parks and bike paths. Other groups scream about the lack of resources being allocated to at-risk youth programs. Still other groups are mobilized against the tax breaks being offered to corporations considering relocating to your area. It’s unfair. . . unconscionable. . . an outrage!

What do you do?

As difficult as it might seem, you ignore the highly-orchestrated hysterics of the political activists and seek a dialogue with the vast majority of residents and potential residents who are focused on their daily lives and not engaged in the life-and-death struggle for public resources.

The surprising fact is that the majority of citizens who constitute the backbone on any community typically neither love nor hate it. They are rational people who recognize that reality lies between the extremes. They’re turned off by the rhetoric and confusion, not by your city.

Unfortunately, their silence doesn’t drown out the apoplectic cries of the highly publicized “victims.”

This is the apathy versus the hate equation. You might think that most people “hate” you and your city. They hate everything you’re doing, and hate everything you stand for.

Nothing could be further from the truth.

But the sad part, is not that people hate you, but that most people don’t seem to care.

On the one hand, you’ve got a super loud, but small group of people. On the other, you’ve got a very quiet, very large group of people who are very detached from what you do.

And that is the group you need to hear from, because they’re the ones who will ultimately help you find the best choice for your city.

As researchers, we are often called on to help city managers learn the opinion of the majority of their city’s population. They want to learn what the majority really wants, rather than ignoring them in favor of those loud few.

That’s why you as a city manager needs to understand the important — very important — difference between hate/anger and apathy from your constituents. One is louder, but the other is more powerful.

Because if you end up angering the apathetic group, they’ll remember it come Election Day.

Check back for our next post, as we discuss some of the different scientific research methods city managers can use to learn the true will of the people in making the best decisions.

view all