Selecting Question Format

Research

Selecting Question Format

Given that every organization is unique, there is no research to qualify the scientifically best strategy for choosing and selecting question format, although there is research offering different effective approaches. Vision Metric’s Consulting Group provided some insight into consumer preferences and strategies for choosing specific questions. This information, however, cannot be extrapolated to every organization as the best practice; it is just one company offering one approach.

It is essential to not make providing feedback a chore with hundreds of monotonous questions. At Vision Metrics, 72% of raters ask more than 35 questions per survey. This is process can still be extremely time intensive filling out surveys for multiple individuals. Keeping feedback simple allows for answers to remain genuine and truthful. It is important to note that a greater number of raters do not mean better, more comprehensive feedback (Ghorpade, 2000). Information compiled by those with an unclear role of expectations can just dilute the data and waste company resources.

It is argued that leaving room for open-ended questions is effective since it leaves room for an interview to elaborate on a specific response and construct a specific solution for personal development. At Vision Metrics, 95% of clients use less than two open-ended questions. In contrast, research by Atwater and Brett (2001) shows that individuals who received text feedback felt significantly less positive and motivated than when they received numeric scores. This could be explained by the fact that open-response feedback carries inevitable bias, allowing an employee to attribute the feedback to a particular individual.

Having clear job descriptions also allows for management to pick particular questions based on necessary job competencies rather than with specific employees in mind. With the third party online feedback systems, selecting questions is simplified by the organization of different key skills and activities. It is suggested not to provide more than six competency groups as a range of important skills to keep the scope of the survey manageable. When choosing questions, it is crucial to select them avoiding generic behaviours that can be interpreted in different perspectives. It is a common mistake to address several behaviours in one question to maximize the breadth of the survey. In this situation, an employee could score high in one behaviour while significantly lagging in the next, which would confuse the data.

Likert’s research stresses the need for “clear, concise, and straight-forward statements” where no other variables can be interpreted (Likert, 1932). It is also suggested to use simple vocabulary and avoid double negatives to ensure clear answers. Action-oriented and positive question formation is also important to avoid negative constructs. Maintaining a positive tone throughout the survey will help to ensure better results keeping the survey optimistic about the company’s development. Questions that allow the subject to take moral sides are generally ineffective (Likert, 1932). For example, the question ‘this person is benevolent to coworkers’ is open to interpretation and cultural differences about the level of reciprocity that should be expressed in the workplace. Therefore, affirmative questions are easier to conceptualize and respond to while also eliminating biases. Thurstone and Murphy (1927) also stress the importance of using present rather than past attitudes when framing questions. This gives an accurate snapshot of the employee in the present with few lingering predispositions of behaviours that occurred months or years ago.

Research from Vision Metrics shows that 93% of consumers use one single survey to form observable traits that can be used generically to all employees receiving comprehensive feedback. This is an example of where selecting 360-degree feedback questions can be seen as a trade-off. Using questions that can apply to all employees lacks specify, but devising individual questions takes significantly more time and effort to design. Generally with a direct company vision, many traits can highlight relevant criteria in a variety of positions. However, when trying to stress goals based on job descriptions these generics behaviours will lack specificity.