Bayesian analysis of ordinal survey data using the Dirichlet process ...

More documents

Recommendations

Info

1 INTRODUCTIONFor the sake of convenience, many surveys consist of ordinal data, often collected on afive-point scale. For example, in a typical course evaluation survey, a student may expresshis view concerning an aspect of the course from a set of five alternatives: 1-poor,2-satisfactory, 3-good, 4-very good, and 5-excellent. Sometimes five-point scales havealternative interpretations. For example, the symmetric Likert scale measures a respondent’slevel of agreement with a statement according to the correspondence: 1-stronglydisagree, 2-disagree, 3-neither agree nor disagree, 4-agree, and 5-strongly agree. Studentfeedback on course evaluation surveys represents a modern approach for measuring quality.Nowadays, a growing number of websites use student feedback as their main performanceindicator in teaching evaluations. As an example, http://www.ratemyprofessors.com/rate over one million professors based on student feedback on a five-point ordinal scale.The scenario is similar in customer satisfaction surveys and social science surveys.The simplest method of summarizing ordinal response data is to report the meanscorresponding to the ordinal scores for each survey question. At a slightly higher level ofstatistical sophistication, standard ANOVA methods may be applied to the ordinal scoresby treating the data as continuous. However, the standard models for the analysis ofordinal data are logistic and loglinear models (Agresti 2010, McCullagh 1980 and Goodmann1979). These models correctly take into account the true measurement scales forordinal data and permit the use of statistical inference procedures for assessing populationcharacteristics. An overview of the methodologies for ordered categorical data is given byLiu and Agresti (2005).The approach in this paper is Bayesian and considers an aspect of ordinal survey datathat is sometimes overlooked. It is widely recognized that respondents may have differingpersonalities. For example, consider a company which conducts a customer satisfaction2
survey where there is a respondent with a negative attitude. The respondent may completethe survey with a preponderance of responses in the 1-2 range. In this case, a response of1 may not truly represent terrible performance on the part of the company. The responsemay reflect more on the disposition of the individual than on the performance of thecompany. As another example of an atypical personality, consider an individual who onlyprovides extreme responses of 1’s and 5’s. It would be useful if statistical analyses couldadjust for personalities. This is the motivation of the paper, and the tool which we use toaccount for personalities is the Dirichlet process, first introduced by Ferguson (1973). As aby-product of the proposed methodology, we attempt to identify areas (survey questions)where performance has been poor or exceptional. In addition, we attempt to identifyquestions that are highly correlated. Clearly, surveyors desire accurate responses and byidentifying highly correlated questions, it allows surveyors to remove redundant questionsfrom the survey which in turn reduces fatigue on the part of the respondents.Our paper is not the first Bayesian paper to consider this problem. AlternativeBayesian approaches include Johnson (1996), Johnson (2003), Dolnicar and Grun (2007),Rossi, Gilula and Allenby (2001), Kottas, Mueller and Quintana (2005), Javaras andRipley (2007) and Emons (2008). Johnson (2003) uses a hierachical ordinal regressionmodel with heterogenious thresholds structure. Dolnicar and Grun (2007) use a ANOVAapproach to assess the inter-cultural differences in responses. Rossi, Gilula and Allenby(2001) address nonidentifiability and parsimony by imposing various complex constraintson the unknown cut-points. Kottas, Mueller and Quintana (2005) propose a nonparametricBayesian approach to model multivariate ordinal data recorded in contingency tables.One of the main features of this paper is that there is a mechanism to cluster subjectsbased on personalities. Most importantly, in our approach, clustering takes place as apart of the model and data determine the clustering structure. Often, clustering is done3
Page 1: Bayesian Analysis of Ordinal Survey
Page 5 and 6: validity of the model. The proposed
Page 7 and 8: and 4’s. We instead consider a st
Page 9 and 10: our rationale for the Uniform(0, 6)
Page 11 and 12: For programming in WinBUGS, the Set
Page 13 and 14: Markov chain simulations. The resul
Page 15 and 16: Table 1: Posterior means and poster
Page 17 and 18: our model gave D = 891. In both the
Page 19 and 20: Table 2: Posterior means and poster
Page 21 and 22: To provide a more stringent test, w
Page 23 and 24: Brooks, S. P. and Gelman, A. (1997)
Page 25 and 26: Parducci, A. (1965). “Category ju
Page 27 and 28: proportion0.00 0.02 0.04 0.06 0.08
Page 29 and 30: elative frequency0.0 0.1 0.2 0.3●

Bayesian analysis of ordinal survey data using the Dirichlet process ...

Create successful ePaper yourself

Delete template?

Save as template?