Validity of a scale for consumer evaluation of mental health service delivery

Objective : The factor structure of a previously developed consumer constructed scale is based on whether the questions are positively or negatively worded and extant literature suggests that this may reflect the wording rather than the genuine structure of the questionnaire. Given the potential usefulness of this type of scale, further investigation of its structure and applicability to the development of person-centered mental health service organisation and delivery is warranted.  Method : The questionnaire, with consumer suggested modifications, was delivered twice across two Australian non-government mental health services. The two data sets were thus independent. The factor structure was investigated using the group 1 data and then applied to both groups using Confirmatory Factor Analysis (CFA) and multiple regression.  Results: The factor structure was affected by the phrasing of the questions.  It was not possible to derive a model using the full set of questions and a reduced set was derived.  Using the reduced set, valid models were produced in both the 2 and 1 factor CFAs, but only the model based on 1 factor was robust when applied to an independent data set.  Conclusions : The 1 factor model with a reduced question set offers the advantages of being quicker to deliver, not use negative phrasing and produces a more robust CFA.  Further work investigating the delivery of such a questionnaire may contribute usefully to the development of person-centered mental health service organisation and delivery.