Does Your Culture Survey Actually Measure Culture?

Jung et al. (2009) conducted the most comprehensive review of organisational culture measurement instruments to date, identifying 70 instruments published in peer-reviewed literature. Their findings were sobering. Many instruments had been developed for a single study and never validated again. Others had been used repeatedly but with limited evidence of construct validity — the extent to which the tool actually measures what it claims to measure. Only a handful had been subjected to confirmatory factor analysis, criterion validity testing, or cross-cultural validation. The review concluded that the field suffers from "a plethora of instruments with limited evidence of validity," making it difficult for practitioners to make informed choices.

Heritage, Pollock and Roberts (2014) published one of the few rigorous validation studies of the Organisational Culture Assessment Instrument (OCAI), the most widely used culture tool in academic research. Their confirmatory factor analysis, published in PLOS ONE, supported the four-factor structure predicted by the Competing Values Framework — clan, adhocracy, market, and hierarchy cultures do emerge as distinct dimensions. However, criterion validity was mixed. The OCAI distinguished between organisations in different sectors and at different life-cycle stages, but its ability to predict specific outcomes like performance or employee satisfaction was not robustly demonstrated.

The OCAI also carries a fundamental measurement limitation: its ipsative format. Respondents distribute 100 points across four culture descriptions, meaning that scoring higher on one culture type automatically forces lower scores on the others. This creates artificial negative correlations between culture types and violates assumptions required by most standard statistical analyses. Researchers have repeatedly flagged this as a problem — ipsative data cannot be factor-analysed in the conventional sense, and comparing individuals or organisations using ipsative scores is statistically problematic. Despite this, the OCAI remains the most commonly used culture instrument in research, largely because it is free and quick to administer (24 items).

The Denison Organisational Culture Survey (DOCS) takes a different approach. Developed by Daniel Denison in the 1990s and refined over more than two decades, the DOCS measures four traits — involvement, consistency, adaptability, and mission — using a normative Likert-scale format that avoids the ipsative problem. Its major advantage is criterion validity: over 15 years of accumulated data from thousands of organisations has linked DOCS scores to financial performance, market share, innovation, employee satisfaction, and quality metrics. Denison and colleagues have published multiple studies showing that organisations scoring in the top quartile on all four traits significantly outperform those in the bottom quartile. The trade-off is that the DOCS is proprietary, requires licensing, and is substantially more expensive than the free OCAI.

The CIPD's 2023 evidence review on organisational culture highlighted a persistent gap in measurement standardisation. Different instruments operationalise culture differently — some measure values and beliefs, others measure behavioural norms, others measure artefacts and practices. This means that two organisations both claiming to have "measured their culture" may have measured fundamentally different things. The review noted that this inconsistency makes it difficult to compare findings across studies and contributes to the impression that culture research is less rigorous than research on constructs like engagement or leadership.

Critically, no meta-analysis has compared the predictive validity of different culture instruments against each other. We have evidence that the OCAI has structural validity and that the DOCS has criterion validity, but we have no systematic evidence telling us which instrument produces more accurate or useful results in practice. This is a significant blind spot given how much organisational money is spent on culture measurement.

Your choice of instrument shapes your conclusions. An ipsative tool like the OCAI forces a zero-sum view of culture — you cannot score high on everything simultaneously. A normative tool like the DOCS allows for the possibility that an organisation has genuinely strong practices across multiple dimensions. These are not just statistical nuances; they lead to different diagnoses and different intervention recommendations.

Free does not mean validated. The OCAI is free and widely cited, which creates a halo of credibility. But frequency of use is not evidence of validity. Its ipsative format is a genuine methodological limitation that most practitioners are unaware of, and its criterion validity — the ability to predict outcomes that matter — remains poorly established.

Proprietary does not mean better, either. The DOCS has stronger validation evidence, but it is also marketed by a consultancy with commercial incentives. The validation studies have largely been conducted by Denison and his collaborators, raising questions about independence. Practitioners should look for validation evidence from researchers without financial ties to the instrument.

Beware of conflating culture with climate. Many instruments marketed as "culture surveys" actually measure organisational climate — employees' perceptions of policies, practices, and reward systems. Climate and culture overlap but are not identical. Climate is more surface-level and easier to change; culture involves deeper shared assumptions and values. If your instrument primarily asks about observable practices and policies, you may be measuring climate and calling it culture.

Measurement is necessary but not sufficient. Even a perfectly validated culture instrument only gives you a snapshot. Culture is dynamic, contextual, and multi-layered. Quantitative surveys should be supplemented with qualitative methods — interviews, observation, artefact analysis — to understand not just what people report but why they report it.