Bayesian estimation of category typicality using ordered probit models

Virtual MathPsych/ICCM 2023

Taylor Curley

taylor.curley@us.af.mil

Air Force Research Laboratory

711th Human Performance Wing

Wright-Patterson AFB, OH

Overview

Humans commonly classify nouns (e.g. chair) as members of superordinate categories (e.g. Furniture).^{1 2}

The degree to which nouns “belong” to a category – referred to as typicality – can be measured using normative data.³

Normative category typicality is an important aspect of linguistic⁴ and cognitive⁵ research.

1. Rosch, E. & Mervis, C. (1975). Family resemblances: Studies in the internal structure of categories. Cognitive Psychology.

2. McCloskey, M. & Glucksber, S.(1979). Decision processes in verifying category membership statements: Implications for models of semantic memory. Cognitive Psychology.

3. Van Overschelde, J., Rawson, K., & Dunlosky, J. (2004). Category norms: An updated and expanded version of the norms. Journal of Memory and Language.

4. Lewellen, M. et al. (1993). Lexical familiarity and processing efficiency: Individual differences in naming, lexical decision, and semantic categorization. JEP: General.

5. Hunt, R., Smith, R., & Toth, J. (2016). Category cued recall evokes a generate-recognize retrieval process. JEP: Learning, Memory, and Cognition.

Overview

Normative typicality is often measured via responses to large-scale surveys.

Example: Van Overschelde et al. (2003) - free response

Category prompt: A four-legged animal

Responses: cat, dog, horse

Measures: % reported, % reported first

Overview

Normative typicality is often measured via responses to large-scale surveys.

Example: Castro, Curley, & Hertzog (2021) - Likert responses

Category prompt: A four-legged animal

Exemplar prompt: cat

Not typical Very typical

1 2 3 4 5 6 7 8 9 10

Overview

Likert response densities (Castro et al., 2021).

Overview

Problem:

• All previous studies employ standard central tendency measures (i.e., averages).
• Response frequencies are NOT normally-distributed!
  • e.g., caterpillar vs. ant in the Insect category

Proposed Solution

Here, we use ordered probit models¹ and Bayesian parameter estimation to better approximate response distributions.

In these models, ordinal responses are represented as bounded areas on estimated distributions.

• Probability of giving a specific response calculated using the CDF of a fitted distribution.
• Parameter recovery maximizes fit of estimated and observed response densities.

1. Liddel, T. & Kruschke, J. (2018). Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology.

Ordered-probit model

Normal distribution (Liddell & Kruschke, 2018)

\[ \scriptsize{ p(y=k|\mu,\sigma,\theta_1,\dots,\theta_{K-1}) = \Phi \left( \frac{\theta_k - \mu}{\sigma} \right) - \Phi \left( \frac{\theta_{k-1} - \mu}{\sigma} \right) } \]

Ordered-probit model

Beta distribution

\[ \scriptsize{ p(y=k|\alpha,\beta,\theta_1,\dots,\theta_{K-1}) = \left( \frac{B(\theta_k;\alpha,\beta)}{B(\alpha,\beta)} \right) - \left( \frac{B(\theta_{k-1};\alpha,\beta)}{B(\alpha,\beta)} \right) } \]

Example: caterpillar

Fit improvements (LL): -33.01 (Gaussian) vs. -24.45 (Beta)

Example: caterpillar

Bayesian Parameter Recovery

Gaussian Probit

Beta Probit

Example: ant

Fit improvements (LL): -36.56 (Gaussian) vs. -21.73 (Beta)

Example: ant

Bayesian Parameter Recovery

Gaussian Probit

Beta Probit

Conclusions

Ordered probit models that estimate response probabilities using Beta distributions provide a novel method of estimating category typicality.

Specifically, distributions estimated from responses can be used to estimate the probability that a given exemplar is rated as more “typical” than another.

Future work will compare responses across different normative samples, e.g. hierarchical parameter recovery.