JCCES General Knowledge Item Structure Analysis

The General Knowledge (GK) subtest of the Jouve Cerebrals Crystallized Educational Scale (JCCES) measures factual breadth — the accumulated stock of information about the world that crystallized intelligence theory treats as a core component of acquired cognitive ability. A subtest of this kind has to satisfy two structural requirements: items should span a meaningful range of difficulty, and they should order along a single underlying continuum of factual breadth rather than tapping multiple unrelated dimensions. This study examined the item structure of the JCCES GK subtest using multidimensional scaling (MDS) on response data from 588 respondents, and recovered the empirical signature of a well-ordered unidimensional construct: a horseshoe-shaped scaling pattern that is the canonical evidence of a Guttman simplex — items ordered cleanly along a single difficulty continuum.

Why item structure matters for cognitive subtests

A cognitive subtest is a measurement instrument with a particular target construct. For the JCCES GK subtest, the target is general factual knowledge — a unidimensional continuum that respondents are positioned on by their cumulative knowledge breadth. If the items are well-aligned with this continuum, harder items are passed only by respondents who pass easier items, and the response pattern across items has the stepwise structure of a Guttman scalogram. If the items tap multiple unrelated dimensions, the response pattern is irregular and the test is measuring a heterogeneous construct rather than a unified ability.

The empirical signature of a clean unidimensional Guttman structure in MDS is the horseshoe-shaped pattern: items arranged along a curved arc in the 2-D solution rather than scattered randomly or clustered into separate groups. The horseshoe shape arises mathematically from the linear ordering of items by difficulty: easier and harder items sit at the two tips of the arc, with intermediate items along the curve. The shape was first documented in Guttman scalogram analyses (see Collins & Cliff, 1990, for the simplex generalization to growth modeling) and is now a recognized diagnostic for unidimensional ordinal item sets.

For a general-knowledge subtest, recovering the horseshoe pattern is the empirical evidence that the items work as intended — they probe the same underlying factual continuum, scaled across a range of difficulty, with no anomalous items that cluster off-axis. The MDS analysis produces this evidence directly without requiring a measurement-model specification, making it a natural exploratory check on item structure.

The JCCES GK subtest design

The JCCES GK subtest is part of the broader JCCES crystallized intelligence battery, alongside Verbal Analogies (VA) and Mathematical Problems (MP). The subtest follows a content-balanced item design: items span historical, scientific, geographical, cultural, and general-factual content, with item difficulty spread across the construct continuum. The administration uses a stopping rule based on consecutive errors — testing terminates after a respondent makes a specified number of errors in a row, similar to the basal/ceiling structure used in Wechsler subtests and the Reynolds Intellectual Assessment Scale (Reynolds & Kamphaus, 2003).

The stopping rule serves two purposes. First, it makes the test more efficient by avoiding administration of items beyond a respondent’s ceiling, where they would predominantly fail and provide little information about ability. Second, it reduces respondent fatigue and discouragement, both of which can bias scores downward through reduced effort or careless responding (Sundre & Kitsantas, 2004). The trade-off is that the stopping rule needs to be calibrated correctly: too aggressive (e.g., terminating after only two consecutive errors) and the test misses information about respondents who recover from a brief difficult patch; too lenient (e.g., requiring ten consecutive errors) and the efficiency benefit is lost. The JCCES GK subtest uses a five-consecutive-error stopping rule, determined to be appropriate for the construct in this sample.

The combination — a unidimensional content design + an efficiency-oriented stopping rule — produces a test that is conceptually sound but requires empirical verification that the items behave as intended. The MDS analysis is the verification step.

Method

Five hundred and eighty-eight respondents completed the JCCES GK subtest under standardized administration. Item-difficulty parameters were estimated using the Rasch model (one-parameter logistic IRT), and items were rearranged in difficulty-ordered sequence based on the Rasch estimates. The rearranged item set was submitted to multidimensional scaling, with Kruskal’s Stress Formula 1 (Kruskal, 1964) as the goodness-of-fit measure and squared correlation (RSQ) between distances and disparities as the complementary index.

The 2-dimensional MDS solution was extracted, supplying the geometric configuration of items in the reduced space. The pattern of item placements within this space — clustered, scattered, or arc-shaped — was the diagnostic outcome of interest. A horseshoe-shaped configuration would confirm unidimensional Guttman-style ordering of items by difficulty; alternative patterns would suggest more complex item structure.

Results

The 2-dimensional MDS solution produced a stress value of 0.18 and an RSQ of 0.87. The stress value falls within the conventional “fair” range (0.10-0.20) and the RSQ exceeds the 0.80 threshold typically used to evaluate adequate fit. The 2-D configuration is therefore a defensible representation of the inter-item relationships.

The substantive finding emerges from the geometric pattern of items in the configuration: the items arrange in a horseshoe-shaped curve, with easier items at one tip, harder items at the other tip, and intermediate items along the arc. This is the canonical Guttman simplex pattern (Collins & Cliff, 1990), and it confirms that the GK subtest items are ordered along a single dominant continuum of factual breadth. Items at adjacent positions on the curve are similar in difficulty and content scope; items at opposite ends differ maximally on the underlying construct.

The pattern is consistent with the constraint that “dissimilarities have all been supported” — meaning no item placement violates the predicted linear ordering, and no item pair appears anomalously close or far compared to the difficulty-based prediction. The result supports the construct validity of the GK subtest as a measure of unidimensional general factual knowledge.

What the horseshoe pattern means for the test

The horseshoe finding has three substantive implications for the JCCES GK subtest. First, the items are well-aligned with the construct: there are no off-axis clusters that suggest items are tapping irrelevant cognitive demands. A test where some items cluster separately would be measuring a multidimensional construct that the GK label does not capture; the horseshoe pattern confirms the unidimensional design intent.

Second, the difficulty ordering is empirical, not just theoretical. The Rasch-estimated difficulty parameters position items along the continuum based on actual response data, and the MDS recovers a configuration consistent with this ordering. The two methods — IRT difficulty estimation and MDS structural analysis — converge on the same conclusion: the items work as a graded difficulty scale.

Third, the stopping rule of five consecutive errors is calibrated to the empirical difficulty structure. With items ordered cleanly along a difficulty continuum, the consecutive-error pattern is a reliable indicator that a respondent has reached their ceiling: five consecutive failures in this ordering means the respondent has crossed from items they can pass to items they cannot, and continued administration would yield mostly failures with little additional information about their ability level.

Where this fits in cognitive-test design

The JCCES GK subtest illustrates a design pattern that is well-established in cognitive assessment but not always implemented cleanly: items selected for graded difficulty along a unidimensional construct, ordered by Rasch-estimated difficulty, administered with a stopping rule calibrated to the difficulty distribution. The pattern produces tests that are efficient, internally consistent, and structurally interpretable.

Comparable design patterns appear in major commercial cognitive batteries. The Wechsler subtests (Wechsler, 2008) use basal-and-ceiling stopping rules calibrated to age-graded difficulty distributions. The Raven’s Progressive Matrices (see Bors & Stokes, 1998 for university-student norms and short-form analyses) order items by empirical difficulty across a unidimensional inductive-reasoning continuum. The Advanced Progressive Matrices in particular uses this pattern as its foundational structure.

The JCCES GK subtest fits this lineage: it adopts the design principles that the broader cognitive-assessment field has converged on, and the present analysis confirms empirically that the implementation works as intended. The horseshoe pattern is the diagnostic that distinguishes well-implemented unidimensional design from poorly-implemented or genuinely multidimensional alternatives.

Methodological caveats

The 588-participant sample is large for MDS — well above the thresholds at which 2-D MDS solutions become stable — and the recovered structure is correspondingly well-supported within this respondent population. The substantive caveat is demographic representativeness rather than sample size: the sample is a voluntary online convenience sample, and the stopping criterion of five consecutive errors was calibrated empirically against this distribution; respondent populations with different ability distributions or test-taking strategies might warrant different thresholds.

MDS makes assumptions about the metric of dissimilarities — most commonly that they are interval-scaled and representable in Euclidean space. For binary-scored item-response data, these assumptions are approximations, and alternative MDS variants (nonmetric MDS, ALSCAL) produce slightly different configurations. The horseshoe pattern is robust across these methodological variations, but specific numerical fit indices vary.

The MDS analysis does not formally test the unidimensional structure against alternatives. A confirmatory analysis using multidimensional IRT with hypothesized one-factor structure would supply the formal model comparison. The MDS result is suggestive of unidimensionality; the IRT analysis would establish whether the data fit a unidimensional model better than a multidimensional one.

Connection to the broader Cogn-IQ research program

The JCCES has been examined under multiple analytical lenses: JCCES + GAMA via MDS, JCCES + GAMA via factor analysis, JCCES + ACT, JCCES + RIAS Verbal Scale, and JCCES vs multiple cognitive and academic measures. The present GK item-structure analysis adds the within-subtest level of resolution: not just whether the JCCES subtests behave as expected at the battery level, but whether the items within each subtest are well-organized along the construct continuum.

The cumulative picture is that the JCCES has empirical support at multiple levels of analytic resolution: the battery-level composite correlates strongly with established cognitive measures, the subtest-level scores load on theoretically expected factors, and the item-level structure within at least the GK subtest aligns with a unidimensional Guttman ordering. This multi-level empirical support is what construct validation in psychometrics aspires to, and the JCCES research program documents it across studies.

Frequently Asked Questions

What is a Guttman scalogram and why does it matter?

A Guttman scalogram is a deterministic ordinal pattern: items are ordered such that respondents who pass any item also pass all easier items. The MDS horseshoe shape is the geometric signature of an approximately Guttman-scaled item set in a noisy real data context. Recovering this pattern confirms the items are ordered along a single underlying difficulty continuum.

Why use MDS instead of factor analysis or IRT?

MDS is exploratory and makes minimal model assumptions. Factor analysis posits a measurement model with latent factors; IRT posits a probabilistic response function. MDS asks only whether the inter-item dissimilarities can be represented in low-dimensional space, which makes it a useful diagnostic step before committing to a measurement-model assumption. The three methods are complementary; MDS often serves as a structural-orientation step before factor analysis or IRT.

What does a stress value of 0.18 indicate?

Kruskal’s stress measures the discrepancy between observed dissimilarities and the distances reproduced by the MDS solution. Conventional benchmarks: stress < 0.05 excellent, 0.05-0.10 good, 0.10-0.20 fair, > 0.20 poor. A value of 0.18 falls within the fair range, indicating that the 2-dimensional solution adequately represents the inter-item relationships without being a perfect fit.

Why use a five-consecutive-error stopping rule rather than another threshold?

The threshold was determined empirically: three consecutive errors was found to terminate testing too aggressively for the JCCES GK construct, while higher thresholds produced diminishing efficiency gains. Five strikes a balance between efficient ceiling determination and giving respondents adequate opportunity to recover from a transient difficult patch.

Does the GK subtest measure crystallized intelligence specifically?

The GK subtest taps factual-knowledge breadth, a core component of crystallized intelligence per Cattell-Horn theory (see Deary, 2000, for the broader framework). The horseshoe-shaped item structure confirms unidimensional ordering, supporting the case that the subtest measures a coherent ability rather than a heterogeneous mix of constructs. Cross-validation against other crystallized-IQ measures further establishes its construct validity.