Dimensional Structure of JCCES Math Items

Mathematical problem-solving is not a single cognitive ability — it draws on numerical reasoning, algorithmic computation, applied analytic thinking, spatial visualization, and quantitative pattern recognition, with different problems weighting these components in different proportions. A test built to measure mathematical problem-solving therefore covers a functional space rather than a single dimension, and the practical question for test development is how many distinct dimensions are needed to represent the structure of real test items adequately. This study used Alternating Least Squares Scaling (ALSCAL; Young, Takane, & Lewyckyj, 1978) — a flexible nonmetric multidimensional-scaling procedure — to investigate the dimensional structure of the 38 Mathematical Problems items from the Jouve Cerebrals Crystallized Educational Scale (JCCES) on a sample of 588 respondents.

Background: dimensions of mathematical problem-solving

The cognitive psychology literature recognizes that mathematical problem-solving is not unidimensional. Geary (1994) framed mathematical development as the interplay of biologically primary number sense, secondary culturally-acquired computation, and higher-order strategy use; the three contribute distinguishably to performance on real problems. Swanson and Beebe-Frankenberger (2004) tied mathematical problem-solving to working-memory subsystems, finding that visuospatial sketchpad and central executive demands are differentially loaded across problems. Hecht and Vagi (2010), looking specifically at fraction skills, identified separable contributions from conceptual understanding, procedural knowledge, and strategic flexibility.

The implication for assessment design is that any reasonably broad mathematical-reasoning test will tap multiple cognitive dimensions, and the question is how many. A unidimensional test would be measuring only one of these dimensions; a hyper-multidimensional test would be redundant. The empirical answer depends on the specific test and the sample, and the appropriate analysis is dimensionality estimation from the data.

The JCCES Mathematical Problems (MP) subtest contains 38 items selected to span numerical, algorithmic, and applied reasoning content. The test is administered as part of the broader JCCES crystallized intelligence battery, which also includes Verbal Analogies and General Knowledge subtests. The MP subtest’s dimensional structure has practical consequences for how its items are interpreted, scored, and used in composite calculations.

Method

Five hundred and eighty-eight respondents completed the JCCES MP subtest under standardized administration. The 38-item correlation matrix was submitted to ALSCAL with dimensional solutions ranging from two through five dimensions. Convergence criteria were set at improvement < 0.001 in successive stress values; Kruskal's Stress Formula 1 (Kruskal, 1964) was the goodness-of-fit measure, with RSQ (proportion of variance in disparities accounted for by the solution) as the complementary index.

The decision criterion for selecting the preferred dimensionality was: the smallest dimensionality that achieves RSQ > 0.80 and stress within the 0.10 to 0.20 range. Higher dimensionalities provide better fit by construction but risk over-fitting; the criteria balance fit against parsimony.

Results

The four candidate dimensionalities produced the following fit statistics:

• 2-dimensional: stress = 0.305, RSQ = 0.647. Failed the criteria — stress too high, RSQ too low.
• 3-dimensional: stress = 0.210, RSQ = 0.739. Stress at the upper limit, RSQ below threshold. Failed.
• 4-dimensional: stress = 0.155, RSQ = 0.815. Both criteria satisfied. Preferred solution.
• 5-dimensional: stress = 0.127, RSQ = 0.851. Better fit but at the cost of an additional dimension.

The 4-dimensional solution captures 81.5% of the variance in inter-item disparities and operates at a stress level (0.155) that ALSCAL conventions characterize as “fair to good” (Kruskal, 1964). The 5-dimensional solution improves both metrics modestly — stress drops by 0.028, RSQ rises by 3.6 percentage points — but the additional dimension does not yield a substantively distinct cluster. Under the parsimony principle, the 4-dimensional solution is preferred.

The four dimensions correspond to distinct cognitive substrates that the MP items collectively tap. A formal dimensional labeling requires substantive analysis of which items load on which dimensions, which is beyond the scope of the present analysis but is the natural follow-up: examining item content within each dimension to identify the cognitive demand each represents.

Interpreting the four-dimensional structure

The 4-D solution implies that the JCCES MP subtest measures four separable but related cognitive dimensions, consistent with the prior literature on mathematical problem-solving (Geary, 1994; Swanson & Beebe-Frankenberger, 2004; Hecht & Vagi, 2010). Plausible candidate dimensions, based on the substantive content of the MP items:

• Numerical reasoning: items requiring direct manipulation of numerical quantities, mental arithmetic, and quantitative comparison. The most basic mathematical demand.
• Algorithmic / procedural fluency: items requiring application of standard mathematical procedures (long-division algorithms, equation-solving routines, formula application).
• Spatial / visualization: items requiring mental visualization of geometric or spatial relationships, distance, area, or directional reasoning.
• Word-problem analytic reasoning: items presented in narrative form requiring extraction of mathematical structure from natural-language description, then application of appropriate mathematical operations.

The four-dimensional structure is consistent with the multi-component frameworks proposed by Geary (1994) and Swanson and Beebe-Frankenberger (2004), and with the more recent fraction-skill decomposition of Hecht and Vagi (2010). The MP items, in aggregate, tap the same general cognitive substrates that the math-cognition literature has identified, suggesting good content validity for the broad construct of mathematical problem-solving rather than a single narrow ability.

Why this matters for the JCCES and for math-test design

The dimensional finding has direct implications for how MP items are scored and reported. If the subtest is unidimensional (one dominant factor), a single composite score adequately summarizes a respondent’s performance. If the subtest is multidimensional with four separable dimensions, a single composite can hide meaningful between-respondent differences in which dimensions are strong or weak. A respondent strong in numerical reasoning but weak in word-problem analysis would receive an average composite score that is less informative than dimension-specific scores.

For composite-score interpretation, the JCCES MP composite is best understood as a broad mathematical-reasoning index rather than a measure of a single ability. The factor analysis of JCCES + GAMA on a different sample showed that the JCCES MP subtest itself loads on a crystallized factor distinct from nonverbal-fluid measures, but the within-MP dimensional structure is finer-grained than that across-battery analysis revealed.

For test development, the 4-dimensional structure suggests that future revisions of the MP subtest could deliberately balance items across the four dimensions, ensuring that the composite represents a defined mix rather than depending on which item types happen to dominate the item pool. Item-development practice that targets each dimension explicitly tends to produce more interpretable composites than item-development that selects for difficulty alone.

Methodological caveats

The 588-participant sample, while substantially larger than typical Cogn-IQ research samples, was a convenience sample drawn from online media. Generalizability to the full population of mathematical-problem-solving respondents — children, professional populations, non-Western cultural contexts — is not established and should be addressed in replication studies.

ALSCAL is an exploratory technique. It identifies a defensible dimensional solution but does not formally test it against alternatives. A confirmatory analysis — through structural equation modeling with hypothesized dimensional structure or item response theory with multidimensional models — would strengthen the inference from “this is the most parsimonious dimensional solution that fits the data” to “this is the structure that the data support more than alternatives”. Multidimensional IRT methods in particular are well-suited to the next analytical step.

The criteria thresholds for stress and RSQ are conventions; reasonable analysts would set them differently and might prefer the 5-dimensional solution under stricter thresholds. The 4-D vs 5-D choice is a parsimony judgment, not a definitive empirical verdict.

Connection to the broader Cogn-IQ research program

This analysis sits within a broader program of psychometric work on the JCCES. The JCCES + GAMA MDS analysis showed that JCCES subtests cluster on a crystallized side distinct from GAMA’s nonverbal cluster. The JCCES + GAMA factor analysis recovered the same two-factor structure under different statistical machinery. The present analysis goes one level deeper: within the MP subtest itself, four dimensions account for the within-subtest structure. The picture across studies is that the JCCES is a hierarchically structured battery — broad crystallized intelligence at the top, distinguishable subtest-level constructs at the next level, and finer item-level dimensions within each subtest at the bottom.

Frequently Asked Questions

What is ALSCAL and how does it differ from other MDS methods?

Alternating Least Squares Scaling (Young, Takane, & Lewyckyj, 1978) is a nonmetric multidimensional-scaling algorithm that allows ordinal-level proximity data and uses an alternating least squares optimization to minimize stress. It is more flexible than classical metric MDS, which assumes interval-level proximities, and is implementable in standard statistical software including SPSS and R packages.

Why prefer the 4-dimensional solution over 5-dimensional?

The 5-D solution improves stress by 0.028 and RSQ by 3.6 percentage points — modest gains for an additional dimension. The parsimony principle favors the simpler model when fit improvements are marginal. The 4-D solution satisfies the predefined criteria (stress 0.80) and is more interpretable.

What does Kruskal’s stress measure?

Kruskal’s Stress Formula 1 (Kruskal, 1964) measures the discrepancy between the observed proximities and the distances reproduced by the MDS solution, normalized by the squared distances. Lower stress indicates better fit. Conventional benchmarks: stress 0.20 poor.

How does the dimensional structure inform test scoring?

If the test is multidimensional with four separable dimensions, a single composite score collapses meaningful information. Reporting dimension-specific scores alongside the composite gives a more granular picture of a respondent’s mathematical-reasoning profile and can identify specific strengths and weaknesses that the composite obscures.

What is the next analytical step beyond ALSCAL for this data?

Confirmatory analysis with multidimensional item response theory (MIRT) or structural equation modeling, using the 4-dimensional structure as a hypothesized model. The exploratory ALSCAL result is suggestive; a confirmatory test against the data would establish whether the structure replicates and whether it fits better than alternative dimensional models.