been reading up on the conceptual spaces a bit. the geometrical lens inspires me a lot when im thinking about multidimensional latent space from llm vs. human.
Indulge me in a thought experiment: imagine concepts in your head as points scattered across a vast, invisible landscape. In this landscape, each concept has coordinates that describe its features. Now, apply this landscape framework to the concepts of fruits. Apples might occupy a region of marked by moderate sweetness, crisp texture, and reddish hues. Lemons would exist elsewhere, sharp-tasting, brightly colored, positioned far from apples on a landscape defined by taste, texture, and color. What feels intuitive to us - that apples are closer to pears than to lemons - becomes precise and measurable within this geometric framework.
It seems that concepts have an innate structure that enables day-to-day, seemingly trivial cognitive tasks possible: comparison, categorizing, analogizing.
This is precisely what cognitive scientist Peter Gärdenfors[1] suggests: our mind organizes knowledge geometrically. Concepts are defined in multidimensional spaces composed of different dimensions or features - sweetness, color, texture, and much more - sometimes even unnamed. These shapes, constantly shifting and evolving as we learn from new experiences, are the cornerstones of our mind. [1]
Dimensions: the Building Blocks
At the heart of Gärdenfors’s framework are “quality dimensions” – the basic ways in which we discern similarity and difference. Consider the dimensions of color: hue, saturation, and brightness. Or for sound: pitch and loudness. They are not necessarily scientific properties - and yet they are fundamental to our perceptual model of this world.
We don’t just do this with natural phenomena - we do it with everything, including ourselves. Take personality for example: we distill the messiness of human traits into dimensions like introversion–extroversion or openness. Tests like the Myers-Briggs or the Big Five plot you along a few chosen axes, locating you in a “personality space” where your position can be compared to others, as if mapping the psyche in coordinates.
Similarity, then, is defined by proximity within these maps; the closer two representations, the more alike they are perceived.
These dimensions coalesce into “domains”, coherent spaces dedicated to particular kinds of qualities. The color domain encompasses its related dimensions, distinct from, say, the domain of spatial relations. Within these domains, individual objects are located as points, their coordinates determined by their specific values along the constituent dimensions. Crucially, properties are not merely labels but are represented as regions within these domains. Gärdenfors argues that the “natural” properties that anchor our thought and language tend to occupy well-behaved, often convex, regions within these spaces. Imagine the property “red” as a defined area within the color space, encompassing the various shades we recognize as red.
This geometrical approach seems to have more explanatory power: it provides an intuitive account of how we judge similarity, a cornerstone of concept formation and categorization. Learning what constitutes a “bird”, for instance, can be seen as carving out a region in a multidimensional space defined by features like size, shape, color, and characteristic sounds. Furthermore, conceptual spaces offer a foundation for a cognitive semantics, where the meaning of words is rooted in these embodied, perceptual structures, potentially illuminating the mechanisms of metaphor as structured mappings between domains. This framework makes me hopeful for a more sophisticated artificial intelligence systems capable of nuanced perception, analogical reasoning, and a more human-like understanding of the world.
Neurobiological Foundation
Is this geometric framework just a convenient metaphor, or does the brain literally use geometric codes?
Interestingly, the theory of conceptual spaces finds neurobiological plausibility in the brain’s inherent use of spatial organization and vector-like representations. Evidence from topographic maps in sensory cortices demonstrates that the brain maintains spatial relationships corresponding to stimulus features, suggesting domain-specific geometrical layouts2. In “the Organization of Learning”, Gallistel argues that the nervous system represents stimulus properties using vectors in anatomical spaces, where dimensions of these spaces correspond to stimulus dimensions. The tensor network theory of Pellionisz and Llinas also aligns with the idea of the brain as a “geometrical object” in its sensorimotor functions3.
Conceptual Spaces in Artificial Minds
But conceptual spaces aren’t limited to biology. Artificial intelligence and language models also appear to “think” geometrically.
Google’s recent research[4] offers an intriguing glimpse into how closely human brains and LLMs might be thinking alike - at least when it comes to language. In the study, researchers found that neural activity in the brain’s language centers during natural speech maps surprisingly well onto the internal representations used by LLMs. Essentially, when we process words and sentences, our brain seems to be navigating a kind of high-dimensional space that mirrors the one an AI uses to make sense of language.
This finding dovetails beautifully with the theory of conceptual spaces. If both our brains and LLMs organize language and meaning as positions in a geometric space—where distances between points reflect relationships and associations - then perhaps geometry really is essential to understanding our mind. It feels like a compelling convergence: humans and machines, each drawing mental maps of meaning, not with definitions or rules, but with spatial intuition.
That convergence isn’t just intellectually satisfying - it may also be practically useful: instead of teaching AI every aspect of the physical world from scratch, we might just align their internal maps to our reality. The AI, in other words, has already sketched out a fairly accurate landscape—it just needs calibration.
Uncharted Maps
In the book, Conceptual Spaces, Peter Gärdenfors also argues that scientific breakthroughs often come not just from new data, but from the introduction of entirely new conceptual dimensions - new ways of slicing up reality. Just as adding “temperature” to our understanding of gases led to thermodynamics, expanding our conceptual framework is often what enables deeper explanations and predictions. Scientific terms are more than just labels - they represent the birth of new cognitive domains, the psychological equivalent of adding new axes to the space in which we understand the world.
LLMs are trained to learn to organize information in high-dimensional embedding spaces. While their internal dimensions aren’t directly interpretable like “color” or “mass,” they might function in a similar way - carving up the space of meaning along statistically derived lines. As the model clusters concepts and learns latent dimensions, it may be constructing its own internal domains, not unlike how science forms new fields. This raises a fascinating possibility: could LLMs, in exploring their own latent spaces, stumble upon novel conceptual structures - relationships or distinctions we haven’t yet formalized? While they lack grounding, understanding, or intention, their ability to generate new patterns and recombine ideas suggests they might be performing, in their own alien way, something not unlike the early stages of scientific discovery.
If LLMs organize knowledge geometrically - positioning meanings in high-dimensional space - then we’re not just training systems that talk. We’re building maps of human thought, drawn in coordinates. And once you have a map, you can do much more than describe the territory. You can edit it, expand it, explore it.
[1] Peter Gärdenfors, “Conceptual Spaces”, 2000.
[2] C. R. Gallistel, “The Organization of Learning”, 1990.
[3] A. Pellionisz & R. Llinás, “Tensor Network Theory Of The Metaorganization Of Functional Geometries In The Central Nervous System”, 1985.
Conceptual spaces, as proposed by cognitive scientist Peter Gärdenfors, suggest that our mind organizes knowledge geometrically. Concepts are defined in multidimensional spaces composed of different dimensions or features, such as sweetness, color, and texture. These dimensions, also known as “quality dimensions,” are the basic ways in which we discern similarity and difference. As we learn from new experiences, these shapes constantly shift and evolve, forming the cornerstones of our mind. (Explanation by AI) ↩︎