The inverted spectrum argument is first found (remarkably full-blown) in Locke’s Essay Concerning Human Understanding (II, xxxii,15, “Of True and False Ideas”):
"Neither would it carry any Imputation of Falshood to our simple Ideas, if by the different Structure of our Organs, it were so ordered, That the same Object should produce in several Men's Minds different Ideas at the same time; v.g. if the Idea, that a Violet produced in one Man's Mind by his Eyes, were the same that a Marigold produces in another Man's, and vice versâ. For since this could never be known: because one Man's Mind could not pass into another Man's Body, to perceive, what Appearances were produced by those Organs; neither the Ideas hereby, nor the Names, would be at all confounded, or any Falshood be in either. For all Things, that had the Texture of a Violet, producing constantly the Idea, which he called Blue, and those that had the Texture of a Marigold, producing constantly the Idea, which he as constantly called Yellow, whatever those Appearances were in his Mind; he would be able as regularly to distinguish Things for his Use by those Appearances, and understand, and signify those distinctions, marked by the Names Blue and Yellow, as if the Appearances, or Ideas in his Mind, received from those two Flowers, were exactly the same, with the Ideas in other Men's Minds."
Locke composed this counterfactual as part of his effort to show that “tertiary” properties (the properties of mental “ideas”) were different from secondary (the causal properties of the object of perception to cause ideas) and primary (the physical properties of the objects themselves). This was the property dualism repudiated by Berkeley and Hume. In the 20th century the inverted spectrum has had a strong career as a demonstration of the failure of functionalism to handle qualitative properties and, more to the point, as a supposed demonstration that there are such properties (in substance this is very much the same as Locke’s original application).
Imagine someone whose color spectrum was inverted (the “invert”): where normal people saw red, the invert saw blue, where blue, red. Such a person, raised among normal, English-speaking people, would be functionally indistinguishable from anyone else: asked to go out to the car and get the blue bag, say, they would perform this task exactly as anyone else would. Neither they nor anyone else would have any way of knowing that the invert’s experience of seeing the blue surface of the bag was the same experience that everyone else had when they saw a red surface, since the invert, like everyone else, would refer to such a surface as "blue." Since the invert would be functionally identical to a normal person, a functionalist is committed to the position that there is nothing different about their mental state. But (the argument goes) of course there is something different about their mental state: the quale, or phenomenal quality of the experience, is different. Thus functionalism is false.
Wittgenstein argues that the absent qualia argument demonstrates just the opposite of what the friend of qualia claims: since it is not even in principle possible for public language (the only kind of language there is) to pick out private sensations, phenomenal properties are not a problem for operationalist approaches. No theory of mind (or science of mind or description of mind) will ever include any actual discussion of the specific quality of any specific private sensations, because they cannot be discussed. As for the alleged discussion of phenomenal experience we find in philosophy, this is an instance of confusing mention with use - just as one can mention “all sentences that have never been expressed,” but cannot cite one. Outside of (misguided) philosophical conversation there is no context for use of indexically subjective language such as “blue-for-me” as opposed to the intersubjective “blue” which, like all words, necessarily has public criteria for appropriate contexts of use. This is why even the very best of the phenomenologists (Husserl, Merleau-Ponty) never seem to get beyond a sort of “Prolegomena to Some Future Actual Practice of Phenomenology”: after the manifesto there is nothing more that can be said.
So far this is a version of what I am calling the soft argument: the argument is about language, not about the ontological furniture of the world. It has conclusions that perhaps all non-philosophers would find definitive. Qualia do not constitute any sort of obstacle to the naturalization of psychology from the point of view of the scientist because science never could be expected to go beyond the limits imposed on language by its public nature in the first place. Nor is there anything inadequate about our ordinary, colloquial speech about qualitative experience (about, that is, the flavor of the sauce or the hue of the sunset), notwithstanding our individuality, for the same reason. But what about that ontological furniture?
The metaphysical argument will be about the identification of consciousness with experience. The Kantian will say (confining ourselves to the terms of the present discussion) that consciousness is a necessary precondition for the possibility of experience, hence not identical to it. This argument might gain some traction if we concede to the Kantian the point that “experiencing” an object entails bringing the object under a concept (although see the discussion of Kant in Chapter Two), but that very distinction between sensation and perception in the case of objects itself entails that sensation (phenomenal experience) is something prior to the formation of a Kantian “representation.” (If the reader is thinking of Aristotle’s nous at this point I beg your indulgence until Chapter Four.)
I have been using the word “consciousness” as synonymous with phenomenal experience, but the word is also sometimes used in regard to intentional states. Used in its intentional sense, to be “conscious” of an object is, on the traditional view, to form a representation of it or, on the view that I advocate, to be in some sort of relationship to it. But it is incoherent to say that to be conscious of pain, say, is to form a representation of pain or to be in a relationship to pain. In its phenomenal sense the word “consciousness” just refers to the sum of phenomenal experience: pain is a constituent of consciousness, not one of its objects. I may be intentionally conscious of pain at some higher level of psychological organization (one that can be picked out with operational criteria), but it makes no sense to say that I have to form a representation of pain or be in a relationship to pain to have an experience of pain.
The same is true for color qualia or for any qualitative experience. Kripke famously pointed out that the word “pain” just refers to the sensation of pain. “Blue,” in its phenomenal sense, just refers to the sensation of blue. It is one of the components of experience, not some object of experience. Experience is qualia; qualia are experience. So (to get back to the supposed metaphysical implications of the inverted spectrum) it makes no sense to say that experience has properties. Only the objects of experience have properties.
Where this leaves us is at the point of distinction between Frank Jackson’s “What Mary Didn’t Know” essay and Thomas Nagel’s equally famous essay “What is it Like to Be a Bat?” In that article Nagel makes an argument very close to but not identical to Jackson’s. No matter how much we come to know (physiologically) about the echolocation organ of the bat, Nagel argues, we will never know “what it is like” to experience the world in the way the bat does. The difference between Nagel and Jackson is that, granting the present argument that (qualitative) experience itself has no properties, experience itself does not constitute any sort of information (experience is rather the ground of information). We can concede that different conscious beings have different experiences without conceding that this entails any ontological implications.
Some writers have argued that the invert cannot, in fact, be conceived (Douglas Hofstadter holds this view, for example). I do not take that position. Since I do not see how to refute the Wittgenstein/Buddhism non-duality version of “solipsism,” I have no motive to try to prove that we do, in fact, grasp the qualitative nature of the experiences of others. Even if we could do so we could not express this “grasp” linguistically. However there is a different application of the “absent qualia” argument, one that holds that we can conceive of beings functionally (“behaviorally” is a more appropriate word here) equivalent to humans that have no qualitative experiences: “zombies.” David Chalmers’ entire argument for qualia-matter dualism hangs on this claim. I do not believe that we can “conceive” of any such thing.