Is the Indian Caste System a form of Racism?

I've only read a few articles on the philosophy of race and critical race theory so I'm not sure about this area of philosophy but it seems that the question of whether the caste system of India (actually, there are many kinds of caste systems in practice today in India but I elide on this unnecessary complication) ought to be classified as a racist institution needs to be seriously addressed. It would obviously have large implications morally and legally for a large swath of humanity if it is deemed such. I don't know if the question has been addressed in the philosophical community but from what I have cursorily read about the caste system from the wikipedia page, the objections to such a classification (usually the one's from Indian sociologists) are quite lame.

The objection from these objectors to the racism charge is that castes are not races and so any discrimination among castes cannot be construed as racism or racial discrimination. Because they say that members of different castes are of the same race, the discrimination lower castes face, though may be repugnant, is still not racial discrimination.

This line of reasoning is based on a misunderstand how contemporary race theorists understand racism (see here for one example of a contemporary understanding of racism). Nearly all contemporary race theorists AFAIK do not understand racism to be defined by mistreatment necessarily from different races. They have good reason for this. For one, race is notoriously difficult to define and thus what constitutes discrimination on that basis is not clear. Furthermore, members of the same race may be racist towards each other. In other words, race component in racism is in the eyes of the beholder, or more specifically, the eyes of the perpetrators of discrimination or the negative intentional, racial attitudes. If members of a group sees racial differences in another group and treats them or have attitudes towards them that are discriminatory then that would be racist even if those differences are illusory and not in anyway biologically, physically instantiated. Also, remember that someone can be racist towards their own race.

Consider the racism that was met out to the Jews during Nazi Germany. That was racism as most race experts would agree though most German Jews are indistinguishable physically (and cluster closely even genetically) from “Aryan” Germans. The racial differences were merely in the eyes of the Nazis, i.e., they were wholly illusory. The Jews (among many other victims such as Gypsies and Slavs) were treated as somehow fundamentally different by their “racial essence” their “blood” according to their “Aryan” counterparts in society. That viewpoint from the perspective of the perpetrators is what counts, not the actual “race” (whatever that is) of the victim vis a vis the perpetrators. Also consider the genocide in Rwanda. The Tutsi and Hutu are physically (and linguistically, genetically and culturally) almost indistinguishable from each other. However the genocides they have committed against each other (in the 70s and 90s) are indeed racial in nature.

It was racial because the perpetrators each time saw their enemies, the people they would go on to exterminate as somehow fundamentally different in essence from them. That essence can be passed down from one generation to the next and is, for the most part, viewed as immutable. That sounds rather like the ideology behind caste system! 

Objectors to the caste-as-racism theory may object that though “actual race” is irrelevant to racism, the caste system is still not racist because the categories of races are a western categorization. But descriptively, there may be little to no difference between how the caste system structures society along the same lines as how racism structures society (i.e. based on perceived inherent abilities of groups that are passed down through generations). One may respond that any semantic hairsplitting is just an apologia for such oppressive practices as the caste system inherently may be.

One may wonder if different intuitions that separates a descriptivist versus a direct reference theory of language is at the heart of this issue. Descriptivists may say, “hey, if it walks like a duck, quacks like duck, it is a duck!” Thus because the caste system has all the qualitative hallmarks of a racialist institution (walking and quacking along perceived similar lines as racism), it is a racist institution. But direct reference theorists such as a causal reference theorist may say that the word “racism” has certain cultural connections to how they have been actually used in the west to refer to specific institutions and do not reference to other institutions no matter how qualitatively similar.

This brings up the issue of whether even if the direct reference theorists (or those who share that kind of intuition) are correct about the caste system, that would only be an empty victory for the caste system apologists because such a system though not “de jure” racist may be justifiably then deemed “de facto” racist especially if the institutions are extremely harmful like they are in actual racism (and there seems to be wide agreement that the caste system is extremely harmful to members of certain lower castes).

So if India has such widespread institutions of racism (or something very much like it) then it ought to motivate the civilized world to action in abolishing such oppressive institutions. India has already begun since the 1950s policies which aim at ameliorating the negative effects of caste discrimination. But it is still a largely caste society; sentiments towards different caste divisions still run deep in society and even if there are no more negative ostensible effects from the caste system, it may still be inherently oppressive system (or set of systems) and worth abolishing completely. Indeed the UN has already denounced Indian caste system as racism before.