Even if it is accepted that polytheists alone are not to be allowed to come in the vicinity of the Ka'bah, the argument remains unanswered. Because at the end of the day, it would still be us humans who would be judging others as 'Mushrikun'. For example, we would certainly suggest that Hindus, for one, are polytheists. But the matter is not as simple. We don't know what is in the hearts of men. Many Hindus claim that they believe in one God alone, and the "statues" are there only to help them visualise some entity - no more, no less. What decision do we make in such cases, and what approach should have, at large?
The basic thing in this regard is the definition of the word Mushrik. As I wrote in my response the word implies those who ascribe themselves to the religion of shirk. This entails that those who due to any misunderstanding indulge in Shirk or do not accept that they are polytheists and present some kind of interpretation and explanation to their involvement in polytheism as some of the Hindu sects and Christians do cannot be called Mushrik. Therefore we see that though the Holy Qur'an says that some of Jews and Christians were involved in polytheistic beliefs yet it did not subject them to the punishment prescribed for the polytheists who had rejected the Messenger of Allah. Therefore the matter does not go undecided and merely subject to our opinion. Rather the final authority in this regard is the person himself.
Keeping the foregoing explanation in mind, it should be clear that there is no subjectivity involved in determining whether or not a particular group is Mushrik. If a group or a person ascribes partners to God and accepts that he believes that God has associates and partners, then such a person is a self-proclaimed Mushrik.