In a democratic state like Athens, all citizens had voting rights, but not all inhabitants were citizens. Citizenship was limited in most poleis to free adult males who had at least one parent from the city in question. This means that in a city like Athens, where the majority of the population was slaves, only a small percentage of the inhabitants were citizens.
There is a thorough discussion of the concept of Greek citizenship in Book III of Aristotle’s “Politics.”