Mita (taxation by labor) is not that weird really. Think of the most basic form of exchange in a barter society. You show up to a market with potatoes you've grown and leave with bread, pots, etc.. Well, what if you don't have potatoes and want bread? You could offer to chop wood for the baker. Work can be bartered just as easily as goods, and credit can be accrued. We do this all the time, even today. Say your sister helps you move. When she next moves, you're going to feel like a scumbag if you're not there to help her! Some campesino villages in Peru still barter labor this way on a rather large scale. Mita in such communities is a bit like a socially enforced volunteer spirit. People are socially obliged to do things for their community. The Inca empire just took that to a completely different scale!

The unusual thing about the Inca is that they preferred to tax labor even from those who produced goods. Potato farmers wouldn't tithe potatoes. They'd spend some of their time working government lands that grew potatoes. The fact that farmers were feeding themselves and the state out of different fields probably had some rather interesting effects.