Retribution and virtue theory

There are many advantages to a retributivist account of punishment as just desert. There are also advantages all around from a virtue framework when it comes to distributive justice relative to other approaches but I don't want to talk about that now. Here's one serious weakness.

Virtue theory says that someone out to be punished when they have certain vices that are deserving of punishment. That central claim appeals to our retributivist intuitions that bad people simply deserve certain ends that others do not (and symmetrically, good people deserve other better ends, etc). However, we can imagine at least in some possible scenario that some possible being will have some vice(s) worthy of punishment on that account but this creature will have the vice(s) in virtue of some law of nature that makes it impossible in that world to rehabilitate. That being is by the nature of that world, recalcitrant. Now imagine that that creature has the vice(s) he has not by some choice of his own but because he had been born with it and he has never committed any act on his vice. We can more plausibly imagine this last clause by imagining that his environment had never given him the opportunity to act on his vices. Now because he has never acted on his vice, he has not committed any immoral acts. But a virtue retributivist would say that he deserves to be continually punished, for as long as he is in existence. If he is an immortal, that means being punished forever for no immoral act. That would seem unfair and cruel.