We are responsible for our actions

We are responsible for our actions whether we have "FreeWill" or not; because it is we who act and we act out of the character of our nature. If our nature is wholesome we will act virtuously. If our nature is corrupt we will act viciously. Moreover every vicious or virtuous act itself modulates our nature - forming habitual propensities to act similarly in the future.

Responsibility isn't so much about choosing (whatever that means!) to do right or wrong; it is about the manifestation of our interior character: "by their fruits shall ye know them," and "all that is hidden will be revealed."

Punishment ought not to be about vengeance or retribution. Vengeance is reserved to God and God has no use for or interest in it. The only reasonable purpose of punishment is to modulate someone's character so as to make them less vicious and more virtuous. This is called reformation or rehabilitation or moral education or repentance. From this perspective it is easier to justify punishment if there is no "could have done otherwise FreeWill" because one might then expect it to be a more simple matter to train or condition a vicious person's future habitual behaviour by a system of rewards and punishments.