the person's human intellect is granted free choice. His soul, therefore, has little influence on his physical body's actions.
The question seems to assume that the soul is an outside force, whereas the intellect is the actual self.
Rabbi Bentzion Shafier defines "The 'I' who is talking to you" as a combination of the animal soul, and the G-dly soul.
This is why a person can simultaneously desire to sin, and desire not to sin. The animal soul provides the drive for the pleasure of doing the sin, and the G-dly soul provides the drive for wanting to do the right thing, and serve G-d properly.
If the animal soul wins out on a given trial, it was still the essential human being - the "I" - who chose to sin. As such, a person can deserve punishment for sinning. Of course, G-d doesn't want to punish us:
Cast away from yourselves all your transgressions whereby you have transgressed, and make yourselves a new heart and a new spirit, and why should you die, O house of Israel! For I do not desire the death of him who dies, says the Lord G-d: so turn away and live!" (Ezekiel 18:31-32)
I'll see if I can find a shorter source, but this concept is explained very well here