David takes over Jerusalem (finally) in Shmuel II, Perek 5. It says that David wants to be rid of the סירך ועורים. This is commonly translated as the blind and the lame. Metzudat David understands this to actually mean that David needed/wanted to get rid of the handicapped people in Jerusalem in order to take it over. Ralbag understands this to mean that they are two statues at the front of the city that make it hard to get in as they each lift and drop their sticks (powered by the aqueduct) (much like mini-golf of today). What does David really mean when he wants to destroy them? He even offers an award to the one who does it. Is it possible he really does want to destroy the handicapped, and if so how to we assimilate that idea with our idea of David?
As you said, Ralbag (and most of the commentaries) understand this to be talking about statues of some kind. (Metzudas David to 5:8 also cites this as a second explanation.) So according to that view of things, David had nothing against the blind and lame people any more than against any of the other Jebusites.
Metzudas David's first explanation (to 5:6), though, is indeed that it means people suffering from these handicaps. However, he explains the mention of them as part of the Jebusites' boast: "Our tower [this is his explanation of צנור in that verse] is so strongly fortified that even if its only remaining defenders were the blind and the lame, you would still be unable to conquer it!" So David's challenge to his men (v. 8) is: "Disprove their claim! First topple the tower while it is still defended by all of the Jebusite army, and only then deal with these insolent Jebusites - including the blind and lame people, all of whom had joined in jeering us."