Of course we generally think that it was conquered by King David. But in Shoftim 1:8 it says that Jerusalem was captured by the tribe of Judah soon after the death of Joshua. So how do we understand this contradiction?
And afterward the children of Judah went down to fight against the Canaanites; the inhabitants of the mountains, of the South, and of the lowland.
Shoftim 1:21 shows that the part of the city under the Jebusites was left alone. As Rashi says:
the Jebusites that inhabited Jerusalem: There was a section in Jerusalem called Jebuse, which was populated by the descendants of Abimelech who were not driven out because of the oath [which Abraham had sworn], until the coming of David. This was because his grandson was still alive and Abraham had sworn [Gen. 21:23] to him, his son, and grandson.
That is the section that remained until the time of David Hamelech. Also see Rashi on Yehoshua 15:63
As for the Jebusites the inhabitants of Jerusalem, the children of Judah could not drive them out; but the Jebusites dwelt with the children of Judah in Jerusalem to this day.
the children of Judah could not drive them out: We learned in Sifrei : Rabbi Joshua the son of Korha says: They really could, but they were not permitted, because of the oath which Abraham had sworn to Abimelech. Now these Jebusites were not of the Jebusite nation, but the Tower of David which was in Jerusalem, was called Jebus, and the inhabitants of that section were of the Philistines. And when the children of Judah conquered Jerusalem, they did not drive out the inhabitants of that section.