Yes kidneys are kosher.
A possible reason they are considered problematic is because they might be viewed as having ribui dam - an abundance of blood, due to their appearance. This would cause issues with the salting process. See Shulchan Aruch Yora De'ah 73 for details on this, under the laws of liver1.
In Siman 74:1, the Mechaber and Rama discuss kidneys and both permit them. The Rama mentions there are some who have a custom to not cook the kidneys, and the Beit Yosef brings the Rokeach as forbidding them. The Darchei Moshe also says, as a result, that lechatchila, they should not be cooked2.
So in addition to Avishai's reason (see also SA Y'D 105:8), this may also be a reason you don't see kidneys in some butchers.
1 - To summarise, there is a gemara in Chullin 110B discussing how to prepare a liver for cooking. There is a principle of כבד אוסרת ואינה נאסרת - liver makes other pieces assur but does not itself become assur (due to its abundance of blood). Abaye asks if liver will assur other meat it is cooked with. Rabbeinu Tam understands his question to be "given that the proper way to kasher meat is by salting, but with a piece that is ribui dam, one shouldn't salt it but use another method, did the Rabbanim worry that by allowing liver in general, people might think you can prepare any meat without salting?". Rashi takes it further and says that Abaye is actually questioning if melicha works at all for these pieces (following the Rosh).
2 - I would be very interested to see if there is any connection with these opinions and a kabbalistic idea regarding emotions etc. like you ask, but haven't found anything.