There is rhyming in Tanach. For example:
Genesis 4:23
וַיֹּאמֶר לֶמֶךְ לְנָשָׁיו: עָדָה וְצִלָּה שְׁמַעַן קוֹלִי \ נְשֵׁי לֶמֶך הַאְזֵנָּה אִמְרָתִי \ כִּי אִישׁ הָרַגְתִּי לְפִצְעִי \ וְיֶלֶד לְחַבֻּרָתִי
Deuteronomy 32:1-2, 6
וְתִשְׁמַע הָאָרֶץ, אִמְרֵי-פִי \ יַעֲרֹף כַּמָּטָר לִקְחִי \ תִּזַּל כַּטַּל אִמְרָתִי
הֲלוֹא-הוּא אָבִיךָ קָּנֶךָ \ הוּא עָשְׂךָ וַיְכֹנְנֶךָ
Proverbs 6:9-10
עַד-מָתַי עָצֵל תִּשְׁכָּב \ מָתַי תָּקוּם מִשְּׁנָתֶךָ \ מְעַט שֵׁנוֹת \ מְעַט תְּנוּמוֹת \ מְעַט חִבֻּק יָדַיִם לִשְׁכָּב
Additional examples can be found in
Ha'Azinu and in V'Zoht ha'Bracha.
[likchi - imrati; k's'irim - v'chirvivim; kanecha - vayechonenecha;
yesovevenhu - y'vonenehu - yitzrenu; tumecha - v'urecha - chasidecha;
etc.] Think of Shirat Devorah (Shoftim 5) [chadashim - she'arim;
mashabim - la'she'arim; lakachu - nilchamu. Also Shmuel Bet 22
(sal'i - um'tzudati - u'mfalti).
Then of course most of Yeshayahu
and in Trei Asar. [e.g. Yeshayahu 1: petza - v'chabura - u'maka tri'a;
3 (ha'avasim - v'hashvisim - v'hasaharonim; ha'machalatzot - v'hama'atafot-
v'hamitpachot). (Source)
The Wikipedia article on Biblical Poetry contains additional information.
Recommended reading: "The Art of Biblical Poetry" by Robert Alter