I read in Real Power on page 47, the following:
“If you are not going upward then you are automatically falling downward.” —Chazal
Does anyone know if this is a Gemara or Medrash etc.?
One place this is mentioned is the Chafetz Chaim's Toras HaBayis (ch. 10). The Chafetz Chaim notes the symbolism in how the angels on the ladder in Ya'akov's dream were either ascending or descending but could not remain static on the same level (B'reishis 28:12).
The Chafetz Chaim generically cites "the holy sefarim" for this interpretation of the verse, but he expands on the concept at length.
Here's my guess from Maseches Avos.
פרק א - משנה יג וּדְלָא מוֹסִיף, יָסֵיף The Bartenura explains
ודלא מוסיף. מי שאינו מוסיף על לימודו: יסיף. יסוף מפיו מה שכבר למד וישכח תלמודו.
Whoever doesn't add to his learning will forget what he already learnt.