Michael Sandler, you have 2 questions:
- Do 10 have to hear kadish? (No)
- Do 10 have to say Amen to kadish? (No)
(OK, maybe that progression should have been the other way around.)
A "davar shebekedusha" is a matter that requires a special presence of G-d (from the pasuk of Hashem nitzav ba'adas e-l we learn that this is 10 adult men). Once Hashem's special presence is assumed, the required kaddish may be said regardless of whether people are listening. The Aruch haShulchan (OC 55:12) uses the above idea to justify why one who can speak but not hear may be counted as one of the 10.
In contrast, the Shulchan Aruch (OC 124:4) states that by the leader's repetition of tefila, if 9 are not paying attention, the brachos are almost as if in vain. I would guess this is because kaddish is a requirement even for 1 person, but there is a technicality that it can only be said among 10 (see end of Aruch haShulchan 55:9). The repetition of tefila is created only through having 10 people, so ten congregants have an equal obligation to say the repetition, a requirement fulfilled by the appointed leader- the shatz.
Note that the Shuchan Aruch's requirement by the repetition (124:4) just says they have to pay attention. Saying amen in not a requirement of the congregation, but a personal requirement that when we hear a bracha we should say amen (see Aruch haShulchan 124:11). Even if one is fulfilling his obligation through the bracha and the amen becomes more pertinent, we say shomea k'oneh (listening is like answering).