The primary commandment applies to the person who is being circumcised. The commandment is that any Jewish male must be circumcised at eight days old. The responsibility of the father is to ensure that the commandment is carried out. If the father cannot do so, then the responsibility moves to other Jewish members of the family.
For example Rambam Milah Chapter 1:1
Circumcision is a positive mitzvah [whose lack of fulfillment] is
punishable by karet, as [Genesis 17:14] states: "And an uncircumcised
male who does not circumcise his foreskin - this soul will be cut off
from his people."
A father is commanded to circumcise his son, and a master, his slaves.
This applies both to those who are born in his home and to those
purchased by him. If the father or the master transgressed and did not
circumcise them, he negated the fulfillment of a positive commandment.
He is not, however, punished by karet, for karet is incurred only by
the uncircumcised person himself. The court is obligated to circumcise
that son or slave at the proper time and should not leave an
uncircumcised male among the Jewish people or their slaves.
Doesn't HaShem teach one should get his son circumcised?
The basic answer is not exactly. As the Rambam says, Hashem commands that one should arrange for his son who is required to be circumcised to get a bris milah. If he fails it falls to the rest of the family or the court.
Halachically, the child of a non-Jewish mother is not related to the (Jewish) father and no such responsibility exists.
If the child is not Jewish, then he is not required to carry out any of the 613 commandments that are not also part of the Sheva Mitzvos Bnai Noach (7 commandment categories non-Jews are subject to). As a result, there is no responsibility for anyone to arrange for a (meaningless) "ceremony". It is only a surgical procedure as with any other non-Jew. If, when the child grows up, he decides to convert, then he would be required to undergo circumcision.
An analogy would be that a non-Jew does not have to eat kosher food and would not be considered as having fulfilled the mitzvah even if he does. Nor would it be a violation of anything if he eats non-kosher food.