He has 3 kicks:
qcf + kick
Used in combos or after a c.RH much like fireball. This is the most useful of the 3.
toward to down + kick
Not much use that I know of.
hcf + kick
This is like Ryu’s s.RH, but it’s faster and generally not as useful.
If you hold the kick button after any of these attacks, it becomes an overhead, which can be useful sometimes.
Using them by accident is a problem everyone has at first. You just need to focus more on doing the motions for the moves you want “cleanly” if that makes sense.