I believe it would depend on which function you are using to display the category list. Try these pages in the codex:

http://codex.wordpress.org/Template_Tags/wp_list_cats

or

http://codex.wordpress.org/Template_Tags/list_cats

Either way, look for the optioncount parameter and set it to 0 or False depending on the function used.