SMTP can be setup and used with sendmail from the CLI on Mac OS X Client.
The easiest way of course is to setup Mac OS X Server and use their built in Mail server but there are problems and limitations in features to it.
If you want, there is also a really great mail server package called exim.
See the links here on how to set them up.
Installing Exim
Spam Assassin
Integrated Spam and Virus Control
Keep in mind that these articles are written on how to install them on Mac OS X Server and not Mac OS X Client. They can be compiled and installed for Mac OS X Client, however some of the config files may be a little different.
You may also want to check out
Eudora Mail Server for Mac OS X, however you have to pay for it and lets all be honest here....free is better