On my setup, the audio out is via a Griffin iMic (USB digital audio). It's selected as the output device, and everything works through it (music plays, etc).
Hitting the mute key kills the startup chime through the built-in speaker in the G4 without affecting audio out through the iMic.
Macman is correct, you could use the mute or volume controls in your Sound Control panel when "Built-in Audio Controller" is active and select your USB device next and adjust the settings again for other sound output (music, etc).
I found it very cool that I didn't have to go to that control panel, the chime would be killed, and music played through my speakers just as before (if you hit the keyboard mute).
If you have no external speakers, hitting the mute key kills the startup chime, but you need to hit it again (un-mute it) to play music, etc through the built-in speaker(s).
If you have external speakers connected via the headphone (analog) out, it's the same procedure as when you use built-in speakers. If there's a useable volume control (that you can get to easily) you probably won't bother with the mute key trick, as you can just turn it down on the speaker to whatever volume you can tolerate.
In general, I have all "system" sounds turned off, i.e: "Alert Volume" to the left [off] and no checkboxes active for "user interface sound effects" or "feedback volume when audio keys pressed". All other volume sliders in control panels or applications are set to maximum [to the right all the way].
The reason for this is that any computer based volume reduction is done by "throwing away" bits of digital data. In other words, reducing the volume in software while playing a CD, for example, would mean perhaps 14-bit resolution of a 16-bit signal. For music to sound best, you need all them bits.
By sending a maximum level full-resolution signal to your external speakers or hi-fi system and then using the analog volume control to adjust sound levels, you are hearing all available signal fidelity down to the noise floor at any volume setting.
The built-in audio controller MAY be 24-bit resolution with Jag (it wasn't on previous releases, but it apparently is capeable) in which case you could reduce the volume in the digital domain by the equivalent of 8 bits and still get a CD's 16-bit resolution. However, I have found no definitive documentation to suggest that 24-bit resolution is enabled. Still, the "extra loud" startup chime in Jag hints at this.