Having VoiceAttack's listening state follow the inverted value of other application's listening states, as Gary proposes, can be done using commands rather than or in addition to the built in option, if you need to trigger that off of more than one hotkey on the same device type (as you can't, for example, have two different mouse buttons assigned to the "Mouse Click Recognition" feature independently)
If you'd rather stick with the behavior you were initially looking for, this command should accomplish that:
Begin Text Compare : [{STATE_LISTENING}] Equals '1'
Stop VoiceAttack listening
Start Loop While : Forward Mouse Button Is Pressed
End Loop
Start VoiceAttack listening
End Condition
Set it up to trigger off your mouse button, as you did with the hotkey feature before, and if listening is already disabled, the command will simply do nothing (thus keeping listening disabled, even after you release the button); If listening is enabled, it will be disabled while the button is down, and re-enabled when it is released.