Does the bar at the bottom of VoiceAttack's main window move when you're not speaking?
This can occur when the speech recognition system is receiving input, as it will wait for the input to finish so it can perform speech recognition on it.
Enabling the "Reject Pending Speech" option on the "Recognition" tab of the VoiceAttack options window should work around that.
If there is background or line noise coming into the speech recognition system, that can affect recognition accuracy as well.
The '"I'm not saying anything but VoiceAttack keeps hearing things"' section of
this topic may have some relevant suggestions.