A .vap file is a text file with XML markup, so the size is determined purely by the amount of text it contains.
If you have a lot of commands, phrase variations, actions per command, long inline functions, etc...that all makes the files bigger.
As the file is UTF-8 encoded, which is part of the Unicode standard, which characters are used also make a minor difference in the file size as certain characters can use up to four bytes(instead of one, for regular character like 'a').
However, unless you're having issues importing the file, the file size shouldn't be an issue to VoiceAttack itself.
If you find importing does hang, you could split up your profile by exporting to multiple .vap files, each containing a number of commands, so you can import one as the profile, and the rest as commands.
Do keep in mind that referenced(directly, not by-name) commands must be imported together or their link will break.