The main (and most common) reason to use the 64bit version is if you have text-to-speech voices that don't have a 32bit version (E.G. "Amazon Polly").
Theoretically a 64bit application can use more memory, but if you're running into memory limits with the 32bit version, the practical difference may not be all that significant.
Being 64bit also means that third-party plugins (and their dependencies) need to be 64bit-compatible.
At this stage the 64bit version is still in beta; You can try it if you want to (profiles should carry over; Possibly settings as well if installed to the same directory), but note that the installer for the 64bit version will overwrite that of the 32bit version even when installed to a different directory, so you'd need to remove the latter manually should you ever find a need to (which just means navigating to the folder manually; The uninstallers only remove the folder; Registry information, settings, and profile data are not affected).