There are generally two reasons why this could happen.  First off, ProTools unfortunately gives that message any time our authorization process doesn't complete for some reason.  So please be sure to click "I'm online" and enter your login info when the Activation prompt comes up during startup. If you click "Quit" instead, you'll get that "not a valid 64 bit plugin" message. Is that what you did?  If not, we'll move on to the second possible reason...  It's possible that the plugin somehow did not install properly, and the aax file became corrupted.  If that's the case, you can just manually uninstall it and then reinstall.  Just go to the Finder, click on the "Go" menu, then click on "Go to folder..." and enter this file path: /Library/Application Support/Avid/Audio/Plug-Ins
A window will open with all your plug-ins, and you can throw away the plugin file.  Then try installing it again.