It's useful to know that the default SSH port is 32022 (not the standard 22). Additionally, the credentials for logging in are different than the LEM Console UI, the username's "cmc" and the password either was set during your activation process or can be set from the virtual appliance console (under "Advanced Configuration" type "appliance" then "password" to change it from the default of "password").
If it's neither of those things, let me know, and I'll suggest some further steps
Thanks Colby! I had the port right, but somewhere I missed that the user name was different, as soon as I tried the cmc user with the password I had set before, it worked like a charm.