We have a few of these set up (Oracle Experience Monitor being polled via Agent), but I don't recall any custom setup we had to perform beyond having the Oracle Client installed as per here:
If it's working agentless I'm assuming you've made it that far.
You may want to try alternating the bitness between x64 and x86 or see if a different Oracle Driver will let it connect. Based on the error I'd guess that it's not finding the driver that it's looking for, so I would try Oracle Data Provider .NET vs Managed Oracle Data Provider, etc.
Just to put it out there, as silly as it seems to say if you're working directly with the Oracle server, does it have the necessary drivers/dlls installed the same?