Will depend greatly on the network connection you have between the two.
High latency, highly utilized sub T1 link? Probably not a good idea. 80 ms, lightly used link? No problem.
The official specs are less than 300 ms.
However, I would certainly install the Microsoft Message Queuing (MSMQ) on the remote poller. If you lose your link, it will continue to hold your data for a short time.
Also, to be especially aware of overloading that poller, since it is remote located.
Credit to Pacetti for his "Stupid Poller Tricks" -- here is a good guide for you:
In general, you can physically put all the pieces in separate locations. The big issue is the connectivity between them. For instance, we have additional pollers in separate states with devices all around the country. It works for us, but the communications between the pollers and DB/Main NPM Poller all ride on our national backbone, so we have good connectivity. This issue is further compounded by how much is being polled by each additional poller and how frequently. However, in my experience, most of the issue comes down to the connectivity.