You can (and probably should) deploy the same WSUS signing certificate to everyone. The easiest way to set up an environment with multiple WSUS servers is to have the root/upstream WSUS server generate a signing certificate (and corresponding private key), install the certificate on all of the downstream servers and clients, and then publish packages to the root/upstream server only. Once the packages are published to the root/upstream server, they are signed by the WSUS server's private key and then replicated to downstream servers like any other update. When clients contact their WSUS server to get the package, they will use the root/upstream certificate to verify the signature that was generated with the root/upstream server's private key. This is the scenario I would recommend.
If you have multiple WSUS servers in Autonomous Mode (doesn't sound like you do - but for the sake of completion), you will have to either 1) generate multiple WSUS signing certificates for the servers and carefully distribute the certificate to only the clients that point to that WSUS server, or 2) manually export the WSUS certificate *and* private key (e.g., to a PKCS#12 package) and import it on all of the WSUS servers. In the latter case, you can deploy the same certificate to all of the clients.