Limit connection to all but me

Hello.

I am running in a big network, with only one WSUS server. Computers
from remote locations are connecting to this server to download
updates, on the WAN (slow) via the webserver.


I have set a speed limit on the IIS configuration, and also a
connection limit to go with it.


Everything go well, in a slow motion, but the problem is that I cannot
access my server throught the web interface because of the connections
limit.


Is there a way to keep the limit settings, for everyone but the admins?

Some kind of white allow list or something.


Thanks.