The solution is to use Network Protection (SWG UI > Configuration > Select Appliance > Network Protection). This will apply to all network traffic, so you will have to configure each port and protocol in use.
Below is a very basic example, which will need to be adjusted to your requirements and network configuration.
By this configuration, you will be able to access SWG UI on port 4712 only if your client IP is 10.207.143.215.
Related articles