The quick answer: Sorry, this is not yet possible.
We receive requests on occasion where users have limited public IP space and have worked around that by placing different servers behind the same IP, but using different ports. For example, Server A might have the IP 1.2.3.4 with a service on port 81. Server B might have the same IP of 1.2.3.4 with the service on port 82. Our customer would like to use a single public cloud IP to proxy and load balance port 80 traffic to both of those servers, performing Port Address Translation (PAT) on the load balancer dynamically, so each server can continue to use the same IP but different ports.
Sadly, our hardware load balancers can only do static one-to-one PAT. So while it is true that we can change the public port from 80 on the client side to any other port you want on the server side without the client knowing, all servers must respond on that same port. This is true for servers that are all active as well as servers that are passive/failover only.
We’re working on this concept, including the ability to fail from one port to another on the same IP all while keeping the public facing IP the same. But it will probably be a while before we’re ready with this concept.