Rotating Residential 2024 Spring
1. Vendors and dimensions
I've chosen the most popular vendors + those who have changed.
- BR - BrightData
- IR - IPRoyal
- OX - Oxylabs
- SM - Smartproxy
Dimentions(comparison criteria)
When looking for Rotating Residential proxies usually people search for:
- Good GEO coverage - places clients can't cover with ISP and DC(there is no DCs near each city) and the big enough amount of IPs in the most popular countries(most developed consumer markets: the US, GB, DE).
- Success Rate - whats the ratios of successful calls? In the case RR(Rotating Resi) it depends on three aspects: quality of your connection with the public internet, vendors' infra(networks, redundancy, scalability, etc), resi devices connectivity, and quality.
- Features - skilled proxy clients can do a lot to increase success rate(TCP, TLS, WEB Browser fingerprints, DNS, WEB RTC leak prevention, etc). But some characteristics in dependent on the vendor and they will be covered in this article.
- Associate UDP(UDP over Socks5) - Ability to tunnel traffic with UDP trasport protocol.
- Ability to choose where to lookup the target domain name on a proxy server or on a residential device.
- Make the session sticky and control its length.
- Ability to protect a client from unwanted ip-addr rotation.
2. General market trends
- The price per megabyte is steadily going down.
- On the one hand, entry of new vendors into the market is difficult due to the constant increase in the cost of attracting new users and very loud promises from large vendors - tens of millions of unique IPs for key countries.
- On the other hand, you can see freshmen constantly appearing and reselling the pool of other vendors. Thus, the client does not always know whose pull he is currently using and can sometimes avoid detailed verification and get a lower price.
3. Theoretical experiment EU -> US
Technical details of the theoretical experiment. We need to build the lab:
-
A proxy client that can provide you a detailed information regarding the latency(its structure: connection to the proxy server, TLS handshake, TTFB as minimum). To choose or create a suitable target that will tell you the client's ip-addr and and the same time will have a predictable response time and performance capacity to not create noise in the measurements.
-
To use an effective scheduler to create a predictable load. it should not have a strong deviation and should not accumulate errors. Good examples are Phantom and Pandora. I think choosing ProxyChick as the proxy client and 100 RPS in 30min time period from EU GEO is a good enough conditions.
-
Of course, we need to repeat the test multiple times to exclude "underground knocks"(external factors independent of us that can create noise). I suggest removing a few of the vest and the word repetitions and taking avg. or most close to the median.
-
To measure latency, Success Rate(SR), and unique ip-addr count - it would be enough to produce 30min 100rps load from the proxy client to proxy service.
3.1 Success Rate
| Vendor | Success Rate % | Amount (ok / error) | Errors |
|---|---|---|---|
| Oxylabs | 98.51 | 177253 / 2680 | TLS 944; EOF 606; TLS handshake TO 526; other 604 |
| Smartproxy | 98.44 | 177089 / 2801 | EOF 1005; TLS 805; TLS handshake TO 504; other 478 |
| BrightData | 99.98 | 179946 / 29 | EOF 13; Internal Server Error 11; TLS handshake TO 4; other 1 |
| IProyal | 98.90 | 177825 / 1972 | EOF 1447; TLS 285; TLS handshake TO 212; other 28 |
3.2 Amount of unique ip-addrs
How much you can get in total in 30min with 100 RPS?
How much can you accumulate by each minute of the test?
What is the ratio of unique addresses to all received each second?
Unique ip-addr intersection between vendors
| BrightData | Oxylabs | Smartproxy | IProyal | |
|---|---|---|---|---|
| BrightData | - | 0 | 0 | 0 |
| Oxylabs | 0 | - | 29507 | 1212 |
| Smartproxy | 0 | 29507 | - | 1224 |
| IProyal | 0 | 1212 | 1224 | - |
For example, if vendor A has 4 unique ip-addr 10.20.30.40 10.20.30.41 10.20.30.42 10.20.30.43
and vendor B has 5 unique ip-addr 10.20.30.42 10.20.30.43 10.20.30.50 10.20.30.51 10.20.30.52
in the intersection of column A and row B (and vice versa), we will have 2 cause they are sharing 2 unique ip-addr
3.3. Latency
Tables with detailed latency breakdown
The request processing pipeline is broken down into sections. Their definitions can be found here you can find them in the 1st column. The first row contains percentiles from 50%p - median to 100%p. You can interpret them this way - if TTFB for 90 percentile equals 2091m then means that with a 90% probability moment when the client gets the 1st byte from the target server will fit into 2091 milliseconds.
All the tests were done with the ProxyChick tool. You can reproduce some tests with it.
Oxylabs
| NAME | 50 | 75 | 85 | 90 | 95 | 99 | 100 |
|---|---|---|---|---|---|---|---|
| TTFB | 1151 | 1439 | 1773 | 2091 | 2852 | 5497 | 26552 |
| DNS resolve | 3 | 4 | 5 | 6 | 732 | 4179 | 5029 |
| Connect | 13 | 14 | 16 | 16 | 17 | 23 | 1049 |
| TLSHandshake | 338 | 391 | 428 | 473 | 581 | 1022 | 13752 |
| ProxyResp | 402 | 532 | 695 | 985 | 1680 | 4663 | 7356 |
BrightData
| NAME | 50 | 75 | 85 | 90 | 95 | 99 | 100 |
|---|---|---|---|---|---|---|---|
| TTFB | 781 | 1065 | 1221 | 1705 | 2670 | 5213 | 10697 |
| DNS resolve | 3 | 4 | 5 | 8 | 878 | 4197 | 5152 |
| Connect | 2 | 2 | 2 | 3 | 8 | 19 | 101 |
| TLSHandshake | 234 | 333 | 340 | 348 | 356 | 418 | 6603 |
| ProxyResp | 285 | 382 | 627 | 1133 | 2100 | 4674 | 8700 |
Smartproxy
| NAME | 50 | 75 | 85 | 90 | 95 | 99 | 100 |
|---|---|---|---|---|---|---|---|
| TTFB | 1421 | 2041 | 2524 | 3120 | 4519 | 6380 | 21350 |
| DNS resolve | 3 | 4 | 4 | 6 | 766 | 4190 | 5178 |
| Connect | 2 | 2 | 2 | 3 | 6 | 18 | 111 |
| TLSHandshake | 389 | 481 | 588 | 681 | 849 | 1870 | 14781 |
| ProxyResp | 524 | 782 | 1397 | 1670 | 3182 | 5168 | 7649 |
IProyal
| NAME | 50 | 75 | 85 | 90 | 95 | 99 | 100 |
|---|---|---|---|---|---|---|---|
| TTFB | 1525 | 1954 | 2314 | 2722 | 5078 | 9028 | 38239 |
| DNS resolve | 3 | 3 | 3 | 4 | 6 | 3639 | 5030 |
| Connect | 2 | 2 | 2 | 2 | 3 | 16 | 1032 |
| TLSHandshake | 310 | 393 | 471 | 578 | 797 | 2006 | 14413 |
| ProxyResp | 857 | 1117 | 1337 | 1621 | 3717 | 6564 | 26713 |
4. Features
GEO of points of presence
Why does this matter?
Talking about content delivery, usually, CDN providers try to put their servers close to the clients to eliminate the problem of the last mile - unstable or low-quality connection of your ISP or ISP that your ISP is connected to.
In the case of residential proxies, there is one more reason to care about it. If the provider has only one point of presence and all resi devices are connected there - it might create a significant latency increase. So placing servers close to the resi nodes and letting clients connect locally(w/o extra hope from one continent to another) might reduce RTT and provide lower latency to the proxy user.
Example of bad GEO of the point of presence(proxy servers) for the client from Asia. At the same time, it's good enough for the client from Europe.
Example of good GEO for the client from Asia
| Vendor | Europe | North and South America | Asia |
|---|---|---|---|
| Oxylabs | Germany / Falkenstein | Canada / Toronto | Singapore / Singapore |
| Smartproxy | Germany / Falkenstein | Canada / Toronto | Singapore / Singapore |
| BrightData | Germany / Frankfurt | US / Ashburn,Wilmington | Germany / Frankfurt |
| IProyal | NL,DE,UK | US / LA, Phila, Ashburn | NL,DE,UK |
As you can see mentioned vendors have points of presence in all major regions. Some of them also have dedicated points for India and China. If the vendor routing traffic is efficient way it should be good. for clients latency. Pay attention to your proxy client DNS settings especially cash cause in all cases client traffic routed with GEO DNS.
Associate UDP
Is the proxy provider capable to tunnel UDP traffic over Socks5 ?
- Oxylabs - Do not support.
- Smartproxy - Unknown
- BrightData - Unknown
- IProyal - Only Static Residential, Sneaker DC, and Datacenter proxies products support it but not Rotating Resi
Configurable DNS lookup
Is it possible to configure a DNS point in the pipeline where to make a DNS lookup? By default, the client can do it himself and use IP for the target address or delegate domain name resolution to the resi device. Some providers offer more options like Proxy Server lookup or customer DNS server lookup.
- Oxylabs - no
- Smartproxy - Unknown
- BrightData - Unknown
- IProyal - no
Session stickiness and length
Nowadays this is a banal feature to use the same ip-addr for new requests and be able to define the length of this association.
Session killswitch
Pretty useful for account management, it might be that you can compromise your account if ip-addr is rotating too often. So my vendors offer solutions - to get defined ip-addr or error. You will need explicit action to restart the session and get rid of the error, but in return, you will receive a guarantee that the target will see a new ip-addr in case of rotation.
- Oxylabs - no
- Smartproxy - Unknown
- BrightData - Unknown
- IProyal - no