I Tried 100s of Free Proxies, Here's the results.
Charles Martinez
- 2024-08-13 17:08
- 2 min read
Challenges of Using Free Proxies
The three main problems that you're going to find by using free proxies like this is the uptime; they don't stay active for that long, the speed; they are generally as you've seen quite slow, it's taking a long time to actually get that response back, and the third one is something that's a bit more nefarious, which I think you need to be aware of. Although I haven't encountered this too much myself, if you're watching this thinking that you need proxies for your project and you don't want to go through the hassle and the egg that I am here with these free ones, you're going to want to check out the sponsor of today's video, IP Royal.
Testing Proxies for Web Scraping
So what we're doing is I have written a retry function decorator that basically just says if it fails we're gonna wait two seconds and then retry the same request again, and we are basically using the proxy list that I've created from the other file to actually try to pull the product data from this URL here. This is again as the same Shopify store that we were testing against.
Risks of Using Free Proxies
Now if that process sounds familiar, this is exactly what we've just been doing using these free proxies. So our entire trust is in whoever's put these free proxies online to actually not be sending us loads of stuff that we really don't want. In some cases it might just be access advertising, which we don't want, we don't want to see, but in other more extreme cases it can be malware and viruses. So you just have to be really careful about where you're getting this from. This is the main reason why I tend to stay away from free proxies because their uptime is bad, they're slow and unreliable, and there's that potential for malware there.
FAQ
Q: What are the challenges of using free proxies?
A: The three main problems that you're going to find by using free proxies like this is the uptime; they don't stay active for that long, the speed; they are generally as you've seen quite slow, it's taking a long time to actually get that response back, and the third one is something that's a bit more nefarious, which I think you need to be aware of. Although I haven't encountered this too much myself, if you're watching this thinking that you need proxies for your project and you don't want to go through the hassle and the egg that I am here with these free ones, you're going to want to check out the sponsor of today's video, IP Royal.
Q: How to test proxies for web scraping?
A: So what we're doing is I have written a retry function decorator that basically just says if it fails we're gonna wait two seconds and then retry the same request again, and we are basically using the proxy list that I've created from the other file to actually try to pull the product data from this URL here. This is again as the same Shopify store that we were testing against.
Q: What are the risks of using free proxies?
A: Now if that process sounds familiar, this is exactly what we've just been doing using these free proxies. So our entire trust is in whoever's put these free proxies online to actually not be sending us loads of stuff that we really don't want. In some cases it might just be access advertising, which we don't want, we don't want to see, but in other more extreme cases it can be malware and viruses. So you just have to be really careful about where you're getting this from. This is the main reason why I tend to stay away from free proxies because their uptime is bad, they're slow and unreliable, and there's that potential for malware there.