How to Set up Residential Proxies for Scrapy Proxy Middleware

Learn how to set up residential proxies on Scrapy Proxy Middleware

Need more help? Contact our team at [email protected]

Smartproxy integration Scrapy Proxy Middleware

Scrapy Proxy Middleware integration:


  • Open Terminal window.
  • Navigate to the main directory of your project folder using:
    
    cd yourprojectname
    
    
  • Download our proxy middleware using the following command:
  • smartproxy integration Scrapy Proxy Middleware
    curl
    https://raw.githubusercontent.com /Smartproxy/Smartproxy/master/ smartproxy_scrapy_middleware/ smartproxy_auth.py > smartproxy_auth.py


  • You should now see your project folder populated with smartproxy_auth.py file.
  • smartproxy integration Scrapy Proxy Middleware
  • Using a file manager, navigate to your project folder, you should see settings.py file located at the bottom of the directory.
  • Edit the settings.py file using an editor of your choice.
  • smartproxy integration Scrapy Proxy Middleware
  • Add the following properties at the bottom:

  • DOWNLOADER_MIDDLEWARES = { 'scrapy.downloadermiddlewares.httpproxy. HttpProxyMiddleware': 110, 'yourprojectname.smartproxy_auth. ProxyMiddleware': 100, } SMARTPROXY_USER = 'username' ## Smartproxy Username (Sub-user) SMARTPROXY_PASSWORD = 'password' ## Password for your user SMARTPROXY_ENDPOINT = 'gate.smartproxy.com' ## Endpoint you'd like to use SMARTPROXY_PORT = '7000' ## Port of the endpoint you are using.
    smartproxy integration Scrapy Proxy Middleware
  • In DOWNLOADER_MIDDLEWARES change yourprojectname line to the name of your project.
  • smartproxy integration Scrapy Proxy Middleware
  • Make sure that you enter your details account details as well as proxy details within punctuation marks ('')
  • Save the file.
  • To find more information about the setup, make sure to visit our Github Page.



    Choose Smartproxy:

    Smartproxy never get flagged

    Never get flagged

    Avoid getting detected by using our vast pool of residential exit nodes.

    smartproxy unlimited threads and connections

    Unlimited connections

    Use an unlimited number of connections, threads and countries at once.

    Smartproxy country targeting

    City and Country level targeting

    Use residential IPs from as many countries or cities as you like.Change locations as often as needed.

    Smartproxy customer support

    24/7 Customer Support

    Experienced 24/7 online technical support with the average response time of 5 minutes. Email us or chat with us at any time.