TeamHG-Memex / scrapy-rotating-proxies
Showing 1 of 1 files from the diff.

@@ -2,8 +2,11 @@
Loading
2 2
from __future__ import absolute_import
3 3
import logging
4 4
import codecs
5 -
import re
6 5
6 +
try:
7 +
    from urllib2 import _parse_proxy
8 +
except ImportError:
9 +
    from urllib.request import _parse_proxy
7 10
from functools import partial
8 11
from six.moves.urllib.parse import urlsplit
9 12
from w3lib.http import basic_auth_header
@@ -145,10 +148,9 @@
Loading
145 148
                    logger.error("No proxies available even after a reset.")
146 149
                    raise CloseSpider("no_proxies_after_reset")
147 150
148 -
        parts = re.match('(\w+://)([^:]+?:[^@]+?@)?(.+)', proxy)
149 -
        if parts.group(2):
150 -
            proxy = parts.group(3)
151 -
            proxy_auth = basic_auth_header(*parts.group(2)[:-1].split(':'))
151 +
        username, password = _parse_proxy(proxy)[1:3]
152 +
        if username and password:
153 +
            proxy_auth = basic_auth_header(username, password)
152 154
            request.headers['Proxy-Authorization'] = proxy_auth
153 155
        request.meta['proxy'] = proxy
154 156
        request.meta['download_slot'] = self.get_proxy_slot(proxy)
Files Coverage
rotating_proxies 51.00%
Project Totals (5 files) 51.00%
Notifications are pending CI completion. Waiting for GitHub's status webhook to queue notifications. Push notifications now.
58.2
TRAVIS_PYTHON_VERSION=3.5
TRAVIS_OS_NAME=linux
TOXENV=py35
58.1
TRAVIS_PYTHON_VERSION=2.7
TRAVIS_OS_NAME=linux
TOXENV=py27
1
comment:
2
  layout: "header, diff, tree"
3

4
coverage:
5
  status:
6
    project: false
Sunburst
The inner-most circle is the entire project, moving away from the center are folders then, finally, a single file. The size and color of each slice is representing the number of statements and the coverage, respectively.
Icicle
The top section represents the entire project. Proceeding with folders and finally individual files. The size and color of each slice is representing the number of statements and the coverage, respectively.
Grid
Each block represents a single file in the project. The size and color of each block is represented by the number of statements and the coverage, respectively.
Loading