TeamHG-Memex / scrapy-rotating-proxies
Showing 1 of 2 files from the diff.
Other files ignored by Codecov
README.rst has changed.

@@ -2,8 +2,11 @@
Loading
2 2
from __future__ import absolute_import
3 3
import logging
4 4
import codecs
5 +
import re
6 +
5 7
from functools import partial
6 8
from six.moves.urllib.parse import urlsplit
9 +
from w3lib.http import basic_auth_header
7 10
8 11
from scrapy.exceptions import CloseSpider, NotConfigured
9 12
from scrapy import signals
@@ -142,6 +145,11 @@
Loading
142 145
                    logger.error("No proxies available even after a reset.")
143 146
                    raise CloseSpider("no_proxies_after_reset")
144 147
148 +
        parts = re.match('(\w+://)([^:]+?:[^@]+?@)?(.+)', proxy)
149 +
        if parts.group(2):
150 +
            proxy = parts.group(3)
151 +
            proxy_auth = basic_auth_header(*parts.group(2)[:-1].split(':'))
152 +
            request.headers['Proxy-Authorization'] = proxy_auth
145 153
        request.meta['proxy'] = proxy
146 154
        request.meta['download_slot'] = self.get_proxy_slot(proxy)
147 155
        request.meta['_rotating_proxy'] = True
Files Coverage
rotating_proxies 50.20%
Project Totals (5 files) 50.20%
Notifications are pending CI completion. Waiting for GitHub's status webhook to queue notifications. Push notifications now.
57.2
TRAVIS_PYTHON_VERSION=3.5
TRAVIS_OS_NAME=linux
TOXENV=py35
57.1
TRAVIS_PYTHON_VERSION=2.7
TRAVIS_OS_NAME=linux
TOXENV=py27
1
comment:
2
  layout: "header, diff, tree"
3

4
coverage:
5
  status:
6
    project: false
Sunburst
The inner-most circle is the entire project, moving away from the center are folders then, finally, a single file. The size and color of each slice is representing the number of statements and the coverage, respectively.
Icicle
The top section represents the entire project. Proceeding with folders and finally individual files. The size and color of each slice is representing the number of statements and the coverage, respectively.
Grid
Each block represents a single file in the project. The size and color of each block is represented by the number of statements and the coverage, respectively.
Loading