Commit Graph
2 Commits
Author SHA1 Message Date
Johannes Knoedtel 3d5f7a3947 [utils] Prevent override of custom headers.
The dict of headers of request objects in urllib has its keys always
capitalized.

This causes the lookup to fail and overwrite the header. If for example
a Extractor tries to add a "User-Agent" header the internal
representation in the request object is "User-agent". The header is
therefore clobbered by the "User-Agent" in std_headers, because the
strings are not equal.
2015-01-12 22:38:51 +01:00
Johannes Knoedtel ff7a07d5c4 [archiveorg] most metadata fields are optional
Example: https://archive.org/details/Cops1922
2014-12-28 20:31:25 +01:00