Commit Graph
100 Commits
Author SHA1 Message Date
Yen Chi Hsuan daac118bf4 [ChangeLog] Update after #11901 2017-02-03 18:56:40 +08:00
Yen Chi HsuanandGitHub 8939f784d9 Merge pull request #11901 from ThomasChr/randonplaylistorder
New parameter --playlist-random to randomize playlist download order. Fixes #11889
2017-02-03 18:53:14 +08:00
Yen Chi Hsuan d7f9242e30 [ChangeLog] Update after #11565 2017-02-03 12:13:24 +08:00
Yen Chi Hsuan a22b2fd19b [youtube] Fix ytsearch* when cookies are provided
Closes #11924

The API with `page` is no longer used in browsers, and YouTube always
returns {'reload': 'now'} when cookies are provided.

See http://youtube.github.io/spfjs/documentation/start/ for how SPF
works. Basically appending static link with a `spf` parameter yields the
corresponding dynamic link.
2017-02-03 01:28:24 +08:00
Yen Chi Hsuan bd8f48c78b [bilibili] Support new Bangumi URLs (closes #11845)
To reduce complexity, I don't support old Bangumi URLs directly via
_VALID_URL. Instead, I choose to let it go to generic redirection. An
example can be found in #10190:

http://bangumi.bilibili.com/anime/v/40062
2017-02-02 21:51:31 +08:00
Yen Chi Hsuan 000f207944 [prosiebensat1] Update ChangeLog 2017-02-01 23:16:35 +08:00
Yen Chi Hsuan fe5aa197b5 [prosiebensat1] PEP8 and update _TESTS 2017-02-01 23:13:45 +08:00
Yen Chi Hsuan 2b2d5d319b [crunchyroll] Remove ScaledBorderAndShadow settings
See https://github.com/rg3/youtube-dl/pull/9028, especially @lachs0r's
comments for the reason behind this change
2017-02-01 16:39:32 +08:00
Yen Chi Hsuan bc35ed3fb6 [nextmedia] Add support for NextTV (壹電視) 2017-01-23 23:38:31 +08:00
Yen Chi Hsuan 0c1c6f4b9f [utils] Add another date format seen in NextTV 2017-01-23 23:37:32 +08:00
Yen Chi Hsuan 7c20b7484c [nextmedia] Support redirected URLs 2017-01-22 02:06:34 +08:00
Yen Chi Hsuan f3c21cb7a7 [cspan] Fix _TESTS 2017-01-20 22:27:13 +08:00
Yen Chi Hsuan 972efe60c3 [generic] Remove a dead test
The web page does not contain a video anymore

Ref: #2694, #2696
2017-01-20 22:27:13 +08:00
Yen Chi Hsuan 4447fb2332 [cspan] Support Ustream embedded videos
Closes #11547
2017-01-20 22:27:13 +08:00
Yen Chi Hsuan d77ac73790 [ustream] Add UstreamIE._extract_url()
Ref: #11547
2017-01-20 22:27:13 +08:00
Yen Chi HsuanandGitHub 1076858f76 Merge pull request #11778 from h4ck3rm1k3/master
Fix typos
2017-01-19 19:32:29 +08:00
Yen Chi Hsuan 16e2c8f771 [brightcove] Recognize another player ID
Closes #11688
2017-01-16 00:06:52 +08:00
Yen Chi Hsuan dcae7b3fdc [niconico] Allow login via cookies
Some codes are borrowed from #7968, which is by @jlhg

Closes #7968
2017-01-15 22:51:54 +08:00
Yen Chi Hsuan 8e4988f1a2 [niconico] Remove codes for downloading anonymously
Apparently Niconico now blocks playing without an account

Closes #11170
2017-01-15 22:10:57 +08:00
Yen Chi Hsuan 1219201143 [ChangeLog] Update after #11581
[ci skip]
2017-01-06 01:07:30 +08:00
Yen Chi Hsuan ec85ded83c Fix "invalid escape sequences" error on Python 3.6 2017-01-06 00:58:56 +08:00
Yen Chi Hsuan 24d8a75982 [discoverygo] Fix JSON data parsing
HTMLParser, which is used by extract_attributes, already unescapes
attribute values with HTMLParser.unescape. They shouldn't be unescaped
again, to there may be parsing errors.

Ref: #11219, #11522
2017-01-05 18:50:34 +08:00
Yen Chi HsuanandGitHub d546d4c8e0 Merge pull request #11578 from rpunkfu/patch-2
Update Readme: Set HOME properly in install command
2017-01-02 13:50:26 +08:00
Yen Chi Hsuan 4606c34e19 [extractor/common] Allow non-lang in subtitles' keys
See 264e77c406
2016-12-25 01:50:50 +08:00
Yen Chi Hsuan 8ab7e6c4cc [kaltura] Improve widget ID extraction (closes #11480) 2016-12-20 18:45:52 +08:00
Yen Chi Hsuan 5c32a5be95 [openload] Recognize oload.tv URLs (#10408) 2016-12-15 17:51:26 +08:00
Yen Chi Hsuan 30918999f5 [facebook] Recognize .onion URLs (closes #11443) 2016-12-15 01:04:49 +08:00
Yen Chi Hsuan 6c20a0bb99 [openload] Fix extraction (closes #10408) 2016-12-09 02:15:16 +08:00
Yen Chi Hsuan 4bd7d9d4ae [socks] Refine exception model for better error handling
1. ProxyError now inherits from socket.error instead of IOError

The only functions socks.py overrides are connect and connect_ex. In
Python 2.x and Python <= 3.2, socket functions raises socket.error. In
newer Python versions, those functions raises OSError instead. The name
socket.error is preserved as an alias of OSError for backward
compability. To keep socks.py compatible with Python's standard library,
it should raise the same exception as raw sockets.

See PEP 3151 (https://www.python.org/dev/peps/pep-3151/) for more
information about the change in Python 3.3.

2. Raise EOFError instead of IOError when the socket receives less data
than it expects

There's no common convention, but both ftplib and telnetlib raises
EOFError for similar situations. socks.py follows them.

Closes #11355

In #11355, only Python 2 is affected. In Python 3, both socket.error and
IOError are alias of OSError, so AbstractHTTPHandler.do_open correctly
catches the error and thus InfoExtractor._is_valid_url works fine.
2016-12-05 00:43:37 +08:00
Yen Chi Hsuan 8344296619 [socks] Fix error reporting (#11355) 2016-12-03 21:53:41 +08:00
Yen Chi Hsuan d17bfe4095 [thisoldhouse] Recognize /tv-episode/ URLs and update _TESTS
Closes #11271
2016-12-01 14:56:52 +08:00
Yen Chi Hsuan 51b1378eed Ignore and clean .swf files
Some videos on NicoNico are swf
2016-11-27 22:01:07 +08:00
Yen Chi Hsuan 44444f0d3b [cbslocal] Support newyork.cbslocal.com
Closes #11285
2016-11-24 20:32:17 +08:00
Yen Chi Hsuan 1d6ae5628f [amcnetworks] Recognize more BBC America URLs
Closes #11263
2016-11-22 20:40:57 +08:00
Yen Chi Hsuan 2574721a81 Clean and ignore more file types
ape is another audio codec seen in kuwo. See
https://en.wikipedia.org/wiki/Monkey's_Audio
2016-11-21 12:50:13 +08:00
Yen Chi Hsuan dbcc4a6b32 [CONTRIBUTING.md] Fix broken links (#11239) 2016-11-21 12:25:19 +08:00
Yen Chi HsuanandGitHub 0bb58a208b Merge pull request #11239 from josephfrazier/patch-1
[CONTRIBUTING.md] Fix broken link
2016-11-21 12:24:11 +08:00
Yen Chi Hsuan 2176e466e0 Merge branch 'DarkstaIkers-master' 2016-11-20 00:07:35 +08:00
Yen Chi Hsuan 303b38fa84 [ChangeLog] Update for #9028 2016-11-20 00:06:44 +08:00
Yen Chi Hsuan fb27d0ce5e Merge branch 'master' of https://github.com/DarkstaIkers/youtube-dl into DarkstaIkers-master 2016-11-20 00:05:11 +08:00
Yen Chi Hsuan 582be35847 Update coding style after pycodestyle 2.1.0
In pycodestyle 2.1.0, E305 was introduced, which requires two blank
lines after top level declarations, too.

See https://github.com/PyCQA/pycodestyle/issues/400

See also #10689; thanks @stepshal for first mentioning this issue and
initial patches
2016-11-17 19:45:42 +08:00
Yen Chi HsuanandGitHub 315cb86a95 Merge pull request #11210 from FooBarQuaxx/patch-2
Strip only args urls
2016-11-16 23:29:37 +08:00
Yen Chi Hsuan d76767c90e [ChangeLog] Update after #11122 landed 2016-11-16 20:47:15 +08:00
Yen Chi HsuanandGitHub eceba9f805 Merge pull request #11122 from kasper93/openload
[openload] Fix extraction.
2016-11-16 20:43:19 +08:00
Yen Chi Hsuan 3eaaa8abac [audioboom] Recognize /posts/ URLs (closes #11149) 2016-11-10 14:52:34 +08:00
Yen Chi Hsuan 5d47b38cf5 [tmz:article] Fix extraction (closes #11052) 2016-11-08 21:53:41 +08:00
Yen Chi Hsuan 3a86b2c51e Ignore and clean .wav files 2016-11-03 18:55:55 +08:00
Yen Chi Hsuan d9ee2e5cf6 [facebook] Remove SWF params so that 1080P are detected
Closes #11073

In the provided link, SWF params give up to 720P, and VideoConfig
gives 1080P for both best and bestvideo. I guess all Facebook videos
supports HTML5 now, so I remove the old detection for SWF params
2016-10-30 18:20:55 +08:00
Yen Chi Hsuan e1a0b3b81c [imgur] Recognize /r/ URLs (closes #11071) 2016-10-30 17:02:03 +08:00
Yen Chi Hsuan f02700a1fa [openload] Fix extraction (#10408)
Thanks @TwelveCharzz again for studying openload codes
2016-10-29 18:00:30 +08:00
Yen Chi Hsuan a5a8877f9c [adultswim] Fix extraction (closes #10979) 2016-10-27 02:16:48 +08:00
Yen Chi Hsuan ec8705117a [hornbunny] Fix extraction (#10981) 2016-10-27 00:10:51 +08:00
Yen Chi Hsuan 5ace137bf4 [dotsub] Support vimeo embed (closes #10964) 2016-10-24 15:13:33 +08:00
Yen Chi Hsuan 9dde0e04e6 [litv] Fix extraction (#11006) 2016-10-23 23:23:40 +08:00
Yen Chi Hsuan e034cbc581 Merge branch 'johnhawkinson-stdin2' 2016-10-22 13:10:27 +08:00
Yen Chi Hsuan 5378f8ce0d [ChangeLog] Update for #10996 2016-10-22 13:08:56 +08:00
Yen Chi Hsuan b64d04c119 [utils] Clarify for redirecting STDIN in get_exe_version() 2016-10-22 13:04:05 +08:00
Yen Chi Hsuan 8e2915d70b Revert "[postprocessor/embedthumbnail] Allow mkv to embed thumbnails"
This reverts commit 7360db05b4.

This commit was added as an attempt to fix #6046. Unfortunately, the fix
is completely wrong. As reported on #10359, embedded thumbnails are not
displayed in VLC, and Se7en on IRC reports that the embedded thumbnail
misleads mpv as well.

The correct way is using -attachment of ffmpeg, while the current
run_ffmpeg_multiple_files API can't handle it cleanly.
2016-10-20 15:07:19 +08:00
Yen Chi Hsuan 19e447150d [ChangeLog] Update for #10971 2016-10-20 04:19:33 +08:00
Yen Chi HsuanandGitHub ad9fd84004 Merge pull request #10971 from kasper93/openload
[openload] Fix extraction.
2016-10-20 04:18:27 +08:00
Yen Chi Hsuan cfc0e7c82b Credit @pyx for the Huajiao extractor (#10917) 2016-10-15 16:48:35 +08:00
Yen Chi Hsuan 4102e64051 Merge branch 'pyx-huajiao' 2016-10-15 14:56:00 +08:00
Yen Chi Hsuan f605242bfc [ChangeLog] Update for #10917 2016-10-15 14:55:36 +08:00
Yen Chi Hsuan d32fa0f12c [huajiao] Coding style 2016-10-15 14:53:53 +08:00
Yen Chi Hsuan a347a0d088 Merge branch 'huajiao' of https://github.com/pyx/youtube-dl into pyx-huajiao 2016-10-15 14:53:05 +08:00
Yen Chi Hsuan 77c5b98dcd [crunchyroll] Skip an invalid _TEST 2016-10-15 14:36:07 +08:00
Yen Chi Hsuan 88ebefc054 [cmt] Fix mgid extraction (closes #10813)
The example in #10813 requires TV provider authentication in Firefox,
while youtube-dl can download it directly with an US proxy.

I'm not sure whether the mgid fix is cmt-specific or it applies to all
mtv-based sites. I keep it in cmt.py until similar patterns are found in
other websites.
2016-10-15 14:27:15 +08:00
Yen Chi Hsuan 8c3e448e80 [clipfish] Update _TEST; the old one is gone 2016-10-15 00:12:21 +08:00
Yen Chi Hsuan 2ecbd2ad6f [chirbit:profile] Fix extraction 2016-10-15 00:01:46 +08:00
Yen Chi Hsuan 62a0b86e4f [carambatv] Fix extraction
The video requested in #9815 now has videomore embeds.
2016-10-14 23:43:18 +08:00
Yen Chi Hsuan 146969e05b [videomore] Support <iframe> embed videos
Seen in CarambaTVPage
2016-10-14 23:42:11 +08:00
Yen Chi Hsuan e2004ccaf7 [canalplus] Fix video_id and update _TESTS
Some tests are gone, and some redirect to different videos
2016-10-14 20:26:12 +08:00
Yen Chi Hsuan a5f8473145 [cbsinteractive] Fix extraction for cnet.com 2016-10-14 18:20:01 +08:00
Yen Chi Hsuan 580d411931 [parliamentliveuk] Recognize lower case URLs
Closes #10912

Seems parliamentliveuk matches URLs case-insentive. For example this URL
also works:
http://parliamentlive.tv/EvEnt/Index/3F24936f-130f-40bf-9a5d-b3d6479da6a4
2016-10-14 00:44:28 +08:00
Yen Chi Hsuan 9feb1c9731 [dailymotion] Fix extraction and update _TESTS
Closes #10901

Seems all videos use player V5 syntax now
2016-10-12 21:45:49 +08:00
Yen Chi Hsuan a093cfc78b [vimeo:review] Fix extraction (#10900)
Now Vimeo Review videos uses React. Thanks @davekaro for analyzing the
problem!
2016-10-12 01:48:06 +08:00
Yen Chi Hsuan 6f20b65e72 [test/test_http] Update tests
After switching to HTML5 extraction helpers in generic.py, the result
info_dict is always a playlist.
2016-10-12 01:41:41 +08:00
Yen Chi Hsuan cea364f70c [extractor/common] Support HTML media elements without child nodes 2016-10-12 01:40:28 +08:00
Yen Chi Hsuan 55642487f0 [nhl] Skip invalid m3u8 formats (closes #10713) 2016-10-11 20:50:52 +08:00
Yen Chi Hsuan 3d643f4cec [hbo] Add HBOEpisodeIE (#10892) 2016-10-11 17:46:52 +08:00
Yen Chi Hsuan c452e69d3d [footyroom] Fix extraction and update _TESTS (closes #10810) 2016-10-11 17:46:13 +08:00
Yen Chi Hsuan 555787d717 [streamable] Add helper for extracting embedded videos 2016-10-11 17:44:35 +08:00
Yen Chi Hsuan f165ca70eb [abc.net.au:iview] Fix for non-series videos (closes #10895) 2016-10-11 12:53:27 +08:00
Yen Chi Hsuan 27b8d2ee95 [hbo] Add display_id and another test (#10892) 2016-10-11 12:41:44 +08:00
Yen Chi Hsuan 71cdcb2331 [hbo] Support episode pages (closes #10892) 2016-10-11 12:30:35 +08:00
Yen Chi Hsuan 176006a120 [allocine] Fix for /video/ videos (closes #10860) 2016-10-09 19:42:42 +08:00
Yen Chi Hsuan 65f4c1de3d [allocine] Fix extraction (closes #10860)
I change the URL of the third test case, because now the original URL
does not contain a video anymore, and there's no easy to get the real
URL from the /film/ one.
2016-10-09 18:58:15 +08:00
Yen Chi Hsuan b0082629a9 [nextmedia] Support action news (動新聞) on Apple Daily 2016-10-09 18:42:15 +08:00
Yen Chi Hsuan 8204c73352 [Makefile] Fix for GNU make < 4 (closes #9387)
Shell assignment operator in BSD make != is ported to GNU make in
version 4.0, so 3.x doesn't work. I choose to drop BSD make support as
installing GNU make on *BSD systems is easier than installing newer GNU
make.
2016-10-09 18:24:45 +08:00
Yen Chi Hsuan 38588ab977 [facebook] Fix for new handleServerJS syntax (closes #10846)
According to the dump file in #10846, handleServerJS() now accepts
an optional second argument. It's a string from available dump files.
2016-10-07 20:04:49 +08:00
Yen Chi Hsuan 85bcdd081c [extractors] Add MmsIE 2016-10-07 19:31:26 +08:00
Yen Chi Hsuan 9dcd6fd3aa [generic,commonprotocols] Move mms suuport from GenericIE
And use _generic_* helpers in those extractors
2016-10-07 19:24:22 +08:00
Yen Chi Hsuan 98763ee354 [extractor/common] Add id and title helpers for generic IEs 2016-10-07 19:20:53 +08:00
Yen Chi Hsuan 3d83a1ae92 [generic] Support direct MMS links (closes #10838) 2016-10-07 17:50:45 +08:00
Yen Chi Hsuan c0a7b9b348 Revert "[Makefilea] Fix for GNU make < 4"
This reverts commit 831a34caa2.

The reverted commit breaks lazy extractors.
2016-10-07 16:03:34 +08:00
Yen Chi Hsuan 831a34caa2 [Makefilea] Fix for GNU make < 4
Closes #9387

The shell assignment operator != was introduced in GNU make 4.0, or
specifically the commit in [1]. This fix removes such usages and
fallback to a more portable syntax. Tested with:

* GNU make 3.82 on CentOS 7.2
* bmake 20150910 on CentOS 7.2, source RPM from Fedora 24 [2]
* GNU make 4.2.1 on Arch Linux (Arch official package)
* bmake 20160926 on Arch Linux (Arch official package)
* GNU make 3.82 on Arch Linux (Compiled from source)
* Apple bsdmake-24 on macOS Sierra, binary package from Homebrew

Thanks @bdeyal for the feedback of the first tests

[1] http://git.savannah.gnu.org/cgit/make.git/commit/?id=b34438bee83ee906a23b881f257e684a0993b9b1
[2] http://koji.fedoraproject.org/koji/buildinfo?buildID=716769
2016-10-07 03:28:41 +08:00
Yen Chi HsuanandGitHub 25291b979a Merge pull request #10829 from TRox1972/pornoxo_improve
[pornoxo] Use JWPlatform to improve metadata extraction
2016-10-02 20:19:34 +08:00
Yen Chi Hsuan 703b3afa93 [amcnetworks] Skip a restricted _TEST 2016-10-02 14:25:06 +08:00
Yen Chi Hsuan 99ed78c79e [jwplatform] Support DASH streams 2016-10-02 14:07:49 +08:00
Yen Chi Hsuan fd15264172 [jwplatform] Support old-style jwplayer playlists 2016-10-02 13:47:06 +08:00