4k Video Downloader and initial VP9 tests

As many of you already know, most YouTube videos are in VP9 format. It's an open-source replacement for H.264 (and arguably H.265) that offers far better compression which can mean either less data for the same quality or more quality for the same data. The downside is that is takes a lot more computational power to process, both compressing and decompressing.

As a side effect of switching browsers, I got my first experience with VP9. I left Internet Explorer in 2014 because I needed to visit sites with HTML5, which it did not yet support. At the time, Opera had about the best support, not to mention faster loading times and low memory use. I don't know if you've seen it in my screenshots, but Opera is still my only browser.

Months ago, or maybe a year, I wanted a better way to download YouTube videos. I knew about Any Video Converter, but it only supports retrieving them as MP4 and I wanted the higher-quality VP9 copies that are served to supporting browsers. I spent a while searching on Google for a way to download YouTube videos as VP9 but got no results.

Fortunately, just as I was about to give up I saw a result deep in the Google pages for a program called 4K Video Downloader. It was quite fortuitous since I don't remember what keywords I used and I can't seem to find it anymore without its name. I'm glad I found it because I was so determined to get VP9 videos that I was considering youtube-dl, a command-line app.

Here's the download link. It's free, which is surprising for a program of this quality. You may want to download it soon if you're interested or it may go the way of Networx and IcoFX, both of which I obtained and archived before you had to pay. One thing I should mention is that "4K" is just what the company prefixes all its titles with; while it can download YouTube videos in 4K, it doesn't have to.

In addition to archiving or watching on a mobile device*, I'm currently doing some experiments that require VP9 videos, and I will describe them after I show you how 4K Video Downloader works.

*Please note that most mobile devices consume FAR less power when playing MP4 rather than VP9. I recommend VP9 for all YouTube archival but only for mobile when space is a concern.

I really like how they designed the interface. It's neither gaudy nor oversimplified like some other video apps I've used. It's like Windows XP: responsive, a few nice icons, some changeable settings, and not in-your-face shiny and trying to be "helpful" like Windows 10.

(Click to enlarge)

I love the flexibility I see here, but let's get to the real reason I use this so much. First, let's copy a YouTube URL.

Now open 4K Video Downloader and notice that the green Paste Link icon at the top now has a YouTube logo. Click the button.

A dialog labeled "Parsing" will open. It normally takes only a few seconds. Sometimes it takes excessively long, and occasionally it will fail, but usually it's successful and you'll get this window:

Alternately, you can also download just the sound:

Obviously you can't open 2 menus at once; I'm just editing to show you everything concisely. MKV is how you get VP9, but sometimes a very new video won't have the MKV option yet.

I like how it shows you the file size before you download, so there are no surprises. You'd be surprised how much smaller MKV can be versus MP4, but don't assume that's always the case. I've seen videos where it's the opposite, so if you want to save space and don't care which format, always compare the MKV sizes to those of MP4.

A quick note: I don't condone stealing music this way. Please buy songs before downloading the music videos. Of course, if you buy the song then you probably don't need the audio download feature.

Once you choose your format and output folder, you can click Download.

It often shows the company's own ads but they can be closed.
After it finishes, it's ready to play in your favorite player. I normally use Media Player Classic because Windows Media Player 10 was the last good version.

As I said near the beginning, I really appreciate the company offering this for free. They did such a good job that I was planning to eventually buy the full version. My only complaint is that there is no way to download YouTube Live streams. For that, I use the command-line app Livestreamer.


Some people are aware of the "ss" YouTube trick, but it's not as good because the site providing the service wants you to pay for any downloads of good quality.

(Below) The "ss" YouTube trick. The free service strips the audio from 480p and 1080p downloads.

And now, a note on the ethics of downloading. Obviously every tool has good and bad uses, so this is not inherently evil. There are often videos I want to own that aren't being sold as a movie or song. As I said above, do not download music videos without supporting the artist. What is okay to save? I personally think it's fine to download:

• anything that's not regular, for-sale movies or music is fine to download. For example, Big Buck Bunny is copyrighted but okay to download.
• Vlogs
• Homemade songs (assuming they aren't being sold somewhere)

If in doubt, use common sense.

And now for my experiments.

I stumbled upon Ronald Bultje's EVE VP9 encoder a while ago. I've always been fascinated by video compression but regarded it as a sort of secret art that required huge teams. I wrote to Ronald Bultje and asked if he wrote his encoder alone or as a team, and he said he did it alone. I was very impressed and inspired to create my own, since his is for sale.

libvpx is the only well-known free VP9 encoder. Its development was primarily done by Google but it has major problems. It's incredibly slow and usually produces blurry output, even at excessive bitrates. Considering the published results of the EVE encoder, which was written by one person, I figured it wouldn't be hard to do better than libvpx.

Pieter Kapsenberg, an expert in this field, informed me via email that libvpx's blurriness was possibly caused by the use of 32x32 transforms and also not using much CPU to compress as well as possible. Although I have not found out for sure why it's slow, I believe it stems from the motion search, which can be solved with GPU-based algorithms.

I recently bought an nVidia GTX 750 Ti and installed the CUDA Toolkit in Visual Studio 2015. I'm actually impressed with how well it integrated with Visual Studio, because most plugins are quite prone to failure. I didn't even have to manually configure directories. After some initial trouble with using the wrong driver version, I was able to start writing CUDA kernels.

My first experiment involves generating residuals (differences between frames) so I can analyze motion and see how best to compress it. I initially thought I'd work with BMP's but who wants a finished encoder that only accepts folders of BMP's? I would've also had to convert from RGB to YUV, which is incredibly fast on CUDA but requires the use of imprecise floating point values, and the U and V are signed, which would require conversion before saving. I quickly saw that it would be better to just accept yuv420p output from FFmpeg, so I recorded some videos and converted them to raw YUV using FFmpeg. I got out my nice Canon camcorder, put it on a tripod to keep the surrounding scene still, and called our dog. In the style of the Xiph.org media collection, I named the file Dog Run. It was taken as 1080p @ 24 Mbit/sec in AVCHD format. I increased the saturation and contrast before downsampling 4x to 960x540 raw YUV.

My program has a flag that toggles between two residual modes. One mode subtracts the current frame from the first frame, and the other mode subtracts the current frame from the one before it. It does a simple subtraction of the 960x540 frames, subtracting the 0-255 Y-plane pixels (U and V are ignored) and producing output in the range of +/- 255. Within the same kernel, this output is divided by 2 and added to 127 to produce the final 8-bit grayscale values used for the images below. According to the CUDA Profiler, it takes about 80 microseconds per frame, making the hard drive the biggest bottleneck. It looked really cool when I finally got to view the 2 types of residuals alongside the original video.

(Click to enlarge)

Notice that the residual on the right is a lot greater because a lot has changed between this frame and the first one.

I also noticed that this is a lot like calculus because we're sort of taking the derivative of the video, by seeing at what points (frames) it's changing the fastest. This should be useful for doing the first pass to find what parts of the video need the most bits.

How 4K Video Downloader ties in

Analyses and algorithms are well and good but to be successful you have to know how to create a bitstream. To better understand VP9, I needed a good VP9 bitstream that I could open in a hex editor. 4K Video Downloader fulfilled the need perfectly.

MKV/WebM are complex container formats so I converted the downloaded video to IVF using FFmpeg. Then I opened it in HHD Hex Editor Neo, extracted a frame into a separate file, and then displayed that file as bits in the hex editor. I copy-pasted the entire bitstream of that frame into a TXT file, opened it in Notepad++, and spent several hours deciphering the uncompressed header while writing a Word document of the exact details. The information is all there in the official VP9 spec PDF, but my document makes it easier to understand since each element in the PDF branches off into a different section, and does this recursively for many levels, making it hard to remember where you are after a while.