Thursday, March 30, 2017

Huge ice maker output

This post is about computer cooling, primarily for GPU's and hard drives. Before I begin, I wanted to suggest that you enter to win an EVGA GTX 1080 Ti. The contest closes 12 days from now. Full disclosure: I do benefit if you enter.

My computer setup happens to be in a room that the previous owners neglected to insulate, so we don't usually run AC there. That presented quite a problem last summer when my computer's aging fan, even with the dust blown out, couldn't keep up and the computer would keep shutting down for its own protection. Losing work randomly made me eventually devise a system of a well-insulated box containing the computer, a fan, and frozen water bottles. It worked pretty well and the box was about as cold as AC, but it still didn't put out nearly as much consistent cold air as was necessary. Using this over the summer, I observed several problems:
  • The bottles didn't have much surface area
  • It quickly goes from frozen bottle to water bottle with ice core. The water was insulating the ice core, and ice "steals" a lot more energy than cold water possibly could (because of the heat of fusion).
  • None of my freezers could freeze bottles as fast as the computer could melt them. I eventually figured out that if I could freeze enough bottles initially, I could swap them in and out and have enough consistently. I worked out the math to find out how many bottles I would need.
I also noticed that our old freezer, a Kenmore Frostless from 1990, froze bottles slightly faster than the brand-new Frigidaire we got in 2014. Considering the old freezer draws less electricity, I suspect the R-12 is responsible. Naturally, I shifted the bulk of my ice production to the Kenmore.

This past winter (2016-2017), I decided to build a better system before it became necessary, so I would have time to perfect it. Since the ice does most of the work and the water (even though it's near-freezing) is virtually useless, I would need my new system to discard the water. Since freezers' automatic ice makers are much faster than freezing bottles, they would be the source of ice. This also solves the surface area problem since a pile of ice cubes will have a lot more area than a cluster of bottles.

I put together a crude system consisting of a plastic container with a hole in the bottom. I fill it with ice and mount a desk fan on top. The fan takes up about half of the top's area and forces air through all the cubes and out the top again. This system worked well. It produced consistent and very cold air, almost like a small AC unit. There are now only two problems:
  • The freezer's ice maker must be able to keep up
  • A bucket must be kept underneath and emptied periodically
This is a lot better than where I was last summer. Lately it's been cold to mild in South Carolina so I haven't had to put it into "production" use yet. However, the old freezer's ice maker was still too slow. A quick Google search for making ice faster revealed something called "Quick Ice", which is a feature on fancy fridges that uses a fan to blow across the ice maker. At one website they write, "For models with the Quick Ice feature, ice production can be increased by nearly 48% to about 6.2 lbs per day." This did not sound impressive, but I knew it couldn't get worse so I decided to try the idea and see how much I could make. Using my strongest CPU fan and the steel wire used for the Inmarsat antenna, I made a mountable fan setup for the old Kenmore freezer. Of course, I first checked to see exactly where the plastic ceiling rail was, so I wouldn't end up making the wrong mount. Then I mounted it and wired it to a 12-volt external hard drive power supply. I emptied the ice maker's bucket and then ran the ice maker and fan for 24 hours before checking it. When I returned, I was very surprised to have 14.3 LB of ice! That happens to be exactly 3 full ice cream buckets (below).

Here's the freezer's ice bucket before being emptied:

And here are my freezer settings:

At 144 BTU/LB for ice, this comes out to 2059.2 BTU of cooling per day, or 85.8 BTU/hour. Nowhere near an AC unit, but still quite useful for small enclosures. Plus, since I'm not there to get the ice during the night, I can get more BTU's per hour using it only during the hottest parts of the day. If that's 8 hours a day, I would get 257.4 BTU/hour.

This is probably not financially feasible in the long term, but it's still a neat project. I was thinking that if I ever set up my GPU for a remote compute service and was still using ice, my website could have a footer that says, "Our GPU setup is proudly cooled with CFC-12, an energy-efficient refrigerant."

New GPU and HDMI audio output

This is just a quick little hack I realized one day when I was listening to P25 police radio. When I first installed my nVidia GPU, Windows set the default audio device to the HDMI audio output, so I set it back to my soundcard. A while later I was about to use Stereo Mix as the input for DSD+ and my USB sound dongle as the output, but decided I might as well use the GPU audio output and used the HDMI audio instead. It worked perfectly once I specified which audio devices for DSD+ to use. Not very exciting, I admit, but in my next post I'll describe my new ice cooling system.

Monday, March 20, 2017

First successful satcom ACARS

Two days ago, using the Satellite AR app,  I noticed that Inmarsat 3F2 at 15W is well above my horizon, probably enough to be received. Yesterday I temporarily put a satellite dish on my roof to try and aim at it. I had no success but I did easily receive the one at 54W. Using JAERO, I was able to decode an AERO channel and get some aircraft messages. When I looked up one of the registration numbers, it turned out to be from a plane flying over Algeria.

Here is an Inmarsat page with coverage maps for their satellites:

Saturday, March 11, 2017

Video Motion Analysis

I updated my video analysis program to output the standard deviation of each residual frame so I could see the motion levels from frame to frame. Then I saved the results into an Excel sheet and generated graphs. Below is a combined graph of BarScene, Dog Run (my own sequence), and snow (99.99% entropy).

(Click to enlarge)

You'll notice that Dog Run had to be repeated since it was much shorter than the others. There are some other things worth mentioning:

  • The spikes in the blue BarScene graph are scene changes; therefore an encoder could, on the first pass, detect scene changes for keyframe purposes using a standard deviation threshold value.
  • At nearly 100% entropy, snow is obviously not going to survive lossy compression very well. Its high standard deviation would also prompt the encoder to make every frame a keyframe. This would be a major problem unless you had good "hard" rate control.
Here's the BarScene graph, which has the most distinct activity.

I found Ronald Bultje's "Overview of the VP9 video codec" very helpful. I'm close to fully understanding the VP9 bitstream and once I do, I'll be able to start my actual encoding experiments.

Monday, March 6, 2017

4k Video Downloader and initial VP9 tests

As many of you already know, most YouTube videos are in VP9 format. It's an open-source replacement for H.264 (and arguably H.265) that offers far better compression which can mean either less data for the same quality or more quality for the same data. The downside is that is takes a lot more computational power to process, both compressing and decompressing.

As a side effect of switching browsers, I got my first experience with VP9. I left Internet Explorer in 2014 because I needed to visit sites with HTML5, which it did not yet support. At the time, Opera had about the best support, not to mention faster loading times and low memory use. I don't know if you've seen it in my screenshots, but Opera is still my only browser.

Months ago, or maybe a year, I wanted a better way to download YouTube videos. I knew about Any Video Converter, but it only supports retrieving them as MP4 and I wanted the higher-quality VP9 copies that are served to supporting browsers. I spent a while searching on Google for a way to download YouTube videos as VP9 but got no results.

Fortunately, just as I was about to give up I saw a result deep in the Google pages for a program called 4K Video Downloader. It was quite fortuitous since I don't remember what keywords I used and I can't seem to find it anymore without its name. I'm glad I found it because I was so determined to get VP9 videos that I was considering youtube-dl, a command-line app.

Here's the download link. It's free, which is surprising for a program of this quality. You may want to download it soon if you're interested or it may go the way of Networx and IcoFX, both of which I obtained and archived before you had to pay. One thing I should mention is that "4K" is just what the company prefixes all its titles with; while it can download YouTube videos in 4K, it doesn't have to.

In addition to archiving or watching on a mobile device*, I'm currently doing some experiments that require VP9 videos, and I will describe them after I show you how 4K Video Downloader works.

*Please note that most mobile devices consume FAR less power when playing MP4 rather than VP9. I recommend VP9 for all YouTube archival but only for mobile when space is a concern.

I really like how they designed the interface. It's neither gaudy nor oversimplified like some other video apps I've used. It's like Windows XP: responsive, a few nice icons, some changeable settings, and not in-your-face shiny and trying to be "helpful" like Windows 10.

(Click to enlarge)

I love the flexibility I see here, but let's get to the real reason I use this so much. First, let's copy a YouTube URL.

Now open 4K Video Downloader and notice that the green Paste Link icon at the top now has a YouTube logo. Click the button.

A dialog labeled "Parsing" will open. It normally takes only a few seconds. Sometimes it takes excessively long, and occasionally it will fail, but usually it's successful and you'll get this window:

Alternately, you can also download just the sound:

Obviously you can't open 2 menus at once; I'm just editing to show you everything concisely. MKV is how you get VP9, but sometimes a very new video won't have the MKV option yet.

I like how it shows you the file size before you download, so there are no surprises. You'd be surprised how much smaller MKV can be versus MP4, but don't assume that's always the case. I've seen videos where it's the opposite, so if you want to save space and don't care which format, always compare the MKV sizes to those of MP4.

A quick note: I don't condone stealing music this way. Please buy songs before downloading the music videos. Of course, if you buy the song then you probably don't need the audio download feature.

Once you choose your format and output folder, you can click Download.

It often shows the company's own ads but they can be closed.
After it finishes, it's ready to play in your favorite player. I normally use Media Player Classic because Windows Media Player 10 was the last good version.

As I said near the beginning, I really appreciate the company offering this for free. They did such a good job that I was planning to eventually buy the full version. My only complaint is that there is no way to download YouTube Live streams. For that, I use the command-line app Livestreamer.

Some people are aware of the "ss" YouTube trick, but it's not as good because the site providing the service wants you to pay for any downloads of good quality.

(Below) The "ss" YouTube trick. The free service strips the audio from 480p and 1080p downloads.

And now, a note on the ethics of downloading. Obviously every tool has good and bad uses, so this is not inherently evil. There are often videos I want to own that aren't being sold as a movie or song. As I said above, do not download music videos without supporting the artist. What is okay to save? I personally think it's fine to download:

  • anything that's not regular, for-sale movies or music is fine to download. For example, Big Buck Bunny is copyrighted but okay to download.
  • Vlogs
  • Homemade songs (assuming they aren't being sold somewhere)
If in doubt, use common sense.

And now for my experiments.

I stumbled upon Ronald Bultje's EVE VP9 encoder a while ago. I've always been fascinated by video compression but regarded it as a sort of secret art that required huge teams. I wrote to Ronald Bultje and asked if he wrote his encoder alone or as a team, and he said he did it alone. I was very impressed and inspired to create my own, since his is for sale.

libvpx is the only well-known free VP9 encoder. Its development was primarily done by Google but it has major problems. It's incredibly slow and usually produces blurry output, even at excessive bitrates. Considering the published results of the EVE encoder, which was written by one person, I figured it wouldn't be hard to do better than libvpx.

Pieter Kapsenberg, an expert in this field, informed me via email that libvpx's blurriness was possibly caused by the use of 32x32 transforms and also not using much CPU to compress as well as possible. Although I have not found out for sure why it's slow, I believe it stems from the motion search, which can be solved with GPU-based algorithms.

I recently bought an nVidia GTX 750 Ti and installed the CUDA Toolkit in Visual Studio 2015. I'm actually impressed with how well it integrated with Visual Studio, because most plugins are quite prone to failure. I didn't even have to manually configure directories. After some initial trouble with using the wrong driver version, I was able to start writing CUDA kernels.

My first experiment involves generating residuals (differences between frames) so I can analyze motion and see how best to compress it. I initially thought I'd work with BMP's but who wants a finished encoder that only accepts folders of BMP's? I would've also had to convert from RGB to YUV, which is incredibly fast on CUDA but requires the use of imprecise floating point values, and the U and V are signed, which would require conversion before saving. I quickly saw that it would be better to just accept yuv420p output from FFmpeg, so I recorded some videos and converted them to raw YUV using FFmpeg. I got out my nice Canon camcorder, put it on a tripod to keep the surrounding scene still, and called our dog. In the style of the media collection, I named the file Dog Run. It was taken as 1080p @ 24 Mbit/sec in AVCHD format. I increased the saturation and contrast before downsampling 4x to 960x540 raw YUV.

My program has a flag that toggles between two residual modes. One mode subtracts the current frame from the first frame, and the other mode subtracts the current frame from the one before it. It does a simple subtraction of the 960x540 frames, subtracting the 0-255 Y-plane pixels (U and V are ignored) and producing output in the range of +/- 255. Within the same kernel, this output is divided by 2 and added to 127 to produce the final 8-bit grayscale values used for the images below. According to the CUDA Profiler, it takes about 80 microseconds per frame, making the hard drive the biggest bottleneck. It looked really cool when I finally got to view the 2 types of residuals alongside the original video.
(Click to enlarge)

Notice that the residual on the right is a lot greater because a lot has changed between this frame and the first one.

I also noticed that this is a lot like calculus because we're sort of taking the derivative of the video, by seeing at what points (frames) it's changing the fastest. This should be useful for doing the first pass to find what parts of the video need the most bits.

How 4K Video Downloader ties in

Analyses and algorithms are well and good but to be successful you have to know how to create a bitstream. To better understand VP9, I needed a good VP9 bitstream that I could open in a hex editor. 4K Video Downloader fulfilled the need perfectly.

MKV/WebM are complex container formats so I converted the downloaded video to IVF using FFmpeg. Then I opened it in HHD Hex Editor Neo, extracted a frame into a separate file, and then displayed that file as bits in the hex editor. I copy-pasted the entire bitstream of that frame into a TXT file, opened it in Notepad++, and spent several hours deciphering the uncompressed header while writing a Word document of the exact details. The information is all there in the official VP9 spec PDF, but my document makes it easier to understand since each element in the PDF branches off into a different section, and does this recursively for many levels, making it hard to remember where you are after a while.

Wednesday, March 1, 2017

ATSC RF recording available

I've been in touch with Light Coder as I tried to perform his requested experiment. I found that 8-bit recording in HDSDR somehow creates a DC spike that isn't present when recording 16-bit. After I told him, Light Coder asked in an email if I would send him an ATSC recording to help him debug HDSDR, and since people had asked for ATSC before, I figured I might as well make a blog post out of it. So here's a screenshot and the download:

This is the first ATSC signal I encountered while scanning with the rabbit ears, and I believe it's PBS.

atsc_rfchannel_33.7z (660 MiB)