ATSC now possible on magnetic drives

I have found a way to reduce the data rate of ATSC IQ recording from 32 MiB/sec to 14 MiB/sec.

Last night I was taking some recordings of Fox, recording clips of World's Funniest. I was annoyed at only being able to take 1 minute at a time and then having to wait almost two minutes transferring it from RAM to my external hard drive. Taking 1 minute clips spaced 2 minutes apart causes you to miss so much and because of this I looked into reducing the required data bandwidth.

To make a long story short, I found a way to take 8-bit IQ recordings and thus reduce data bandwidth to the point where you can actually take IQ recordings to an ordinary hard drive. Why is this important? Because with hard drives being so cheap versus SSD's, you can buy an 8 TB model like mine and (at least in theory) take recordings of virtually unlimited length.

This new way of recording will take the required data from the 32 MiB/sec in my original work down to 14 MiB/sec, while still yielding good decodes! This should be well within what a magnetic drive can handle, plus it takes less data so your hard drive space goes further.

A few notes/tips:

• Use 7 MHz bandwidth on the SDRPlay. Although I said to use 8 before, 7 has been proven to provide flawless recordings while taking up less data.
• If you can find one, I would suggest you use an older version of SDRSharp. All my recent perfect decodes were recorded in SDRSharp. As I said before, it has superior DC cancellation.
• SDRSharp also has an 8-bit recording option

Some math...

Last night I was limited to 1 minute 12 seconds on a 2-GiB RAM drive. I was using 7 MHz bandwidth at 16 bits per sample. I used a stopwatch app on my phone to time how long it takes Windows to move this onto my external drive, since Windows' reported transfer rate is inaccurate. It took me 102 seconds to transfer. The files were about 1.92 GiB each.

1.92 GiB divided by 102 = about 19 MiB / sec average transfer rate. At 16-bit sampling, this would only allow you to record 4.75 MHz of spectrum, which is obviously not enough for TV. When I calculated this, I saw that if I could switch to 8-bit recordings, my hard drives would be sufficiently fast.

Recall in my last post that I said it would take 16 MiB/sec to do an 8 MHz @ 8 bit recording, which is below the 19 MiB/sec I just calculated. But now that I'm using 7 MHz, it would only take 14 MiB/sec, well below the limit.

But 8-bit recordings didn't work last time. Then it dawned on me: if GNURadio can't natively decode 8-bit ATSC recordings, why not trick it by taking an 8-bit recording and upsampling to 16 bits?

After some more tests I noticed SDRSharp is limited to 2 GiB, which is about 2 minutes. This is a hard limit of the official WAV format, and SDRSharp doesn't support RAW or WAV64. If anyone finds a fix for this please comment below.

How I did it:

Today I took an 8-bit recording in SDRSharp. It was a WAV file, not RAW, so I was able to open it in Audacity and merely save it as a 16-bit WAV. I knew Audacity likes to dither, but that doesn't seem to happen on upsampling. I verified the 16-bit WAV I had just generated by opening it in SDRSharp and, sure enough, it looked like a great signal, so I knew Audacity hadn't messed it up.

Below: live waterfall of WACH Fox.

Below: 8-to-16 bit converted WAV recording

As you can see, there's very little difference.

I fed the 16-bit converted WAV through GNURadio as usual, and waited expectantly. For all I knew, I could get the dreaded "!!! atsc_fs_checker" errors, or simply an empty TS file. Neither happened (see below) but I still had no proof it would play.

To my surprise it actually played! (although it did skip in 2 spots)
(pictured below)

I think the skips were caused by minor sample dropping, so it should work perfectly with a very fast and defragmented hard drive. Of course, PrimoCache would still be a great idea to smooth out any hard drive latency, such as what occurs during recalibration, but only as long as the IQ data rate doesn't exceed your drive's average write speed.

I did more tests on Sunday, but with a RAM drive this time to avoid latency. 8-bit recording is indeed very reliable, not to mention space-efficient, as I was able to fit 2 minutes 33 seconds into my 2-GiB limit. I had been concerned about the reduced dynamic range afforded by 8-bit sampling, but my fears seem to be unfounded.

It would be amazing to leave this recording, latency smoothed out, to my 8 TB drive, to catch shows I'd miss. Below are two ideas for software fixes that would allow us to set up a full software-based DVR.

1. HDSDR supports unlimited file length but has a minimum bit depth of 16. It should be updated to support 8-bit recording,
2. SDR#, on the other hand, supports 8-bit recording but [its native recorder] is limited to 2 GiB. It should be updated to use a format other than WAV. See new recorder for SDRsharp breaks 2GiB limit. With a certain 3rd-party plugin, we can now take 8-bit recordings of unlimited length in Wave RF64 format using SDRsharp.