New fiber optic lines

A few days ago (May 3) I saw some colored tubes sticking out of the ground at the corner of the local post office. I asked inside about them and the lady at the counter hadn't even noticed them. She said they must have been put in during her lunch break, which seems odd considering how long it would take. Anyway, I took some pictures because I knew it must have something to do with fiber optics.

I wondered why the town hadn't been dug up but some Googling revealed that they have horizontal drilling machines for this job.

I took a different route home and noticed that there was some new road paint. In addition to new dashed white lines (not shown), someone had spray-painted MH next to a BellSouth manhole cover.

A manhole cover directly across the street from my yard.

This photo doesn't show the fiberglass junction box nearby, or the fiber optic cable on a utility pole. The cable on the pole leads into the ground near the fiberglass box. There are a bunch of poles carrying fiber optic cable around this part of town. I saw an AT&T truck putting new fiber on the poles several months ago, and the manhole cover has a BellSouth logo so I believe this section is managed by them. With all of this infrastructure available right by my yard, I wonder why AT&T refuses to connect me to their fiber.

Anyway, I don't think it's a coincidence that the paint appeared at the same time as those tubes.

Today I went to check if anything had happened and, to my surprise, there was a call-before-you-dig marker beside a new fiberglass junction box set partway into the sidewalk.

Apparently this is being done by the Palmetto Rural Telephone Co-op, a company I hadn't heard of before.

This is only 1/4 mile from my house so perhaps the neighborhood will be offered better Internet service. I can't imagine why a small-town post office, or any post office for that matter, would need its own dedicated fiber lines.

UTSC Datagram Specification

I finalized this in 2017 but forgot to release it. This is the format for sending files (aka datacasting) across a dedicated UTSC channel. With such a setup you could send about 10 GiB of files per day.

utsc_datagram_finished_release.txt

New data fuzzer

The main focus of my UTSC standard is reliability so I needed a way to test how it responds to bit errors. Randomly corrupting data is called "fuzzing" but I couldn't find a program that was easy to use so I wrote one in Liberty BASIC.

My program takes a file to be fuzzed, and another file containing high-quality random bytes. It uses 24-bit values from the random file to get byte positions to fuzz, and uses "mod 8" to get the bit position to flip. This means it can randomly flip bits in files up to 16 MiB.

You can fuzz anything you like, but I wanted to fuzz audio so I could see how it would sound when the signal is weak and bits are being corrupted. The sound worked on analog TV when the signal was too weak for the picture to come through, and I want UTSC to do the same. First I tried Opus files. They can handle some bit errors but they stop playing altogether if there are too many. And if the header is corrupted, they won't play at all. I added a header-skipping feature to my fuzzer but obviously a real-world signal could lose the header.

Then I got to thinking about WAV audio. It has a very small header (44 bytes) and the audio portion can withstand unlimited bit errors without stopping. Of course, you need the header to know the sample rate and format, but what if "best practices" were defined for UTSC that define a default WAV format? After some tests I found that 24 kHz mono 8-bit WAV files are a good compromise between quality and bandwidth. As I wrote on the LostCarrier.Online Discord channel today, "With 17% bit errors, it degrades like analog sound and fades into the noise rather than glitching." My code was off by a factor of 8, so I meant 2.125% bit errors.

I already specified Opus as the recommended audio format on UTSC, but it can't handle anywhere near enough bit errors to be reliable in bad conditions. Let me demonstrate with a 10-second clip from Syn Cole's "Feel Good" from NoCopyrightSounds.

This falls outside of YouTube use, so hopefully 10 seconds is short enough to fall under Fair Use, but if not I'm including the attribution and I'll gladly swap the clip for something else if the owner complains.

"Syn Cole - Feel Good [NCS Release]"
https://www.youtube.com/watch?v=q1ULJ92aldE
Syn Cole
https://soundcloud.com/syncole
https://www.facebook.com/SynCole
https://twitter.com/SynColeOfficial
https://www.instagram.com/SynCole/




Notice that with only 0.2% of the bits flipped, the Opus file is barely playable. In contrast, the WAV files still contain obvious music even with about 50% errors. I say "about" because since this is a random process, some bits may be flipped twice and be unchanged, so 50% is only a reasonable figure. We can assume it's very close to 50% because of the high quality of the randomness used.

These results make me think that 24 kHz mono 8-bit audio is the optimal format to use if you want to ensure audio reliability at low bandwidth. However, the bandwidth is much higher than Opus. With Opus at 48 kbit/sec, the audio takes about 5.7% of the channel bandwidth, counting overhead. Using WAV as I've described would take 192 kbit/sec, or about 19.24%. That's roughly 4 times as much bandwidth just to make sure the sound gets through.

UTSC offers 1 Mbit/sec of bandwidth. So with Opus audio, about 94% of the bandwidth is available for video compared to 80.75% when using WAV. It's up to the broadcaster to decide if losing 135.8 kbit/sec of video bandwidth is worth it. If extra-high quality is desired, it may not be.

LimeSDR Mini unboxing

Yesterday my 2 LimeSDR Mini's arrived. Before I show its performance, let's see some unboxing photos.

Driver Setup

The LimeSDR is not well documented. You can't just Google "limesdr mini drivers" and expect to find anything. After I put in a lot of trial and error, Jeff from LostCarrier.Online linked me a USB controller driver that somehow makes this work.

So to install the drivers, it looks like you need to start by installing PothosSDR. It won't install drivers, but it will provide a Start Menu link to something called Zadig. Use that to install a driver for your LimeSDR Mini. Then download the USB controller driver and have Device Manager update your LimeSDR with the new drivers. Windows should prefer the USB controller driver and quickly begin upgrading to it once you choose the Update Driver Software option.

This does not work for EXTIO-based programs like HDSDR. You'll need SDRConsole v3 to try your LimeSDR Mini. It has good support for receiving from a LimeSDR, but can't transmit even though there is a Transmit tab. Also, I heard somewhere that 20 MHz is the most bandwidth you can do over USB 2.

I have an NVidia GTX 750 Ti so SDRConsole can use CUDA to accelerate the FFT that generates the waterfall. Still, for me it stutters over 7.5 MHz.

One more thing: choose an antenna once your LimeSDR Mini is running in SDRConsole. I spent a few minutes troubleshooting the blank waterfall before realizing that no antenna was selected.

Reception tests

3 ATSC TV pilots (spikes) at once:

LTE at 2.1 GHz:

I couldn't get SDRAngel or Foobar2000 (with jocover's plugin) to transmit. I'm still trying to figure out how to transmit and when I do I'll write another post showing how to do it.

Jammer on 2018 AM Rally

Last night I heard a looping recorded message jamming AM transmissions on 80 meters. It was mixed in with other QSO's, but here's what I managed to get:

"Why don't you narrow 'er up, because like narrow it up, I'll have a sideband QSO below me, one above me, and I'm the ****** in the middle."

As I said, it was looping but one contestant seemed to think it was a live person. I think it was controlled by a live person, because I remember it following the contest when it went lower in the band, but it was looping, not to mention it was the exact same tone of voice each time. The real proof came during one cycle when it started stuttering like a slow computer. Everyone else's voice was fine, so it wasn't an issue on my end.

Here's a bit I managed to record.

Reading power meters with RTL-AMR

Today I got the RTL-AMR tool working and monitored some power meters. To install it, you'll need to install 2 programs: Git and Go. This program needs to be compiled by Go and that can't happen without Git.

The install instructions are mostly copied from RTL-AMR's official GitHub page.

Download Go from here. Once you've installed it, download Git from here and install it. Lastly, you'll need to make sure your RTL-SDR drivers are installed and then download and extract this ZIP file.

Go to the extracted files, open the "rtl-sdr-release" folder, choose your architecture (x32 is fine by default) and run "rtl_tcp.exe". This will produce a Command Prompt window. Leave it running.

Open another Command Prompt and enter (or Copy-Paste) go get github.com/bemasher/rtlamr.

This will produce a file called "rtlamr.exe". It should be in your "C:\Users\[username]\go\bin" folder. Drag it onto a command prompt window, press Space, and add (without brackets) [> "C:\pop.csv"]. Press Enter to run this command.

This is what it should look like when you press Enter:

That command with the right caret and "C:\pop.csv" means we want to pipe the output into a file.

Now, as long as an antenna is hooked up, it will collect power usage stats from the nearest power meters and save them into the *.CSV file.

Here's what that file should look like in Notepad++:

The Type field in the middle tells us what kind of meter we're reading. Type 7 means electric. Reddit user dongledorr says that gas meters are type 12, but I'm not picking up any of those.

I've written some programs in Liberty BASIC to process these files.

To make it easy to keep this process running, I left RTL-AMR running and just did copy-paste on C:\pop.csv. In Windows 7, this produced "C:\pop - Copy.csv". You'll need to open this copy in Notepad++ and convert the line endings to Windows-style using Edit->EOL Conversion->Windows (CR LF), or the following code won't read it.

Here's the first program to process it into an actual CSV file:

This produces C:\pop2.csv, which may be opened in Excel. You'll want to sort the data. Excel can sort by more than one column, so you'll want to specify that there's a header column, then sort primarily by ID's and secondarily by time. Then save your Excel sheet. I saved mine as C:\pop2sorted.csv.

Now you can process it into a power graph with the following program. It takes the power consumed between intervals and converts it to an average at the end of the interval.

This program doesn't produce a proper CSV file. The values are actually tab-delimited so you can easily copy-paste into Excel.

Here's what the output should look like in Notepad++:

You can copy-paste it into Excel and select a group of lines having the same ID, but only select the last 2 columns. Then insert a data graph and you should get this:

Now you have a graph showing wattage at various times. This is just the simplest way to do it. Other people have much more elaborate ways of collecting and showing the data.

VP9 ELI5 Part 7

Now we need to write the data for the first 32x32 block. Remember that the last chapter ended with us recursively calling decode_partition(). Now we need partition to be PARTITION_NONE because we don't want to split the 32x32 block any further.

Let's go to the beginning of decode_partition(), section 6.4.3 of the spec PDF.

Skip the first IF statement since we're not on any right or bottom boundaries.

num8x8 will be 4 since you could fit 4 8x8 blocks inside the width of a 32x32 block.
halfBlock8x8 is rather obvious: 2.
hasRows and hasCols are both TRUE.

Now let's take care of our partition field. Remember, we need it to be PARTITION_NONE.

Plugging our values into this method, we find that our first probability is 17. For PARTITION_NONE, it's quite easy. We just need to write a 0 with probability 17. Let's add that to our running tally of bits.

Running tally of bits

1, p = 10

1, p = 7

1, p = 6
0, p = 17

Continuing, subsize is going to be BLOCK_32X32, or numerically, 9.

The next line is an IF block. We want to check if subsize is smaller than 8x8 OR if partition is PARTITION_NONE. In our case, the second condition is met so we will execute the statement inside the block, decode_block(r, c, subsize).

MiRow = 0

MiCol = 0

MiSize = BLOCK_32X32

AvailU = FALSE

AvailL = FALSE

Now we need to do mode_info().

This is an intra frame so we'll execute intra_frame_mode_info().

We can skip intra_segment_id() since we chose not to enable segmentation. intra_segment will be 0.

read_skip() reads a value called skip. We want skip to be 0 because we don't want to skip over this block.

Skip is a Tree value, so we need to figure out the probabilities to use when writing it.

Since AvailU and AvailL are both FALSE, ctx will be 0. This means our probability will be skip_prob[0]. We want default_skip_prob[0] since we didn't change any probabilities. default_skip_prob[0] is 192.

The Note at the bottom is interesting because it means that this is not a tree, but instead a normal bit flag expressed as a bool value. That means all we have to do is write 0 with probability 192.

Running tally of bits

1, p = 10

1, p = 7

1, p = 6
0, p = 17
0, p = 192

Now it's time for read_tx_size(1).

allowSelect is 1, but TX_MODE_SELECT is FALSE so we'll execute the code in the ELSE block. tx_size will be the lesser of maxTxSize and tx_mode_to_biggest_tx_size[tx_mode]. tx_mode, as you may recall, has been set to ALLOW_32X32.

maxTxSize is TX_32X32.  tx_mode_to_biggest_tx_size[ALLOW_32X32] = TX_32X32. So, tx_size will be TX_32X32.

Now we have some more state variables:

     ref_frame[0] = INTRA_FRAME

     ref_frame[1] = NONE

     is_inter = 0

Now we reach an IF block that checks to see that we're working on an 8x8 block or larger. Since we are, we'll execute the statements in here and skip the ELSE block underneath.

We need to figure out default_intra_mode, which is a Tree value. Let's look at what intra modes are available.

Before we go on, you may be wondering what these intra modes do. Well, intra modes are smudge functions. You see, compressing a video isn't just about finding differences between frames. You also have to compress as much as you can within each frame. That's what "intra" compression is all about.

So in an intra frame (like the one we're encoding now), you don't just write image data. That would take a lot of space. Instead, you grab the top, left, or both top and left edges of the block and smudge them across, and then only save the difference between the actual image and the smudged block. Here is an example of all the possible smudge functions:

Let's start by getting the necessary probability.

Both abovemode and leftmode will be DC_PRED. That means our probability will be kf_y_mode_probs[DC_PRED][DC_PRED][0] which equals 137.

For simplicity, let's just write a 0 bit, which translates to DC_PRED.

Running tally of bits

1, p = 10

1, p = 7

1, p = 6
0, p = 17
0, p = 192
0, p = 137

Now we have to add some more state variables.

     y_mode = DC_PRED

     sub_modes[0 - 3] = DC_PRED

To finish up intra_frame_mode_info(), we need to write our default_uv_mode. We'll set it to DC_PRED so we can write as little as possible.

kf_uv_mode_probs[DC_PRED][0] = 144, so we need to write a 0 bit with probability 144.

Running tally of bits

1, p = 10

1, p = 7

1, p = 6
0, p = 17
0, p = 192
0, p = 137
0, p = 144

Another state variable:

     uv_mode = DC_PRED

Finally, we're done with mode_info() and we're back in decode_block().

We have another state variable,

     EobTotal = 0

The next step is residual(), which will be covered in the next chapter.