{"@attributes":{"version":"2.0"},"channel":{"title":"K3XEC","link":"https:\/\/k3xec.com\/","description":"Recent content on K3XEC","generator":"Hugo -- gohugo.io","language":"en-us","lastBuildDate":"Tue, 19 May 2026 11:00:00 -0500","item":[{"title":"designing arf, an sdr iq encoding format \ud83d\udc36","link":"https:\/\/k3xec.com\/arf\/","pubDate":"Wed, 15 Apr 2026 11:43:00 -0400","guid":"https:\/\/k3xec.com\/arf\/","description":"<div class=\"hz-alert-ok\">\nInterested in future updates? Follow me on mastodon at\n<a href=\"https:\/\/soylent.green\/@paul\">@paul@soylent.green<\/a>. Posts about\n<code>hz.tools<\/code> will be tagged\n<a href=\"https:\/\/soylent.green\/@paul\/tagged\/hztools\">#hztools<\/a>.<br \/>\n<br \/>\n\ud83d\udc36 Want to jump right to the draft? I'll be maintaining ARF going forward at\n<a href=\"https:\/\/k3xec.com\/draft-tagliamonte-arf-00.txt\">\/draft-tagliamonte-arf-00.txt<\/a>.\n<\/div>\n<p>It&rsquo;s true &ndash; processing data from software defined radios can be a bit\n<a href=\"http:\/\/localhost:1313\/packrat-processing-iq\/\">complex<\/a>\n\ud83d\udc48\ud83d\ude0f\ud83d\udc48 &ndash; which tends to keep all but the most grizzled experts and bravest\nsouls from playing with it. While I wouldn&rsquo;t describe myself as either, I will\nsay that I&rsquo;ve stuck with it for longer than most would have expected of me.\nOne of the biggest takeaways I have from my adventures with software defined\nradio is that there&rsquo;s a lot of cool crossover opportunity between RF and\nnearly every other field of engineering.<\/p>\n<p>Fairly early on, I decided on a very light metadata scheme to track SDR\ncaptures, called <a href=\"https:\/\/k3xec.com\/rfcap\/\">rfcap<\/a>. rfcap has withstood my test\nof time, and I can go back to even my earliest captures and still make sense of\nwhat they are &ndash; IQ format, capture frequencies, sample rates, etc. A huge\npart of this was the simplicity of the scheme (fixed-lengh header, byte-aligned\nto supported capture formats), which made it roughly as easy to work with as a\nraw file of IQ samples.<\/p>\n<p>However, rfcap has a number of downsides. It&rsquo;s only a single, fixed-length\nheader. If the frequency of operation changed during the capture, that change\nis not represented in the capture information. It&rsquo;s not possible to easily\nrepresent mulit-channel coherent IQ streams, and additional metadata is\ncondemned to adjacent text files.<\/p>\n<h1 id=\"arf-archive-of-rf\">ARF (Archive of RF)<\/h1>\n<p>A few years ago, I needed to finally solve some of these shortcomings and tried\nto see if a new format would stick. I sat down and wrote out my design goals\nbefore I started figuring out what it looked like.<\/p>\n<p>First, whatever I come up with must be capable of being streamed and processed\nwhile being streamed. This includes streaming across the network or merely\nwritten to disk as it&rsquo;s being created. No post-processing required. This is\nmostly an artifact of how I&rsquo;ve built all my tools and how I intereact with my\nSDRs. I use them extensively over the network (both locally, as well\nas remotely by friends across my <a href=\"https:\/\/tpl.house\">wider<\/a>\n<a href=\"https:\/\/notes.pault.ag\/tpl\/\">lan<\/a>). This decision sometimes even\nprompts me to do some <a href=\"https:\/\/k3xec.com\/sparky-rtlsdr\/\">crazy things<\/a> from time\nto time.<\/p>\n<p>I need actual, real support for multiple IQ channels from my multi-channel SDRs\n(Ettus, Kerberos\/Kracken SDR, etc) for playing with things like\n<a href=\"https:\/\/k3xec.com\/simulating-phased-arrays\/\">beamforming<\/a>.\nMy new format must be capable of storing\nmultiple streams in a single capture file, rather than a pile of files in\na directory (and hope they&rsquo;re aligned).<\/p>\n<p>Finally, metadata must be capable of being stored in-band. The initial set of\nmetadata I needed to formalize in-stream were <code>Frequency Changes<\/code> and\n<code>Discontinuities<\/code>. Since then, ARF has grown a few more.<\/p>\n<p>After getting all that down, I opted to start at what I thought the simplest\ncontainer would look like,\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Type%E2%80%93length%E2%80%93value\">TLV<\/a>\n(tag-length-value) encoded packets. This is a fairly well trodden path,\nand used by a bunch of existing protocols\n<a href=\"https:\/\/datatracker.ietf.org\/doc\/html\/rfc4880\">we<\/a>\n<a href=\"https:\/\/datatracker.ietf.org\/doc\/html\/rfc4253\">all<\/a>\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/ASN.1\">know<\/a>\nand\n<a href=\"https:\/\/datatracker.ietf.org\/doc\/html\/rfc6184\">love<\/a>.\nEach ARF file (or stream) was a set of\nencoded &ldquo;packets&rdquo; (sometimes called data units in other specs). This means that\nunknown packet types may be skipped (since the length is included) and\nadditional data can be added after the existing fields without breaking\nexisting decoders.<\/p>\n<div class=\"hz-abi\">\n<div type=\"u8\" class=\"hz-abi-yellow hz-abi-2b\">\n<a href=\"#tag-field\">tag<\/a>\n<\/div>\n<div type=\"u8\" class=\"hz-abi-yellow hz-abi-2b\">\n<a href=\"#flags\">flags<\/a>\n<\/div>\n<div type=\"u16\" class=\"hz-abi-yellow hz-abi-4b\">\nlength\n<\/div>\n<div type=\"[]u8\" class=\"hz-abi-yellow hz-abi-Nb\">\nvalue\n<\/div>\n<\/div>\n<div class=\"hz-alert-error\">\n<b>Heads up!<\/b>\nOnce this is posted, I'm not super likely to update this page. Once this\ngoes out, the latest stable copy of the ARF spec is maintained at\n<a href=\"https:\/\/k3xec.com\/draft-tagliamonte-arf-00.txt\">draft-tagliamonte-arf-00.txt<\/a>.\nThis page may quickly become out of date, so if you're actually interested in\nimplementing this, I've put a lot of effort into making the draft\ncomprehensive, and I plan to maintain it as I edit the format.\n<\/div>\n<p>Unlike a &ldquo;traditional&rdquo; TLV structure, I opted to add &ldquo;flags&rdquo; to the top-level\npacket. This gives me a bit of wiggle room down the line, and gives me a\nfeature that I like from ASN.1 &ndash; a &ldquo;critical&rdquo; bit. The critical bit indicates\nthat the packet must be understood fully by implementers, which allows future\nbackward incompatible changes by marking a new packet type as critical. This\nwould only really be done if something meaningfully changed the interpretation\nof the backwards compatible data to follow.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Flag<\/td>\n<td class=\"hz-5-6th\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr><td>0x01<\/td><td>Critical (tag must be understood)<\/td><\/tr>\n<\/tbody>\n<\/table>\n<p>Within each Packet is a <code>tag<\/code> field. This tag indicates how the contents of the\n<code>value<\/code> field should be interpreted.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Tag ID<\/td>\n<td class=\"hz-5-6th\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr><td>0x01<\/td><td><a href=\"#header\">Header<\/a><\/td><\/tr>\n<tr><td>0x02<\/td><td><a href=\"#stream-header\">Stream Header<\/a><\/td><\/tr>\n<tr><td>0x03<\/td><td><a href=\"#samples\">Samples<\/a><\/td><\/tr>\n<tr><td>0x04<\/td><td><a href=\"#frequency-change\">Frequency Change<\/a><\/td><\/tr>\n<tr><td>0x05<\/td><td>Timing<\/td><\/tr>\n<tr><td>0x06<\/td><td><a href=\"#discontinuity\">Discontinuity<\/a><\/td><\/tr>\n<tr><td>0x07<\/td><td><a href=\"#location\">Location<\/a><\/td><\/tr>\n<tr><td>0xFE<\/td><td><a href=\"#vendor-extension\">Vendor Extension<\/a><\/td><\/tr>\n<\/tbody>\n<\/table>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is an example packet which should parse without error.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span> 00, \/\/ tag (0; no subpacket is 0 yet)\n<\/span><\/span><span style=\"display:flex;\"><span> 00, \/\/ flags (0; no flags)\n<\/span><\/span><span style=\"display:flex;\"><span> 00, 00 \/\/ length (0; no data)\n<\/span><\/span><span style=\"display:flex;\"><span> \/\/ data would go here, but there is none\n<\/span><\/span><\/code><\/pre><\/div><p>Additionally, throughout the rest of the subpackets, there are a few unique and\nshared datatypes. I document them all more clearly in the draft, but to quickly\nrun through them here too:<\/p>\n<h3 id=\"uuid\">UUID<\/h3>\n<p>This field represents a globally unique idenfifer, as defined by RFC 9562, as\n16 raw bytes.<\/p>\n<h3 id=\"frequency\">Frequency<\/h3>\n<p>Data encoded in a Frequency field is stored as microhz (1 Hz is stored as\n1000000, 2 Hz is stored as 2000000) as an unsigned 64 bit integer. This has a\nminimum value of 0 Hz, and a maximum value of 18446744073709551615 uHz, or just\nabove 18.4 THz. This is a bit of a tradeoff, but it&rsquo;s a set of issues that I\nwould gladly contend with rather than deal with the related issues with storing\nfrequency data as a floating point value downstream. Not a huge factor, but as\nan aside, this is also how my current generation SDR processing code (<code>sparky<\/code>)\nstores Frequency data internally, which makes conversion between the two\nnatural.<\/p>\n<h3 id=\"iq-samples\">IQ samples<\/h3>\n<p>ARF supports IQ samples in a number of different formats. Part of the idea here\nis I want it to be easy for capturing programs to encode ARF for a specific\nradio without mandating a single iq format representation. For IQ types with\na scalar value which takes more than a single byte, this is always paired\nwith a Byte Order field, to indicate if the IQ scalar values are little or\nbig endian.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">ID<\/td>\n<td class=\"hz-1-6th\">Name<\/td>\n<td class=\"hz-2-3rd\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr><td>0x01<\/td><td>f32<\/td><td>interleaved 32 bit floating point scalar values<\/td><\/tr>\n<tr><td>0x02<\/td><td>i8<\/td> <td>interleaved 8 bit signed integer scalar values<\/td><\/tr>\n<tr><td>0x03<\/td><td>i16<\/td><td>interleaved 16 bit signed integer scalar values<\/td><\/tr>\n<tr><td>0x04<\/td><td>u8<\/td> <td>interleaved 8 bit unsigned integer scalar values<\/td><\/tr>\n<tr><td>0x05<\/td><td>f64<\/td><td>interleaved 64 bit floating point scalar values<\/td><\/tr>\n<tr><td>0x06<\/td><td>f16<\/td><td>interleaved 16 bit floating point scalar values<\/td><\/tr>\n<\/tbody>\n<\/table>\n<h2 id=\"header\">Header<\/h2>\n<p>Each ARF file must start with a specific Header packet. The header contains\ninformation about the ARF stream writ large to follow. Header packets are\nalways marked as &ldquo;critical&rdquo;.<\/p>\n<div class=\"hz-abi\">\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-2b\">\nmagic\n<\/div>\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-2b\">\nflags\n<\/div>\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-2b\">\nstart\n<\/div>\n<div type=\"uuid\" class=\"hz-abi-blue hz-abi-4b\">\nguid\n<\/div>\n<div type=\"uuid\" class=\"hz-abi-blue hz-abi-4b\">\nsite guid\n<\/div>\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\n#st\n<\/div>\n<\/div>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is an example header subpacket (when encoded or decoded this\nwill be found inside an ARF packet as described above) which should parse\nwithout error, with known values.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>00, 00, 00, fa, de, dc, ab, 1e, \/\/ magic\n<\/span><\/span><span style=\"display:flex;\"><span>00, 00, 00, 00, 00, 00, 00, 00, \/\/ flags\n<\/span><\/span><span style=\"display:flex;\"><span>18, 27, a6, c0, b5, 3b, 06, 07, \/\/ start time (1740543127)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\/\/ guid (fb47f2f0-957f-4545-94b3-75bc4018dd4b)\n<\/span><\/span><span style=\"display:flex;\"><span>fb, 47, f2, f0, 95, 7f, 45, 45,\n<\/span><\/span><span style=\"display:flex;\"><span>94, b3, 75, bc, 40, 18, dd, 4b,\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\/\/ site_id (ba07c5ce-352b-4b20-a8ac-782628e805ca)\n<\/span><\/span><span style=\"display:flex;\"><span>ba, 07, c5, ce, 35, 2b, 4b, 20,\n<\/span><\/span><span style=\"display:flex;\"><span>a8, ac, 78, 26, 28, e8, 05, ca\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"stream-header\">Stream Header<\/h2>\n<p>Immediately after the arf <a href=\"#header\">Header<\/a>, some number of Stream Headers\nfollow. There must be exactly the same number of Stream Header packets as are\nindicated by the <code>num streams<\/code> field of the Header. This has the nice effect of\nenabling clients to read all the stream headers without requiring buffering of\n&ldquo;unread&rdquo; packets from the stream.<\/p>\n<div class=\"hz-abi\">\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\nid\n<\/div>\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-2b\">\nflags\n<\/div>\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\nfmt\n<\/div>\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\nbo\n<\/div>\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-2b\">\nrate\n<\/div>\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-2b\">\nfreq\n<\/div>\n<div type=\"uuid\" class=\"hz-abi-blue hz-abi-2b\">\nguid\n<\/div>\n<div type=\"uuid\" class=\"hz-abi-blue hz-abi-2b\">\nsite\n<\/div>\n<\/div>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is an example stream header subpacket (when encoded or decoded\nthis will be found inside an ARF packet as described above) which should parse\nwithout error, with known values.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>00, 01, \/\/ id (1)\n<\/span><\/span><span style=\"display:flex;\"><span>00, 00, 00, 00, 00, 00, 00, 00, \/\/ flags\n<\/span><\/span><span style=\"display:flex;\"><span>01, \/\/ format (float32)\n<\/span><\/span><span style=\"display:flex;\"><span>01, \/\/ byte order (Little Endian)\n<\/span><\/span><span style=\"display:flex;\"><span>00, 00, 01, d1, a9, 4a, 20, 00, \/\/ rate (2 MHz)\n<\/span><\/span><span style=\"display:flex;\"><span>00, 00, 5a, f3, 10, 7a, 40, 00, \/\/ frequency (100 MHz)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\/\/ guid (7b98019d-694e-417a-8f18-167e2052be4d)\n<\/span><\/span><span style=\"display:flex;\"><span>7b, 98, 01, 9d, 69, 4e, 41, 7a,\n<\/span><\/span><span style=\"display:flex;\"><span>8f, 18, 16, 7e, 20, 52, be, 4d,\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\/\/ site_id (98c98dc7-c3c6-47fe-bc05-05fb37b2e0db)\n<\/span><\/span><span style=\"display:flex;\"><span>98, c9, 8d, c7, c3, c6, 47, fe,\n<\/span><\/span><span style=\"display:flex;\"><span>bc, 05, 05, fb, 37, b2, e0, db,\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"samples\">Samples<\/h2>\n<p>Block of IQ samples in the format indicated by this stream&rsquo;s <code>format<\/code> and\n<code>byte_order<\/code> field sent in the related <a href=\"#stream-header\">Stream Header<\/a>.<\/p>\n<div class=\"hz-abi\">\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\nid\n<\/div>\n<div type=\"[]iq\" class=\"hz-abi-blue hz-abi-Nb\">\niq samples\n<\/div>\n<\/div>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is an samples subpacket (when encoded or decoded\nthis will be found inside an ARF packet as described above). The IQ values\nhere are notional (and are either 2 8 bit samples, or 1 16 bit sample,\ndepending on what the related <a href=\"#stream-header\">Stream Header<\/a> was).<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>01, \/\/ id\n<\/span><\/span><span style=\"display:flex;\"><span>ab, cd, ab, cd, \/\/ iq samples\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"frequency-change\">Frequency Change<\/h2>\n<p>The center frequency of the IQ stream has changed since the\n<a href=\"#stream-header\">Stream Header<\/a> or last <a href=\"#frequency-change\">Frequency Change<\/a>\nhas been sent. This is useful to capture IQ streams that are jumping\naround in frequency during the duration of the capture, rather than\nstarting and stopping them.<\/p>\n<div class=\"hz-abi\">\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\nid\n<\/div>\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-4b\">\nfrequency\n<\/div>\n<\/div>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is a frequency change subpacket (when encoded or decoded\nthis will be found inside an ARF packet as described above).<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>01, \/\/ id\n<\/span><\/span><span style=\"display:flex;\"><span>00, 00, b5, e6, 20, f4, 80, 00 \/\/ frequency (200 MHz)\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"discontinuity\">Discontinuity<\/h2>\n<p>Since the last Samples packet for this stream, samples have been dropped\nor not encoded to this stream. This can be used for a stream that has\ndropped samples for some reason, a large gap (radio was needed for something\nelse), or communicating &ldquo;iq snippits&rdquo;.<\/p>\n<div class=\"hz-abi\">\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\nid\n<\/div>\n<\/div>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is a discontinuity subpacket (when encoded or decoded this will\nbe found inside an ARF packet as described above).<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>01, \/\/ id\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"location\">Location<\/h2>\n<p>Up-to-date location as of this moment of the IQ stream, usually from a GPS.\nThis allows for in-band geospatial information to be marked in the IQ stream.\nThis can be used for all sorts of things (detected IQ packet snippits aligned\nwith a time and location or a survey of rf noise in an area)<\/p>\n<div class=\"hz-abi\">\n<div type=\"u64\" class=\"hz-abi-blue hz-abi-2b\">\nflags\n<\/div>\n<div type=\"u8\" class=\"hz-abi-blue hz-abi-1b\">\n<a href=\"#location-geodetic-systems\">sys<\/a>\n<\/div>\n<div type=\"f64\" class=\"hz-abi-blue hz-abi-2b\">\nlat\n<\/div>\n<div type=\"f64\" class=\"hz-abi-blue hz-abi-2b\">\nlong\n<\/div>\n<div type=\"f64\" class=\"hz-abi-blue hz-abi-2b\">\nel\n<\/div>\n<div type=\"f64\" class=\"hz-abi-blue hz-abi-2b\">\naccuracy\n<\/div>\n<\/div>\n<p>The <code>sys<\/code> field indicates the Geodetic system to be used for the provided\n<code>latitude<\/code>, <code>longitude<\/code> and <code>elevation<\/code> fields. The full list of supported\ngeodetic systems is currently just WGS84, but in case something meaningfully\nchanges in the future, it&rsquo;d be nice to migrate forward.<\/p>\n<p>Unfortunately, being a bit of a coward here, the accuracy field is a bit of a\ncop-out. I&rsquo;d really rather it be what we see out of kinematic state estimation\ntools like a kalman filter, or at minimum, some sort of ellipsoid. This is\nneither of those - it&rsquo;s a perfect sphere of error where we pick the largest\nerror in any direction and use that. Truthfully, I can&rsquo;t be bothered to model\nthis accurately, and I don&rsquo;t want to contort myself into half-assing something\nI know I will half-ass just because I know better.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">System<\/td>\n<td class=\"hz-5-6th\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr><td>0x01<\/td><td>\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/World_Geodetic_System#WGS_84\">WGS84 - World Geodetic System 1984<\/a>\n<\/td><\/tr>\n<\/tbody>\n<\/table>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is a location subpacket (when encoded or decoded this will be\nfound inside an ARF packet as described above).<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>00, 00, 00, 00, 00, 00, 00, 00, \/\/ flags\n<\/span><\/span><span style=\"display:flex;\"><span>01, \/\/ system (wgs84)\n<\/span><\/span><span style=\"display:flex;\"><span>3f, f3, be, 76, c8, b4, 39, 58, \/\/ latitude (1.234)\n<\/span><\/span><span style=\"display:flex;\"><span>40, 02, c2, 8f, 5c, 28, f5, c3, \/\/ longitude (2.345)\n<\/span><\/span><span style=\"display:flex;\"><span>40, 59, 00, 00, 00, 00, 00, 00, \/\/ elevation (100)\n<\/span><\/span><span style=\"display:flex;\"><span>40, 24, 00, 00, 00, 00, 00, 00 \/\/ accuracy (10)\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"vendor-extension\">Vendor Extension<\/h2>\n<p>In addition to the fields I put in the spec, I expect that I may need custom\npacket types I can&rsquo;t think of now. There&rsquo;s all sorts of useful data that could\nbe encoded into the stream, so I&rsquo;d rather there be an officially sanctioned\nmechanism that allows future work on the spec without constraining myself.<\/p>\n<p>Just an example, I&rsquo;ve used a custom subpacket to create test vectors, the data\nis encoded into a Vendor Extension, followed by the IQ for the modulated\npacket. If the demodulated data and in-band original data don&rsquo;t match, we&rsquo;ve\nregressed. You could imagine in-band speech-to-text, antenna rotator azimuth\ninformation, or demodulated digital sideband data (like FM HDR data) too. Or\neven things I can&rsquo;t even think of!<\/p>\n<div class=\"hz-abi\">\n<div type=\"uuid\" class=\"hz-abi-blue hz-abi-2b\">\nid\n<\/div>\n<div type=\"[]u8\" class=\"hz-abi-blue hz-abi-Nb\">\ndata\n<\/div>\n<\/div>\n<p>In order to help with checking the basic parsing and encoding of this format,\nthe following is a vendor extension subpacket (when encoded or decoded this\nwill be found inside an ARF packet as described above).<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>\/\/ extension id (b24305f6-ff73-4b7a-ae99-7a6b37a5d5cd)\n<\/span><\/span><span style=\"display:flex;\"><span>b2, 43, 05, f6, ff, 73, 4b, 7a,\n<\/span><\/span><span style=\"display:flex;\"><span>ae, 99, 7a, 6b, 37, a5, d5, cd,\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\/\/ data (0x01, 0x02, 0x03, 0x04, 0x05)\n<\/span><\/span><span style=\"display:flex;\"><span>01, 02, 03, 04, 05\n<\/span><\/span><\/code><\/pre><\/div><h1 id=\"tradeoffs\">Tradeoffs<\/h1>\n<p>The biggest tradeoff that I&rsquo;m not <em>entirely<\/em> happy with is limiting the length\nof a packet to <code>u16<\/code> &ndash; 65535 bytes. Given the u8 sample header, this limits us\nto 8191 32 bit sample pairs at a time. I wound up believing that the overhead in\nterms of additional packet framing is worth it &ndash; because always encoding 4\nbyte lengths felt like overkill, and a dynamic length scheme ballooned\ncodepaths in the decoder that I was trying to keep as easy to change as\npossible as I worked with the format.<\/p>"},{"title":"librtlsdr.so for fun and profit","link":"https:\/\/k3xec.com\/sparky-rtlsdr\/","pubDate":"Fri, 27 Mar 2026 13:30:00 -0400","guid":"https:\/\/k3xec.com\/sparky-rtlsdr\/","description":"<div class=\"hz-alert-ok\">\nInterested in future updates? Follow me on mastodon at\n<a href=\"https:\/\/soylent.green\/@paul\">@paul@soylent.green<\/a>. Posts about\n<code>hz.tools<\/code> will be tagged\n<a href=\"https:\/\/soylent.green\/@paul\/tagged\/hztools\">#hztools<\/a>.<br \/>\n<\/div>\n<p>It&rsquo;s well known and universally agreed that radios are cool. Among the\ncontested field of coolest radios, Software Defined Radios (SDRs) are\ndefinitely the most interesting to me. Out of all of my (entirely too many)\nSDRs I own, the <code>rtlsdr<\/code> is still my #1. It&rsquo;s just <em>good<\/em>. It&rsquo;s a great price,\nextremely capable, reliable, well-supported, and compact. Why bother with\nanything else? Sure, it can&rsquo;t transmit, uses a (fairly weird) <a href=\"https:\/\/k3xec.com\/packrat-processing-iq\/#rtl-sdr\">8 bit unsigned\ninteger IQ representation<\/a>,\nlimited sampling rate, limited frequency range &ndash; but even with all that, it&rsquo;s\nstill the radio I will pack first. Don&rsquo;t get me wrong, I love my Ettus radios,\nPlutoSDRs, HackRFs, my AirspyHF+ - they&rsquo;re great! I just always find myself\nfalling back to an <code>rtl-sdr<\/code>, every time.<\/p>\n<p>Perhaps the best reason to use an <code>rtlsdr<\/code> is the absolutely mind-boggling\namount of cool stuff people have written for it. The <code>rtlsdr<\/code> API is super easy\nto use, widely supported if you&rsquo;re building on top of existing radio processing\nframeworks &ndash; it&rsquo;s still a <em>shock<\/em> to me when something omits <code>rtlsdr<\/code> support.<\/p>\n<h1 id=\"sparky\">sparky<\/h1>\n<p>Over the last 7 years, I&rsquo;ve been learning about radios &ndash; I got my ham radio\nlicense (<code>de K3XEC<\/code>), <a href=\"https:\/\/k3xec.com\/christmas\/\">hacked<\/a>\n<a href=\"https:\/\/k3xec.com\/power-output\/\">on<\/a> <a href=\"https:\/\/k3xec.com\/hztools\/\">some<\/a>\n<a href=\"https:\/\/k3xec.com\/td158\/\">cool<\/a> <a href=\"https:\/\/k3xec.com\/su68g\/\">stuff<\/a> where I&rsquo;ve\nlearned how radios work by &ldquo;doing&rdquo;, and even was lucky enough to give my first\nrf-centric <a href=\"https:\/\/k3xec.com\/paging-all-radio-curious-hackers\/\">talk at districtcon<\/a>.\nEmbarrassingly, I still haven&rsquo;t gotten around to learning how the fancy stuff\nlike <a href=\"https:\/\/www.gnuradio.org\/\">GNU Radio<\/a> works. I&rsquo;m sure I&rsquo;m going to love\nit when I do.<\/p>\n<p>As part of this, I&rsquo;ve also cooked up some very unprofessional formats and\nprotocols I use for convenience. Locally, all my on-disk captures are stored in\n<a href=\"https:\/\/k3xec.com\/rfcap\/\">rfcap<\/a> or more recently <a href=\"https:\/\/k3xec.com\/arf\/\">arf<\/a>,\nwhile direct SDR access at my house is almost entirely a mix of\nthe widely used <a href=\"https:\/\/k3xec.com\/rtl-tcp\/\">rtl-tcp<\/a> protocol, and my\n&ldquo;<code>riq<\/code>&rdquo; protocol (post on this coming soon). Both <code>rtl-tcp<\/code> and <code>riq<\/code> operate\nover the network, so I don&rsquo;t have to bother with plugging things into USB ports,\nand I can share my radios with <a href=\"https:\/\/notes.pault.ag\/tpl\/\">my friends<\/a>.<\/p>\n<p>All of that work sits in my current generation of radio processing code,\n&ldquo;sparky&rdquo; (a reference to\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Spark-gap_transmitter\">spark-gap transmitters<\/a>),\nwhich is a heap of Rust, supporting everything from <code>no_std<\/code> for embedded\nexperiments, conditional support for interfacing with all the radios I\nown, and <code>tokio<\/code>-based async support in addition to blocking i\/o\nfor highly concurrent daemons. This quickly advanced beyond my old Go-based\ncode (<a href=\"https:\/\/github.com\/hztools\/go-sdr\">hz.tools\/go-sdr<\/a>), which I archived\nso I can focus on learning. I still think Go is a great language to write RF\ncode in &ndash; but I can&rsquo;t focus on that tech tree anymore.<\/p>\n<p>Of course, this now poses a new problem &ndash; no one supports my format(s) or\nradio protocol(s), since, well, I&rsquo;m the only one using them. I&rsquo;ve committed a\nfair amount of my hardware to this setup, and yanking it from the rack to try\nsomething out does pose a bit of a pickle. This isn&rsquo;t a huge deal for learning,\nbut it does make it tedious to try out something from the internets.<\/p>\n<h1 id=\"librtlsdrso\">librtlsdr.so<\/h1>\n<p>Thankfully, Rust has robust support for\n<a href=\"https:\/\/faultlore.com\/blah\/c-isnt-a-language\/\">wrap[ping itself] in a grotesque simulacra of C\u2019s skin and mak[ing its] flesh undulate<\/a>,\nwhich is an attractive nuisance if i&rsquo;ve ever seen one. Naturally, my ability\nto restrain myself from engaging in ill-advised rf adventures is basically\nzero, so it&rsquo;s time to do the thing any similarly situated person would do &ndash;\nreimplement the API and ABI of <code>librtlsdr.so<\/code>, backed with <code>sparky<\/code> instead.<\/p>\n<p>Since enumeration of devices is going to be annoying (specifically, they&rsquo;re\nover the network), I decided early-on to rely on an explicit list of\ndevices via a configuration file. I&rsquo;d rather only load that once so programs\ndon&rsquo;t get confused, so I opted to use a\n<a href=\"https:\/\/ftp.gnu.org\/old-gnu\/Manuals\/ld-2.9.1\/html_node\/ld_26.html\">CTOR<\/a>\nto run a stub when the ELF is linked at runtime.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ lightly edited for clarity\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[used]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[expect(unused)]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[unsafe(link_section = <\/span><span style=\"color:#e6db74\">&#34;.init_array&#34;<\/span><span style=\"color:#75715e\">)]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">pub<\/span> <span style=\"color:#66d9ef\">static<\/span> <span style=\"color:#66d9ef\">INITIALIZE<\/span>: <span style=\"color:#a6e22e\">extern<\/span> <span style=\"color:#e6db74\">&#34;C&#34;<\/span> <span style=\"color:#66d9ef\">fn<\/span>() <span style=\"color:#f92672\">=<\/span> sparky_rtlsdr_ctor;\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[unsafe(no_mangle)]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">pub<\/span> <span style=\"color:#66d9ef\">extern<\/span> <span style=\"color:#e6db74\">&#34;C&#34;<\/span> <span style=\"color:#66d9ef\">fn<\/span> <span style=\"color:#a6e22e\">sparky_rtlsdr_ctor<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> config: <span style=\"color:#a6e22e\">Config<\/span> <span style=\"color:#f92672\">=<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#66d9ef\">let<\/span> Ok(config_bytes) <span style=\"color:#f92672\">=<\/span> std::fs::read(<span style=\"color:#e6db74\">&#34;\/etc\/sparky-rtlsdr.toml&#34;<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span> toml::from_slice(<span style=\"color:#f92672\">&amp;<\/span>config_bytes).unwrap()\n<\/span><\/span><span style=\"display:flex;\"><span> } <span style=\"color:#66d9ef\">else<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> Config { device: <span style=\"color:#a6e22e\">vec<\/span><span style=\"color:#f92672\">!<\/span>[] }\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> };\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">CONFIG<\/span>.set(config);\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>Next, it&rsquo;s time to start with the basics. Opening and closing a handle using\n<code>rtlsdr_open<\/code> and <code>rtlsdr_close<\/code>. Given we don&rsquo;t control the runtime, and the\n<code>rtl-sdr<\/code> device handle is opaque (for good reason!), I opted to smuggle a rust\n<code>Box&lt;Device&gt;<\/code> non-FFI safe heap-allocated struct through the device handle\npointer, and let C take ownership of the <code>Box<\/code>. No one should be looking in\nthere anyway.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ lightly edited for clarity\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[unsafe(no_mangle)]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">pub<\/span> <span style=\"color:#66d9ef\">unsafe<\/span> <span style=\"color:#66d9ef\">extern<\/span> <span style=\"color:#e6db74\">&#34;C&#34;<\/span> <span style=\"color:#66d9ef\">fn<\/span> <span style=\"color:#a6e22e\">rtlsdr_open<\/span>(dev: <span style=\"color:#f92672\">*<\/span><span style=\"color:#66d9ef\">mut<\/span> <span style=\"color:#f92672\">*<\/span><span style=\"color:#66d9ef\">mut<\/span> Handle, index: <span style=\"color:#66d9ef\">u32<\/span>) -&gt; <span style=\"color:#a6e22e\">int<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> config <span style=\"color:#f92672\">=<\/span> <span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#66d9ef\">CONFIG<\/span>.device[index <span style=\"color:#66d9ef\">as<\/span> <span style=\"color:#66d9ef\">usize<\/span>];\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> sdr <span style=\"color:#f92672\">=<\/span> <span style=\"color:#66d9ef\">match<\/span> config.load() {\n<\/span><\/span><span style=\"display:flex;\"><span> Ok(v) <span style=\"color:#f92672\">=&gt;<\/span> v,\n<\/span><\/span><span style=\"display:flex;\"><span> Err(err) <span style=\"color:#f92672\">=&gt;<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">1<\/span>;\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> };\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> handle <span style=\"color:#f92672\">=<\/span> Box::new(Handle { config, sdr });\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">unsafe<\/span> { <span style=\"color:#f92672\">*<\/span>dev <span style=\"color:#f92672\">=<\/span> Box::into_raw(handle) };\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[unsafe(no_mangle)]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">pub<\/span> <span style=\"color:#66d9ef\">unsafe<\/span> <span style=\"color:#66d9ef\">extern<\/span> <span style=\"color:#e6db74\">&#34;C&#34;<\/span> <span style=\"color:#66d9ef\">fn<\/span> <span style=\"color:#a6e22e\">rtlsdr_close<\/span>(dev: <span style=\"color:#f92672\">*<\/span><span style=\"color:#66d9ef\">mut<\/span> Handle) -&gt; <span style=\"color:#a6e22e\">int<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> dev <span style=\"color:#f92672\">=<\/span> <span style=\"color:#66d9ef\">unsafe<\/span> { Box::from_raw(dev) };\n<\/span><\/span><span style=\"display:flex;\"><span> drop(dev);\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>With that in place, we can chip away at the API surface, translating calls\nas best as we can. I won&rsquo;t bother listing it all, since it&rsquo;s not very\ninteresting &ndash; but here&rsquo;s an example implementation of <code>rtlsdr_set_sample_rate<\/code>\nand <code>rtlsdr_get_sample_rate<\/code>. These calls are translating from an rtl-sdr\nfrequency (which is a <code>u32<\/code> containing the value as Hz) into a sparky Frequency\ntype, and invoking <code>get_sample_rate<\/code> or <code>set_sample_rate<\/code> on the device&rsquo;s\nrust handle. Since each device implements the sparky <code>Sdr<\/code> trait, the actual\nunderlying device doesn&rsquo;t matter much here.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[unsafe(no_mangle)]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">pub<\/span> <span style=\"color:#66d9ef\">unsafe<\/span> <span style=\"color:#66d9ef\">extern<\/span> <span style=\"color:#e6db74\">&#34;C&#34;<\/span> <span style=\"color:#66d9ef\">fn<\/span> <span style=\"color:#a6e22e\">rtlsdr_set_sample_rate<\/span>(dev: <span style=\"color:#f92672\">*<\/span><span style=\"color:#66d9ef\">mut<\/span> Handle, rate: <span style=\"color:#66d9ef\">u32<\/span>) -&gt; <span style=\"color:#a6e22e\">int<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> dev <span style=\"color:#f92672\">=<\/span> <span style=\"color:#66d9ef\">unsafe<\/span> { <span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#66d9ef\">mut<\/span> <span style=\"color:#f92672\">*<\/span>dev };\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> rate <span style=\"color:#f92672\">=<\/span> Frequency::from_hz(rate <span style=\"color:#66d9ef\">as<\/span> <span style=\"color:#66d9ef\">i64<\/span>);\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#66d9ef\">let<\/span> Err(err) <span style=\"color:#f92672\">=<\/span> dev.sdr.set_sample_rate(dev.channel, rate) {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">1<\/span>;\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">#[unsafe(no_mangle)]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">pub<\/span> <span style=\"color:#66d9ef\">unsafe<\/span> <span style=\"color:#66d9ef\">extern<\/span> <span style=\"color:#e6db74\">&#34;C&#34;<\/span> <span style=\"color:#66d9ef\">fn<\/span> <span style=\"color:#a6e22e\">rtlsdr_get_sample_rate<\/span>(dev: <span style=\"color:#f92672\">*<\/span><span style=\"color:#66d9ef\">mut<\/span> Handle) -&gt; <span style=\"color:#66d9ef\">u32<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> dev <span style=\"color:#f92672\">=<\/span> <span style=\"color:#66d9ef\">unsafe<\/span> { <span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#66d9ef\">mut<\/span> <span style=\"color:#f92672\">*<\/span>dev };\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> freq <span style=\"color:#f92672\">=<\/span> <span style=\"color:#66d9ef\">match<\/span> dev.sdr.get_sample_rate(dev.channel) {\n<\/span><\/span><span style=\"display:flex;\"><span> Ok(freq) <span style=\"color:#f92672\">=&gt;<\/span> freq,\n<\/span><\/span><span style=\"display:flex;\"><span> Err(err) <span style=\"color:#f92672\">=&gt;<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#ae81ff\">0<\/span>;\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> };\n<\/span><\/span><span style=\"display:flex;\"><span> freq.as_hz() <span style=\"color:#66d9ef\">as<\/span> <span style=\"color:#66d9ef\">u32<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>After repeating this process for the rest of the stubs I could (and otherwise\nsetting error conditions if the functionality is not supported), I was ready to\ntry it out. Within sparky, I patched my &ldquo;MockSDR&rdquo; (basically a <code>Sdr<\/code> traited\nMock type) to implement the same testmode IQ protocol that the RTL-SDR has, and\ndecided to see if <code>rtl_test<\/code> from <code>apt<\/code> without any changes could be fooled.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>$ rtl_test\n<\/span><\/span><span style=\"display:flex;\"><span>No supported devices found.\n<\/span><\/span><\/code><\/pre><\/div><p>Great, cool. No devices plugged in. Looks great. Let&rsquo;s try it with my\n<code>librtlsdr.so<\/code> <code>LD_PRELOAD<\/code>-ed into the binary first:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>$ LD_PRELOAD=target\/release\/librtlsdr.so rtl_test\n<\/span><\/span><span style=\"display:flex;\"><span>Found 1 device(s):\n<\/span><\/span><span style=\"display:flex;\"><span> 0: hz.tools, mock sdr, SN: totally legit no tricks\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>Using device 0: sparky mock sdr\n<\/span><\/span><span style=\"display:flex;\"><span>Supported gain values (0):\n<\/span><\/span><span style=\"display:flex;\"><span>Sampling at 2048000 S\/s.\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>Info: This tool will continuously read from the device, and report if\n<\/span><\/span><span style=\"display:flex;\"><span>samples get lost. If you observe no further output, everything is fine.\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>Reading samples in async mode...\n<\/span><\/span><span style=\"display:flex;\"><span>^CSignal caught, exiting!\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>User cancel, exiting...\n<\/span><\/span><span style=\"display:flex;\"><span>Samples per million lost (minimum): 0\n<\/span><\/span><span style=\"display:flex;\"><span>$\n<\/span><\/span><\/code><\/pre><\/div><p>Outstanding. Even more outstandingly, if I change my testmode implementation to\nskip samples, <code>rtl_test<\/code> correctly reports the errors &ndash; I think it&rsquo;s showing\npromise! On to try the real endgame here &ndash; let&rsquo;s have our new <code>librtlsdr.so<\/code>\nconnect to an <code>rtl-tcp<\/code> endpoint and see if <code>rtl_fm<\/code> works:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>LD_PRELOAD=target\/release\/librtlsdr.so \\\n<\/span><\/span><span style=\"display:flex;\"><span> rtl_fm -d 1 -s 120k -E deemp -M fm -f 90.9M | \\\n<\/span><\/span><span style=\"display:flex;\"><span> ffplay -f s16le -ar 120k -i -\n<\/span><\/span><span style=\"display:flex;\"><span>Found 2 device(s):\n<\/span><\/span><span style=\"display:flex;\"><span> 0: hz.tools, mock sdr, SN: totally legit no tricks\n<\/span><\/span><span style=\"display:flex;\"><span> 1: hz.tools, rtl-tcp, SN: node2.rf.lan:1202\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>Using device 1: sparky rtltcp node2\n<\/span><\/span><span style=\"display:flex;\"><span>Tuner gain set to automatic.\n<\/span><\/span><span style=\"display:flex;\"><span>Tuned to 91170000 Hz.\n<\/span><\/span><span style=\"display:flex;\"><span>Oversampling input by: 9x.\n<\/span><\/span><span style=\"display:flex;\"><span>Oversampling output by: 1x.\n<\/span><\/span><span style=\"display:flex;\"><span>Buffer size: 7.59ms\n<\/span><\/span><span style=\"display:flex;\"><span>Sampling at 1080000 S\/s.\n<\/span><\/span><span style=\"display:flex;\"><span>Output at 120000 Hz.\n<\/span><\/span><\/code><\/pre><\/div><p>And there it was! Not the best audio quality (mostly due to my inability to\ncorrectly read the <code>rtl_fm<\/code> manpage to tune the filter and\ndownsample\/oversampling rates to audio), but it&rsquo;s <em>definitely<\/em> passable.\nI figured I&rsquo;d try something that was a bit more interesting next &ndash; <code>gqrx<\/code>,\nsince it&rsquo;s super handy, I use it a ton, and will definitely amuse me to no\nend. To my surprise and delight, <code>LD_PRELOAD=target\/release\/librtlsdr.so gqrx<\/code>\nwound up running, and I saw my devices pop right up in the setting menu:<\/p>\n<img src=\"https:\/\/k3xec.com\/imgs\/sparky-rtlsdr\/gqrx-settings.png\" alt=\"\" style=\"max-width: 20em;\" \/>\n<p>Huge. Huge. Amazing. It did crash as soon as I tried to actually <em>use<\/em> the\nradio, but after fixing a few dangling bugs in the API surface (and some\nassumptions I think some underlying gnuradio driver may be making that I need\nto double check in the code), I was able to get a super solid stream of\nbroadcast fm radio, with gqrx being none the wiser. It thought it was\n&ldquo;just&rdquo; talking to the device it knows as <code>rtl=1<\/code>.<\/p>\n<img src=\"https:\/\/k3xec.com\/imgs\/sparky-rtlsdr\/gqrx-waterfall.png\" alt=\"\" \/>\n<p>Nice. I can&rsquo;t wait to try this with the rest of the rtl-sdr based tools I like\nhaving around using my <code>riq<\/code> protocol next. I don&rsquo;t think that&rsquo;ll be worth a\npost, but hopefully I&rsquo;ll get around to publishing details on that stack next.<\/p>\n<h1 id=\"epilogue\">epilogue<\/h1>\n<p>Well. That&rsquo;s it. End of story. A bit anti-climatic, sure. While this new shim\nwill provide me endless minutes of mild amusement, I could see using this to\nexpose my sparky testing utilities via <code>librtlsdr.so<\/code> &ndash; my &ldquo;mock sdr&rdquo; driver\nallows for replaying captures off disk, which could be interesting to make sure\nthat signals are still properly decoded after changes, or instrument\nperformance changes (via SNR, BER, packets observed, etc) on reference samples\nI have on my NAS. Maybe that&rsquo;ll come in handy one day!<\/p>\n<p>Truth be told, I&rsquo;m not sure I actually want to encourage anyone to do this for\nreal (although I think I&rsquo;ll definitely be using it on my LAN to see what\nhappens). I also don&rsquo;t have a repo to share &ndash; I don&rsquo;t particularly feel with\ndealing with the secondary effects of publishing <code>sparky<\/code> (and <code>sparky-rtlsdr<\/code>)\nyet, since i&rsquo;m still getting my feet under me on the radio aspect of all this.<\/p>\n<p>I&rsquo;ll be sure to post updates if anything changes with this here (tagged\n<a href=\"https:\/\/k3xec.com\/tags\/sparky\/\">sparky<\/a>) and at\n<a href=\"https:\/\/soylent.green\/@paul\">@paul@soylent.green<\/a>.\nI can&rsquo;t wait to post more about some of the odd sidequests (like this one!)\ni&rsquo;ve completed over the last few years &ndash; I&rsquo;ve been waiting to feel\nconfident that my work has matured and was withstood the new problems i&rsquo;ve\nthrown at it, and it largely has.<\/p>\n<p>It&rsquo;s my hope that these projects (and this project in particular) has provided\na glimpse into the world of software defined radio for my systems friends, and\na bit about systems for my radio friends. It&rsquo;s not <em>all<\/em> magic, and I hope\nsomeone out there feels inclined to have some fun with radios themselves!<\/p>"},{"title":"Paging all Radio Curious Hackers","link":"https:\/\/k3xec.com\/paging-all-radio-curious-hackers\/","pubDate":"Mon, 02 Feb 2026 09:50:00 -0500","guid":"https:\/\/k3xec.com\/paging-all-radio-curious-hackers\/","description":"<p>After years of thinking about and learning about how radios work, I figured it\nwas high-time to start to more aggressively share the things i&rsquo;ve been\nlearning. I had a ton of fun at <a href=\"https:\/\/www.districtcon.org\/\">DistrictCon<\/a>\nyear 0, so it was a pretty natural place to pitch an RF-focused introductory\ntalk.<\/p>\n<p>I was selected for Year 1, and able to give my first ever RF related talk about\nhow to set off restaurant pagers (including one on stage!) by reading and\nwriting IQ directly using a little bit of stdlib only Python.<\/p>\n<p>This talk is based around the work I&rsquo;ve written about previously\n(<a href=\"https:\/\/k3xec.com\/christmas\/\">here<\/a>, <a href=\"https:\/\/k3xec.com\/td158\/\">here<\/a> and\n<a href=\"https:\/\/k3xec.com\/su68g\/\">here<\/a>), but the &ldquo;all-in-one&rdquo; form factor was\nsomething I was hoping would help encourage folks out there to take a look\nunder the hood of some of the gear around them.<\/p>\n<iframe width=\"750\" height=\"400\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/2EgNs88Ombo\" title=\"Paul Tagliamonte | Paging all radio curious hackers!\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<p>(In case the iframe above isn&rsquo;t working, direct link to the YouTube video\nrecording is <a href=\"https:\/\/www.youtube.com\/watch?v=jEYYnWuRAH8\">here<\/a>)<\/p>\n<p>I&rsquo;ve posted my slides from the talk at\n<a href=\"https:\/\/k3xec.com\/pdf\/PARCH.pdf\">PARCH.pdf<\/a> to hopefully give folks some time\nto flip through them directly.<\/p>\n<p>All in all, the session was great &ndash; It was truly humbling to see so many\nfolks interested in hearing me talk about radios. I had a bit of an own-goal in\npicking a 20 minute form-factor, so the talk is paced wrong (it feels like it\nwent way too fast). Hopefully being able to see the slides and pause the video\nis helpful.<\/p>\n<p>We had a short ad-hoc session after where I brought two sets of pagers and my\npower switch; but unfortunately we didn&rsquo;t have anyone who was able to trigger\nany of the devices on their own (due to a mix of time between sessions and\ncomputer set-up). Hopefully it was enough to get folks interested in trying\nthis on their own!<\/p>"},{"title":"Reverse Engineering (another) Restaurant Pager system \ud83c\udf7d\ufe0f","link":"https:\/\/k3xec.com\/su68g\/","pubDate":"Tue, 04 Mar 2025 10:00:00 -0500","guid":"https:\/\/k3xec.com\/su68g\/","description":"<p>Some of you may remember that I recently felt a bit underwhelmed\nby the last <a href=\"https:\/\/k3xec.com\/td158\/\">pager<\/a> I reverse engineered &ndash; the Retekess TD-158,\nmostly due to how intuitive their design decions were. It was pretty easy\nto jump to conclusions because they had made some pretty good decisions on\nhow to do things.<\/p>\n<p>I figured I&rsquo;d spin the wheel again and try a new pager system &ndash; this time I\nwent for a SU-68G-10 pager, since I recognized the form factor as another\nfairly common unit I&rsquo;ve seen around town. Off to Amazon I went, bought a set,\nand got to work trying to track down the FCC filings on this model. I\neventually found what seemed to be the right make\/model, and it, once again,\nindicated that this system should be operating in the <code>433 MHz<\/code> ISM band likely\nusing OOK modulation. So, figured I&rsquo;d start with the center of the band (again)\nat <code>433.92 MHz<\/code>, take a capture, test my luck, and was greeted with a now very\nfamiliar sight.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/su68g\/packet.png\" alt=\"\"><\/p>\n<p>Same as the last goarounds, except the premable here is a <code>0<\/code> symbol followed\nby 6-ish symbol durations of no data, followed by 25 bits of a packet. Careful\nreaders will observe 26 symbols above after the preamble &ndash; I did too! The last\n<code>0<\/code> in the screenshot above is not actually a part of the packet &ndash; rather,\nit&rsquo;s part of the next packet&rsquo;s preamble. Each packet is packed in pretty tight.<\/p>\n<h1 id=\"by-hand-demodulation\">By Hand Demodulation<\/h1>\n<p>Going off the same premise as last time, I figured i&rsquo;d give it a manual demod\nand see what shakes out (again). This is now the third time i&rsquo;ve run this play,\nso check out either of my prior <a href=\"https:\/\/k3xec.com\/christmas\/\">two<\/a> <a href=\"https:\/\/k3xec.com\/td158\/\">posts<\/a> for a\nbetter written description of what&rsquo;s going on here &ndash; I&rsquo;ll skip all the details\nsince i&rsquo;d just be copy-pasting from those posts into here. Long story short, I\ndemodulated a call for pager 1, call for pager 10, and a power off command.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">What<\/td>\n<td class=\"hz-5-6th\">Bits<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr><td>Call 1 <\/td><td><code>1101111111100100100000000<\/code><\/td><\/tr>\n<tr><td>Call 10<\/td><td><code>1101111111100100010100000<\/code><\/td><\/tr>\n<tr><td>Off <\/td><td><code>1101111111100111101101110<\/code><\/td><\/tr>\n<\/tbody>\n<\/table>\n<p>A few things jump out at me here &ndash; the first 14 bits are fixed (in my case,\n<code>11011111111001<\/code>), which means some mix of preamble, system id, or other\nsystem-wide constant. Additionally, The last 9 bits also look like they are our\npager &ndash; the <code>1<\/code> and <code>10<\/code> pager numbers (LSB bit order) jump right out\n(<code>100000000<\/code> and <code>010100000<\/code>, respectively). That just leaves the two remaining\nbits which look to be the &ldquo;action&rdquo; &ndash; <code>00<\/code> for a &ldquo;Call&rdquo;, and <code>11<\/code> for a &ldquo;Power\noff&rdquo;. I don&rsquo;t super love this since command has two bits rather than one, the\nbase station ID seems really long, and a 9-bit Pager ID is just weird. Also,\nwhat is up with that power-off pager id? Weird. So, let&rsquo;s go and see what we\ncan do to narrow down and confirm things by hand.<\/p>\n<h1 id=\"testing-bit-flips\">Testing bit flips<\/h1>\n<p>Rather than call it a day at that, I figure it&rsquo;s worth a bit of diligence to\nmake sure it&rsquo;s all correct &ndash; so I figured we should try sending packets to\nmy pagers and see how they react to different messages after flipping bits\nin parts of the packet.<\/p>\n<p>I implemented a simple base station for the pagers using my Ettus B210mini, and\nthrew together a simple OOK modulator and transmitter program which allows me\nto send specifically crafted test packets on frequency. Implementing the base\nstation is pretty straightforward, because of the modulation of the signal\n(OOK), it&rsquo;s mostly a matter of setting a buffer to <code>1<\/code> and <code>0<\/code> for where the\ncarrier signal is on or off timed to the sample rate, and sending that off to\nthe radio. If you&rsquo;re interested in a more detailed writeup on the steps\ninvolved, there&rsquo;s a bit more in my <a href=\"https:\/\/k3xec.com\/christmas\/\">christmas tree post<\/a>.<\/p>\n<p>First off, I&rsquo;d like to check the base id. I want to know if all the bits in\nwhat I&rsquo;m calling the &ldquo;base id&rdquo; are truly part of the base station ID, or\nperhaps they have some other purpose (version, preamble?). I wound up following\na three-step process for every base station id:<\/p>\n<ul>\n<li>Starting with an unmodified call packet for the pager under test:\n<ul>\n<li>Flip the Nth bit, and transmit the call. See if the pager reacts.<\/li>\n<li>Hold &ldquo;SET&rdquo;, and pair the pager with the new packet.<\/li>\n<li>Transmit the call. See if the pager reacts.<\/li>\n<li>After re-setting the ID, transmit the call with the physical base station,\nsee if the pager reacts.<\/li>\n<\/ul>\n<\/li>\n<li>Starting with an unmodified off packet for the pager system<\/li>\n<li>Flip the Nth bit, transmit the off, see if the pager reacts.<\/li>\n<\/ul>\n<p>What wound up happening is that changing any bit in the first 14 bits meant\nthat the packet no longer worked with any pager until it was re-paired, at\nwhich point it begun to work again. This likely means the first 14 bits are\npart of the base station ID &ndash; and not static between base stations, or some\nconstant like a version or something. All bits appear to be used.<\/p>\n<p>I repeated the same process with the &ldquo;command&rdquo; bits, and found that only <code>11<\/code>\nand <code>00<\/code> caused the pagers to react for the pager ids i&rsquo;ve tried.<\/p>\n<p>I repeated this process one last time with the &ldquo;pager id&rdquo; bits this time, and\nfound the last bit in the packet isn&rsquo;t part of the pager ID, and can be either\na <code>1<\/code> or a <code>0<\/code> and still cause the pager to react as if it were a 0. This means\nthat the last bit is unknown but it has no impact on either a power off or\ncall, and all messages sent by my base station always have a 0 set. It&rsquo;s not\nclear if this is used by anything &ndash; likely not since setting a bit there\ndoesn&rsquo;t result in any change of behavior I can see yet.<\/p>\n<h1 id=\"final-packet-structure\">Final Packet Structure<\/h1>\n<p>After playing around with flipping bits and testing, the final structure\nI was able to come up with based on behavior I was able to observe from\ntransmitting hand-crafted packets and watching pagers buzz:<\/p>\n<div class=\"hz-abi\">\n<div type=\"14 bits\" class=\"hz-abi-green hz-abi-2b\">base id<\/div>\n<div type=\"2 bits\" class=\"hz-abi-green hz-abi-2b\">command<\/div>\n<div type=\"8 bits\" class=\"hz-abi-green hz-abi-2b\">pager id<\/div>\n<div type=\"1 bit\" class=\"hz-abi-green hz-abi-2b\">???<\/div>\n<\/div>\n<h2 id=\"commands\">Commands<\/h2>\n<p>The <code>command<\/code> section bit comes in two flavors &ndash; either a &ldquo;call&rdquo; or an &ldquo;off&rdquo;\ncommand.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Type<\/td>\n<td class=\"hz-1-6th\">Id (2 bits)<\/td>\n<td class=\"hz-4-6th\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr><td>Call<\/td><td>00<\/td><td>Call the pager identified by the id in <code>pager id<\/code><\/td><\/tr>\n<tr><td>Off<\/td><td>11<\/td><td>Request pagers power off, <code>pager id<\/code> is always <code>10110111<\/code><\/td><\/tr>\n<\/tbody>\n<\/table>\n<p>As for the actual RF PHY characteristics, here&rsquo;s my best guesses at what&rsquo;s\ngoing on with them:<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">What<\/td>\n<td class=\"hz-5-6th\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Center Frequency<\/td>\n<td>433.92 MHz<\/td>\n<\/tr>\n<tr>\n<td>Modulation<\/td>\n<td>OOK<\/td>\n<\/tr>\n<tr>\n<td>Symbol Duration<\/td>\n<td>1300us<\/td>\n<\/tr>\n<tr>\n<td>Bits<\/td>\n<td>25<\/td>\n<\/tr>\n<tr>\n<td>Preamble<\/td>\n<td>325us of carrier, followed by 8800us of no carrier<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>I&rsquo;m not 100% on the timings, but they appear to be close enough to work\nreliably. Same with the center frequency, it&rsquo;s roughly right but there\nmay be a slight difference i&rsquo;m missing.<\/p>\n<h1 id=\"lingering-questions\">Lingering Questions<\/h1>\n<p>This was all generally pretty understandable &ndash; another system that had some\ngood decisions, and wasn&rsquo;t too bad to reverse engineer. This was a bit more fun\nto do, since there was a bit more ambiguity here, but still not crazy. At least\nthis one was a bit more ambiguous that needed a bit of followup to confirm\nthings, which made it a bit more fun.<\/p>\n<p>I am left with a few questions, though &ndash; which I&rsquo;m kinda interested in\nunderstanding, but I&rsquo;ll likely need a lot more data and\/or original source:<\/p>\n<p>Why is the &ldquo;command&rdquo; two bits here? This was a bit tough to understand because\nof the number of bits they have at their disposal &ndash; given the one last bit at\nthe end of the packet that doesn&rsquo;t seem to do anything, there&rsquo;s no reason this\ncouldn&rsquo;t have been a 16 bit base station id, and an 8 bit pager id along with a\nsingle bit command (call or off).<\/p>\n<p>When sending an &ldquo;off&rdquo; &ndash; why is power off that bit pattern? Other pager IDs\ndon&rsquo;t seem to work with &ldquo;off&rdquo;, so it has some meaning, but I&rsquo;m not sure what\nthat is. You press and hold 9 on the physical base station, but the code winds\nup coming out to <code>0xED<\/code>, <code>237<\/code> or maybe <code>-19<\/code> if it&rsquo;s signed. I can&rsquo;t quite\nfigure out <em>why<\/em> it&rsquo;s this value. Are there other codes?<\/p>\n<p>Finally &ndash; what&rsquo;s up with the last bit? Why is it 25 bits and not 24? It must\ntake more work to process something that isn&rsquo;t 8 bit aligned &ndash; and all for\nsomething that&rsquo;s not being used!<\/p>"},{"title":"Reverse Engineering a Restaurant Pager system \ud83c\udf7d\ufe0f","link":"https:\/\/k3xec.com\/td158\/","pubDate":"Fri, 14 Jun 2024 01:07:00 -0400","guid":"https:\/\/k3xec.com\/td158\/","description":"<p>It&rsquo;s been a while since I played with something new &ndash; been stuck in a bit of a\nrut with radios recently - working on refining and debugging stuff I mostly\nunderstand for the time being. The other day, I was out getting some food and I\nidly wondered how the restaurant pager system worked. Idle curiosity gave way\nto the realization that I, in fact, likely had the means and ability to answer\nthis question, so I bought the first set of the most popular looking restaurant\npagers I could find on eBay, figuring it&rsquo;d be a fun multi-week adventure.<\/p>\n<h1 id=\"order-up\">Order up!<\/h1>\n<p>I wound up buying a Retekess brand TD-158 Restaurant Pager System (they looked\nlike ones I&rsquo;d seen before and seemed to be low-cost and popular), and quickly\nafter, had a pack of 10 pagers and a base station in-hand. The manual stated\nthat the radios operated at <code>433 MHz<\/code> (cool! can do! Love a good ISM band\ndevice), and after taking an initial read through the manual for tips on the\nPHY, I picked out a few interesting things. First is that the base station ID\nwas limited to 0-999, which is weird because it means the limiting factor is\nlikely the base-10 display on the base station, not the protocol &ndash; we need\nenough bits to store 999 &ndash; at least 10 bits. Nothing else seemed to catch my\neye, so I figured may as well jump right to it.<\/p>\n<p>Not being the type to mess with success, I did exactly the same thing as I did\nin my <a href=\"https:\/\/k3xec.com\/christmas\">christmas tree post<\/a>, and took a capture at <code>433.92MHz<\/code>\nsince it was in the middle of the band, and immediately got deja-vu. Not only\nwas the signal at <code>433.92MHz<\/code>, but throwing the packet into <code>inspectrum<\/code> gave\nme the <em>identical<\/em> plot of the OOK encoding scheme.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/td158\/td158-packet.png\" alt=\"\"><\/p>\n<p>Not just similar &ndash; identical. The only major difference was the baud rate and\nbit structure of the packets, and the only minor difference was the existence\nof what I think is a wakeup preamble packet (of all zeros), rather than a\npreamble symbol that lasted longer than usual PHY symbol (which makes this\npager system a bit easier to work with than my tree, IMHO).<\/p>\n<p>Getting down to work, I took some measurements to determine what the symbol\nduration was over the course of a few packets, I was able to determine the\nsymbol rate was somewhere around <code>858<\/code> microseconds (<code>0.000858<\/code> seconds per\nsymbol), which is a weird number, but maybe I&rsquo;m slightly off or there&rsquo;s some\nlarger math I&rsquo;m missing that makes this number satisfyingly round (internal\nlow cost crystal clock or something? I assume this is some hardware constraint\nwith the pager?)<\/p>\n<p>Anyway, good enough. Moving along, let&rsquo;s try our hand at a demod &ndash; let&rsquo;s just\nassume it&rsquo;s all the same as the chrismas tree post and demod ones and zeros\nthe same way here. That gives us 26 bits:<\/p>\n<pre tabindex=\"0\"><code>00001101110000001010001000\n<\/code><\/pre><p>Now, I know we need at least 10 bits for the base station ID, some number\nof bits for the pager ID, and some bits for the command. This was a capture\nof me hitting &ldquo;call&rdquo; from a base station ID of 55 to a pager with the ID of\n10, so let&rsquo;s blindly look for 10 bit chunks with the numbers we&rsquo;re looking for:<\/p>\n<pre tabindex=\"0\"><code>0000110111 0000001010 001000\n<\/code><\/pre><p>Jeez. First try. 10 bits for the base station ID (55 in binary is\n<code>0000110111<\/code>), 10 bits for the pager ID (10 in binary is <code>0000001010<\/code>), which\nleaves us with 6 bits for a command (and maybe something else too?) &ndash; which is\n<code>8<\/code> here. Great, cool, let&rsquo;s work off that being the case and revisit it if\nwe hit bugs.<\/p>\n<p>Besides our data packet, there&rsquo;s also a &ldquo;preamble&rdquo; packet that I&rsquo;ll add in,\nin case it&rsquo;s used for signal detection or wakeup or something &ndash; which is\nfairly easy to do since it&rsquo;s the same packet structure as the above, just\nall zeros. Very kind of them to leave it with the same number of bits and\nencoding scheme &ndash; it&rsquo;s nice that it can live outside the PHY.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/td158\/td158-preamble.png\" alt=\"\"><\/p>\n<p>Once I got here, I wrote a quick and dirty modulator, and was able to ring up\npagers! Unmitigated success and good news &ndash; only downside was that it took me\na single night, and not the multi-week adventure I was looking for. Well, let&rsquo;s\nfinish the job and document what we&rsquo;ve found for the sake of completeness.<\/p>\n<h1 id=\"boxing-everything-up\">Boxing everything up<\/h1>\n<p>My best guess on the packet structure is as follows:<\/p>\n<div class=\"hz-abi\">\n<div type=\"10 bits\" class=\"hz-abi-green hz-abi-2b\">base id<\/div>\n<div type=\"10 bits\" class=\"hz-abi-green hz-abi-2b\">argument<\/div>\n<div type=\"6 bits\" class=\"hz-abi-green hz-abi-2b\">command<\/div>\n<\/div>\n<p>For a <code>call<\/code> or <code>F2<\/code> operation, the <code>argument<\/code> is the Pager&rsquo;s ID code,\nbut for other commands it&rsquo;s a value or an enum, depending. Here&rsquo;s a table\nof my by-hand demodulation of all the packet types the base station\nproduces:<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Type<\/td>\n<td class=\"hz-1-6th\">Cmd Id<\/td>\n<td class=\"hz-4-6th\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr><td>Call<\/td><td>8<\/td><td>Call the pager identified by the id in <code>argument<\/code><\/td><\/tr>\n<tr><td>Off<\/td><td>60<\/td><td>Request any pagers on the charger power off when power is removed, <code>argument<\/code> is all zero<\/td><\/tr>\n<tr><td>F2<\/td><td>40<\/td><td>Program a pager to the specified Pager ID (in <code>argument<\/code>) and base station<\/td><\/tr>\n<tr><td>F3<\/td><td>44<\/td><td>Set the reminder duration in seconds specified in <code>argument<\/code><\/td><\/tr>\n<tr><td>F4<\/td><td>48<\/td><td>Set the pager's beep mode to the one in <code>argument<\/code> (<code>0<\/code> is disabled, <code>1<\/code> is slow, <code>2<\/code> is medium, <code>3<\/code> is fast)<\/td><\/tr>\n<tr><td>F5<\/td><td>52<\/td><td>Set the pager's vibration mode to the one in <code>argument<\/code> (<code>0<\/code> is disabled, <code>1<\/code> is enabled)<\/td><\/tr>\n<\/tbody>\n<\/table>\n<h1 id=\"kitchens-closed-for-the-night\">Kitchen&rsquo;s closed for the night<\/h1>\n<p>I&rsquo;m not going to be publishing this code since I can&rsquo;t think of a good use\nanyone would have for this besides folks using a low cost SDR and annoying\nlocal restaurants; but there&rsquo;s enough here for folks who find this interesting\nto try modulating this protocol on their own hardware if they want to buy their\nown pack of pagers and give it a shot, which I do encourage! It&rsquo;s fun! Radios\nare great, and this is a good protocol to hack with &ndash; it&rsquo;s really nice.<\/p>\n<p>All in all, this wasn&rsquo;t the multi-week adventure I was looking for, this was\nstill a great exercise and a fun reminder that I&rsquo;ve come a far way from when\nI&rsquo;ve started. It felt a lot like cheating since I was able to infer a lot about\nthe PHY because I&rsquo;d seen it before, but it was still a great time. I may grab a\nfew more restaurant pagers and see if I can find one with a more exotic PHY to\nemulate next. I mean why not, I&rsquo;ve already got the\n<a href=\"https:\/\/github.com\/paultag\/go-epson\">thermal<\/a>\n<a href=\"https:\/\/crates.io\/crates\/epson\">printer<\/a>\nlibraries\n<a href=\"https:\/\/soylent.green\/@paul\/111805908699444082\">working<\/a>\n\ud83d\udda8\ufe0f<\/p>"},{"title":"Writing a simulator to check phased array beamforming \ud83c\udf00","link":"https:\/\/k3xec.com\/simulating-phased-arrays\/","pubDate":"Mon, 22 Jan 2024 15:11:41 +0000","guid":"https:\/\/k3xec.com\/simulating-phased-arrays\/","description":"<div class=\"hz-alert-ok\">\nInterested in future updates? Follow me on mastodon at\n<a href=\"https:\/\/soylent.green\/@paul\">@paul@soylent.green<\/a>. Posts about\n<code>hz.tools<\/code> will be tagged\n<a href=\"https:\/\/soylent.green\/@paul\/tagged\/hztools\">#hztools<\/a>.\n<br \/>\n<br \/>\nIf you're on the Fediverse, I'd very much appreciate boosts on\n<a href=\"https:\/\/soylent.green\/@paul\/111788830823384684\">my toot<\/a>!\n<\/div>\n<p>While working on <a href=\"https:\/\/k3xec.com\/hztools\">hz.tools<\/a>, I started to move my beamforming\ncode from 2-D (meaning, beamforming to some specific angle on the X-Y plane for\nwaves on the X-Y plane) to 3-D. I&rsquo;ll have more to say about that once I get\naround to publishing the code as soon as I&rsquo;m sure it&rsquo;s not completely wrong,\nbut in the meantime I decided to write a simple simulator to visually\ncheck the beamformer against the textbooks. The results were pretty rad,\nso I figured I&rsquo;d throw together a post since it&rsquo;s interesting all on its\nown outside of beamforming as a general topic.<\/p>\n<p>I figured I&rsquo;d write this in Rust, since I&rsquo;ve been using Rust as my primary\nlanguage over at <a href=\"https:\/\/zoo.dev\/\">zoo<\/a>, and it&rsquo;s a good chance\nto learn the language better.<\/p>\n<div class=\"hz-alert-warning\">\n\u26a0\ufe0f This post has some large GIFs<br \/>\n<br \/>\nIt make take a little bit to load depending\non your internet connection. Sorry about that, I'm not clever enough to\ndo better without doing tons of complex engineering work. They may be\nchoppy while they load or something. I tried to compress an ensmall them,\nso if they're loaded but fuzzy, click on them to load a slightly larger\nversion.\n<\/div>\n<p>This post won&rsquo;t cover the basics of how\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Phased_array\">phased arrays work<\/a>\nor the specifics of calculating the phase offsets for each antenna,\nbut I&rsquo;ll dig into how I wrote a simple &ldquo;simulator&rdquo; and how I wound up\nchecking my phase offsets to generate the renders below.<\/p>\n<h1 id=\"assumptions\">Assumptions<\/h1>\n<p>I didn&rsquo;t want to build a general purpose RF simulator, anything particularly\ngeneric, or something that would solve for any more than the things right\nin front of me. To do this as simply (and quickly &ndash; all this code took about\na day to write, including the beamforming math) &ndash; I had to reduce the\namount of work in front of me.<\/p>\n<p>Given that I was concerend with visualizing what the antenna pattern would look\nlike in 3-D given some antenna geometry, operating frequency and configured\nbeam, I made the following assumptions:<\/p>\n<p>All anetnnas are perfectly isotropic &ndash; they receive a signal that is\nexactly the same strength no matter what direction the signal originates\nfrom.<\/p>\n<p>There&rsquo;s a single point-source isotropic emitter in the far-field (I modeled\nthis as being 1 million meters away &ndash; 1000 kilometers) of the antenna system.<\/p>\n<p>There is no noise, multipath, loss or distortion in the signal as it travels\nthrough space.<\/p>\n<p>Antennas will never interfere with each other.<\/p>\n<h1 id=\"2-d-polar-plots\">2-D Polar Plots<\/h1>\n<p>The <a href=\"https:\/\/soylent.green\/@paul\/109263478055233925\">last time I wrote something like this<\/a>,\nI generated 2-D GIFs which show a radiation pattern, not unlike the\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Microphone#Polar_patterns\">polar plots you&rsquo;d see on a microphone<\/a>.<\/p>\n<p>These are handy because it lets you visualize what the directionality of\nthe antenna looks like, as well as in what direction emissions are\ncaptured, and in what directions emissions are nulled out. You can see\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Radiation_pattern\">these plots<\/a>\non spec sheets for antennas in both 2-D and 3-D form.<\/p>\n<p>Now, let&rsquo;s port the 2-D approach to 3-D and see how well it works out.<\/p>\n<h1 id=\"writing-the-3-d-simulator\">Writing the 3-D simulator<\/h1>\n<p>As an EM wave travels through free space, the place at which you sample\nthe wave controls that phase you observe at each time-step. This means,\nassuming perfectly synchronized clocks, a transmitter and receiver exactly\none RF wavelength apart will observe a signal in-phase, but a transmitter\nand receiver a half wavelength apart will observe a signal 180 degrees\nout of phase.<\/p>\n<p>This means that if we take the distance between our point-source and\nantenna element, divide it by the wavelength, we can use the fractional\npart of the resulting number to determine the phase observed. If we\nmultiply that number (in the range of 0 to just under 1) by\n<a href=\"https:\/\/tauday.com\/\">tau<\/a>, we can generate a complex number by taking the\n<code>cos<\/code> and <code>sin<\/code> of the multiplied phase (in the range of 0 to tau), assuming\nthe transmitter is emitting a carrier wave at a static amplitude and all\nclocks are in perfect sync.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> observed_phases: Vec<span style=\"color:#f92672\">&lt;<\/span>Complex<span style=\"color:#f92672\">&gt;<\/span> <span style=\"color:#f92672\">=<\/span> antennas\n<\/span><\/span><span style=\"display:flex;\"><span> .iter()\n<\/span><\/span><span style=\"display:flex;\"><span> .map(<span style=\"color:#f92672\">|<\/span>antenna<span style=\"color:#f92672\">|<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> distance <span style=\"color:#f92672\">=<\/span> (antenna <span style=\"color:#f92672\">-<\/span> tx).magnitude();\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> distance <span style=\"color:#f92672\">=<\/span> distance <span style=\"color:#f92672\">-<\/span> (distance <span style=\"color:#66d9ef\">as<\/span> <span style=\"color:#66d9ef\">i64<\/span> <span style=\"color:#66d9ef\">as<\/span> <span style=\"color:#66d9ef\">f64<\/span>);\n<\/span><\/span><span style=\"display:flex;\"><span> ((distance <span style=\"color:#f92672\">\/<\/span> wavelength) <span style=\"color:#f92672\">*<\/span> <span style=\"color:#66d9ef\">TAU<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> })\n<\/span><\/span><span style=\"display:flex;\"><span> .map(<span style=\"color:#f92672\">|<\/span>phase<span style=\"color:#f92672\">|<\/span> Complex(phase.cos(), phase.sin()))\n<\/span><\/span><span style=\"display:flex;\"><span> .collect();\n<\/span><\/span><\/code><\/pre><\/div><p>At this point, given some synthetic transmission point and each antenna, we\nknow what the expected complex sample would be at each antenna. At this point,\nwe can adjust the phase of each antenna according to the beamforming\nphase offset configuration, and add up every sample in order to determine\nwhat the entire system would collectively produce a sample as.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> beamformed_phases: Vec<span style=\"color:#f92672\">&lt;<\/span>Complex<span style=\"color:#f92672\">&gt;<\/span> <span style=\"color:#f92672\">=<\/span> <span style=\"color:#f92672\">..<\/span>.;\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">let<\/span> magnitude <span style=\"color:#f92672\">=<\/span> beamformed_phases\n<\/span><\/span><span style=\"display:flex;\"><span> .iter()\n<\/span><\/span><span style=\"display:flex;\"><span> .zip(observed_phases.iter())\n<\/span><\/span><span style=\"display:flex;\"><span> .map(<span style=\"color:#f92672\">|<\/span>(beamformed, observed)<span style=\"color:#f92672\">|<\/span> observed <span style=\"color:#f92672\">*<\/span> beamformed)\n<\/span><\/span><span style=\"display:flex;\"><span> .reduce(<span style=\"color:#f92672\">|<\/span>acc, el<span style=\"color:#f92672\">|<\/span> acc <span style=\"color:#f92672\">+<\/span> el)\n<\/span><\/span><span style=\"display:flex;\"><span> .unwrap()\n<\/span><\/span><span style=\"display:flex;\"><span> .abs();\n<\/span><\/span><\/code><\/pre><\/div><p>Armed with this information, it&rsquo;s straight forward to generate some number\nof (Azimuth, Elevation) points to sample, generate a transmission point\nfar away in that direction, resolve what the resulting Complex sample would be,\ntake its magnitude, and use that to create an <code>(x, y, z)<\/code> point at\n<code>(azimuth, elevation, magnitude)<\/code>. The color attached two that point is\nbased on its distance from <code>(0, 0, 0)<\/code>. I opted to use the\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/The_Life_Aquatic_with_Steve_Zissou\">Life Aquatic<\/a>\ntable for this one.<\/p>\n<p>After this process is complete, I have a\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Point_cloud\">point cloud<\/a> of\n<code>((x, y, z), (r, g, b))<\/code> points. I wrote a small program using\n<a href=\"https:\/\/docs.rs\/kiss3d\/\">kiss3d<\/a> to render point cloud using tons of\nsmall spheres, and write out the frames to a set of PNGs, which get compiled\ninto a GIF.<\/p>\n<p>Now for the fun part, let&rsquo;s take a look at some radiation patterns!<\/p>\n<h1 id=\"1x4-phased-array\">1x4 Phased Array<\/h1>\n<p>The first configuration is a phased array where all the elements are\nin perfect alignment on the <code>y<\/code> and <code>z<\/code> axis, and separated by some\noffset in the <code>x<\/code> axis. This configuration can sweep 180 degrees (not\nthe full 360), but can&rsquo;t be steared in elevation at all.<\/p>\n<p>Let&rsquo;s take a look at what this looks like for a well constructed\n1x4 phased array:<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/1x4_overview.png\" alt=\"\"><\/p>\n<p>And now let&rsquo;s take a look at the renders as we play with the configuration of\nthis array and make sure things look right. Our initial quarter-wavelength\nspacing is very effective and has some outstanding performance characteristics.\nLet&rsquo;s check to see that everything looks right as a first test.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/1x4_quarter_lambda_sweep_az_small.gif\" alt=\"\"><\/p>\n<p>Nice. Looks perfect. When pointing forward at <code>(0, 0)<\/code>, we&rsquo;d expect to see a\ntorus, which we do. As we sweep between 0 and 360, astute observers will notice\nthe pattern is mirrored along the axis of the antennas, when the beam is facing\nforward to 0 degrees, it&rsquo;ll also receive at 180 degrees just as strong. There&rsquo;s\na small sidelobe that forms when it&rsquo;s configured along the array, but\nit also becomes the most directional, and the sidelobes remain fairly small.<\/p>\n<h2 id=\"long-compared-to-the-wavelength-1-\u03bb\">Long compared to the wavelength (1\u00bc \u03bb)<\/h2>\n<p>Let&rsquo;s try again, but rather than spacing each antenna \u00bc of a wavelength\napart, let&rsquo;s see about spacing each antenna 1\u00bc of a wavelength apart instead.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/1x4_long_baseline_sweep_az_small.gif\" alt=\"\"><\/p>\n<p>The main lobe is a lot more narrow (not a bad thing!), but some significant\nsidelobes have formed (not ideal). This can cause a lot of confusion when doing\nthings that require a lot of directional resolution unless they&rsquo;re compensated\nfor.<\/p>\n<h2 id=\"going-from--to-5-\u03bb\">Going from (\u00bc to 5\u00bc \u03bb)<\/h2>\n<p>The last model begs the question - what do things look like when you separate\nthe antennas from each other but without moving the beam? Let&rsquo;s simulate moving\nour antennas but not adjusting the configured beam or operating frequency.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/1x4_baseline_quarter_to_5_lambda_small.gif\" alt=\"\"><\/p>\n<p>Very cool. As the spacing becomes longer in relation to the operating frequency,\nwe can see the sidelobes start to form out of the end of the antenna system.<\/p>\n<h1 id=\"2x2-phased-array\">2x2 Phased Array<\/h1>\n<p>The second configuration I want to try is a phased array where the elements\nare in perfect alignment on the <code>z<\/code> axis, and separated by a fixed offset\nin either the <code>x<\/code> or <code>y<\/code> axis by their neighbor, forming a square when\nviewed along the x\/y axis.<\/p>\n<p>Let&rsquo;s take a look at what this looks like for a well constructed\n2x2 phased array:<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/2x2_overview.png\" alt=\"\"><\/p>\n<p>Let&rsquo;s do the same as above and take a look at the renders as we play with the\nconfiguration of this array and see what things look like. This configuration\nshould suppress the sidelobes and give us good performance, and even give us\nsome amount of control in elevation while we&rsquo;re at it.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/2x2_quarter_lambda_sweep_az_el_small.gif\" alt=\"\"><\/p>\n<p>Sweet. Heck yeah. The array is quite directional in the configured direction,\nand can even sweep a little bit in elevation, a definite improvement\nfrom the 1x4 above.<\/p>\n<h2 id=\"long-compared-to-the-wavelength-1-\u03bb-1\">Long compared to the wavelength (1\u00bc \u03bb)<\/h2>\n<p>Let&rsquo;s do the same thing as the 1x4 and take a look at what happens when the\ndistance between elements is long compared to the frequency of operation &ndash;\nsay, 1\u00bc of a wavelength apart? What happens to the sidelobes given this\nspacing when the frequency of operation is much different than the physical\ngeometry?<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/2x2_long_baseline_sweep_az_el_small.gif\" alt=\"\"><\/p>\n<p>Mesmerising. This is my favorate render. The sidelobes are very fun to\nwatch come in and out of existence. It looks absolutely other-worldly.<\/p>\n<h2 id=\"going-from--to-5-\u03bb-1\">Going from (\u00bc to 5\u00bc \u03bb)<\/h2>\n<p>Finally, for completeness&rsquo; sake, what do things look like when you separate the\nantennas from each other just as we did with the 1x4? Let&rsquo;s simulate moving our\nantennas but not adjusting the configured beam or operating frequency.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/phased-array-simulations\/2x2_baseline_quarter_to_5_lambda_small.gif\" alt=\"\"><\/p>\n<p>Very very cool. The sidelobes wind up turning the very blobby cardioid into\nan electromagnetic dog toy. I think we&rsquo;ve proven to ourselves that using\na phased array much outside its designed frequency of operation seems like\na real bad idea.<\/p>\n<h1 id=\"future-work\">Future Work<\/h1>\n<p>Now that I have a system to test things out, I&rsquo;m a bit more confident that\nmy beamforming code is close to right! I&rsquo;d love to push that code over\nthe line and blog about it, since it&rsquo;s a really interesting topic on its own.\nOnce I&rsquo;m sure the code involved isn&rsquo;t full of lies, I&rsquo;ll put it up on the\n<a href=\"https:\/\/github.com\/hztools\">hztools org<\/a>, and post about it here and on\nmastodon.<\/p>"},{"title":"Overview of the AudioSocket protocol \ud83d\udcde","link":"https:\/\/k3xec.com\/audio-socket\/","pubDate":"Wed, 13 Dec 2023 00:55:09 -0500","guid":"https:\/\/k3xec.com\/audio-socket\/","description":"<p>The <a href=\"https:\/\/www.asterisk.org\/\">asterisk<\/a> VoIP project has a protocol\nbuilt-in called &ldquo;AudioSocket&rdquo;. AudioSocket is built on top of TCP,\nstreaming int16 values at a sample rate of 8 kHz, neither of those\noptions are configurable (by design). AudioSocket will stream audio\nfrom the connected phone to the tcp server, and play audio samples\nsent from the tcp server to the phone.<\/p>\n<div class=\"hz-alert-warning\">\nThis documentation is a work in progress, and a result of\nsource code spelunking or reverse engineering. It may contain\nerrors or outright lies. The names may not match the original\nname, but it's been documented on a best-effort basis to help\nfuture engineering efforts.\n<\/div>\n<h2 id=\"audiosocket-packet\">AudioSocket Packet<\/h2>\n<p>Data is exchanged over <code>AudioSocket<\/code> by framing data into\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Type%E2%80%93length%E2%80%93value\">TLV<\/a>\npackets. This should be a pretty natural concept for anyone\nwho&rsquo;s worked on other line encoding schemes like <code>ASN.1<\/code>, <code>SSH<\/code>, <code>PGP<\/code>,\nor <code>protobuf<\/code>.<\/p>\n<p>The <code>type<\/code> is a <code>uint8<\/code>, length is transmitted as a <code>uint16<\/code>, and the\npayload is a variable sized block of data.<\/p>\n<p>The <strong>header<\/strong> is encoded using\n<span class=\"hz-highlight\">network byte order<\/span> (big endian).\nThe only field this really matters for is the <code>length<\/code> field, since\nthe <code>type<\/code> field is <code>uint8<\/code>. The <code>payload<\/code> format is dependent on the\n<code>type<\/code> of message.<\/p>\n<div class=\"hz-abi\">\n<div type=\"uint8\" class=\"hz-abi-yellow hz-abi-1b\">\ntype\n<\/div>\n<div type=\"uint16\" class=\"hz-abi-yellow hz-abi-2b\">\nlength\n<\/div>\n<div type=\"[]uint8\" class=\"hz-abi-yellow hz-abi-Nb\">\npayload\n<\/div>\n<\/div>\n<p>A full list of Commands, and the semantics of their Argument\nis detailed on the table below.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Command<\/td>\n<td class=\"hz-1-6th\">Definition<\/td>\n<td class=\"hz-2-3rd\">Payload<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>0x00<\/td>\n<td>Terminate<\/td>\n<td>none<\/td>\n<\/tr>\n<tr>\n<td>0x01<\/td>\n<td>UUID<\/td>\n<td>16-byte UUID encoded as raw bytes.<\/td>\n<\/tr>\n<tr>\n<td>0x10<\/td>\n<td>Audio Samples<\/td>\n<td>variable length buffer of <b>little endian<\/b> signed 16 bit integers sampled at 8 kHz<\/td>\n<\/tr>\n<tr>\n<td>0xFF<\/td>\n<td>Error<\/td>\n<td>byte (see table below)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The most simple (and also shortest) command for AudioSocket is the &ldquo;<code>Terminate<\/code>&rdquo;\ncommand, which can be used to indicate that the connection should be tore down,\nwhich is a type of <code>0x00<\/code>, and no payload (length of <code>0<\/code>, no body). This would\nbe encoded as <code>[0x00, 0x00, 0x00]<\/code>.<\/p>\n<h2 id=\"well-known-error-codes\">Well known Error Codes<\/h2>\n<p>The length of the <code>Error<\/code> packet is not defined, and may be any length.\nAccording to an <code>AudioSocket<\/code> Go library\n(<a href=\"https:\/\/github.com\/CyCoreSystems\/audiosocket\">github.com\/CyCoreSystems\/audiosocket<\/a>),\nAsterisk has the following well known error codes (although I can&rsquo;t seem to\nfind these in the source, if anyone has a link). Given the most common\nimplementation is asterisk, I suspect mandating a 1-byte Error code is not\na bad idea.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Code<\/td>\n<td class=\"hz-1-6th\">Impl<\/td>\n<td class=\"hz-2-3rd\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>0x01<\/td>\n<td>Asterisk<\/td>\n<td>Caller has hung up the Connection<\/td>\n<\/tr>\n<tr>\n<td>0x02<\/td>\n<td>Asterisk<\/td>\n<td>Error forwarding the Frame to the caller<\/td>\n<\/tr>\n<tr>\n<td>0x04<\/td>\n<td>Asterisk<\/td>\n<td>Internal memory allocation error<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h1 id=\"example-packets\">Example Packets<\/h1>\n<p>Terminate the connection:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-text\" data-lang=\"text\"><span style=\"display:flex;\"><span>0x00 0x00 0x00\n<\/span><\/span><\/code><\/pre><\/div><p>Indicate an error state of <code>0x11<\/code><\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-text\" data-lang=\"text\"><span style=\"display:flex;\"><span>0xFF 0x00 0x01 0x11\n<\/span><\/span><\/code><\/pre><\/div><p>Send 2 audio samples of <code>+1<\/code> and <code>-1<\/code><\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-text\" data-lang=\"text\"><span style=\"display:flex;\"><span>0x10 0x00 0x04 0x01 0x00 0xFF 0xFF\n<\/span><\/span><\/code><\/pre><\/div><h1 id=\"handshake\">Handshake<\/h1>\n<p>After a TCP connection is established, the client is expected to send a\n<code>UUID<\/code> Packet to the server, which has an application dependent meaning. It\ncould indicate the audio stream to attach to, an identity, or an API key\ndepending on how the server uses it.<\/p>\n<p>After the <code>UUID<\/code> packet is sent, both the Client and the Server begin to send\n<code>Audio<\/code> packets to their peer until the TCP connection is closed, the\n<code>Terminate<\/code> command is issued, or an <code>Error<\/code> packet is sent.<\/p>\n<h1 id=\"implementation-notes\">Implementation Notes<\/h1>\n<p>Because the aduio stream needs to be very low latency, it&rsquo;s advisable to\nset <code>TCP_NODELAY<\/code>, in order to disable Nagle&rsquo;s algorithm on the TCP connection.\nThe reason is that we&rsquo;re sending many small packets with time sensitive\naudio information which need to be sent right away, even if there is more data\nto be sent very shortly after.<\/p>\n<p>Additionally, Asterisk specifically will be very upset if you send headers,\nand reading the body takes more than 5ms, even if there&rsquo;s a buffer you never\nexhaust. This state is hard to hit when the audio data is contained in an IP\npacket, but it&rsquo;s very easy to trigger when you&rsquo;re operating under Nagle&rsquo;s\nalgorithm, since your packet is likely to be split along non-packet boundaries.<\/p>"},{"title":"Announcing hz.tools","link":"https:\/\/k3xec.com\/hztools\/","pubDate":"Wed, 22 Feb 2023 21:00:00 -0500","guid":"https:\/\/k3xec.com\/hztools\/","description":"<div class=\"hz-alert-ok\">\nInterested in future updates? Follow me on mastodon at\n<a href=\"https:\/\/soylent.green\/@paul\">@paul@soylent.green<\/a>. Posts about\n<code>hz.tools<\/code> will be tagged\n<a href=\"https:\/\/soylent.green\/@paul\/tagged\/hztools\">#hztools<\/a>.<br \/>\n<br \/>\nIf you're on the Fediverse, I'd very much appreciate boosts on\n<a href=\"https:\/\/soylent.green\/@paul\/109911598120551418\">my announcement toot<\/a>!\n<\/div>\n<p>Ever since 2019, I&rsquo;ve been learning about how radios work, and trying to learn\nabout using them &ldquo;the hard way&rdquo; &ndash; by writing as much of the stack as is\npractical (for some value of practical) myself. I wrote my first &ldquo;Hello World&rdquo;\nin 2018, which was a simple FM radio player, which used <code>librtlsdr<\/code> to read in\nan IQ stream, did some filtering, and played the real valued audio stream via\n<code>pulseaudio<\/code>. Over 4 years this has slowly grown through persistence, lots of\nquestions to too many friends to thank (although I will try), and the eternal\npatience of my wife hearing about radios nonstop &ndash; for years &ndash; into a number\nof Go repos that can do quite a bit, and support a handful of radios.<\/p>\n<p><strong>I&rsquo;ve resisted making the repos public not out of embarrassment or a desire to\nkeep secrets, but rather, an attempt to keep myself free of any maintenance\nobligations to users &ndash; so that I could freely break my own API, add and remove\nAPI surface as I saw fit. The worst case was to have this project feel like\nwork, and I can&rsquo;t imagine that will happen if I feel frustrated by PRs\nthat are &ldquo;getting ahead of me&rdquo; &ndash; solving problems I didn&rsquo;t yet know about, or\nbugs I didn&rsquo;t understand the fix for.<\/strong><\/p>\n<p>As my rate of changes to the most central dependencies has slowed, i&rsquo;ve begun\nto entertain the idea of publishing them. After a bit of back and forth, I&rsquo;ve\ndecided\n<span class=\"hz-highlight\">it&rsquo;s time to make a number of them public<\/span>,\nand to start working on them in the open, as I&rsquo;ve built up a bit of knowledge\nin the space, and I and feel confident that the repo doesn&rsquo;t contain overt\nlies. That&rsquo;s not to say it doesn&rsquo;t contain lies, but those lies are likely\nhidden and lurking in the dark. Beware.<\/p>\n<p>That being said, it shouldn&rsquo;t be a surprise to say I&rsquo;ve not published\neverything yet &ndash; for the same reasons as above. I plan to open repos as the\nrate of changes slows and I understand the problems the library solves well\nenough &ndash; or if the project &ldquo;dead ends&rdquo; and I&rsquo;ve stopped learning.<\/p>\n<h2 id=\"intention-behind-hztools\">Intention behind hz.tools<\/h2>\n<p>It&rsquo;s my sincere hope that my repos help to make Software Defined Radio (SDR)\ncode a bit easier to understand, and serves as an understandable framework to\nlearn with. It&rsquo;s a large codebase, but one that is possible to sit down and\nunderstand because, well, it was written by a single person. Frankly, I&rsquo;m also\nnot productive enough in my free time in the middle of the night and on\nweekends and holidays to create a codebase that&rsquo;s too large to understand, I\nhope!<\/p>\n<p>I remain wary of this project turning into work, so my goal is to be very\nupfront about my boundaries, and the limits of what classes of contributions\ni&rsquo;m interested in seeing.<\/p>\n<p>Here&rsquo;s some <strong>goals<\/strong> of open sourcing these repos:<\/p>\n<ul>\n<li>I <strong>do<\/strong> want this library to be used to learn with. Please go through it\nall and use it to learn about radios and how software can control them!<\/li>\n<li>I <strong>am<\/strong> interested in bugs if there&rsquo;s a problem you discover. Such bugs\nare likely a great chance for me to fix something I&rsquo;ve misunderstood or\ntypoed.<\/li>\n<li>I <strong>am<\/strong> interested in PRs fixing bugs you find. I may need a bit of a\nback and forth to fully understand the problem if I do not understand\nthe bug and fix yet. I hope you may have some grace if it&rsquo;s taking a long\ntime.<\/li>\n<\/ul>\n<p>Here&rsquo;s a list of some <strong>anti-goals<\/strong> of open sourcing these repos.<\/p>\n<ul>\n<li>I do <strong>not<\/strong> want this library to become a critical dependency of an\nimportant project, since I do not have the time to deal with the maintenance\nburden. Putting me in that position is going to make me very uncomfortable.<\/li>\n<li>I am <strong>not<\/strong> interested in feature requests, the features have grown as\nI&rsquo;ve hit problems, I&rsquo;m not interested in building or maintaining features\nfor features sake. The API surface should be exposed enough to allow others\nto experiment with such things out-of-tree.<\/li>\n<li>I&rsquo;m <strong>not<\/strong> interested in clever code replacing clear code without a very\ncompelling reason.<\/li>\n<li>I use GNU\/Linux (specifically Debian\n<img src=\"https:\/\/k3xec.com\/imgs\/emojo\/Debian.png\" style=\"width: 1em; height: 1em;\"\/>),\nand from time-to-time I&rsquo;ve made sure that my code runs on OpenBSD\n<img src=\"https:\/\/k3xec.com\/imgs\/emojo\/OpenBSD.png\" style=\"width: 1em; height: 1em;\" \/> too.\nPlatforms beyond that will likely not be supported at the expense of either\nof those two. I&rsquo;ll take fixes for bugs that fix a problem on another\nplatform, but not damage the code to work around issues \/ lack of features\non other platforms (like Windows).<\/li>\n<\/ul>\n<p><span class=\"hz-highlight\">I&rsquo;m not saying all this to be a jerk, I do it to\nmake sure I can continue on my journey to learn about how radios work without\nmy full time job becoming maintaining a radio framework single-handedly for\nother people to use &ndash; even if it means I need to close PRs or bugs without\nmerging it or fixing the issue.<\/span><\/p>\n<p><strong>With all that out of the way, I&rsquo;m very happy to announce that the repos are now\npublic under <a href=\"https:\/\/github.com\/hztools\">github.com\/hztools<\/a>.<\/strong><\/p>\n<h2 id=\"should-you-use-this\">Should you use this?<\/h2>\n<p><strong>Probably not<\/strong>. The intent here is not to provide a general purpose Go SDR\nframework for everyone to build on, although I am keenly aware it looks and\nfeels like it, since that what it is to me. This is a learning project, so for\nany use beyond joining me in learning should use something like\n<a href=\"https:\/\/www.gnuradio.org\/\">GNU Radio<\/a>\nor a similar framework that has a community behind it.<\/p>\n<p>In fact, I suspect most contributors ought to be contributing to GNU Radio, and\nnot this project. If I can encourage people to do so,\n<a href=\"https:\/\/wiki.gnuradio.org\/index.php?title=HowToGetInvolved\">contribute to GNU Radio<\/a>!\nNothing makes me happier than seeing GNU Radio continue to be the go-to, and\nwell supported. Consider\n<a href=\"https:\/\/donorbox.org\/gnuradio\">donating<\/a> to GNU Radio!<\/p>\n<h2 id=\"hztoolsrf---frequency-types\">hz.tools\/rf - Frequency types<\/h2>\n<p>The <code>hz.tools\/rf<\/code> library contains the abstract concept of frequency, and\nsome very basic helpers to interact with frequency ranges (such as helpers\nto deal with frequency ranges, or frequency range math) as well as frequencies\nand some very basic conversions (to meters, etc) and parsers (to parse values\nlike <code>10MHz<\/code>). This ensures that all the <code>hz.tools<\/code> libraries have a shared\nunderstanding of Frequencies, a standard way of representing ranges of\nFrequencies, and the ability to handle the IO boundary with things like CLI\narguments, JSON or YAML.<\/p>\n<p>The git repo can be found at\n<a href=\"https:\/\/github.com\/hztools\/go-rf\">github.com\/hztools\/go-rf<\/a>, and is\nimportable as <a href=\"https:\/\/pkg.go.dev\/hz.tools\/rf\">hz.tools\/rf<\/a>.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Parse a frequency using hz.tools\/rf.ParseHz, and print it to stdout.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">freq<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">MustParseHz<\/span>(<span style=\"color:#e6db74\">&#34;-10kHz&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;Frequency: %s\\n&#34;<\/span>, <span style=\"color:#a6e22e\">freq<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">MHz<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Prints: &#39;Frequency: 990kHz&#39;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Return the Intersection between two RF ranges, and print<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ it to stdout.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">r1<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">Range<\/span>{<span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">KHz<\/span>, <span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">MHz<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">r2<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">Range<\/span>{<span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">Hz<\/span>(<span style=\"color:#ae81ff\">10<\/span>), <span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">KHz<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#ae81ff\">100<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;Range: %s\\n&#34;<\/span>, <span style=\"color:#a6e22e\">r1<\/span>.<span style=\"color:#a6e22e\">Intersection<\/span>(<span style=\"color:#a6e22e\">r2<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Prints: Range: 1000Hz-&gt;100kHz<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>These can be used to represent tons of things - ranges can be used for things\nlike the tunable range of an SDR, the bandpass of a filter or the frequencies\nthat correspond to a bin of an FFT, while frequencies can be used for things\nsuch as frequency offsets or the tuned center frequency.<\/p>\n<h2 id=\"hztoolssdr---sdr-io-and-iq-types\">hz.tools\/sdr - SDR I\/O and IQ Types<\/h2>\n<p>This&hellip; is the big one. This library represents the majority of the shared\ntypes and bindings, and is likely the most useful place to look at when\nlearning about the IO boundary between a program and an SDR.<\/p>\n<p>The git repo can be found at\n<a href=\"https:\/\/github.com\/hztools\/go-sdr\">github.com\/hztools\/go-sdr<\/a>, and is\nimportable as <a href=\"https:\/\/pkg.go.dev\/hz.tools\/sdr\">hz.tools\/sdr<\/a>.<\/p>\n<p>This library is designed to look (and in some cases, mirror) the Go <code>io<\/code> idioms\nso that this library feels as idiomatic as it can, so that Go builtins interact\nwith IQ in a way that&rsquo;s possible to reason about, and to avoid reinventing the\nwheel by designing new API surface. While some of the API looks (and is even\ncalled) the same thing as a similar function in <code>io<\/code>, the implementation is\nusually a <strong>lot<\/strong> more naive, and may have unexpected sharp edges such as\nconcurrency issues or performance problems.<\/p>\n<p>The following IQ types are implemented using the <code>sdr.Samples<\/code> interface. The\n<code>hz.tools\/sdr<\/code> package contains helpers for conversion between types, and some\nbasic manipulation of IQ streams.<\/p>\n<table>\n<thead>\n<tr>\n<th>IQ Format<\/th>\n<th>hz.tools Name<\/th>\n<th>Underlying Go Type<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Interleaved uint8 (rtl-sdr)<\/td>\n<td><code>sdr.SamplesU8<\/code><\/td>\n<td><code>[][2]uint8<\/code><\/td>\n<\/tr>\n<tr>\n<td>Interleaved int8 (hackrf, uhd)<\/td>\n<td><code>sdr.SamplesI8<\/code><\/td>\n<td><code>[][2]int8<\/code><\/td>\n<\/tr>\n<tr>\n<td>Interleaved int16 (pluto, uhd)<\/td>\n<td><code>sdr.SamplesI16<\/code><\/td>\n<td><code>[][2]int16<\/code><\/td>\n<\/tr>\n<tr>\n<td>Interleaved float32 (airspy, uhd)<\/td>\n<td><code>sdr.SamplesC64<\/code><\/td>\n<td><code>[]complex64<\/code><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The following SDRs have implemented drivers in-tree.<\/p>\n<table>\n<thead>\n<tr>\n<th>SDR<\/th>\n<th>Format<\/th>\n<th>RX\/TX<\/th>\n<th>State<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>rtl<\/td>\n<td>u8<\/td>\n<td>RX<\/td>\n<td>Good<\/td>\n<\/tr>\n<tr>\n<td>HackRF<\/td>\n<td>i8<\/td>\n<td>RX\/TX<\/td>\n<td>Good<\/td>\n<\/tr>\n<tr>\n<td>PlutoSDR<\/td>\n<td>i16<\/td>\n<td>RX\/TX<\/td>\n<td>Good<\/td>\n<\/tr>\n<tr>\n<td>rtl kerberos<\/td>\n<td>u8<\/td>\n<td>RX<\/td>\n<td>Old<\/td>\n<\/tr>\n<tr>\n<td>uhd<\/td>\n<td>i16\/c64\/i8<\/td>\n<td>RX\/TX<\/td>\n<td>Good<\/td>\n<\/tr>\n<tr>\n<td>airspyhf<\/td>\n<td>c64<\/td>\n<td>RX<\/td>\n<td>Exp<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The following major packages and subpackages exist at the time of writing:<\/p>\n<table>\n<thead>\n<tr>\n<th>Import<\/th>\n<th>What is it?<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>hz.tools\/sdr<\/td>\n<td>Core IQ types, supporting types and implementations that interact with the byte boundary<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/rtl<\/td>\n<td><code>sdr.Receiver<\/code> implementation using <code>librtlsdr<\/code>.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/rtl\/kerberos<\/td>\n<td>Helpers to enable coherent RX using the Kerberos SDR.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/rtl\/e4k<\/td>\n<td>Helpers to interact with the <a href=\"https:\/\/hz.tools\/e4k\/\">E4000<\/a> RTL-SDR dongle.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/fft<\/td>\n<td>Interfaces for performing an FFT, which are implemented by other packages.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/rtltcp<\/td>\n<td><code>sdr.Receiver<\/code> implementation for <a href=\"https:\/\/hz.tools\/rtl_tcp\/\">rtl_tcp<\/a> servers.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/pluto<\/td>\n<td><code>sdr.Transceiver<\/code> implementation for the PlutoSDR using <code>libiio<\/code>.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/uhd<\/td>\n<td><code>sdr.Transceiver<\/code> implementation for UHD radios, specifically the B210 and B200mini<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/hackrf<\/td>\n<td><code>sdr.Transceiver<\/code> implementation for the HackRF using <code>libhackrf<\/code>.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/mock<\/td>\n<td>Mock SDR for testing purposes.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/airspyhf<\/td>\n<td><code>sdr.Receiver<\/code> implementation for the AirspyHF+ Discovery with <code>libairspyhf<\/code>.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/internal\/simd<\/td>\n<td>SIMD helpers for IQ operations, written in Go ASM. This isn&rsquo;t the best to learn from, and it contains pure go implemtnations alongside.<\/td>\n<\/tr>\n<tr>\n<td>hz.tools\/sdr\/stream<\/td>\n<td>Common Reader\/Writer helpers that operate on IQ streams.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2 id=\"hztoolsfftw---hztoolssdrfft-implementation\">hz.tools\/fftw - hz.tools\/sdr\/fft implementation<\/h2>\n<p>The <code>hz.tools\/fftw<\/code> package contains bindings to <code>libfftw3<\/code> to implement\nthe <code>hz.tools\/sdr\/fft.Planner<\/code> type to transform between the time and\nfrequency domain.<\/p>\n<p>The git repo can be found at\n<a href=\"https:\/\/github.com\/hztools\/go-fftw\">github.com\/hztools\/go-fftw<\/a>, and is\nimportable as <a href=\"https:\/\/pkg.go.dev\/hz.tools\/fftw\">hz.tools\/fftw<\/a>.<\/p>\n<p>This is the default throughout most of my codebase, although that default is\nonly expressed at the &ldquo;leaf&rdquo; package &ndash; libraries should not be hardcoding the\nuse of this library in favor of taking an <code>fft.Planner<\/code>, unless it&rsquo;s used as\npart of testing. There are a bunch of ways to do an FFT out there, things like\n<code>clFFT<\/code> or a pure-go FFT implementation could be plugged in depending on what&rsquo;s\nbeing solved for.<\/p>\n<h2 id=\"hztoolsfmam---analog-audio-demodulation-and-modulation\">hz.tools\/{fm,am} - analog audio demodulation and modulation<\/h2>\n<p>The <code>hz.tools\/fm<\/code> and <code>hz.tools\/am<\/code> packages contain demodulators for\nAM analog radio, and FM analog radio. This code is a bit old, so it has\na lot of room for cleanup, but it&rsquo;ll do a very basic demodulation of IQ\nto audio.<\/p>\n<p>The git repos can be found at\n<a href=\"https:\/\/github.com\/hztools\/go-fm\">github.com\/hztools\/go-fm<\/a> and\n<a href=\"https:\/\/github.com\/hztools\/go-am\">github.com\/hztools\/go-am<\/a>,\nand are importable as\n<a href=\"https:\/\/pkg.go.dev\/hz.tools\/fm\">hz.tools\/fm<\/a> and\n<a href=\"https:\/\/pkg.go.dev\/hz.tools\/am\">hz.tools\/am<\/a>.<\/p>\n<p>As a bonus, the <code>hz.tools\/fm<\/code> package also contains a modulator, which has been\ntested &ldquo;on the air&rdquo; and with some of my handheld radios. This code is a bit\nold, since the <code>hz.tools\/fm<\/code> code is effectively the first IQ processing code\nI&rsquo;d ever written, but it still runs and I run it from time to time.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Basic sketch for playing FM radio using a reader stream from<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ an SDR or other IQ stream.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">bandwidth<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#ae81ff\">150<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">rf<\/span>.<span style=\"color:#a6e22e\">KHz<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">reader<\/span>, <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">stream<\/span>.<span style=\"color:#a6e22e\">ConvertReader<\/span>(<span style=\"color:#a6e22e\">reader<\/span>, <span style=\"color:#a6e22e\">sdr<\/span>.<span style=\"color:#a6e22e\">SampleFormatC64<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">demod<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">fm<\/span>.<span style=\"color:#a6e22e\">Demodulate<\/span>(<span style=\"color:#a6e22e\">reader<\/span>, <span style=\"color:#a6e22e\">fm<\/span>.<span style=\"color:#a6e22e\">DemodulatorConfig<\/span>{\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Deviation<\/span>: <span style=\"color:#a6e22e\">bandwidth<\/span> <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#ae81ff\">2<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Downsample<\/span>: <span style=\"color:#ae81ff\">8<\/span>, <span style=\"color:#75715e\">\/\/ some value here depending on sample rate<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Planner<\/span>: <span style=\"color:#a6e22e\">fftw<\/span>.<span style=\"color:#a6e22e\">Plan<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> })\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">speaker<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">pulseaudio<\/span>.<span style=\"color:#a6e22e\">NewWriter<\/span>(<span style=\"color:#a6e22e\">pulseaudio<\/span>.<span style=\"color:#a6e22e\">Config<\/span>{\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Format<\/span>: <span style=\"color:#a6e22e\">pulseaudio<\/span>.<span style=\"color:#a6e22e\">SampleFormatFloat32NE<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Rate<\/span>: <span style=\"color:#a6e22e\">demod<\/span>.<span style=\"color:#a6e22e\">SampleRate<\/span>(),\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">AppName<\/span>: <span style=\"color:#e6db74\">&#34;rf&#34;<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">StreamName<\/span>: <span style=\"color:#e6db74\">&#34;fm&#34;<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Channels<\/span>: <span style=\"color:#ae81ff\">1<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">SinkName<\/span>: <span style=\"color:#e6db74\">&#34;&#34;<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> })\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">buf<\/span> <span style=\"color:#f92672\">:=<\/span> make([]<span style=\"color:#66d9ef\">float32<\/span>, <span style=\"color:#ae81ff\">1024<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#ae81ff\">64<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">i<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">demod<\/span>.<span style=\"color:#a6e22e\">Read<\/span>(<span style=\"color:#a6e22e\">buf<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">==<\/span> <span style=\"color:#ae81ff\">0<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> panic(<span style=\"color:#e6db74\">&#34;...&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">speaker<\/span>.<span style=\"color:#a6e22e\">Write<\/span>(<span style=\"color:#a6e22e\">buf<\/span>[:<span style=\"color:#a6e22e\">i<\/span>]); <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"hztoolsrfcap---byte-serialization-for-iq-data\">hz.tools\/rfcap - byte serialization for IQ data<\/h2>\n<p>The <code>hz.tools\/rfcap<\/code> package is the reference implementation of the\n<a href=\"https:\/\/hz.tools\/rfcap\/\">rfcap<\/a> &ldquo;spec&rdquo;, and is how I store IQ captures\nlocally, and how I send them across a byte boundary.<\/p>\n<p>The git repo can be found at\n<a href=\"https:\/\/github.com\/hztools\/go-rfcap\">github.com\/hztools\/go-rfcap<\/a>, and is\nimportable as <a href=\"https:\/\/pkg.go.dev\/hz.tools\/rfcap\">hz.tools\/rfcap<\/a>.<\/p>\n<p>If you&rsquo;re interested in storing IQ in a way others can use, the better approach\nis to use <a href=\"https:\/\/github.com\/sigmf\/SigMF\">SigMF<\/a> &ndash; <code>rfcap<\/code> exists for cases\nlike using UNIX pipes to move IQ around, through APIs, or when I send\nIQ data through an OS socket, to ensure the sample format (and other metadata)\nis communicated with it.<\/p>\n<p><code>rfcap<\/code> has a number of limitations, for instance, it can not express a change\nin frequency or sample rate during the capture, since the header is fixed at\nthe beginning of the file.<\/p>"},{"title":"Decoding LDPC: k-Bit Brute Forcing","link":"https:\/\/k3xec.com\/ldpc-k-bit\/","pubDate":"Tue, 01 Nov 2022 19:00:00 -0400","guid":"https:\/\/k3xec.com\/ldpc-k-bit\/","description":"<div class=\"hz-alert-error\">\n<b>Before you go on:<\/b> I've been\n<a href=\"https:\/\/mastodon.social\/@destevez\/109272986530413236\">warned off<\/a>\nimplementing this in practice on a few counts; namely, the space tradeoff isn't\nworth it, and it's unlikely to correct meaningful errors. I'm going to leave\nthis post up, but please do take the content with a very large grain of salt!\n<\/div>\n<p>My initial efforts to build a <a href=\"https:\/\/en.wikipedia.org\/wiki\/Physical_layer\">PHY<\/a>\nand <a href=\"https:\/\/en.wikipedia.org\/wiki\/Data_link_layer\">Data Link<\/a> layer &ndash; from\nscratch using my own code &ndash; have been progressing nicely since the initial\nBPSK based protocol I&rsquo;ve documented under the PACKRAT series. As part of that,\nI&rsquo;ve been diving deep into <a href=\"https:\/\/en.wikipedia.org\/wiki\/Error_correction_code\">FEC<\/a>,\nand in particular, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Low-density_parity-check_code\">LDPC<\/a>.<\/p>\n<p>I won&rsquo;t be able to do an overview of LDPC justice in this post &ndash; with any\nluck that&rsquo;ll come in a later post to come as part of the RATPACK series,\nso some knowledge is assumed. As such this post is less useful for those\nlooking to learn about LDPC, and a bit more targeted to those who enjoy\ntalking and thinking about FEC.<\/p>\n<div class=\"hz-alert-warning\">\n<b>Hey, heads up!<\/b> - This post contains extremely unvalidated and back of\nthe napkin quality work without any effort to prove this out generally.\nHopefully this work can be of help to others, but please double check anything\nbelow if you need it for your own work!\n<\/div>\n<p>While implementing LDPC, I&rsquo;ve gotten an encoder and checker working, enough\nto use LDPC like a checksum. The next big step is to write a Decoder, which\ncan do error correction. The two popular approaches for the actual correction\nthat I&rsquo;ve seen while reading about LDPC are\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Belief_propagation\">Belief Propagation<\/a>,\nand some class of linear programming that I haven&rsquo;t dug into yet.\nI&rsquo;m not thrilled at how expensive this all is in software, so while implementing\nthe stack I&rsquo;ve been exploring every shady side ally to try and learn more about\nhow encoders and decoders work, both in theory - and in practice.<\/p>\n<h2 id=\"processing-an-ldpc-message\">Processing an LDPC Message<\/h2>\n<p>Checking if a message is correct is fairly straightforward with LDPC (as with\nencoding, I&rsquo;ll note). As a quick refresher &ndash; given the LDPC <code>H<\/code> (check) matrix\nof width <code>N<\/code>, you can check your message vector (<code>msg<\/code>) of length <code>N<\/code> by\nmultiplying <code>H<\/code> and <code>msg<\/code>, and checking if the output vector is all zero.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ scheme contains our G (generator) and<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ H (check) matrices.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">scheme<\/span> <span style=\"color:#f92672\">:=<\/span> {<span style=\"color:#a6e22e\">G<\/span>: <span style=\"color:#a6e22e\">Matrix<\/span>{<span style=\"color:#f92672\">...<\/span>}, <span style=\"color:#a6e22e\">H<\/span>: <span style=\"color:#a6e22e\">Matrix<\/span>{<span style=\"color:#f92672\">...<\/span>}}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ msg contains our LDPC message (data and<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ check bits).<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">msg<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">Vector<\/span>{<span style=\"color:#f92672\">...<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ N is also the length of the encoded<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ msg vector after check bits have been<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ added.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">N<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">scheme<\/span>.<span style=\"color:#a6e22e\">G<\/span>.<span style=\"color:#a6e22e\">Width<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Now, let&#39;s generate our &#39;check&#39; vector.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">ch<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">Multiply<\/span>(<span style=\"color:#a6e22e\">scheme<\/span>.<span style=\"color:#a6e22e\">H<\/span>, <span style=\"color:#a6e22e\">msg<\/span>)\n<\/span><\/span><\/code><\/pre><\/div><p>We can now see if the message is correct or not:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ if the ch vector is all zeros, we know<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ that the message is valid, and we don&#39;t<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ need to do anything.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">ch<\/span>.<span style=\"color:#a6e22e\">IsZero<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ handle the case where the message<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ is fine as-is.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Expensive decode here<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>This is great for getting a thumbs up \/ thumbs down on the message being\ncorrect, but correcting errors still requires pulling the LDPC matrix values\nfrom the <code>g<\/code> (generator) matrix out, building a bipartite graph, and\niteratively reprocessing the bit values, until constraints are satisfied and\nthe message has been corrected.<\/p>\n<p>This got me thinking - what <em>is<\/em> the output vector when it&rsquo;s not all zeros?\nSince <code>1<\/code> values in the output vector indicates consistency problems in the\nmessage bits as they relate to the check bits, I wondered if this could be used\nto speed up my LDPC decoder. It appears to work, so this post is half an attempt\nto document this technique before I put it in my hot path, and half a plea for\nthose who <em>do<\/em> like to talk about FEC to tell me what name this technique\nactually is.<\/p>\n<h2 id=\"k-bit-brute-forcing\">k-Bit Brute Forcing<\/h2>\n<p>Given that the output Vector&rsquo;s non-zero bit pattern is set due to the position\nof errors in the message vector, let&rsquo;s use that fact to build up a table\nof k-Bit errors that we can index into.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ for clarity&#39;s sake, the Vector<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ type is being used as the lookup<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ key here, even though it may<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ need to be a hash or string in<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ some cases.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">idx<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">map<\/span>[<span style=\"color:#a6e22e\">Vector<\/span>]<span style=\"color:#66d9ef\">int<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#ae81ff\">0<\/span>; <span style=\"color:#a6e22e\">i<\/span> &lt; <span style=\"color:#a6e22e\">N<\/span>; <span style=\"color:#a6e22e\">i<\/span><span style=\"color:#f92672\">++<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Create a vector of length N<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">v<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">Vector<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">v<\/span>.<span style=\"color:#a6e22e\">FlipBit<\/span>(<span style=\"color:#a6e22e\">i<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Now, let&#39;s use the generator matrix to encode<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ the data with checksums, and then use the<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ check matrix on the message to figure out what<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ bit pattern results<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">ev<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">Multiply<\/span>(<span style=\"color:#a6e22e\">scheme<\/span>.<span style=\"color:#a6e22e\">H<\/span>, <span style=\"color:#a6e22e\">Multiply<\/span>(<span style=\"color:#a6e22e\">v<\/span>, <span style=\"color:#a6e22e\">scheme<\/span>.<span style=\"color:#a6e22e\">G<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">idx<\/span>[<span style=\"color:#a6e22e\">ev<\/span>] = <span style=\"color:#a6e22e\">i<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><p>This can be extended to multiple bits (hence: k-Bits), but I&rsquo;ve only done\none here for illustration. Now that we have our <code>idx<\/code> mapping, we can now\ngo back to the hot path on Checking the incoming message data:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ if the ch vector is all zeros, we know<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ that the message is valid, and we don&#39;t<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ need to do anything.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">ch<\/span>.<span style=\"color:#a6e22e\">IsZero<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ handle the case where the message<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ is fine as-is.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">errIdx<\/span>, <span style=\"color:#a6e22e\">ok<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">idx<\/span>[<span style=\"color:#a6e22e\">ch<\/span>]\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">ok<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">msg<\/span>.<span style=\"color:#a6e22e\">FlipBit<\/span>(<span style=\"color:#a6e22e\">errIdx<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Verify the LDPC message using<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ H again here.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Expensive decode here<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Since map lookups wind up a heck of a lot faster than message-passing\nbit state, the hope here is this will short-circuit easy to solve errors\nfor k-Bits, for some value of k that the system memory can tolerate.<\/p>\n<h2 id=\"does-this-work\">Does this work?<\/h2>\n<p>Frankly &ndash; I have no idea. I&rsquo;ve written a small program and brute forced\nsingle-bit errors in all bit positions using random data to start with, and\nI&rsquo;ve not been able to find any collisions in the 1-bit error set, using the\nLDPC matrix from <code>802.3an-2006<\/code>. Even if I was to find a collision for a\nhigher-order k-Bit value, I&rsquo;m tempted to continue with this approach, and treat\neach set of bits in the Vector&rsquo;s bin (like a hash-table), checking the LDPC\nvalidity after each bit set in the bin. As long as the collision rate is small\nenough, it should be possible to correct k-Bits of error faster than the more\nexpensive Belief Propagation approach. That being said, I&rsquo;m not entirely\nconvinced collisions will be very common, but it&rsquo;ll take a bit more time\nworking through the math to say that with any confidence.<\/p>\n<p><b>Have you seen this approach called something official in publications? See\nan obvious flaw in the system? Send me a tip, please!<\/b><\/p>"},{"title":"k3xec.com\/patty: Go bindings to patty","link":"https:\/\/k3xec.com\/patty\/","pubDate":"Mon, 11 Apr 2022 19:33:00 -0400","guid":"https:\/\/k3xec.com\/patty\/","description":"<p>AX.25 is a tough protocol to use on UNIX systems. A lot of the support in\nLinux, specifically, is pretty hard to use, and tends to be built into the\nreptilian brain of the kernel. <a href=\"https:\/\/kz3rox.us\/\">KZ3ROX<\/a> built a userland\nAX.25 stack called <a href=\"https:\/\/scm.xan.host\/patty.git\/\">patty<\/a>, for which I have\nnow built some Go bindings on top of.<\/p>\n<p>Code needed to create AX.25 Sockets via Go can be found at\n<a href=\"https:\/\/github.com\/k3xec\/go-patty\">github.com\/k3xec\/go-patty<\/a>,\nand imported by Go source as\n<a href=\"https:\/\/pkg.go.dev\/k3xec.com\/patty\">k3xec.com\/patty<\/a>.<\/p>\n<h1 id=\"overview\">Overview<\/h1>\n<p>Clint patty programs (including consumers of this Go library) work by\ncommunicating with a userland daemon (<code>pattyd<\/code>) via a UNIX named socket.\nThat daemon will communicate with a particular radio using a\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/KISS_(TNC)\">KISS TNC<\/a> serial device.<\/p>\n<p>The Go bindings implement as many standard Go library interfaces as is\npractical, allowing for the &ldquo;plug and play&rdquo; use of <code>patty<\/code> (and AX.25) in\nplaces where you would expect a network socket (such as TCP) to work, such\nas Go&rsquo;s http library.<\/p>\n<h1 id=\"example\">Example<\/h1>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">package<\/span> <span style=\"color:#a6e22e\">main<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">import<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;fmt&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;log&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;net&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;os&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;time&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;k3xec.com\/patty&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">main<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">callsign<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#e6db74\">&#34;N0CALL-10&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">client<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">patty<\/span>.<span style=\"color:#a6e22e\">Open<\/span>(<span style=\"color:#e6db74\">&#34;patty.sock&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> panic(<span style=\"color:#a6e22e\">err<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">l<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">client<\/span>.<span style=\"color:#a6e22e\">Listen<\/span>(<span style=\"color:#e6db74\">&#34;ax25&#34;<\/span>, <span style=\"color:#a6e22e\">callsign<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> panic(<span style=\"color:#a6e22e\">err<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">log<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;Listening for requests to %s&#34;<\/span>, <span style=\"color:#a6e22e\">l<\/span>.<span style=\"color:#a6e22e\">Addr<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">conn<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">l<\/span>.<span style=\"color:#a6e22e\">Accept<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">log<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;Error accepting: %s&#34;<\/span>, <span style=\"color:#a6e22e\">err<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">continue<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">go<\/span> <span style=\"color:#a6e22e\">handle<\/span>(<span style=\"color:#a6e22e\">conn<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">handle<\/span>(<span style=\"color:#a6e22e\">c<\/span> <span style=\"color:#a6e22e\">net<\/span>.<span style=\"color:#a6e22e\">Conn<\/span>) <span style=\"color:#66d9ef\">error<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">defer<\/span> <span style=\"color:#a6e22e\">c<\/span>.<span style=\"color:#a6e22e\">Close<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">log<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;New connection from %s (local: %s)&#34;<\/span>, <span style=\"color:#a6e22e\">c<\/span>.<span style=\"color:#a6e22e\">RemoteAddr<\/span>(), <span style=\"color:#a6e22e\">c<\/span>.<span style=\"color:#a6e22e\">LocalAddr<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Fprintf<\/span>(<span style=\"color:#a6e22e\">c<\/span>, <span style=\"color:#e6db74\">`\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">Hello! This is Paul&#39;s experimental %s node. Feel free\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">to poke around. Let me know if you spot anything funny.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">Five pings are to follow!\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">`<\/span>, <span style=\"color:#a6e22e\">c<\/span>.<span style=\"color:#a6e22e\">LocalAddr<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#ae81ff\">0<\/span>; <span style=\"color:#a6e22e\">i<\/span> &lt; <span style=\"color:#ae81ff\">5<\/span>; <span style=\"color:#a6e22e\">i<\/span><span style=\"color:#f92672\">++<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Sleep<\/span>(<span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Second<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#ae81ff\">5<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Fprintf<\/span>(<span style=\"color:#a6e22e\">c<\/span>, <span style=\"color:#e6db74\">&#34;Ping!\\n&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">nil<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div>"},{"title":"Proxying Ethernet Frames to PACKRAT (Part 5\/5) \ud83d\udc00","link":"https:\/\/k3xec.com\/packrat-proxy\/","pubDate":"Mon, 06 Dec 2021 11:00:00 -0500","guid":"https:\/\/k3xec.com\/packrat-proxy\/","description":"<div class=\"hz-alert-ok\">\n\ud83d\udc00 This post is part of a series called \"PACKRAT\". If this is the first post\nyou've found, it'd be worth reading the\n<a href=\"https:\/\/k3xec.com\/packrat-intro\/\">intro post<\/a> first and then looking over\n<a href=\"https:\/\/k3xec.com\/tags\/packrat\/\">all posts in the series<\/a>.\n<\/div>\n<p>In the <a href=\"https:\/\/k3xec.com\/packrat-framing\/\">last post<\/a>, we left off at being able to send and receive PACKRAT frames\nto and from devices. Since we can transport IPv4 packets over the network,\nlet&rsquo;s go ahead and see if we can read\/write\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Ethernet\">Ethernet<\/a> frames from a Linux network\ninterface, and on the backend, read and write PACKRAT frames over the air.\nThis has the benefit of continuing to allow Linux userspace tools to work (like\ncURL, as we&rsquo;ll try!), which means we don&rsquo;t have to do a lot of work to\nimplement higher level protocols or tactics to get a connection established\nover the link.<\/p>\n<p>Given that this post is less RF and more Linuxy, I&rsquo;m going to include more code\nsnippits than in prior posts, and those snippits are closer to runable Go, but\nstill not complete examples. There&rsquo;s also a lot of different ways to do this,\nI&rsquo;ve just picked the easiest one for me to implement and debug given my\nexisting tooling &ndash; for you, you may find another approach easier to implement!<\/p>\n<p>Again, deviation here is very welcome, and since this segment is the least\nRF centric post in the series, the pace and tone is going to feel different.\nIf you feel lost here, that&rsquo;s OK. This isn&rsquo;t the most important part of\nthe series, and is mostly here to give a concrete ending to the story arc.\nAny way you want to finish your own journey is the best way for you to finish\nit!<\/p>\n<h4 id=\"implement-ethernet-conversion-code\">Implement Ethernet conversion code<\/h4>\n<p>This assumes an importable package with a Frame struct, which we can use to\nconvert a Frame to\/from Ethernet. Given that the PACKRAT frame has a field that\nEthernet doesn&rsquo;t (namely, <code>Callsign<\/code>), that will need to be explicitly passed\nin when turning an Ethernet frame into a PACKRAT Frame.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ ToPackrat will create a packrat frame from an Ethernet frame.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">ToPackrat<\/span>(<span style=\"color:#a6e22e\">callsign<\/span> [<span style=\"color:#ae81ff\">8<\/span>]<span style=\"color:#66d9ef\">byte<\/span>, <span style=\"color:#a6e22e\">frame<\/span> <span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">ethernet<\/span>.<span style=\"color:#a6e22e\">Frame<\/span>) (<span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">packrat<\/span>.<span style=\"color:#a6e22e\">Frame<\/span>, <span style=\"color:#66d9ef\">error<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">frameType<\/span> <span style=\"color:#a6e22e\">packrat<\/span>.<span style=\"color:#a6e22e\">FrameType<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">switch<\/span> <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">EtherType<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">case<\/span> <span style=\"color:#a6e22e\">ethernet<\/span>.<span style=\"color:#a6e22e\">EtherTypeIPv4<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">frameType<\/span> = <span style=\"color:#a6e22e\">packrat<\/span>.<span style=\"color:#a6e22e\">FrameTypeIPv4<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">default<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Errorf<\/span>(<span style=\"color:#e6db74\">&#34;ethernet: unsupported ethernet type %x&#34;<\/span>, <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">EtherType<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#a6e22e\">packrat<\/span>.<span style=\"color:#a6e22e\">Frame<\/span>{\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Destination<\/span>: <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Destination<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Source<\/span>: <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Source<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Type<\/span>: <span style=\"color:#a6e22e\">frameType<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Callsign<\/span>: <span style=\"color:#a6e22e\">callsign<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Payload<\/span>: <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Payload<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> }, <span style=\"color:#66d9ef\">nil<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ FromPackrat will create an Ethernet frame from a Packrat frame.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">FromPackrat<\/span>(<span style=\"color:#a6e22e\">frame<\/span> <span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">packrat<\/span>.<span style=\"color:#a6e22e\">Frame<\/span>) (<span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">ethernet<\/span>.<span style=\"color:#a6e22e\">Frame<\/span>, <span style=\"color:#66d9ef\">error<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">etherType<\/span> <span style=\"color:#a6e22e\">ethernet<\/span>.<span style=\"color:#a6e22e\">EtherType<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">switch<\/span> <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Type<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">case<\/span> <span style=\"color:#a6e22e\">packrat<\/span>.<span style=\"color:#a6e22e\">FrameTypeRaw<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Errorf<\/span>(<span style=\"color:#e6db74\">&#34;ethernet: unsupported packrat type &#39;raw&#39;&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">case<\/span> <span style=\"color:#a6e22e\">packrat<\/span>.<span style=\"color:#a6e22e\">FrameTypeIPv4<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">etherType<\/span> = <span style=\"color:#a6e22e\">ethernet<\/span>.<span style=\"color:#a6e22e\">EtherTypeIPv4<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">default<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Errorf<\/span>(<span style=\"color:#e6db74\">&#34;ethernet: unknown packrat type %x&#34;<\/span>, <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Type<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ We lose the Callsign here, which is sad.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#a6e22e\">ethernet<\/span>.<span style=\"color:#a6e22e\">Frame<\/span>{\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Destination<\/span>: <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Destination<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Source<\/span>: <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Source<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">EtherType<\/span>: <span style=\"color:#a6e22e\">etherType<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Payload<\/span>: <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">Payload<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> }, <span style=\"color:#66d9ef\">nil<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>Our helpers, <code>ToPackrat<\/code> and <code>FromPackrat<\/code> can now be used to transmorgify\nPACKRAT into Ethernet, or Ethernet into PACKRAT. Let&rsquo;s put them into use!<\/p>\n<h4 id=\"implement-a-tap-interface\">Implement a TAP interface<\/h4>\n<p>On Linux, the networking stack can be exposed to userland using <a href=\"https:\/\/en.wikipedia.org\/wiki\/TUN\/TAP\">TUN or\nTAP<\/a> interfaces. TUN devices allow a\nuserspace program to read and write data at the Layer 3 \/ IP layer. TAP devices\nallow a userspace program to read and write data at the Layer 2 Data Link \/\nEthernet layer. Writing data at Layer 2 is what we want to do, since we&rsquo;re\nlooking to transform our Layer 2 into Ethernet&rsquo;s Layer 2 Frames. Our first job\nhere is to create the actual TAP interface, set the MAC address, and set the IP\nrange to our pre-coordinated IP range.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">import<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;net&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;github.com\/mdlayher\/ethernet&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;github.com\/songgao\/water&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#e6db74\">&#34;github.com\/vishvananda\/netlink&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">config<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">water<\/span>.<span style=\"color:#a6e22e\">Config<\/span>{<span style=\"color:#a6e22e\">DeviceType<\/span>: <span style=\"color:#a6e22e\">water<\/span>.<span style=\"color:#a6e22e\">TAP<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">Name<\/span> = <span style=\"color:#e6db74\">&#34;rat0&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">iface<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">water<\/span>.<span style=\"color:#a6e22e\">New<\/span>(<span style=\"color:#a6e22e\">config<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">netIface<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">LinkByName<\/span>(<span style=\"color:#e6db74\">&#34;rat0&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Pick a range here that works for you!<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ For my local network, I&#39;m using some IPs<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ that AMPR (ampr.org) was nice enough to<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ allocate to me for ham radio use. Thanks,<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ AMPR!<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Let&#39;s just use 10.* here, though.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">ip<\/span>, <span style=\"color:#a6e22e\">cidr<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">net<\/span>.<span style=\"color:#a6e22e\">ParseCIDR<\/span>(<span style=\"color:#e6db74\">&#34;10.0.0.1\/24&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">cidr<\/span>.<span style=\"color:#a6e22e\">IP<\/span> = <span style=\"color:#a6e22e\">ip<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">AddrAdd<\/span>(<span style=\"color:#a6e22e\">netIface<\/span>, <span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">Addr<\/span>{\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">IPNet<\/span>: <span style=\"color:#a6e22e\">cidr<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Peer<\/span>: <span style=\"color:#a6e22e\">cidr<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> })\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Add all our neighbors to the ARP table<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">_<\/span>, <span style=\"color:#a6e22e\">neighbor<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">neighbors<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">NeighAdd<\/span>(<span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">Neigh<\/span>{\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">LinkIndex<\/span>: <span style=\"color:#a6e22e\">netIface<\/span>.<span style=\"color:#a6e22e\">Attrs<\/span>().<span style=\"color:#a6e22e\">Index<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Type<\/span>: <span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">FAMILY_V4<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">State<\/span>: <span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">NUD_PERMANENT<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">IP<\/span>: <span style=\"color:#a6e22e\">neighbor<\/span>.<span style=\"color:#a6e22e\">IP<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">HardwareAddr<\/span>: <span style=\"color:#a6e22e\">neighbor<\/span>.<span style=\"color:#a6e22e\">MAC<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> })\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Pick a MAC that is globally unique here, this is<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ just used as an example!<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">addr<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">net<\/span>.<span style=\"color:#a6e22e\">ParseMAC<\/span>(<span style=\"color:#e6db74\">&#34;FA:DE:DC:AB:LE:01&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">LinkSetHardwareAddr<\/span>(<span style=\"color:#a6e22e\">netIface<\/span>, <span style=\"color:#a6e22e\">addr<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">netlink<\/span>.<span style=\"color:#a6e22e\">LinkSetUp<\/span>(<span style=\"color:#a6e22e\">netIface<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">frame<\/span> = <span style=\"color:#f92672\">&amp;<\/span><span style=\"color:#a6e22e\">ethernet<\/span>.<span style=\"color:#a6e22e\">Frame<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">buf<\/span> = make([]<span style=\"color:#66d9ef\">byte<\/span>, <span style=\"color:#ae81ff\">1500<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">n<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">iface<\/span>.<span style=\"color:#a6e22e\">Read<\/span>(<span style=\"color:#a6e22e\">buf<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">UnmarshalBinary<\/span>(<span style=\"color:#a6e22e\">buf<\/span>[:<span style=\"color:#a6e22e\">n<\/span>])\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ process frame here (to come)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Now that our network stack can resolve an IP to a MAC Address (via <code>ip neigh<\/code>\naccording to our pre-defined neighbors), and send that IP packet to our daemon,\nit&rsquo;s now on us to send IPv4 data over the airwaves. Here, we&rsquo;re going to take\npackets coming in from our TAP interface, and marshal the Ethernet frame into a\nPACKRAT Frame and transmit it. As with the rest of the RF code, we&rsquo;ll leave\nthat up to the implementer, of course, using what was built during <a href=\"https:\/\/k3xec.com\/packrat-transmitting\/\">Part 2:\nTransmitting BPSK symbols<\/a> and <a href=\"https:\/\/k3xec.com\/packrat-framing\/\">Part 4: Framing\ndata<\/a>.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ continued from above<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">n<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">iface<\/span>.<span style=\"color:#a6e22e\">Read<\/span>(<span style=\"color:#a6e22e\">buf<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">UnmarshalBinary<\/span>(<span style=\"color:#a6e22e\">buf<\/span>[:<span style=\"color:#a6e22e\">n<\/span>])\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">switch<\/span> <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">EtherType<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">case<\/span> <span style=\"color:#ae81ff\">0x0800<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ ipv4 packet<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">pack<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">ToPackrat<\/span>(\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Add my callsign to all Frames, for now<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> [<span style=\"color:#ae81ff\">8<\/span>]<span style=\"color:#66d9ef\">byte<\/span>{<span style=\"color:#e6db74\">&#39;K&#39;<\/span>, <span style=\"color:#e6db74\">&#39;3&#39;<\/span>, <span style=\"color:#e6db74\">&#39;X&#39;<\/span>, <span style=\"color:#e6db74\">&#39;E&#39;<\/span>, <span style=\"color:#e6db74\">&#39;C&#39;<\/span>},\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">frame<\/span>,\n<\/span><\/span><span style=\"display:flex;\"><span> )\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">transmitPacket<\/span>(<span style=\"color:#a6e22e\">pack<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Now that we have transmitting covered, let&rsquo;s go ahead and handle the receive\npath here. We&rsquo;re going to listen on frequency using the code built in\n<a href=\"https:\/\/k3xec.com\/packrat-receiving\/\">Part 3: Receiving BPSK symbols<\/a> and\n<a href=\"https:\/\/k3xec.com\/packrat-framing\/\">Part 4: Framing data<\/a>. The Frames we decode from\nthe airwaves are expected to come back from the call\n<code>packratReader.Next<\/code> in the code below, and the exact way that works\nis up to the implementer.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ pull the next packrat frame from<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ the symbol stream as we did in the<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ last post<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">packet<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">packratReader<\/span>.<span style=\"color:#a6e22e\">Next<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ check for CRC errors and drop invalid<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ packets<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">packet<\/span>.<span style=\"color:#a6e22e\">Check<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">bytes<\/span>.<span style=\"color:#a6e22e\">Equal<\/span>(<span style=\"color:#a6e22e\">packet<\/span>.<span style=\"color:#a6e22e\">Source<\/span>, <span style=\"color:#a6e22e\">addr<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ if we&#39;ve heard ourself transmitting<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ let&#39;s avoid looping back<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">continue<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ create an ethernet frame<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">frame<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">FromPackrat<\/span>(<span style=\"color:#a6e22e\">packet<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">buf<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">frame<\/span>.<span style=\"color:#a6e22e\">MarshalBinary<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ and inject it into the tap<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">iface<\/span>.<span style=\"color:#a6e22e\">Write<\/span>(<span style=\"color:#a6e22e\">buf<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Phew. Right. Now we should be able to listen for PACKRAT frames on the\nair and inject them into our TAP interface.<\/p>\n<h4 id=\"putting-it-all-together\">Putting it all Together<\/h4>\n<p>After all this work &ndash; weeks of work! &ndash; we can finally get around to putting\nsome real packets over the air. For me, this was an incredibly satisfying\nmilestone, and tied together months of learning!<\/p>\n<p>I was able to start up a UDP server on a remote machine with an RTL-SDR dongle\nattached to it, listening on the TAP interface&rsquo;s host IP with my defined MAC\naddress, and send UDP packets to that server via PACKRAT using my laptop,\n<code>\/dev\/udp<\/code> and an Ettus B210, sending packets into the TAP interface.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/packrat\/packrat-hello-world.png\" alt=\"\"><\/p>\n<p>Now that UDP was working, I was able to get TCP to work using two PlutoSDRs,\nwhich allowed me to run the cURL command I pasted in the first post (both\nsimultaneously listen and transmit on behalf of my TAP interface).<\/p>\n<p>It&rsquo;s my hope that someone out there will be inspired to implement their own\nLayer 1 and Layer 2 as a learning exercise, and gets the same sense of\ngratification that I did! If you&rsquo;re reading this, and at a point\nwhere you&rsquo;ve been able to send IP traffic over your own Layer 1 \/ Layer 2,\nplease get in touch! I&rsquo;d be thrilled to hear all about it. I&rsquo;d love to link\nto any posts or examples you publish here!<\/p>"},{"title":"Framing data (Part 4\/5) \ud83d\udc00","link":"https:\/\/k3xec.com\/packrat-framing\/","pubDate":"Sun, 05 Dec 2021 11:00:00 -0500","guid":"https:\/\/k3xec.com\/packrat-framing\/","description":"<div class=\"hz-alert-ok\">\n\ud83d\udc00 This post is part of a series called \"PACKRAT\". If this is the first post\nyou've found, it'd be worth reading the\n<a href=\"https:\/\/k3xec.com\/packrat-intro\/\">intro post<\/a> first and then looking over\n<a href=\"https:\/\/k3xec.com\/tags\/packrat\/\">all posts in the series<\/a>.\n<\/div>\n<p>In the <a href=\"https:\/\/k3xec.com\/packrat-receiving\/\">last post<\/a>, we we were able to build a functioning\nLayer 1 PHY where we can encode symbols to transmit, and receive symbols on the\nother end, we&rsquo;re now at the point where we can encode and decode those symbols\nas bits and frame blocks of data, marking them with a Sender and a Destination\nfor routing to the right host(s). This is a &ldquo;Layer 2&rdquo; scheme in the OSI model,\nwhich is otherwise known as the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Data_link_layer\">Data Link\nLayer<\/a>. You&rsquo;re using one to\nview this website right now &ndash; I&rsquo;m willing to bet your data is going\nthrough an Ethernet layer 2 as well as WiFi or maybe a cellular data\nprotocol like 5G or LTE.<\/p>\n<p>Given that this entire exercise is hard enough without designing a complex\nLayer 2 scheme, I opted for simplicity in the hopes this would free me from the\ncomplexity and research that has gone into this field for the last 50 years. I\nsettled on stealing a few ideas from Ethernet Frames &ndash; namely, the use of <a href=\"https:\/\/en.wikipedia.org\/wiki\/MAC_address\">MAC\naddresses<\/a> to identify parties, and\nthe <code>EtherType<\/code> field to indicate the Payload type. I also stole the idea of\nusing a CRC at the end of the Frame to check for corruption, as well as the\nspecific CRC method (<code>crc32<\/code> using <code>0xedb88320<\/code> as the polynomial).<\/p>\n<p>Lastly, I added a <code>callsign<\/code> field to make life easier on ham radio frequencies\nif I was ever to seriously attempt to use a variant of this protocol over the\nair with multiple users. However, given this scheme is not a commonly used\nscheme, it&rsquo;s best practice to use a nearby radio to identify your transmissions\non the same frequency while testing &ndash; or use a Faraday box to test without\ntransmitting over the airwaves. I added the callsign field in an effort to lean\ninto the spirit of the Part 97 regulations, even if I relied on a phone\nemission to identify the Frames.<\/p>\n<p>As an aside, I asked the ARRL for input here, and their stance to me over email\nwas I&rsquo;d be OK according to the regs if I were to stick to UHF and put my\ncallsign into the BPSK stream using a widely understood encoding (even with no\nknowledge of PACKRAT, the callsign is ASCII over BPSK and should be easily\ndemodulatable for followup with me). Even with all this, I opted to use FM\nphone to transmit my callsign when I was active on the air (specifically, using\nan SDR and a small bash script to automate transmission while I watched for\ninterference or other band users).<\/p>\n<p>Right, back to the Frame:<\/p>\n<div class=\"hz-abi\">\n<div type=\"[3]byte\" class=\"hz-abi-green hz-abi-2b\">sync<\/div>\n<div type=\"[6]byte\" class=\"hz-abi-green hz-abi-2b\">dest<\/div>\n<div type=\"[6]byte\" class=\"hz-abi-green hz-abi-2b\">source<\/div>\n<div type=\"[8]byte\" class=\"hz-abi-green hz-abi-2b\">callsign<\/div>\n<div type=\"[2]byte\" class=\"hz-abi-green hz-abi-2b\">type<\/div>\n<div type=\"[]byte\" class=\"hz-abi-green hz-abi-Nb\">payload<\/div>\n<div type=\"uint32\" class=\"hz-abi-green hz-abi-1b\">crc<\/div>\n<\/div>\n<p>With all that done, I put that layout into a struct, so that we can marshal\nand unmarshal bytes to and from our Frame objects, and work with it\nin software.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">type<\/span> <span style=\"color:#a6e22e\">FrameType<\/span> [<span style=\"color:#ae81ff\">2<\/span>]<span style=\"color:#66d9ef\">byte<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">type<\/span> <span style=\"color:#a6e22e\">Frame<\/span> <span style=\"color:#66d9ef\">struct<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Destination<\/span> <span style=\"color:#a6e22e\">net<\/span>.<span style=\"color:#a6e22e\">HardwareAddr<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Source<\/span> <span style=\"color:#a6e22e\">net<\/span>.<span style=\"color:#a6e22e\">HardwareAddr<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Callsign<\/span> [<span style=\"color:#ae81ff\">8<\/span>]<span style=\"color:#66d9ef\">byte<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Type<\/span> <span style=\"color:#a6e22e\">FrameType<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">Payload<\/span> []<span style=\"color:#66d9ef\">byte<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">CRC<\/span> <span style=\"color:#66d9ef\">uint32<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><h4 id=\"time-to-pick-some-consts\">Time to pick some consts<\/h4>\n<p>I picked a unique and distinctive <code>sync<\/code> sequence, which the sender\nwill transmit before the Frame, while the receiver listens for that\nsequence to know when it&rsquo;s in byte alignment with the symbol stream.\nMy <code>sync<\/code> sequence is <code>[3]byte{'U', 'f', '~'}<\/code> which works out to be a\nvery pleasant bit sequence of <code>01010101 01100110 01111110<\/code>. It&rsquo;s important\nto have soothing preambles for your Frames. We need all the good energy\nwe can get at this point.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">var<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">FrameStart<\/span> = [<span style=\"color:#ae81ff\">3<\/span>]<span style=\"color:#66d9ef\">byte<\/span>{<span style=\"color:#e6db74\">&#39;U&#39;<\/span>, <span style=\"color:#e6db74\">&#39;f&#39;<\/span>, <span style=\"color:#e6db74\">&#39;~&#39;<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">FrameMaxPayloadSize<\/span> = <span style=\"color:#ae81ff\">1500<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><\/code><\/pre><\/div><p>Next, I defined some <code>FrameType<\/code> values for the <code>type<\/code> field,\nwhich I can use to determine what is done with that data next,\nsomething Ethernet was originally missing, but has since grown\nto depend on (who needs Length anyway? Not me. See below!)<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">FrameType<\/td>\n<td class=\"hz-2-6th\">Description<\/td>\n<td class=\"hz-1-3th\">Bytes<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Raw<\/td>\n<td>Bytes in the Payload field are opaque and not to be parsed.<\/td>\n<td>[2]byte{0x00, 0x01}<\/td>\n<\/tr>\n<tr>\n<td>IPv4<\/td>\n<td>Bytes in the Payload field are an IPv4 packet.<\/td>\n<td>[2]byte{0x00, 0x02}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>And finally, I decided on a maximum length of the Payload, and decided on\nlimiting it to 1500 bytes to align with the MTU of Ethernet.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">var<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">FrameTypeRaw<\/span> = <span style=\"color:#a6e22e\">FrameType<\/span>{<span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#ae81ff\">1<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">FrameTypeIPv4<\/span> = <span style=\"color:#a6e22e\">FrameType<\/span>{<span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#ae81ff\">2<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><\/code><\/pre><\/div><p>Given we know how we&rsquo;re going to marshal and unmarshal binary data to and from\nFrames, we can now move on to looking through the bit stream for our Frames.<\/p>\n<h4 id=\"why-is-there-no-length-field\">Why is there no Length field?<\/h4>\n<p>I was initially a bit surprised that Ethernet Frames didn&rsquo;t have a Length field\nin use, but the more I thought about it, the more it seemed like a big ole'\nfailure mode without a good implementation outcome. Either the Length is right\n(resulting in no action and used bits on every packet) or the Length is not the\nlength of the Payload and the driver needs to determine what to do with the\npacket &ndash; does it try and trim the overlong payload and ignore the rest? What\nif both the end of the read bytes and the end of the subset of the packet\ndenoted by Length have a valid CRC? Which is used? Will everyone agree? What\nif Length is longer than the Payload but the CRC is good where we detected a\nlost carrier?<\/p>\n<p>I decided on simplicity. The end of a Frame is denoted by the loss of the BPSK\ncarrier &ndash; when the signal is no longer being transmitted (or more correctly,\nwhen the signal is no longer received), we know we&rsquo;ve hit the end of a packet.\nMissing a single symbol will result in the Frame being finalized. This can\ncause some degree of corruption, but it&rsquo;s also a lot easier than doing tricks\nlike bit stuffing to create an end of symbol stream delimiter.<\/p>\n<h4 id=\"finding-the-frame-start-in-a-symbol-stream\">Finding the Frame start in a Symbol Stream<\/h4>\n<p>First thing we need to do is find our <code>sync<\/code> bit pattern in the symbols we&rsquo;re\nreceiving from our BPSK demodulator. There&rsquo;s some smart ways to do this, but\ngiven that I&rsquo;m not much of a smart man, I again decided to go for simple\ninstead. Given our incoming vector of symbols (which are still <code>float<\/code> values)\nprepend one at a time to a vector of floats that is the same length as the\n<code>sync<\/code> phrase, and compare against the <code>sync<\/code> phrase, to determine if we&rsquo;re in\nsync with the byte boundary within the symbol stream.<\/p>\n<p>The only trick here is that because we&rsquo;re using BPSK to modulate and demodulate\nthe data, post phaselock we can be 180 degrees out of alignment (such that a +1\nis demodulated as -1, or vice versa). To deal with that, I check against both\nthe <code>sync<\/code> phrase as well as the inverse of the <code>sync<\/code> phrase (both <code>[1, -1, 1]<\/code> as well as <code>[-1, 1, -1]<\/code>) where if the inverse sync is matched, all symbols\nto follow will be inverted as well. This effectively turns our symbols back\ninto bits, even if we&rsquo;re flipped out of phase. Other techniques like\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Non-return-to-zero\">NRZI<\/a> will represent a 0 or\n1 by a change in phase state &ndash; which is great, but can often cascade into long\nruns of bit errors, and is generally more complex to implement. That\nrepresentation isn&rsquo;t ambiguous, given you look for a phase change, not the\nabsolute phase value, which is incredibly compelling.<\/p>\n<p>Here&rsquo;s a notional example of how I&rsquo;ve been thinking about the phrase sliding\nwindow &ndash; and how I&rsquo;ve been thinking of the checks. Each row is a new symbol\ntaken from the BPSK receiver, and pushed to the head of the sliding window,\nmoving all symbols back in the vector by one.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">sync<\/span> = []<span style=\"color:#66d9ef\">float<\/span>{ <span style=\"color:#f92672\">...<\/span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">buf<\/span> = make([]<span style=\"color:#66d9ef\">float<\/span>, len(<span style=\"color:#a6e22e\">sync<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">incomingSymbols<\/span> = []<span style=\"color:#66d9ef\">float<\/span>{ <span style=\"color:#f92672\">...<\/span> }\n<\/span><\/span><span style=\"display:flex;\"><span> )\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">_<\/span>, <span style=\"color:#a6e22e\">el<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">incomingSymbols<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> copy(<span style=\"color:#a6e22e\">buf<\/span>, <span style=\"color:#a6e22e\">buf<\/span>[<span style=\"color:#ae81ff\">1<\/span>:])\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">buf<\/span>[len(<span style=\"color:#a6e22e\">buf<\/span>)<span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">1<\/span>] = <span style=\"color:#a6e22e\">el<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">compare<\/span>(<span style=\"color:#a6e22e\">sync<\/span>, <span style=\"color:#a6e22e\">buf<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ we&#39;re synced!<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">break<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><p>Given the pseudocode above, let&rsquo;s step through what the checks would be doing\nat each step:<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-3ed\">Buffer<\/td>\n<td class=\"hz-1-3ed\">Sync<\/td>\n<td class=\"hz-1-3ed\">Inverse Sync<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>[\u2026]float{0,\u2026,0}<\/td>\n<td>\u274c [\u2026]float{-1,\u2026,-1}<\/td>\n<td>\u274c [\u2026]float{1,\u2026,1}<\/td>\n<\/tr>\n<tr>\n<td>[\u2026]float{0,\u2026,1}<\/td>\n<td>\u274c [\u2026]float{-1,\u2026,-1}<\/td>\n<td>\u274c [\u2026]float{1,\u2026,1}<\/td>\n<\/tr>\n<tr>\n<td>[more bits in]<\/td>\n<td>\u274c [\u2026]float{-1,\u2026,-1}<\/td>\n<td>\u274c [\u2026]float{1,\u2026,1}<\/td>\n<\/tr>\n<tr>\n<td>[\u2026]float{1,\u2026,1}<\/td>\n<td>\u274c [\u2026]float{-1,\u2026,-1}<\/td>\n<td>\u2705 [\u2026]float{1,\u2026,1}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>After this notional set of comparisons, we know that at the last step, we are\nnow aligned to the frame and byte boundary &ndash; the next symbol \/ bit will be the\nMSB of the 0th Frame byte. Additionally, we know we&rsquo;re also 180 degrees out of\nphase, so we need to flip the symbol&rsquo;s sign to get the bit. From this point on\nwe can consume 8 bits at a time, and re-assemble the byte stream. I don&rsquo;t know\nwhat this technique is called &ndash; or even if this is used in real grown-up\nimplementations, but it&rsquo;s been working for my toy implementation.<\/p>\n<h4 id=\"next-steps\">Next Steps<\/h4>\n<p>Now that we can read\/write Frames to and from PACKRAT, the next steps here are\ngoing to be implementing code to encode and decode Ethernet traffic into\nPACKRAT, coming next in <a href=\"https:\/\/k3xec.com\/packrat-proxy\/\">Part 5!<\/a><\/p>"},{"title":"Receiving BPSK symbols (Part 3\/5) \ud83d\udc00","link":"https:\/\/k3xec.com\/packrat-receiving\/","pubDate":"Sat, 04 Dec 2021 11:00:00 -0500","guid":"https:\/\/k3xec.com\/packrat-receiving\/","description":"<div class=\"hz-alert-ok\">\n\ud83d\udc00 This post is part of a series called \"PACKRAT\". If this is the first post\nyou've found, it'd be worth reading the\n<a href=\"https:\/\/k3xec.com\/packrat-intro\/\">intro post<\/a> first and then looking over\n<a href=\"https:\/\/k3xec.com\/tags\/packrat\/\">all posts in the series<\/a>.\n<\/div>\n<p>In the <a href=\"https:\/\/k3xec.com\/packrat-transmitting\/\">last post<\/a>, we worked through how to generate\na BPSK signal, and hopefully transmit it using one of our SDRs.\nLet&rsquo;s take that and move on to Receiving BPSK and turning that back\ninto symbols!<\/p>\n<p>Demodulating BPSK data is a bit more tricky than transmitting BPSK data, mostly\ndue to tedious facts of life such as space, time, and hardware built with\ncompromises because not doing that makes the problem impossible. Unfortunately,\nit&rsquo;s now our job to work within our imperfect world to recover perfect data. We\nneed to handle the addition of noise, differences in frequency, clock\nsynchronization and interference in order to recover our information. This\nmakes life a lot harder than when we transmit information, and as a result, a\nlot more complex.<\/p>\n<h4 id=\"coarse-sync\">Coarse Sync<\/h4>\n<p>Our starting point for this section will be working from a capture of a number\nof generated PACKRAT packets\n<a href=\"https:\/\/k3xec.s3.amazonaws.com\/receive\/1-rx-debug.rfcap.xz\">as heard by a PlutoSDR at (xz compressed interleaved int16, 2,621,440 samples per second)<\/a><\/p>\n<p>Every SDR has its own oscillator, which eventually controls a number of\ndifferent components of an SDR, such as the IF (if it&rsquo;s a superheterodyne\narchitecture) and the sampling rate. Drift in oscillators lead to drifts in\nfrequency &ndash; such that what one SDR may think is 100MHz may be 100.01MHz for\nanother radio. Even if the radios were perfectly in sync, other artifacts such\nas <a href=\"https:\/\/en.wikipedia.org\/wiki\/Relativistic_Doppler_effect\">doppler time dilation due to motion<\/a>\ncan cause the frequency to appear higher or lower in frequency than it was\ntransmitted.<\/p>\n<p>All this is a long way of saying, we need to determine when we see a strong\nsignal that&rsquo;s close-ish to our tuned frequency, and take steps to roughly\ncorrect it to our center frequency (in the order of 100s of Hz to kHz) in order\nto acquire a phase lock on the signal to attempt to decode information contained\nwithin.<\/p>\n<p>The easiest way of detecting the loudest signal of interest is to use an\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Fast_Fourier_transform\">FFT<\/a>. Getting into how\nFFTs work is out of scope of this post, so if this is the first time you&rsquo;re\nseeing mention of an FFT, it may be a good place to take a quick break to learn\na bit about the time domain (which is what the IQ data we&rsquo;ve been working with\nso far is), frequency domain, and how the FFT and iFFT operations can convert\nbetween them.<\/p>\n<p>Lastly, because FFTs average power over the window, swapping phases such that\nthe transmitted wave has the same number of in-phase and inverted-phase symbols\nthe power would wind up averaging to zero. This is not helpful, so I took a tip\nfrom <a href=\"https:\/\/pysdr.org\/content\/sync.html#coarse-frequency-synchronization\">Dr. Marc Lichtman&rsquo;s PySDR\nproject<\/a>\nand used <a href=\"https:\/\/ventrella.com\/ComplexSquaring\/\">complex squaring<\/a> to drive\nour BPSK signal into a single detectable carrier by squaring the IQ data.\nBecause points are on the unit circle and at <code>tau\/2<\/code> (specifically, <code>tau\/(2^1)<\/code>\nfor BPSK, <code>2^2<\/code> for QPSK) angles, and given that squaring has the effect of\ndoubling the angle, and angles are all mod <code>tau<\/code>, this will drive our wave\ncomprised of two opposite phases back into a continuous wave &ndash; effectively\nremoving our BPSK modulation, making it much easier to detect in the\nfrequency domain. Thanks to <a href=\"https:\/\/github.com\/tomberek\/\">Tom Bereknyei<\/a>\nfor helping me with that!<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">iq<\/span> []<span style=\"color:#a6e22e\">complex<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">freq<\/span> []<span style=\"color:#a6e22e\">complex<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">iq<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>]\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ perform an fft, computing the frequency<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ domain vector in `freq` given the iq data<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ contained in `iq`.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">fft<\/span>(<span style=\"color:#a6e22e\">iq<\/span>, <span style=\"color:#a6e22e\">freq<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ get the array index of the max value in the<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ freq array given the magnitude value of the<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ complex numbers.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">binIdx<\/span> = max(<span style=\"color:#a6e22e\">abs<\/span>(<span style=\"color:#a6e22e\">freq<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Now, most FFT operations will lay the frequency domain data out a bit\ndifferently than you may expect (as a human), which is that the 0th element of\nthe FFT is 0Hz, not the most negative number (like in a waterfall). Generally\nspeaking, &ldquo;zero first&rdquo; is the most common frequency domain layout (and\ngenerally speaking the most safe assumption if there&rsquo;s no other documentation\non fft layout). &ldquo;Negative first&rdquo; is usually used when the FFT is being rendered\nfor human consumption \u2013 such as a waterfall plot.<\/p>\n<p>Given that we now know which FFT bin (which is to say, which index into the\nFFT array) contains the strongest signal, we&rsquo;ll go ahead and figure out what\nfrequency that bin relates to.<\/p>\n<p>In the time domain, each complex number is the next time instant. In the\nfrequency domain, each bin is a discrete frequency \u2013 or more specifically \u2013 a\nfrequency range. The bandwidth of the bin is a function of the sampling rate\nand number of time domain samples used to do the FFT operation. As you increase\nthe amount of time used to perform the FFT, the more precise the FFT\nmeasurement of frequency can be, but it will cover the same bandwidth, as\ndefined by the sampling rate.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">sampleRate<\/span> = <span style=\"color:#ae81ff\">2<\/span>,<span style=\"color:#ae81ff\">621<\/span>,<span style=\"color:#ae81ff\">440<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ bandwidth is the range of frequencies<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ contained inside a single FFT bin,<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ measured in Hz.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">bandwidth<\/span> = <span style=\"color:#a6e22e\">sampleRate<\/span><span style=\"color:#f92672\">\/<\/span>len(<span style=\"color:#a6e22e\">freq<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Now that we know we have a zero-first layout and the bin bandwidth,\nwe can compute what our frequency offset is in Hz.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ binIdx is the index into the freq slice<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ containing the frequency domain data.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">binIdx<\/span> = <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ binFreq is the frequency of the bin<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ denoted by binIdx<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">binFreq<\/span> = <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">binIdx<\/span> &gt; len(<span style=\"color:#a6e22e\">freq<\/span>)<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">2<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ This branch covers the case where the bin<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ is past the middle point - which is to say,<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ if this is a negative frequency.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">binFreq<\/span> = <span style=\"color:#a6e22e\">bandwidth<\/span> <span style=\"color:#f92672\">*<\/span> (<span style=\"color:#a6e22e\">binIdx<\/span> <span style=\"color:#f92672\">-<\/span> len(<span style=\"color:#a6e22e\">freq<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span> } <span style=\"color:#66d9ef\">else<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ This branch covers the case where the bin<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ is in the first half of the frequency array,<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ which is to say - if this frequency is<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ a positive frequency.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">binFreq<\/span> = <span style=\"color:#a6e22e\">bandwidth<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">binIdx<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>However, sice we squared the IQ data, we&rsquo;re off in frequency by twice the\nactual frequency &ndash; if we are reading 12kHz, the bin is actually 6kHz. We\nneed to adjust for that before continuing with processing.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">binFreq<\/span> = <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ [compute the binFreq as above]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Adjust for the squaring of our IQ data<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">binFreq<\/span> = <span style=\"color:#a6e22e\">binFreq<\/span> <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#ae81ff\">2<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Finally, we need to shift the frequency by the inverse of the <code>binFreq<\/code>\nby generating a carrier wave at a specific frequency and rotating every\nsample by our carrier wave &ndash; so that a wave at the same frequency will\nslow down (or stand still!) as it approaches 0Hz relative to the carrier\nwave.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">tau<\/span> = <span style=\"color:#a6e22e\">pi<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#ae81ff\">2<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ ts tracks where in time we are (basically: phase)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">ts<\/span> <span style=\"color:#66d9ef\">float<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ inc is the amount we step forward in time (seconds)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ each sample.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">inc<\/span> <span style=\"color:#66d9ef\">float<\/span> = (<span style=\"color:#ae81ff\">1<\/span> <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#a6e22e\">sampleRate<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ amount to shift frequencies, in Hz,<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ in this case, shift +12 kHz to 0Hz<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">shift<\/span> = <span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">12<\/span>,<span style=\"color:#ae81ff\">000<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">iq<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">ts<\/span> <span style=\"color:#f92672\">+=<\/span> <span style=\"color:#a6e22e\">inc<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">ts<\/span> &gt; <span style=\"color:#a6e22e\">tau<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ not actually needed, but keeps ts within<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ 0 to 2*pi (since it is modulus 2*pi anyway)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">ts<\/span> <span style=\"color:#f92672\">-=<\/span> <span style=\"color:#a6e22e\">tau<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Here, we&#39;re going to create a carrier wave<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ at the provided frequency (in this case,<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ -12kHz)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">cwIq<\/span> = complex(<span style=\"color:#a6e22e\">cos<\/span>(<span style=\"color:#a6e22e\">tau<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">shift<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">ts<\/span>), <span style=\"color:#a6e22e\">sin<\/span>(<span style=\"color:#a6e22e\">tau<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">shift<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">ts<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">cwIq<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><p>Now we&rsquo;ve got the strong signal we&rsquo;ve observed (which may or may not be\nour BPSK modulated signal!) close enough to 0Hz that we ought to be able\nto Phase Lock the signal in order to begin demodulating the signal.<\/p>\n<h4 id=\"filter\">Filter<\/h4>\n<p>After we&rsquo;re roughly in the neighborhood of a few kHz, we can now take some\nsteps to cut out any high frequency components (both positive high frequencies\nand negative high frequencies). The normal way to do this would be to do an\nFFT, apply the filter in the frequency domain, and then do an iFFT to turn it\nback into time series data. This will work in loads of cases, but I&rsquo;ve\nfound it to be incredibly tricky to get right when doing PSK. As such, I&rsquo;ve\nopted to do this the old fashioned way in the time domain.<\/p>\n<p>I&rsquo;ve &ndash; again &ndash; opted to go simple rather than correct, and haven&rsquo;t used\nnearly any of the advanced level trickery I&rsquo;ve come across for fear of using\nit wrong. As a result, our process here is going to be generating a\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Sinc_function\">sinc<\/a> filter by computing a\nnumber of taps, and applying that in the time domain directly on the IQ\nstream.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ Generate sinc taps<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">sinc<\/span>(<span style=\"color:#a6e22e\">x<\/span> <span style=\"color:#66d9ef\">float<\/span>) <span style=\"color:#66d9ef\">float<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">x<\/span> <span style=\"color:#f92672\">==<\/span> <span style=\"color:#ae81ff\">0<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">v<\/span> = <span style=\"color:#a6e22e\">pi<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">x<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#a6e22e\">sin<\/span>(<span style=\"color:#a6e22e\">v<\/span>) <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#a6e22e\">v<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">dst<\/span> []<span style=\"color:#66d9ef\">float<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">length<\/span> = float(len(<span style=\"color:#a6e22e\">dst<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> int(<span style=\"color:#a6e22e\">length<\/span>)<span style=\"color:#f92672\">%<\/span><span style=\"color:#ae81ff\">2<\/span> <span style=\"color:#f92672\">==<\/span> <span style=\"color:#ae81ff\">0<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">length<\/span><span style=\"color:#f92672\">++<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">j<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">dst<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> float(<span style=\"color:#a6e22e\">j<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">dst<\/span>[<span style=\"color:#a6e22e\">j<\/span>] = <span style=\"color:#a6e22e\">sinc<\/span>(<span style=\"color:#ae81ff\">2<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">cutoff<\/span> <span style=\"color:#f92672\">*<\/span> (<span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">-<\/span> (<span style=\"color:#a6e22e\">length<\/span><span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">1<\/span>)<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">2<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>then we apply it in the time domain<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ Apply sinc taps to an IQ stream<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">iq<\/span> []<span style=\"color:#a6e22e\">complex<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ taps as created in `dst` above<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">taps<\/span> []<span style=\"color:#66d9ef\">float<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">delay<\/span> = make([]<span style=\"color:#a6e22e\">complex<\/span>, len(<span style=\"color:#a6e22e\">taps<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">iq<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ let&#39;s shift the next sample into<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ the delay buffer<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> copy(<span style=\"color:#a6e22e\">delay<\/span>[<span style=\"color:#ae81ff\">1<\/span>:], <span style=\"color:#a6e22e\">delay<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">delay<\/span>[<span style=\"color:#ae81ff\">0<\/span>] = <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>]\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">phasor<\/span> <span style=\"color:#a6e22e\">complex<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">j<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">delay<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ for each sample in the buffer, let&#39;s<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ weight them by the tap values, and<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ create a new complex number based on<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ filtering the real and imag values.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">phasor<\/span> <span style=\"color:#f92672\">+=<\/span> complex(\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">taps<\/span>[<span style=\"color:#a6e22e\">j<\/span>] <span style=\"color:#f92672\">*<\/span> real(<span style=\"color:#a6e22e\">delay<\/span>[<span style=\"color:#a6e22e\">j<\/span>]),\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">taps<\/span>[<span style=\"color:#a6e22e\">j<\/span>] <span style=\"color:#f92672\">*<\/span> imag(<span style=\"color:#a6e22e\">delay<\/span>[<span style=\"color:#a6e22e\">j<\/span>]),\n<\/span><\/span><span style=\"display:flex;\"><span> )\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ now that we&#39;ve run this sample<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ through the filter, we can go ahead<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ and scale it back (since we multiply<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ above) and drop it back into the iq<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ buffer.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = complex(\n<\/span><\/span><span style=\"display:flex;\"><span> real(<span style=\"color:#a6e22e\">phasor<\/span>) <span style=\"color:#f92672\">\/<\/span> len(<span style=\"color:#a6e22e\">taps<\/span>),\n<\/span><\/span><span style=\"display:flex;\"><span> imag(<span style=\"color:#a6e22e\">phasor<\/span>) <span style=\"color:#f92672\">\/<\/span> len(<span style=\"color:#a6e22e\">taps<\/span>),\n<\/span><\/span><span style=\"display:flex;\"><span> )\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>After running IQ samples through the taps and back out, we&rsquo;ll have a signal\nthat&rsquo;s been filtered to the shape of our designed Sinc filter &ndash; which will\ncut out captured high frequency components (both positive and negative).<\/p>\n<p>Astute observers will note that we&rsquo;re using the real (float) valued taps\non both the real and imaginary values independently. I&rsquo;m sure there&rsquo;s a way\nto apply taps using complex numbers, but it was a bit confusing to work\nthrough without being positive of the outcome. I may revisit this in the\nfuture!<\/p>\n<h4 id=\"downsample\">Downsample<\/h4>\n<p>Now, post-filter, we&rsquo;ve got a lot of extra RF bandwidth being represented in\nour IQ stream at our high sample rate All the high frequency values are now\nfiltered out, which means we can reduce our sampling rate without losing much\ninformation at all. We can either do nothing about it and process at the fairly\nhigh sample rate we&rsquo;re capturing at, or we can drop the sample rate down and\nhelp reduce the volume of numbers coming our way.<\/p>\n<p>There&rsquo;s two big ways of doing this; either you can take every Nth sample (e.g.,\ntake every other sample to half the sample rate, or take every 10th to decimate\nthe sample stream to a 10th of what it originally was) which is the easiest to\nimplement (and easy on the CPU too), or to average a number of samples to\ncreate a new sample.<\/p>\n<p>A nice bonus to averaging samples is that you can trade-off some CPU time for a\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Effective_number_of_bits\">higher effective number of bits\n(ENOB)<\/a> in your IQ\nstream, which helps reduce noise, among other things. Some hardware does\nexactly this (called &ldquo;Oversampling&rdquo;), and like many things, it has\n<a href=\"https:\/\/www.ti.com\/lit\/an\/slaa594a\/slaa594a.pdf\">some pros and some cons<\/a>.\nI&rsquo;ve opted to treat our IQ stream like an oversampled IQ stream\nand average samples to get a marginal bump in ENOB.<\/p>\n<p>Taking a group of 4 samples and averaging them results in a bit of added\nprecision. That means that a stream of IQ data at 8 ENOB can be bumped to 9\nENOB of precision after the process of oversampling and averaging. That\nresulting stream will be at 1\/4 of the sample rate, and this process can be\nrepeated 4 samples can again be taken for a bit of added precision; which is\ngoing to be 1\/4 of the sample rate (again), or 1\/16 of the original sample\nrate. If we again take a group of 4 samples, we&rsquo;ll wind up with another bit and\na sample rate that&rsquo;s 1\/64 of the original sample rate.<\/p>\n<h4 id=\"phase-lock\">Phase Lock<\/h4>\n<p>Our starting point for this section is the same capture as above, but\n<a href=\"https:\/\/k3xec.s3.amazonaws.com\/receiving\/4-downsample-debug.rfcap.xz\">post-coarse sync, filtering downsampling (xz compressed interleaved float32, 163,840 samples per second)<\/a><\/p>\n<p>The <a href=\"https:\/\/en.wikipedia.org\/wiki\/Phase-locked_loop\">PLL<\/a> in PACKRAT was one\nof the parts I spent the most time stuck on. There&rsquo;s no shortage of discussions\nof how hardware PLLs work, or even a few software PLLs, but very little by way\nof how to apply them and\/or troubleshoot them. After getting frustrated trying\nto follow the well worn path, I decided to cut my own way through the bush\nusing what I had learned about the concept, and hope that it works well enough\nto continue on.<\/p>\n<p>PLLs, in concept are fairly simple &ndash; you generate a carrier wave at a\nfrequency, compare the real-world SDR IQ sample to where your carrier wave is\nin phase, and use the difference between the local wave and the observed wave\nto adjust the frequency and phase of your carrier wave. Eventually, if all goes\nwell, that delta is driven as small as possible, and your carrier wave can be\nused as a reference clock to determine if the observed signal changes in\nfrequency or phase.<\/p>\n<p>In reality, tuning PLLs is a total pain, and basically no one outlines how to\napply them to BPSK signals in a descriptive way. I&rsquo;ve had to steal an approach\nI&rsquo;ve seen in hardware to implement my software PLL, with any hope it&rsquo;s close\nenough that this isn&rsquo;t a hazard to learners. The concept is to generate the\ncarrier wave (as above) and store some rolling averages to tune the carrier\nwave over time. I use two constants, &ldquo;alpha&rdquo; and &ldquo;beta&rdquo; (which appear to be\ntraditional PLL variable names for this function) which control how quickly the\nfrequency and phase is changed according to observed mismatches. Alpha is set\nfairly high, which means discrepancies between our carrier and observed data\nare quickly applied to the phase, and a lower constant for Beta, which will\ntake long-term errors and attempt to use that to match frequency.<\/p>\n<p>This is all well and good. Getting to this point isn&rsquo;t all that obscure, but\nthe trouble comes when processing a BPSK signal. Phase changes kick the PLL\nout of alignment and it tends to require some time to get back into phase lock,\nwhen we really shouldn&rsquo;t even be losing it in the first place. My attempt is\nto generate two predicted samples, one for each phase of our BPSK signal. The\ndelta is compared, and the lower error of the two is used to adjust the PLL,\nbut the carrier wave itself is used to rotate the sample.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">alpha<\/span> = <span style=\"color:#ae81ff\">0.1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">beta<\/span> = (<span style=\"color:#a6e22e\">alpha<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">alpha<\/span>) <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#ae81ff\">2<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">phase<\/span> = <span style=\"color:#ae81ff\">0.0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">frequency<\/span> = <span style=\"color:#ae81ff\">0.0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">iq<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">predicted<\/span> = complex(<span style=\"color:#a6e22e\">cos<\/span>(<span style=\"color:#a6e22e\">phase<\/span>), <span style=\"color:#a6e22e\">sin<\/span>(<span style=\"color:#a6e22e\">phase<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">sample<\/span> = <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">conj<\/span>(<span style=\"color:#a6e22e\">predicted<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">delta<\/span> = <span style=\"color:#a6e22e\">phase<\/span>(<span style=\"color:#a6e22e\">sample<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">predicted2<\/span> = complex(<span style=\"color:#a6e22e\">cos<\/span>(<span style=\"color:#a6e22e\">phase<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">pi<\/span>), <span style=\"color:#a6e22e\">sin<\/span>(<span style=\"color:#a6e22e\">phase<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">pi<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">sample2<\/span> = <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">conj<\/span>(<span style=\"color:#a6e22e\">predicted2<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">delta2<\/span> = <span style=\"color:#a6e22e\">phase<\/span>(<span style=\"color:#a6e22e\">sample2<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">abs<\/span>(<span style=\"color:#a6e22e\">delta2<\/span>) &lt; <span style=\"color:#a6e22e\">abs<\/span>(<span style=\"color:#a6e22e\">delta<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ note that we do not update &#39;sample&#39;.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">delta<\/span> = <span style=\"color:#a6e22e\">delta2<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">phase<\/span> <span style=\"color:#f92672\">+=<\/span> <span style=\"color:#a6e22e\">alpha<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">delta<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">frequency<\/span> <span style=\"color:#f92672\">+=<\/span> <span style=\"color:#a6e22e\">beta<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#a6e22e\">delta<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ adjust the iq sample to the PLL rotated<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ sample.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = <span style=\"color:#a6e22e\">sample<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>If all goes well, this loop has the effect of driving a BPSK signal&rsquo;s imaginary\nvalues to 0, and the real value between +1 and -1.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/packrat\/packrat-phase-locked.png\" alt=\"\"><\/p>\n<h4 id=\"average-idle--carrier-detect\">Average Idle \/ Carrier Detect<\/h4>\n<p>Our starting point for this section is the same capture as above, but\n<a href=\"https:\/\/k3xec.s3.amazonaws.com\/receiving\/5-phase-debug.rfcap.xz\">post-PLL (xz compressed interleaved float32, 163,840 samples per second)<\/a><\/p>\n<p>When we start out, we have IQ samples that have been mostly driven to an\nimaginary component of 0 and real value range between +1 and -1 for each symbol\nperiod. Our goal now is to determine if we&rsquo;re receiving a signal, and if so,\ndetermine if it&rsquo;s +1 or -1. This is a deceptively hard problem given it spans a\nlot of other similarly entertaining hard problems. I&rsquo;ve opted to not solve the\nhard problems involved and hope that in practice my very haphazard\nimplementation works well enough. This turns out to be both good (not solving a\nproblem is a great way to not spend time on it) and bad (turns out it does\nmaterially impact performance). This segment is the one I plan on revisiting,\nfirst. Expect more here at some point!<\/p>\n<p>Given that I want to be able to encapsulate three states in the output from\nthis section (our Symbols are no carrier detected (&ldquo;0&rdquo;), real value 1 (&ldquo;1&rdquo;) or\nreal value -1 (&quot;-1&quot;)), which means spending cycles to determine what the\nbaseline noise is to try and identify when a signal breaks through the noise\nbecomes incredibly important.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">idleThreshold<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">thresholdFactor<\/span> = <span style=\"color:#ae81ff\">10<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ sigThreshold is used to determine if the symbol<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ is -1, +1 or 0. It&#39;s 1.3 times the idle signal<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ threshold.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">sigThreshold<\/span> = (<span style=\"color:#a6e22e\">idleThreshold<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#ae81ff\">0.3<\/span>) <span style=\"color:#f92672\">+<\/span> <span style=\"color:#a6e22e\">idleThreshold<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ iq contains a single symbol&#39;s worth of IQ samples.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ clock alignment isn&#39;t really considered; so we&#39;ll<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ get a bad packet if we have a symbol transition<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ in the middle of this buffer. No attempt is made<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ to correct for this yet.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">iq<\/span> []<span style=\"color:#a6e22e\">complex<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ avg is used to average a chunk of samples in the<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ symbol buffer.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">avg<\/span> <span style=\"color:#66d9ef\">float<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">mid<\/span> = len(<span style=\"color:#a6e22e\">iq<\/span>) <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#ae81ff\">2<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ midNum is used to determine how many symbols to<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ average at the middle of the symbol.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">midNum<\/span> = len(<span style=\"color:#a6e22e\">iq<\/span>) <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#ae81ff\">50<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">j<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">mid<\/span>; <span style=\"color:#a6e22e\">j<\/span> &lt; <span style=\"color:#a6e22e\">mid<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">midNum<\/span>; <span style=\"color:#a6e22e\">j<\/span><span style=\"color:#f92672\">++<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">avg<\/span> <span style=\"color:#f92672\">+=<\/span> real(<span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#a6e22e\">j<\/span>])\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">avg<\/span> <span style=\"color:#f92672\">\/=<\/span> <span style=\"color:#a6e22e\">midNum<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">symbol<\/span> <span style=\"color:#66d9ef\">float<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">switch<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">case<\/span> <span style=\"color:#a6e22e\">avg<\/span> &gt; <span style=\"color:#a6e22e\">sigThreshold<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">symbol<\/span> = <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">case<\/span> <span style=\"color:#a6e22e\">avg<\/span> &lt; <span style=\"color:#f92672\">-<\/span><span style=\"color:#a6e22e\">sigThreshold<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">symbol<\/span> = <span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">default<\/span>:\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">symbol<\/span> = <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ update the idleThreshold using the thresholdFactor<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ to average the idleThreshold over more samples to<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ get a better idea of average noise.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">idleThreshold<\/span> = (\n<\/span><\/span><span style=\"display:flex;\"><span> (<span style=\"color:#a6e22e\">idleThreshold<\/span><span style=\"color:#f92672\">*<\/span>(<span style=\"color:#a6e22e\">thresholdFactor<\/span><span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">1<\/span>) <span style=\"color:#f92672\">+<\/span> <span style=\"color:#a6e22e\">symbol<\/span>) \\\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#a6e22e\">thresholdFactor<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> )\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ write symbol to output somewhere<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><\/code><\/pre><\/div><h4 id=\"next-steps\">Next Steps<\/h4>\n<p>Now that we have a stream of values that are either <code>+1<\/code>, <code>-1<\/code> or <code>0<\/code>, we\ncan frame \/ unframe the data contained in the stream, and decode Packets\ncontained inside, coming next in <a href=\"https:\/\/k3xec.com\/packrat-framing\/\">Part 4!<\/a><\/p>"},{"title":"Transmitting BPSK symbols (Part 2\/5) \ud83d\udc00","link":"https:\/\/k3xec.com\/packrat-transmitting\/","pubDate":"Fri, 03 Dec 2021 11:00:00 -0500","guid":"https:\/\/k3xec.com\/packrat-transmitting\/","description":"<div class=\"hz-alert-ok\">\n\ud83d\udc00 This post is part of a series called \"PACKRAT\". If this is the first post\nyou've found, it'd be worth reading the\n<a href=\"https:\/\/k3xec.com\/packrat-intro\/\">intro post<\/a> first and then looking over\n<a href=\"https:\/\/k3xec.com\/tags\/packrat\/\">all posts in the series<\/a>.\n<\/div>\n<p>In the <a href=\"https:\/\/k3xec.com\/packrat-processing-iq\/\">last post<\/a>, we worked through what IQ is,\nand different formats that it may be sent or received in. Let&rsquo;s take that\nand move on to Transmitting BPSK using IQ data!<\/p>\n<p>When we transmit and receive information through RF using an SDR, data is\ntraditionally encoded into a stream of\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Symbol_rate#Symbols\">symbols<\/a>\nwhich are then used by a program to modulate the IQ stream, and sent over\nthe airwaves.<\/p>\n<p>PACKRAT uses <a href=\"https:\/\/en.wikipedia.org\/wiki\/Phase-shift_keying\">BPSK<\/a> to\nencode Symbols through RF. BPSK is the act of modulating the phase of a\nsine wave to carry information. The transmitted wave swaps between two\nstates in order to convey a 0 or a 1. Our symbols modulate the transmitted\nsine wave&rsquo;s phase, so that it moves between in-phase with the SDR&rsquo;s transmitter\nand 180 degrees (or \u03c0 radians) out of phase with the SDR&rsquo;s transmitter.<\/p>\n<p>The difference between a &ldquo;Bit&rdquo; and a &ldquo;Symbol&rdquo; in PACKRAT is not incredibly\nmeaningful, and I&rsquo;ll often find myself slipping up when talking about them.\nI&rsquo;ve done my best to try and use the right word at the right stage, but it&rsquo;s\nnot as obvious where the line between bit and symbol is &ndash; at least not as\nobvious as it would be with QPSK or QAM. The biggest difference is that there\nare three meaningful states for PACKRAT over BPSK - a 1 (for &ldquo;In phase&rdquo;), -1\n(for &ldquo;180 degrees out of phase&rdquo;) and 0 (for &ldquo;no carrier&rdquo;). For my\nimplementation, a stream of all zeros will not transmit data over the airwaves,\na stream of all 1s will transmit all &ldquo;1&rdquo; bits over the airwaves, and a stream\nof all -1s will transmit all &ldquo;0&rdquo; bits over the airwaves.<\/p>\n<p>We&rsquo;re not going to cover turning a byte (or bit) into a symbol yet &ndash; I&rsquo;m going\nto write more about that in a later section. So for now, let&rsquo;s just worry about\nsymbols in, and symbols out.<\/p>\n<h4 id=\"transmitting-a-sine-wave-at-0hz\">Transmitting a Sine wave at 0Hz<\/h4>\n<p>If we go back to thinking about IQ data as a precisely timed measurements of\nenergy over time at some particular specific frequency, we can consider what\na sine wave will look like in IQ. Before we dive into antennas and RF, let&rsquo;s go\nto something a bit more visual.<\/p>\n<p>For the first example, you can see an example of a camera who&rsquo;s frame rate (or\nSampling Rate!) matches the exact number of rotations per second (or\nFrequency!) of the propeller and it <a href=\"https:\/\/www.youtube.com\/watch?v=g5sPG6Ss5_E\">appears to stand exactly\nstill<\/a>. Every time the Camera\ntakes a frame, it&rsquo;s catching the propeller in the exact same place in space,\neven though it&rsquo;s made a complete rotation.<\/p>\n<p>The second example is very similar, it&rsquo;s a light strobing (in this case, our\nsampling rate, since the darkness is ignored by our brains) at the same rate\n(frequency) as <a href=\"https:\/\/www.youtube.com\/watch?v=OtxlQTmx1LE\">water dropping from a\nfaucet<\/a> &ndash; and the video creator\nis even nice enough to change the sampling frequency to have the droplets move\nboth forward and backward (positive and negative frequency) in comparison to the\nfaucet.<\/p>\n<p>IQ works the same way. If we catch something in perfect frequency alignment\nwith our radio, we&rsquo;ll wind up with readings that are the same for the entire\nstream of data. This means we can transmit a sine wave by setting all of the\nIQ samples in our buffer to <code>1+0i<\/code>, which will transmit a pure sine wave at\nexactly the center frequency of the radio.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">sine<\/span> []<span style=\"color:#a6e22e\">complex<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">sine<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">sine<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = complex(<span style=\"color:#ae81ff\">1.0<\/span>, <span style=\"color:#ae81ff\">0.0<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><p>Alternatively, we can transmit a Sine wave (but with the opposite phase) by\nflipping the real value from 1 to -1. The same Sine wave is transmitted on\nthe same Frequency, except when the wave goes high in the example above, the\nwave will go low in the example below.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">sine<\/span> []<span style=\"color:#a6e22e\">complex<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">sine<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">sine<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = complex(<span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">1.0<\/span>, <span style=\"color:#ae81ff\">0.0<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><p>In fact, we can make a carrier wave at any phase angle and amplitude by using a\nbit of trig.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ angle is in radians - here we have<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ 1.5 Pi (0.75 Tau) or 270 degrees.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">angle<\/span> = <span style=\"color:#a6e22e\">pi<\/span> <span style=\"color:#f92672\">*<\/span> <span style=\"color:#ae81ff\">1.5<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ amplitude controls the transmitted<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ strength of the carrier wave.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">amplitude<\/span> = <span style=\"color:#ae81ff\">1.0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ output buffer as above<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">sine<\/span> []<span style=\"color:#a6e22e\">complex<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">sine<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">sine<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = complex(\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">amplitude<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">cos<\/span>(<span style=\"color:#a6e22e\">angle<\/span>),\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">amplitude<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">sin<\/span>(<span style=\"color:#a6e22e\">angle<\/span>),\n<\/span><\/span><span style=\"display:flex;\"><span> )\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><p>The amplitude of the transmitted wave is the absolute value of the IQ\nsample (sometimes called magnitude), and the phase can be computed as the\nangle (or argument). The amplitude remains constant (at 1) in both cases.\nRemember back to the airplane propeller or water droplets &ndash; we&rsquo;re controlling\nwhere we&rsquo;re observing the sine wave. It looks like a consistent value\nto us, but in reality it&rsquo;s being transmitted as a pure carrier wave at the\nprovided frequency. Changing the angle of the number we&rsquo;re transmitting will\ncontrol where in the sine wave cycle we&rsquo;re &ldquo;observing&rdquo; it at.<\/p>\n<h4 id=\"generating-bpsk-modulated-iq-data\">Generating BPSK modulated IQ data<\/h4>\n<p>Modulating our carrier wave with our symbols is fairly straightforward to\ndo &ndash; we can multiply the symbol by 1 to get the real value to be used\nin the IQ stream. Or, more simply - we can just use the symbol directly in\nthe constructed IQ data.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">sampleRate<\/span> = <span style=\"color:#ae81ff\">2<\/span>,<span style=\"color:#ae81ff\">621<\/span>,<span style=\"color:#ae81ff\">440<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">baudRate<\/span> = <span style=\"color:#ae81ff\">1024<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ This represents the number of IQ samples<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ required to send a single symbol at the<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ provided baud and sample rate. I picked<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ two numbers in order to avoid half samples.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ We will transmit each symbol in blocks of<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ this size.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">samplesPerSymbol<\/span> = <span style=\"color:#a6e22e\">sampleRate<\/span> <span style=\"color:#f92672\">\/<\/span> <span style=\"color:#a6e22e\">baudRate<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">samples<\/span> = make([]<span style=\"color:#a6e22e\">complex<\/span>, <span style=\"color:#a6e22e\">samplesPerSymbol<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ symbol is one of 1, -1 or 0.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">each<\/span> <span style=\"color:#a6e22e\">symbol<\/span> <span style=\"color:#a6e22e\">in<\/span> <span style=\"color:#a6e22e\">symbols<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">samples<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">samples<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = complex(<span style=\"color:#a6e22e\">symbol<\/span>, <span style=\"color:#ae81ff\">0<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ write the samples out to an output file<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ or radio.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">write<\/span>(<span style=\"color:#a6e22e\">samples<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span> }\n<\/span><\/span><\/code><\/pre><\/div><p>If you want to check against a baseline capture, here&rsquo;s\n<a href=\"https:\/\/k3xec.s3.amazonaws.com\/transmit\/10_raw_packets_2048000sps_c64.rfcap\">10 example packets at 204800 samples per second<\/a>.<\/p>\n<h4 id=\"next-steps\">Next Steps<\/h4>\n<p>Now that we can transmit data, we&rsquo;ll start working on a receive path in Part 3,\nin order to check our work when transmitting the packets, as well as being\nable to hear packets we transmit from afar, coming up next in\n<a href=\"https:\/\/k3xec.com\/packrat-receiving\/\">Part 3!<\/a>!<\/p>"},{"title":"Processing IQ data formats (Part 1\/5) \ud83d\udc00","link":"https:\/\/k3xec.com\/packrat-processing-iq\/","pubDate":"Thu, 02 Dec 2021 12:00:00 -0500","guid":"https:\/\/k3xec.com\/packrat-processing-iq\/","description":"<div class=\"hz-alert-ok\">\n\ud83d\udc00 This post is part of a series called \"PACKRAT\". If this is the first post\nyou've found, it'd be worth reading the\n<a href=\"https:\/\/k3xec.com\/packrat-intro\/\">intro post<\/a> first and then looking over\n<a href=\"https:\/\/k3xec.com\/tags\/packrat\/\">all posts in the series<\/a>.\n<\/div>\n<p>When working with SDRs, information about the signals your radio is receiving\nare communicated by streams of\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/In-phase_and_quadrature_components\">IQ<\/a>\ndata. IQ is short for &ldquo;In-phase&rdquo; and &ldquo;Quadrature&rdquo;, which means 90 degrees\nout of phase. Values in the IQ stream are\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Complex_number\">complex numbers<\/a>, so\nconverting them to a native complex type in your language\nhelps greatly when processing the IQ data for meaning.<\/p>\n<p>I won&rsquo;t get too deep into what IQ is or why complex numbers (mostly since I\ndon&rsquo;t think I fully understand it well enough to explain it yet), but here&rsquo;s\nsome basics in case this is your first interaction with IQ data before\ngoing off and reading more.<\/p>\n<div class=\"hz-alert-warning\">\nBefore we get started &mdash;\nat any point, if you feel lost in this post, it's OK to take a break to do\na bit of learning elsewhere in the internet. I'm still new to this, so I'm\nsure my overview in one paragraph here won't help clarify things too much.\nThis took me months to sort out on my own. It's not you, really! I particularly\nenjoyed reading\n<a href=\"https:\/\/visual-dsp.switchb.org\/presentation\">visual-dsp.switchb.org<\/a>\nwhen it came to learning about how IQ represents signals, and\n<a href=\"https:\/\/www.analog.com\/en\/education\/education-library\/software-defined-radio-for-engineers.html\">Software-Defined Radio for Engineers<\/a>\nfor a more general reference.\n<\/div>\n<p>Each value in the stream is taken at a precisely spaced sampling interval\n(called the sampling rate of the radio). Jitter in that sampling interval, or a\ndrift in the requested and actual sampling rate (usually represented in PPM, or\nparts per million &ndash; how many samples out of one million are missing) can cause\nerrors in frequency. In the case of a PPM error, one radio may think it&rsquo;s\n100.1MHz and the other may think it&rsquo;s 100.2MHz, and jitter will result in added\nnoise in the resulting stream.<\/p>\n<p>A single IQ sample is both the real and imaginary values, together. The complex\nnumber (both parts) is the sample. The number of samples per second is the\nnumber of real and imaginary value pairs per second.<\/p>\n<p>Each sample is reading the electrical energy coming off the antenna at that\nexact time instant. We&rsquo;re looking to see how that goes up and down over time to\ndetermine what frequencies we&rsquo;re observing around us. If the IQ stream is only\nreal-valued measures (e.g., float values rather than complex values reading\nvoltage from a wire), you can still send and receive signals, but those signals\nwill be mirrored across your 0Hz boundary. That means if you&rsquo;re tuned to\n100MHz, and you have a nearby transmitter at 99.9MHz, you&rsquo;d see it at 100.1MHz.\nIf you want to get an intuitive understanding of this concept before getting\ninto the heavy math, a good place to start is looking at how <a href=\"https:\/\/en.wikipedia.org\/wiki\/Incremental_encoder#Quadrature_outputs\">Quadrature\nencoders<\/a>\nwork. Using complex numbers means we can see &ldquo;up&rdquo; in frequency as well as\n&ldquo;down&rdquo; in frequency, and understand that those are different signals.<\/p>\n<p>The reason why we need negative frequencies is that our 0Hz is the center of\nour SDR&rsquo;s tuned frequency, not actually at 0Hz in nature. Generally speaking,\nit&rsquo;s doing loads in hardware (and firmware!) to mix the raw RF signals with a\nlocal oscillator to a frequency that can be sampled at the requested rate\n(fundamentally the same concept as a <a href=\"https:\/\/en.wikipedia.org\/wiki\/Superheterodyne_receiver\">superheterodyne\nreceiver<\/a>), so a\nfrequency of &lsquo;-10MHz&rsquo; means that signal is 10 MHz below the center of our SDR&rsquo;s\ntuned frequency.<\/p>\n<p>The sampling rate dictates the amount of frequency representable in the data\nstream. You&rsquo;ll sometimes see this called the\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Nyquist_frequency\">Nyquist frequency<\/a>. The Nyquist\nFrequency is one half of the sampling rate. Intuitively, if you think about the\namount of bandwidth observable as being 1:1 with the sampling rate of the stream,\nand the middle of your bandwidth is 0 Hz, you would only have enough space to\ngo up in frequency for half of your bandwidth &ndash; or half of your sampling rate.\nSame for going down in frequency.<\/p>\n<h4 id=\"float-32--complex-64\">Float 32 \/ Complex 64<\/h4>\n<p>IQ samples that are being processed by software are commonly processed as\nan interleaved pair of 32 bit floating point numbers, or a 64 bit complex\nnumber. The first float32 is the real value, and the second is the imaginary\nvalue.<\/p>\n<div class=\"hz-abi\">\n<div type=\"f32\" class=\"hz-abi-green hz-abi-2b\">I#0<\/div>\n<div type=\"f32\" class=\"hz-abi-green hz-abi-2b\">Q#0<\/div>\n<div type=\"f32\" class=\"hz-abi-green hz-abi-2b\">I#1<\/div>\n<div type=\"f32\" class=\"hz-abi-green hz-abi-2b\">Q#1<\/div>\n<div type=\"f32\" class=\"hz-abi-green hz-abi-2b\">I#2<\/div>\n<div type=\"f32\" class=\"hz-abi-green hz-abi-2b\">Q#2<\/div>\n<\/div>\n<p>The complex number <code>1+1i<\/code> is represented as <code>1.0 1.0<\/code> and the complex number\n<code>-1-1i<\/code> is represented as <code>-1.0 -1.0<\/code>. <b>Unless otherwise specified, all the\nIQ samples and pseudocode to follow assumes interleaved float32 IQ data\nstreams.<\/b><\/p>\n<p><a href=\"https:\/\/k3xec.s3.amazonaws.com\/iq\/10hz_1024sps_c64.rfcap\">Example interleaved float32 file (10Hz Wave at 1024 Samples per Second)<\/a><\/p>\n<h4 id=\"rtl-sdr\">RTL-SDR<\/h4>\n<p>IQ samples from the RTL-SDR are encoded as a stream of interleaved unsigned\n8 bit integers (uint8 or u8). The first sample is the real (in-phase or I)\nvalue, and the second is the imaginary (quadrature or Q) value. Together each\npair of values makes up a complex number at a specific time instant.<\/p>\n<div class=\"hz-abi\">\n<div type=\"u8\" class=\"hz-abi-green hz-abi-1b\">I#0<\/div>\n<div type=\"u8\" class=\"hz-abi-green hz-abi-1b\">Q#0<\/div>\n<div type=\"u8\" class=\"hz-abi-green hz-abi-1b\">I#1<\/div>\n<div type=\"u8\" class=\"hz-abi-green hz-abi-1b\">Q#1<\/div>\n<div type=\"u8\" class=\"hz-abi-green hz-abi-1b\">I#2<\/div>\n<div type=\"u8\" class=\"hz-abi-green hz-abi-1b\">Q#2<\/div>\n<\/div>\n<p>The complex number <code>1+1i<\/code> is represented as <code>0xFF 0xFF<\/code> and the complex number\n<code>-1-1i<\/code> is represented as <code>0x00 0x00<\/code>. The complex number <code>0+0i<\/code> is not easily\nrepresentable &ndash; since half of <code>0xFF<\/code> is <code>127.5<\/code>.<\/p>\n<table>\n<thead>\n<tr>\n<td>Complex Number<\/td>\n<td>Representation<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>1+1i<\/td>\n<td>[]uint8{0xFF, 0xFF}<\/td>\n<\/tr>\n<tr>\n<td>-1+1i<\/td>\n<td>[]uint8{0x00, 0xFF}<\/td>\n<\/tr>\n<tr>\n<td>-1-1i<\/td>\n<td>[]uint8{0x00, 0x00}<\/td>\n<\/tr>\n<tr>\n<td>0+0i<\/td>\n<td>[]uint8{0x80, 0x80} or []uint8{0x7F, 0x7F}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>And finally, here&rsquo;s some pseudocode to convert an rtl-sdr style IQ sample\nto a floating point complex number:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">in<\/span> = []<span style=\"color:#66d9ef\">uint8<\/span>{<span style=\"color:#ae81ff\">0x7F<\/span>, <span style=\"color:#ae81ff\">0x7F<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">real<\/span> = (float(<span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#ae81ff\">0<\/span>])<span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">127.5<\/span>)<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">127.5<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">imag<\/span> = (float(<span style=\"color:#a6e22e\">iq<\/span>[<span style=\"color:#ae81ff\">1<\/span>])<span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">127.5<\/span>)<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">127.5<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">out<\/span> = complex(<span style=\"color:#a6e22e\">real<\/span>, <span style=\"color:#a6e22e\">imag<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>.\n<\/span><\/span><\/code><\/pre><\/div><p><a href=\"https:\/\/k3xec.s3.amazonaws.com\/iq\/10hz_1024sps_u8.rfcap\">Example interleaved uint8 file (10Hz Wave at 1024 Samples per Second)<\/a><\/p>\n<h4 id=\"hackrf\">HackRF<\/h4>\n<p>IQ samples from the HackRF are encoded as a stream of interleaved signed\n8 bit integers (int8 or i8). The first sample is the real (in-phase or I)\nvalue, and the second is the imaginary (quadrature or Q) value. Together each\npair of values makes up a complex number at a specific time instant.<\/p>\n<div class=\"hz-abi\">\n<div type=\"i8\" class=\"hz-abi-green hz-abi-1b\">I#0<\/div>\n<div type=\"i8\" class=\"hz-abi-green hz-abi-1b\">Q#0<\/div>\n<div type=\"i8\" class=\"hz-abi-green hz-abi-1b\">I#1<\/div>\n<div type=\"i8\" class=\"hz-abi-green hz-abi-1b\">Q#1<\/div>\n<div type=\"i8\" class=\"hz-abi-green hz-abi-1b\">I#2<\/div>\n<div type=\"i8\" class=\"hz-abi-green hz-abi-1b\">Q#2<\/div>\n<\/div>\n<p>Formats that use signed integers do have one quirk due to\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Two%27s_complement\">two&rsquo;s complement<\/a>,\nwhich is that the smallest negative number representable&rsquo;s absolute\nvalue is one more than the largest positive number. <code>int8<\/code> values can\nrange between <code>-128<\/code> to <code>127<\/code>, which means there&rsquo;s bit of ambiguity in\nhow +1, 0 and -1 are represented. Either you can create perfectly symmetric\nranges of values between +1 and -1, but 0 is not representable, have more\npossible values in the negative range, or allow values above (or just below)\nthe maximum in the range to be allowed.<\/p>\n<p>Within my implementation, my approach has been to scale based on the max\ninteger value of the type, so the lowest possible signed value is actually\nslightly smaller than <code>-1<\/code>. Generally, if your code is seeing values that low\nthe difference in step between -1 and slightly less than -1 isn&rsquo;t very\nsignificant, even with only 8 bits. Just a curiosity to be aware of.<\/p>\n<table>\n<thead>\n<tr>\n<td>Complex Number<\/td>\n<td>Representation<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>1+1i<\/td>\n<td>[]int8{127, 127}<\/td>\n<\/tr>\n<tr>\n<td>-1+1i<\/td>\n<td>[]int8{-128, 127}<\/td>\n<\/tr>\n<tr>\n<td>-1-1i<\/td>\n<td>[]int8{-128, -128}<\/td>\n<\/tr>\n<tr>\n<td>0+0i<\/td>\n<td>[]int8{0, 0}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>And finally, here\u2019s some pseudocode to convert a hackrf style IQ sample to a\nfloating point complex number:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">in<\/span> = []<span style=\"color:#66d9ef\">int8<\/span>{<span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">5<\/span>, <span style=\"color:#ae81ff\">112<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">real<\/span> = (float(<span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">0<\/span>]))<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">127<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">imag<\/span> = (float(<span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">1<\/span>]))<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">127<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">out<\/span> = complex(<span style=\"color:#a6e22e\">real<\/span>, <span style=\"color:#a6e22e\">imag<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>.\n<\/span><\/span><\/code><\/pre><\/div><p><a href=\"https:\/\/k3xec.s3.amazonaws.com\/iq\/10hz_1024sps_i8.rfcap\">Example interleaved int8 file (10Hz Wave at 1024 Samples per Second)<\/a><\/p>\n<h4 id=\"plutosdr\">PlutoSDR<\/h4>\n<p>IQ samples from the PlutoSDR are encoded as a stream of interleaved signed\n16 bit integers (int16 or i16). The first sample is the real (in-phase or I)\nvalue, and the second is the imaginary (quadrature or Q) value. Together each\npair of values makes up a complex number at a specific time instant.<\/p>\n<p>Almost no SDRs capture at a 16 bit depth natively, often you&rsquo;ll see 12 bit\nintegers (as is the case with the PlutoSDR) being sent around as 16 bit\nintegers. This leads to the next possible question, which is are values LSB\nor MSB aligned? The PlutoSDR sends data LSB aligned (which is to say, the\nlargest real or imaginary value in the stream will not exceed 4095), but\nexpects data being transmitted to be MSB aligned (which is to say the lowest\nset bit possible is the 5th bit in the number, or values can only be set in\nincrements of 16).<\/p>\n<p>As a result, the quirk observed with the HackRF (that the range of values\nbetween 0 and -1 is different than the range of values between 0 and +1) does\nnot impact us so long as we do not use the whole 16 bit range.<\/p>\n<table>\n<thead>\n<tr>\n<td>Complex Number<\/td>\n<td>Representation<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>1+1i<\/td>\n<td>[]int16{32767, 32767}<\/td>\n<\/tr>\n<tr>\n<td>-1+1i<\/td>\n<td>[]int16{-32768, 32767}<\/td>\n<\/tr>\n<tr>\n<td>-1-1i<\/td>\n<td>[]int16{-32768, -32768}<\/td>\n<\/tr>\n<tr>\n<td>0+0i<\/td>\n<td>[]int16{0, 0}<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>And finally, here\u2019s some pseudocode to convert a PlutoSDR style IQ sample to a\nfloating point complex number, including moving the sample from LSB to MSB\naligned:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">in<\/span> = []<span style=\"color:#66d9ef\">int16<\/span>{<span style=\"color:#f92672\">-<\/span><span style=\"color:#ae81ff\">15072<\/span>, <span style=\"color:#ae81ff\">496<\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ shift left 4 bits (16 bits - 12 bits = 4 bits)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#75715e\">\/\/ to move from LSB aligned to MSB aligned.<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">0<\/span>] = <span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">0<\/span>] <span style=\"color:#f92672\">&lt;&lt;<\/span> <span style=\"color:#ae81ff\">4<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">1<\/span>] = <span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">1<\/span>] <span style=\"color:#f92672\">&lt;&lt;<\/span> <span style=\"color:#ae81ff\">4<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">real<\/span> = (float(<span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">0<\/span>]))<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">32767<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">imag<\/span> = (float(<span style=\"color:#a6e22e\">in<\/span>[<span style=\"color:#ae81ff\">1<\/span>]))<span style=\"color:#f92672\">\/<\/span><span style=\"color:#ae81ff\">32767<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#a6e22e\">out<\/span> = complex(<span style=\"color:#a6e22e\">real<\/span>, <span style=\"color:#a6e22e\">imag<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">...<\/span>.\n<\/span><\/span><\/code><\/pre><\/div><p><a href=\"https:\/\/k3xec.s3.amazonaws.com\/iq\/10hz_1024sps_i16.rfcap\">Example interleaved i16 file (10Hz Wave at 1024 Samples per Second)<\/a><\/p>\n<h4 id=\"next-steps\">Next Steps<\/h4>\n<p>Now that we can read (and write!) IQ data, we can get started first on the\ntransmitter, which we can (in turn) use to test receiving our own BPSK signal,\n<a href=\"https:\/\/k3xec.com\/packrat-transmitting\/\">coming next in Part 2!<\/a><\/p>"},{"title":"Intro to PACKRAT (Part 0\/5) \ud83d\udc00","link":"https:\/\/k3xec.com\/packrat-intro\/","pubDate":"Thu, 02 Dec 2021 11:00:00 -0500","guid":"https:\/\/k3xec.com\/packrat-intro\/","description":"<p>Hello! Welcome. I&rsquo;m so thrilled you&rsquo;re here.<\/p>\n<p>Some of you may know this (as I&rsquo;ve written about in the past), but if you&rsquo;re\nnew to my RF travels, I\u2019ve spent nights and weekends over the last two years\ndoing some self directed learning on how radios work. I\u2019ve gone from a very\nbasic understanding of wireless communications, all the way through the process\nof learning about and implementing a set of libraries to modulate and\ndemodulate data using my now formidable stash of\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Software-defined_radio\">SDRs<\/a>. I\u2019ve been\nimplementing all of the RF processing code from first principals and purely\nbased on other primitives I\u2019ve written myself to prove to myself that I\nunderstand each concept before moving on.<\/p>\n<p>I&rsquo;ve just finished a large personal milestone &ndash; I was able to successfully\nsend a cURL HTTP request through a network interface into my stack of\nlibraries, through my own\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Phase-shift_keying#Binary_phase-shift_keying_(BPSK)\">BPSK<\/a>\nimplementation, framed in my own artisanal hand crafted Layer 2 framing scheme,\ndemodulated by my code on the other end, and sent into a Linux network\ninterface. The combination of the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Physical_layer\">Layer 1\nPHY<\/a> and <a href=\"https:\/\/en.wikipedia.org\/wiki\/Data_link_layer\">Layer 2 Data\nLink<\/a> is something that I&rsquo;ve\nbeen calling &ldquo;PACKRAT&rdquo;.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-txt\" data-lang=\"txt\"><span style=\"display:flex;\"><span>$ curl http:\/\/44.127.0.8:8000\/\n<\/span><\/span><span style=\"display:flex;\"><span>* Connected to 44.127.0.8 (44.127.0.8) port 8000 (#0)\n<\/span><\/span><span style=\"display:flex;\"><span>&gt; GET \/ HTTP\/1.1\n<\/span><\/span><span style=\"display:flex;\"><span>&gt; Host: localhost:1313\n<\/span><\/span><span style=\"display:flex;\"><span>&gt; User-Agent: curl\/7.79.1\n<\/span><\/span><span style=\"display:flex;\"><span>&gt; Accept: *\/*\n<\/span><\/span><span style=\"display:flex;\"><span>&gt;\n<\/span><\/span><span style=\"display:flex;\"><span>* Mark bundle as not supporting multiuse\n<\/span><\/span><span style=\"display:flex;\"><span>* HTTP\/1.0, assume close after body\n<\/span><\/span><span style=\"display:flex;\"><span>&lt; HTTP\/1.0 200 OK\n<\/span><\/span><span style=\"display:flex;\"><span>&lt; Content-Length: 236\n<\/span><\/span><span style=\"display:flex;\"><span>&lt;\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> ____ _ ____ _ ______ _ _____\n<\/span><\/span><span style=\"display:flex;\"><span>| _ \\ \/ \\ \/ ___| |\/ \/ _ \\ \/ \\|_ _|\n<\/span><\/span><span style=\"display:flex;\"><span>| |_) \/ _ \\| | | &#39; \/| |_) | \/ _ \\ | |\n<\/span><\/span><span style=\"display:flex;\"><span>| __\/ ___ \\ |___| . \\| _ &lt; \/ ___ \\| |\n<\/span><\/span><span style=\"display:flex;\"><span>|_| \/_\/ \\_\\____|_|\\_\\_| \\_\\\/_\/ \\_\\_|\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>* Closing connection 0\n<\/span><\/span><\/code><\/pre><\/div><p>In an effort to &ldquo;pay it forward&rdquo; to thank my friends for their time walking me\nthrough huge chunks of this, and those who publish their work, I&rsquo;m now spending\nsome time documenting how I was able to implement this protocol. I would never\nhave gotten as far as I did without the incredible patience and kindness of\nfriends spending time working with me, and educators publishing their hard work\nfor the world to learn from. Please accept my deepest thanks and appreciation.<\/p>\n<p>The PACKRAT posts are written from the perspective of a novice radio engineer,\nbut experienced software engineer. <span class=\"hz-highlight\">I&rsquo;ll be leaving\nout a lot of the technical details on the software end and specific software\nimplementation, focusing on the general gist of the implementation in the radio\ncritical components exclusively. The idea here is this is intended to be a\nframework &ndash; a jumping off point &ndash; for those who are interested in doing this\nthemselves.<\/span> I hope that this series of blog posts will come to be useful\nto those who embark on this incredibly rewarding journey after me.<\/p>\n<p>This is the first post in the series, and it will contain links to all the\nposts to follow. This is going to be the landing page I link others to &ndash; as I\npublish additional posts, I&rsquo;ll be updating the links on this page. The posts\nwill also grow a tag, which you can check back on, or follow along with\n<a href=\"https:\/\/k3xec.com\/tags\/packrat\/\">here<\/a>.<\/p>\n<h4 id=\"tau\">Tau<\/h4>\n<p>Tau (\ud835\udf0f) is a <a href=\"https:\/\/tauday.com\/tau-manifesto\">much more natural expression of the mathematical constant used\nfor circles<\/a> which I use rather than Pi (\u03c0).\nYou may see me use Tau in code or text &ndash; Tau is the same as 2\u03c0, so if you\nsee a Tau and don&rsquo;t know what to do, feel free to mentally or textually\nreplace it with 2\u03c0. I just hate always writing 2\u03c0 everywhere &ndash; and only\nusing \u03c0 (or worse yet &ndash; 2\u03c0\/2) .when I mean 1\/2 of a circle (or, \ud835\udf0f\/2).<\/p>\n<h4 id=\"pseudo-code\">Pseudo-code<\/h4>\n<p>Basically none of the code contained in this series is valid on its own. It&rsquo;s\nvery lightly basically Go, and only meant to express concepts in term of\nsoftware. The examples in the post shouldn&rsquo;t be taken on their own as working\nsnippits to process IQ data, but rather, be used to guide implementations\nto process the data in question. I&rsquo;d love to invite all readers to try to\n&ldquo;play at home&rdquo; with the examples, and try and work through the example data\ncaptures!<\/p>\n<h4 id=\"captures\">Captures<\/h4>\n<p>Speaking of captures, I&rsquo;ve included live on-the-air captures of PACKRAT\npackets, as transmitted from my implementation, in different parts of these\nposts. This means you can go through the process of building code to parse and\nreceive PACKRAT packets, and then build a transmitter that is validated by your\nreceiver. It&rsquo;s my hope folks will follow along at home and experiment with\nsoftware to process RF data on their own!<\/p>\n<h4 id=\"posts-in-this-series\">Posts in this series<\/h4>\n<ul>\n<li>Part 1: <a href=\"https:\/\/k3xec.com\/packrat-processing-iq\/\">Processing IQ data<\/a><\/li>\n<li>Part 2: <a href=\"https:\/\/k3xec.com\/packrat-transmitting\/\">Transmitting BPSK symbols<\/a><\/li>\n<li>Part 3: <a href=\"https:\/\/k3xec.com\/packrat-receiving\/\">Receiving BPSK symbols<\/a><\/li>\n<li>Part 4: <a href=\"https:\/\/k3xec.com\/packrat-framing\/\">Framing data<\/a><\/li>\n<li>Part 5: <a href=\"https:\/\/k3xec.com\/packrat-proxy\/\">Proxying Ethernet Frames to PACKRAT<\/a><\/li>\n<\/ul>"},{"title":"Measuring the Power Output of my SDRs \u26a1","link":"https:\/\/k3xec.com\/power-output\/","pubDate":"Mon, 15 Nov 2021 22:06:00 -0500","guid":"https:\/\/k3xec.com\/power-output\/","description":"<p>Over the last few years, I\u2019ve often wondered what the true power output of my\nSDRs are. It\u2019s a question with a shocking amount of complexity in the\nresponse, due to a number of factors (mostly Frequency). The ranges given in\nspec sheets are often extremely vague, and if I\u2019m being honest with myself,\nnot incredibly helpful for being able to determine what specific filters and\namplifiers I\u2019ll need to get a clean signal transmitted.<\/p>\n<div class=\"hz-alert-warning\">\n<b>Hey, heads up!<\/b> - This post contains extremely unvalidated and back of\nthe napkin quality work to understand how my equipment works. Hopefully this\nwork can be of help to others, but please double check any information you need\nfor your own work!\n<\/div>\n<p>I was specifically interested in what gain output (in dBm) looks like across\nthe frequency range &ndash; in particular, how variable the output dBm is when I\nchange frequencies. The second question I had was understanding how linear the\noutput gain is when adjusting the requested gain from the radio. Does a 2 dB\nincrease on a HackRF API mean 2 dB of gain in dBm, no matter what the absolute\nvalue of the gain stage is?<\/p>\n<p>I\u2019ve finally bit the bullet and undertaken work to characterize the hardware I\ndo have, with some outdated laboratory equipment I found on eBay. Of course, if\nit\u2019s worth doing, it\u2019s worth overdoing, so I spent a bit of time automating\na handful of components in order to collect the data that I need from my\nSDRs.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/HP437B.png\" alt=\"\"><\/p>\n<p>I bought an HP 437B, which is the cutting edge of 30 years ago, but still\naccurate to within 0.01dBm. I paired this Power Meter with an Agilent 8481A\nPower Sensor (-30 dBm to 20 dBm from 10MHz to 18GHz). For some of my radios, I\nwas worried about exceeding the 20 dBm mark, so I used a 20db attenuator while\nI waited for a higher power power sensor. Finally, I was able to find a GPIB to\nUSB interface, and get that interface working with the GPIB Kernel driver on my\nsystem.<\/p>\n<p>With all that out of the way, I was able to write <a href=\"https:\/\/hz.tools\/gpib\/\">Go bindings to my HP\n437B<\/a> to allow for totally headless and automated\ncontrol in sync with my SDR\u2019s RF output. This allowed me to script the\ntransmission of a sine wave at a controlled amplitude across a defined gain\nrange and frequency range and read the Power Sensor\u2019s measured dBm output to\ncharacterize the Gain across frequency and configured Gain.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/test-setup.png\" alt=\"\"><\/p>\n<h1 id=\"hackrf\">HackRF<\/h1>\n<p>Looking at configured Gain against output power, the requested gain appears to\nhave a fairly linear relation to the output signal power. The measured dBm\nranged between the sensor noise floor to approx +13dBm. The average standard\ndeviation of all tested gain values over the frequency range swept was +\/-2dBm,\nwith a minimum standard deviation of +\/-0.8dBm, and a maximum of +\/-3dBm.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/hackrf-gain-gain.png\" alt=\"\"><\/p>\n<p>When looking at output power over the frequency range swept, the HackRF\ncontains a distinctive (and frankly jarring) ripple across the Frequency range,\nwith a clearly visible jump in gain somewhere around 2.1GHz. I have no idea\nwhat is causing this massive jump in output gain, nor what is causing these\ndistinctive ripples. I\u2019d love to know more if anyone\u2019s familiar with HackRF&rsquo;s\nRF internals!<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/hackrf-freq-gain.png\" alt=\"\"><\/p>\n<h1 id=\"plutosdr\">PlutoSDR<\/h1>\n<p>The power output is very linear when operating above -20dB TX channel\ngain, but can get quite erratic the lower the output power is configured. The\nPlutoSDR\u2019s output power is directly related to the configured power level,\nand is generally predictable once a minimum power level is reached. The\nmeasured dBm ranged from the noise floor to 3.39 dBm, with an average standard\ndeviation of +\/-1.98 dBm, a minimum standard deviation of +\/-0.91 dBm and a\nmaximum standard deviation of +\/-3.37 dBm.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/pluto-gain-gain.png\" alt=\"\"><\/p>\n<p>Generally, the power output is quite stable, and looks to have very even and\nwideband gain control. There\u2019s a few artifacts, which I have not confidently\nisolated to the SDR TX gain, noise (transmit artifacts such as intermodulation)\nor to my test setup. They appear fairly narrowband, so I\u2019m not overly worried\nabout them yet. If anyone has any ideas what this could be, I\u2019d very much\nappreciate understanding why they exist!<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/pluto-freq-gain.png\" alt=\"\"><\/p>\n<h1 id=\"ettus-b210\">Ettus B210<\/h1>\n<p>The power output on the Ettus B210 is higher (in dBm) than any of my other\nradios, but it has a very odd quirk where the power becomes nonlinear\nsomewhere around -55dB TX channel gain. After that point, adding gain has no\neffect on the measured signal output in dBm up to 0 dB gain. The measured dBm\nranged from the noise floor to 18.31 dBm, with an average standard deviation of\n+\/-2.60 dBm, a minimum of +\/-1.39 dBm and a maximum of +\/-5.82 dBm.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/ettus-gain-gain.png\" alt=\"\"><\/p>\n<p>When the Gain is somewhere around the noise floor, the measured gain is\nincredibly erratic, which throws the maximum standard deviation significantly.\nI haven\u2019t isolated that to my test setup or the radio itself. I\u2019m inclined to\nbelieve it\u2019s my test setup. The radio has a fairly even and wideband gain, and\nso long as you\u2019re operating between -70dB to -55dB, fairly linear as well.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/ettus-freq-gain.png\" alt=\"\"><\/p>\n<h1 id=\"summary\">Summary<\/h1>\n<p>Of all my radios, the Ettus B210 has the highest output (in dBm) over the\nwidest frequency range, but the HackRF is a close second, especially after the\ngain bump kicks in around 2.1GHz. The Pluto SDR feels the most predictable and\nconsistent, but also a very low output, comparatively - right around 0 dBm.<\/p>\n<table>\n<thead>\n<tr>\n<th>Name<\/th>\n<th>Max dBm<\/th>\n<th>stdev dBm<\/th>\n<th>stdev min dBm<\/th>\n<th>stdev max dBm<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>HackRF<\/td>\n<td>+12.6<\/td>\n<td>+\/-2.0<\/td>\n<td>+\/-0.8<\/td>\n<td>+\/-3.0<\/td>\n<\/tr>\n<tr>\n<td>PlutoSDR<\/td>\n<td>+3.3<\/td>\n<td>+\/-2.0<\/td>\n<td>+\/-0.9<\/td>\n<td>+\/-3.7<\/td>\n<\/tr>\n<tr>\n<td>B210<\/td>\n<td>+18.3<\/td>\n<td>+\/-2.6<\/td>\n<td>+\/-1.4<\/td>\n<td>+\/-6.0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/power-output\/all-freq-gain.png\" alt=\"\"><\/p>"},{"title":"Reverse Engineering my Christmas Tree \ud83c\udf84","link":"https:\/\/k3xec.com\/christmas\/","pubDate":"Sat, 26 Dec 2020 11:24:00 -0500","guid":"https:\/\/k3xec.com\/christmas\/","description":"<p>Over the course of the last year and a half, I&rsquo;ve been doing some self-directed\nlearning on how radios work. I&rsquo;ve gone from a very basic understanding of\nwireless communications (there&rsquo;s usually some sort of antenna, I guess?) all\nthe way through the process of learning about and implementing a set of\nlibraries to modulate and demodulate data using my now formidable stash of SDRs.\nI&rsquo;ve been implementing all of the RF processing code from first principals and\npurely based on other primitives I&rsquo;ve written myself to prove to myself that I\nunderstand each concept before moving on.<\/p>\n<p>I figured that there was a fun &ldquo;capstone&rdquo; to be done here - the blind reverse\nengineering and implementation of the protocol my cheep Amazon power switch\nuses to turn on and off my Christmas Tree. All the work described in this post\nwas done over the course of a few hours thanks to help during the demodulation\nfrom <a href=\"https:\/\/github.com\/tomberek\/\">Tom Bereknyei<\/a> and\n<a href=\"https:\/\/blog.setec.io\/\">hlieberman<\/a>.<\/p>\n<h1 id=\"going-in-blind\">Going in blind<\/h1>\n<p>When I first got my switch, I checked it for any FCC markings in order to look\nup the FCC filings to determine the operational frequency of the device, and\nmaybe some other information such as declared modulation or maybe even part\nnumbers and\/or diagrams. However, beyond a few regulatory stickers, there were\nno FCC ids or other distinguishing IDs on the device. Worse yet, it appeared to\nbe a whitelabeled version of another product, so searching Google for the\nproduct name was very unhelpful.<\/p>\n<p>Since operation of this device is unlicensed, I figured I&rsquo;d start looking in\nthe ISM band. The most common band used that I&rsquo;ve seen is the band starting\nat <code>433.05MHz<\/code> up to <code>434.79MHz<\/code>. I fired up my trusty waterfall tuned to a\ncenter frequency of <code>433.92MHz<\/code> (since it&rsquo;s right in the middle of the band, and\nit let me see far enough up and down the band to spot the remote) and pressed\na few buttons. Imagine my surprise when I realize the operational frequency of\nthis device is <code>433.920MHz<\/code>, exactly dead center. Weird, but lucky!<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/waterfall.gif\" alt=\"\"><\/p>\n<p>After taking a capture, I started to look at understanding what the modulation\ntype of the signal was, and how I may go about demodulating it.\nUsing <a href=\"https:\/\/github.com\/miek\/inspectrum\">inspectrum<\/a>, I was able to clearly\nsee the signal in the capture, and it immediately stuck out to my eye to be\nencoded using OOK \/ ASK.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/ook.png\" alt=\"\"><\/p>\n<p>Next, I started to measure the smallest pulse, and see if I could infer the\nsymbols per second, and try to decode it by hand. These types of signals are\ngenerally pretty easy to decode by eye.<\/p>\n<p><a href=\"https:\/\/k3xec.com\/imgs\/christmas\/inspectrum-christmas-lights-4-bit.png\"><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/inspectrum-christmas-lights-4-strip.png\" alt=\"\"><\/a><\/p>\n<p>This wound up giving me symbol rate of 2.2 Ksym\/s, which is a lot faster than I\nexpected. While I was working by hand, <a href=\"https:\/\/github.com\/tomberek\/\">Tom<\/a>\ndemodulated a few messages in Python, and noticed that if you grouped the bits\ninto groups of 4, you either had a <code>1000<\/code> or a <code>1110<\/code> &ndash; which caused me to\nrealize this was encoded using something I saw documented elsewhere, where the\n0 is a &ldquo;short&rdquo; pulse, and a 1 is a &ldquo;long&rdquo; pulse, not unlike morse code, but\nwhere each symbol takes up a fixed length of time (monospace morse code?).\nWorking on that assumption, I changed my inspectrum symbol width, and\ndemodulated a few more by hand. This wound up demodulating nicely (and the\npreamble \/ clock sync could be represented as repeating <code>0<\/code>s, which is handy!)\nand gave us a symbol rate of 612(ish) symbols per second &ndash; a lot closer to\nwhat I was expecting.<\/p>\n<p><a href=\"https:\/\/k3xec.com\/imgs\/christmas\/inspectrum-christmas-lights.png\"><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/inspectrum-christmas-lights-strip.png\" alt=\"\"><\/a><\/p>\n<p>If we take the code for &lsquo;on&rsquo; in the inspectrum capture above and demodulate\nit by hand, we get <code>0000000000110101100100010<\/code> (treat a short pulse as a 0, and\na long pulse as a 1). If you&rsquo;re interested in following along at home, click on\nthe inspectrum image, and write down the bits you see, and compare it to what\nI have!<\/p>\n<p>Right, so it looks like from what we can tell so far that the packet looks\nsomething like this:<\/p>\n<div class=\"hz-abi\">\n<div type=\"10 bits\" class=\"hz-abi-green hz-abi-4b\">preamble \/ sync<\/div>\n<div type=\"15-16 bits\" class=\"hz-abi-green hz-abi-8b\">stuff<\/div>\n<\/div>\n<p>Next, I took a capture of all the button presses and demodulated them by hand,\nand put them into a table to try and understand the format of the messages:<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-3rd\">Button<\/td>\n<td class=\"hz-2-3rd\">Demod'd Bits<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>On<\/td>\n<td>0000000000110101100100010<\/td>\n<\/tr>\n<tr>\n<td>Off<\/td>\n<td>00000000001101011001010000<\/td>\n<\/tr>\n<tr>\n<td>Dim Up<\/td>\n<td>0000000000110101100110100<\/td>\n<\/tr>\n<tr>\n<td>Dim Down<\/td>\n<td>0000000000110101100100100<\/td>\n<\/tr>\n<tr>\n<td>Timer 1h<\/td>\n<td>0000000000110101100110010<\/td>\n<\/tr>\n<tr>\n<td>Timer 2h<\/td>\n<td>0000000000110101100100110<\/td>\n<\/tr>\n<tr>\n<td>Timer 4h<\/td>\n<td>0000000000110101100100000<\/td>\n<\/tr>\n<tr>\n<td>Dim 100%<\/td>\n<td>0000000000110101000101010<\/td>\n<\/tr>\n<tr>\n<td>Dim 75%<\/td>\n<td>00000000001101010001001100<\/td>\n<\/tr>\n<tr>\n<td>Dim 50%<\/td>\n<td>00000000001101010001001000<\/td>\n<\/tr>\n<tr>\n<td>Dim 25%<\/td>\n<td>0000000000110101000100000<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Great! So, this is enough to attempt to control the tree with, I think &ndash; so I\nwrote a simple modulator. My approach was to use the fact that I can break down\na single symbol into 4 &ldquo;sub-symbol&rdquo; components &ndash; which is to say, go back to\nrepresenting a <code>1<\/code> as <code>1110<\/code>, and a <code>0<\/code> as <code>1000<\/code>. This let me allocate IQ\nspace for the symbol, break the bit into 4 symbols, and if that symbol is 1,\nwrite out values from a carrier wave (<code>cos<\/code> in the <code>real<\/code> values, and <code>sin<\/code> in\nthe <code>imaginary<\/code> values) to the buffer. Now that I can go from bits to IQ data,\nI can transmit that IQ data using my PlutoSDR or HackRF and try and control my\ntree. I gave it a try, and the tree blinked off!<\/p>\n<p>\ud83c\udf89\ud83c\udf8a Success! \ud83c\udf8a\ud83c\udf89<\/p>\n<p>But wait &ndash; that&rsquo;s not enough for me &ndash; I know I can&rsquo;t just demodulate bits and\ntry and replay the bits forever &ndash; there&rsquo;s stuff like addresses and keys and\nstuff, and I want to get a second one of these working. Let&rsquo;s take a look at\nthe bits to see if we spot anything fun &amp; interesting.<\/p>\n<p>At first glance, a few things jumped out at me as being&hellip; weird? First is\nthat the preamble is 10 bits long (fine, let&rsquo;s move along - maybe it\njust needs 8 in a row and there&rsquo;s two to ensure clocks sync?). Next is that\nthe messages are not all the same length. I double (and triple!) checked\nthe messages, and it&rsquo;s true, the messages are not all the same length. Adding\nan extra bit at the end didn&rsquo;t break anything, but I wonder if that&rsquo;s just due\nto the implementation rather than the protocol.<\/p>\n<p>But, good news, it looks like we have a stable prefix to the messages from the\nremote &ndash; must be my device&rsquo;s address! The stable 6 bits that jump out right\naway are <code>110101<\/code>. Something seems weird, though, 6 bits is a bit awkward, even\nfor a bit limited embedded device. Why 6? But hey, wait, we had 10 bits in the\npreamble, what if we have an 8 bit address &ndash; meaning my device is <code>00110101<\/code>,\nand the preamble is 8 <code>0<\/code> symbols! Those are numbers that someone working on\nan 8 bit aligned platform would pick! To test this, I added a <code>0<\/code> to the\npreamble to see if the message starts at the first <code>1<\/code>, or if it requires all\nthe bits to be fully decoded, and lo and behold, the tree did not turn on or\noff. This would seem to me to confirm that the 0s are part of the address,\nand I can assume we have two 8 bit aligned bytes in the prefix of the message.<\/p>\n<div class=\"hz-abi\">\n<div type=\"byte\" class=\"hz-abi-green hz-abi-4b\">preamble \/ sync<\/div>\n<div type=\"byte\" class=\"hz-abi-green hz-abi-4b\">address<\/div>\n<div type=\"9-10 bits\" class=\"hz-abi-green hz-abi-4b\">stuff<\/div>\n<\/div>\n<p>Now, when we go through the 9-10 bits of &ldquo;stuff&rdquo;, we see all sorts of weird\nbits floating all over the place. The first 4 bits look like it&rsquo;s either\n<code>1001<\/code> or <code>0001<\/code>, but other than that, there&rsquo;s a lot of chaos. This is where\nthings get really squishy. I needed more information to try and figure this out,\nbut no matter how many times I sent a command it was always the same bits (so,\nno counters), and things feel very opaque still.<\/p>\n<p>The only way I was going to make any progress is to get another switch and see\nhow the messages from the remote change. Off to Amazon I went, and ordered\nanother switch from the same page, and eagerly waited its arrival.<\/p>\n<h2 id=\"switch-2\">Switch #2<\/h2>\n<p>The second switch showed up, and I hurriedly unboxed the kit, put batteries\ninto the remote, and fired up my SDR to take a capture. After I captured the\nfirst button (&ldquo;Off&rdquo;), my heart sunk as I saw my lights connected to\nSwitch #1 flicker off. Apparently the new switch and the old switch have the\nsame exact address. To be sure, I demodulated the messages as before, and\ncame out with the exact same bit pattern. This is a setback and letdown &ndash; I\nwas hoping to independently control my switches, but it also means I got no\nadditional information about the address or button format.<\/p>\n<p>The upside to all of this, though, is that because the switches are controlled\nby either remote, I only needed one remote, so why not pull it apart and see if\nI can figure out what components it&rsquo;s using to transmit, and find any\ndatasheets I can. The PCB was super simple, and I wound up finding a &ldquo;WL116SC&rdquo;\nIC on the PCB.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/remote-teardown.png\" alt=\"\"><\/p>\n<p>After some googling, I found a single lone\n<a href=\"https:\/\/web.archive.org\/web\/20201023121143\/https:\/\/datasheet.lcsc.com\/szlcsc\/2007311538_WL-WL116AC-1527_C708721.pdf\">datasheet<\/a>,\nentirely in Chinese. Thankfully, Google Translate seems to have worked well\nenough on technical words, and I was able to put together at least a little bit\nof understanding based on the documentation that was made available. I took a\nfew screenshots below - I put the google translated text above the hanzi. From\nthat sheet, we can see we got the basics of the &ldquo;1&rdquo; and &ldquo;0&rdquo; symbol encoding\nright (I was halfway expecting the bits to be flipped), and a huge find by way\nof a description of the bits in the message!<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/wl116sc-pulse-width.png\" alt=\"\"><\/p>\n<p>It&rsquo;s a bummer that we missed the clock sync \/ preamble pulse before the data\nmessage, but that&rsquo;s OK somehow. It also turns out that 8 or 10 bit series of of\n&ldquo;0&quot;s wasn&rsquo;t clock sync at all - it was part of the address! Since it also turns\nout that all devices made by this manufacturer have the hardcoded address of\n<code>[]byte{0x00, 0x35}<\/code>, that means that the vast majority of bits sent are always\ngoing to be the same for any button press on any remote made by this vendor.\nSeems like a waste of bits to me, but hey, what do I know.<\/p>\n<p>Additionally, this also tells us the trailing zeros are not part of the data\nencoding scheme, which is progress!<\/p>\n<div class=\"hz-abi\">\n<div type=\"uint16\" class=\"hz-abi-green hz-abi-8b\">address<\/div>\n<div type=\"byte\" class=\"hz-abi-green hz-abi-4b\">keycode<\/div>\n<\/div>\n<p>Now, working on the assumptions validated by the datasheet, here&rsquo;s the updated\nlist of scancodes we&rsquo;ve found:<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-3rd\">Button<\/td>\n<td class=\"hz-1-3rd\">Scancode Bits<\/td>\n<td class=\"hz-1-3rd\">Integer<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>On<\/td>\n<td>10010001<\/td>\n<td>145 \/ 0x91<\/td>\n<\/tr>\n<tr>\n<td>Off<\/td>\n<td>10010100<\/td>\n<td>148 \/ 0x94<\/td>\n<\/tr>\n<tr>\n<td>Dim Up<\/td>\n<td>10011010<\/td>\n<td>154 \/ 0x9A<\/td>\n<\/tr>\n<tr>\n<td>Dim Down<\/td>\n<td>10010010<\/td>\n<td>146 \/ 0x92<\/td>\n<\/tr>\n<tr>\n<td>Timer 1h<\/td>\n<td>10011001<\/td>\n<td>154 \/ 0x99<\/td>\n<\/tr>\n<tr>\n<td>Timer 2h<\/td>\n<td>10010011<\/td>\n<td>147 \/ 0x93<\/td>\n<\/tr>\n<tr>\n<td>Timer 4h<\/td>\n<td>10010000<\/td>\n<td>144 \/ 0x90<\/td>\n<\/tr>\n<tr>\n<td>Dim 100%<\/td>\n<td>00010101<\/td>\n<td>21 \/ 0x15<\/td>\n<\/tr>\n<tr>\n<td>Dim 75%<\/td>\n<td>00010011<\/td>\n<td>19 \/ 0x13<\/td>\n<\/tr>\n<tr>\n<td>Dim 50%<\/td>\n<td>00010010<\/td>\n<td>18 \/ 0x12<\/td>\n<\/tr>\n<tr>\n<td>Dim 25%<\/td>\n<td>00010000<\/td>\n<td>16 \/ 0x10<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Interestingly, I think the &ldquo;Dim&rdquo; keys may have a confirmation that we have\na good demod &ndash; the codes on the bottom are missing the most significant\nbit, and when I look back at the scancode table in the datasheet, they make an\ninteresting pattern &ndash; the bottom two rows, right and left side values match\nup! If you take a look, Dim 100% is &ldquo;S1&rdquo;, Dim 75% is &ldquo;S19&rdquo;, Dim 50% is &ldquo;S8&rdquo;,\nand Dim 25% is &ldquo;S20&rdquo;. Cool!<\/p>\n<p>Since none of the other codes line up, I am willing to bet the most significant\nbit is a &ldquo;Combo&rdquo; indicator, and not part of the button (leaving 7 bits for the\nkeycode).<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/wl116sc-scancode.png\" alt=\"\"><\/p>\n<p>And even more interestingly, one of our scancodes (&ldquo;Off&rdquo;, which is 0x94) shows up just\nbelow this table, in the examples.<\/p>\n<p><img src=\"https:\/\/k3xec.com\/imgs\/christmas\/wl116sc-math.png\" alt=\"\"><\/p>\n<p>Over all, I think this tells us we have the right bits to look at for\ndetermining the scan code! Great news there!<\/p>\n<h2 id=\"back-to-the-modulation\">Back to the modulation!<\/h2>\n<p>So, armed with this knowledge, I was able to refactor my code to match the\ntimings and understanding outlined by the datasheet and ensure things still work.\nThe switch itself has a high degree of tolerance, so being wildly off frequency\nor a wildly wrong symbol rate may actually still work. It&rsquo;s hard to know if\nthis is more or less correct, but matching documentation seems like a more stable\nfoundation if nothing else.<\/p>\n<p>This code has been really reliable, and tends to work just as well as the\nremote from what I&rsquo;ve been able to determine. I&rsquo;ve been using incredibly low\npower to avoid any interference, and it&rsquo;s been very robust - a testament to the\nengineering that went into the outlet hardware, even though it cost less than\nof a lot of other switches! I have a lot of respect for the folks who built\nthis device - it&rsquo;s incredibly simple, reliable and my guess is this thing will\nkeep working even in some fairly harsh RF environments.<\/p>\n<p>The only downside is the fact the manufacturer used the same address for all\ntheir devices, rather than programming a unique address for each outlet and\nremote when the underlying WL116SC chip supports it. I&rsquo;m sure this was done to\navoid complexity in assembly (e.g. pairing the remote and outlet, and having to\nkeep those two items together during assembly), but it&rsquo;s still a bummer. I took\napart the switch to see if I could dump an EEPROM and change the address in\nROM, but the entire thing was potted in waterproof epoxy, which is a very nice\nfeature if this was ever used outdoors. Not good news for tinkering, though!<\/p>\n<h2 id=\"unsolved-mysteries\">Unsolved Mysteries<\/h2>\n<p>At this point, even though I understand the protocol enough to control the\ndevice, it still feels like I hit a dead end in my understanding. I&rsquo;m not able\nto figure out how exactly the scancodes are implemented, and break them down\ninto more specific parts. They are stable and based on the physical wiring of\nthe remote, so I think I&rsquo;m going to leave it a magic number. I have what I was\nlooking for, and these magic constants appear to be the right one to use, even\nif I did understand how to create the codes itself.<\/p>\n<p>This does leave us with a few bits we never resolved, which I&rsquo;ll memorialize\nbelow just to be sure I don&rsquo;t forget about them.<\/p>\n<p><strong>Question #1:<\/strong> According to the datasheet there should be a preamble. Why do\nI not see one leading the first message?<\/p>\n<p>My hunch is that the trailing &ldquo;0&rdquo; at the end of the payload is actually just\nthe preamble for the next message (always rendering the first message\ninvalid?). This would let us claim there&rsquo;s an engineering reason why we are\nignoring the weird bit, and also explain away something from the documentation.\nIt&rsquo;s just weird that it wouldn&rsquo;t be present on the first message.<\/p>\n<p>This theory is mostly confirmed by measuring the timing and comparing it to the\ndatasheet, but it&rsquo;s not <em>exactly<\/em> in line with the datasheet timings either\n(specifically, it&rsquo;s off by 200\u00b5s, which is kinda a lot for a system using 400\u00b5s\ntimings). I think I could go either way on the last &ldquo;0&rdquo; being the preamble for\nthe next message. It could be that the first message is technically invalid, or\nit could also be that this was not implemented or actively disabled by the\nvendor for this specific application \/ device. It&rsquo;s really hard to know\nwithout getting the source code for the WL116SC chip in this specific remote\nor the source in the outlet itself.<\/p>\n<p><strong>Question #2:<\/strong> Why are some keycodes 8 bits and others 9 bits?<\/p>\n<p>I still have no idea why there sometimes 8 bits (for instance, &ldquo;On&rdquo;) and\nother times there are 9 bits (for instance, &ldquo;Off&rdquo;) in the 8 bit keycode\nfield.<\/p>\n<p>I spent some time playing with the &ldquo;trailing&rdquo; zeros, when I try and send an\n&ldquo;Off&rdquo; with the most significant 8 bits (without the least significant \/ last\n9th bit, which is a &ldquo;0&rdquo;), it does not turn the tree off. If I send an &ldquo;On&rdquo; with\n9 bits (an additional 0 after the least significant bit), it does work,\nbut both &ldquo;On&rdquo; and &ldquo;Off&rdquo; work when I send 10, 11 or 12 bits padded with trailing\nzeros. I suspect my outlet will ignore data after the switch is &ldquo;done&rdquo; reading\nbits regardless of trailing zeros. The docs tell me there should only be 8 bits,\nbut it won&rsquo;t work unless I send 9 bits for some commands. There&rsquo;s something\nfishy going on here, and the datasheet isn&rsquo;t exactly right either way.<\/p>\n<p><strong>Question #3:<\/strong> How in the heck do those scancodes work?<\/p>\n<p>This one drove me <em>nuts<\/em>. I&rsquo;ve spent countless hours on trying to figure this\nout, including emailing the company that makes the WL116SC (they&rsquo;re really\nnice!), and even though they were super kind and generous with documentation\nand example source, I&rsquo;m still having a hard time lining up their documentation\nand examples with what I see from my remote. I think the manufacturer of my\nremote and switch has modified the protocol enough to where there&rsquo;s actually\nsomething different going on here. Bummer.<\/p>\n<p>I wound up in my place of last resort &ndash; asking friends over Signal to try and\nsee if they could find a pattern, as well as making\nmultiple please to the twittersphere, to no avail (but thank you to\n<a href=\"https:\/\/phasor.dev\/\">Ben Hilburn<\/a>,\n<a href=\"https:\/\/twitter.com\/devnulling\">devnulling<\/a>,\n<a href=\"https:\/\/activelow.net\/\">Andreas Bombe<\/a> and <a href=\"https:\/\/twitter.com\/larme\">Larme<\/a>\nfor your repiles, help and advice!)<\/p>\n<p>I still don&rsquo;t understand how they assemble the scan code &ndash; for instance,\nif you merely add, you won&rsquo;t know if a key press of <code>0x05<\/code> is <code>0x03<\/code> + <code>0x02<\/code>\nor if it&rsquo;s <code>0x01<\/code> + <code>0x04<\/code>. On the other hand, treating it as two 4-bit\nintegers won&rsquo;t work for <code>0x10<\/code> to <code>0x15<\/code> (since they need 5 bits to\nrepresent). It&rsquo;s also likely the most significant bit is a combo indicator,\nwhich only leaves 7 bits for the actual keypress data. Stuffing 10 bits of data\ninto 7 bits is likely resulting in some really intricate bit work.\nOn a last ditch whim, I tried to XOR the math into working, but some initial\nbrute forcing to make the math work given the provided examples did not result\nin anything. It could be a bitpacked field that I don&rsquo;t understand, but I don&rsquo;t\nthink I can make progress on that without inside knowledge and much more work.<\/p>\n<p>Here&rsquo;s the table containing the numbers I was working off of:<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-3rd\">Keys<\/td>\n<td class=\"hz-1-3rd\">Key Codes<\/td>\n<td class=\"hz-1-3rd\">Scancode<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>S3 + S9<\/td>\n<td>0x01 + 0x03<\/td>\n<td>0x96<\/td>\n<\/tr>\n<tr>\n<td>S6 + S12<\/td>\n<td>0x07 + 0x09<\/td>\n<td>0x94<\/td>\n<\/tr>\n<tr>\n<td>S22 + S10<\/td>\n<td>0x0D + 0x0F<\/td>\n<td>0x3F<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>If anyone has thoughts on how these codes work, I&rsquo;d love to hear about it! Send\nme an email or a tweet or something - I&rsquo;m a bit stumped.<\/p>\n<p>There&rsquo;s some trick here that is being used to encode the combo key in a way\nthat is decodeable. If it&rsquo;s actually not decodeable (which is a real\npossibility!), this may act as a unique button combo &ldquo;hash&rdquo; which allows the\nreceiver to not actually determine which keys are pressed, but have a unique\n&ldquo;button&rdquo; that gets sent when a combo is used. I&rsquo;m not sure I know enough to\nhave a theory as to which it may be.<\/p>"},{"title":"Overview of the E4000 RTL SDR Tuner's IF stage \ud83c\udf9a\ufe0f","link":"https:\/\/k3xec.com\/e4k\/","pubDate":"Tue, 03 Nov 2020 09:47:48 -0500","guid":"https:\/\/k3xec.com\/e4k\/","description":"<p>This post is all about the E4000 (e4k) RTL-SDR Tuner, commonly found in the\nNooelec RTL-SDR. It&rsquo;s one of my favorite RTL-SDR tuners, but it can be\nincredibly frustrating to work with if it&rsquo;s not left on\n<abbr title=\"Automatic Gain Control\">AGC<\/abbr>.<\/p>\n<div class=\"hz-alert-warning\">\nThis documentation is a work in progress, and a result of\nsource code spelunking or reverse engineering. It may contain\nerrors or outright lies. The names may not match the original\nname, but it's been documented on a best-effort basis to help\nfuture engineering efforts.\n<\/div>\n<p>Specifically, this post covers a quirk of e4k rtlsdr dongles,\nthe addition of an IF gain stage.<\/p>\n<h2 id=\"what-is-the-if-stage\">What is the IF stage?<\/h2>\n<p>The IF (or intermediate frequency) stage is where the input signal has been\nshifted to a common frequency. This allows internal components to be designed\nto work at one frequency (such as a single oscillator that works on a specific\nfrequency), and have a single component that takes any frequency and shifts it\nto the common frequency before processing that signal. Transceivers that convert\nsignals to an IF for processing are sometimes called &ldquo;Superheterodyne&rdquo;\ntransceivers.<\/p>\n<h2 id=\"how-does-gain-work-on-the-e4k-rtl-sdr\">How does gain work on the e4k rtl-sdr?<\/h2>\n<p>e4k based rtl-sdr devices have two gain stages, the first is the tuner gain stage,\nwhich can be set using <code>rtlsdr_set_tuner_gain<\/code>. Valid gain values\nfor the connected\nrtl-sdr device can be queried via <code>rtlsdr_get_tuner_gains<\/code>.\nThe second gain stage is the IF gain, which performs amplification on the\nsignal after it&rsquo;s been converted to a single intermediate frequency.\nUnfortunately, setting and creating value IF gain configurations is not quite\nas easy as working with tuner gain. There&rsquo;s no way to get the supported\nif_tuner_gains, and confusingly, <code>rtlsdr_set_tuner_if_gain<\/code> takes an\nextra argument \u2014 stage!<\/p>\n<p>Without a bit of deeper knowledge about the e4k, It&rsquo;s not super clear what\nstage should be set to, nor what gain range or gain values are supported,\nand documentation on this is very lacking if you&rsquo;re searching for RTL-SDR\ndocumentation. Don&rsquo;t panic!<\/p>\n<p>Internally, the IF Gain on the e4k is made up of 6 stages, each of which can\nbe set to a specific set of values, but in practice (and as a user) you generally\nset all 6 gains together.<\/p>\n<p>Each stage has a set number of gain values that are supported (which differ\nper-stage), and can be a bit confusing to understand at first glace.<\/p>\n<table>\n<tbody>\n<tr>\n<td>Stage 1<\/td><td>-3dB<\/td><td>6db<\/td><td><\/td><td><\/td><td><\/td>\n<\/tr>\n<tr>\n<td>Stage 2<\/td><td>0dB<\/td><td>3dB<\/td><td>6dB<\/td><td>9dB<\/td><td><\/td>\n<\/tr>\n<tr>\n<td>Stage 3<\/td><td>0dB<\/td><td>3dB<\/td><td>6dB<\/td><td>9dB<\/td><td><\/td>\n<\/tr>\n<tr>\n<td>Stage 4<\/td><td>0dB<\/td><td>1db<\/td><td>2dB<\/td><td><\/td><td><\/td>\n<\/tr>\n<tr>\n<td>Stage 5<\/td><td>3dB<\/td><td>6db<\/td><td>9dB<\/td><td>12dB<\/td><td>15dB<\/td>\n<\/tr>\n<tr>\n<td>Stage 6<\/td><td>3dB<\/td><td>6db<\/td><td>9dB<\/td><td>12dB<\/td><td>15dB<\/td>\n<\/tr>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Each of these gain stages can be added together to determine the total\ngain of the IF stage. As an example, if we used the 0th values of the table\nabove, the final gain would be -3 + 0 + 0 + 0 + 3 + 3, giving us\na final gain of 3dB. You can set any valid values for each gain stage, and\nit should provide you the right output gain amount.<\/p>\n<p>However, the e4k datasheet provides two tables, one optimized for linearity,\nand one optimized for sensitivity. I&rsquo;ve converted those tables into Go for use in my\nSDR library. You can find the <a href=\"\">Sensitivity Table<\/a> and\n<a href=\"\">Linerarity Table<\/a> in the bottom of this post.<\/p>\n<p>The reason why those tables are different (and have different performance\nprofiles) is that although the total gain is simply additive as shown above,\ndoing large gains at the early stage results in a different signal than doing large\ngains at the final stages. Neither are wrong - but if you&rsquo;re doing something like\n<abbr title=\"Frequency Modulation\">FM<\/abbr>,\n<abbr title=\"Frequency Shift Keying\">FSK<\/abbr>, or\n<abbr title=\"Orthogonal frequency division multiplexing\">OFDM<\/abbr>,\nlinerarity matters a lot less than if you&rsquo;re processing\n<abbr title=\"amplitude modulation\">AM<\/abbr>.\nNeither is a more correct configuration, but there are trade-offs, so do make a\nconscious decision as to which table you use!<\/p>\n<h2 id=\"do-rlt-sdr-tuners-other-than-the-e4k-have-an-if-gain-control\">Do RLT-SDR tuners other than the e4k have an IF gain control?<\/h2>\n<p>Not that I&rsquo;m aware of! This scheme is only supported with the E4000 as far as\nI understand. Other SDRs (like the HackRF) do have IF gain controls, but that&rsquo;s\nout of scope for this post.<\/p>\n<h2 id=\"tables\">Tables<\/h2>\n<p>The following are two tables that have been transcribed from the elonics\ne4000 datasheet into Go. If you&rsquo;re using another language, you may need\nto translate these values to a different format.<\/p>\n<p>Each table stores values in the same way RTL-SDR Tuner gains are stored,\nwhich is in tenths of a DB. When you see -30, that&rsquo;s really -3dB. Be\ncareful when computing total gain!<\/p>\n<p>The gain values in the comment are a bit confusing (even to me now) - they&rsquo;re\ntaken from the E4000 documentation directly, but they don&rsquo;t align with the\nprovided stage values. Here, the -3 + 3 + 3 is computed as 6dB,\nwhich doesn&rsquo;t match up the table values. I&rsquo;ll update this post if I ever figure\nout why the -3 dB attenuation isn&rsquo;t being factored in. In my own code,\nI&rsquo;m using the values that I compute when adding values, in direct\ncontradiction of the datasheet. It&rsquo;s likely wrong, but it&rsquo;s not a large difference\nfor now.<\/p>\n<h3 id=\"sensitivity-table\">Sensitivity Table<\/h3><\/h3>\n<pre>\nsenIFGains = []Stages{\nStages{-30, 00, 00, 00, 30, 30}, \/\/ 6 dB gain\nStages{-30, 00, 00, 10, 30, 30}, \/\/ 7 dB gain\nStages{-30, 00, 00, 20, 30, 30}, \/\/ 8 dB gain\nStages{-30, 30, 00, 00, 30, 30}, \/\/ 9 dB gain\nStages{-30, 30, 00, 10, 30, 30}, \/\/ 10 dB gain\nStages{-30, 30, 00, 20, 30, 30}, \/\/ 11 dB gain\nStages{-30, 60, 00, 00, 30, 30}, \/\/ 12 dB gain\nStages{-30, 60, 00, 10, 30, 30}, \/\/ 13 dB gain\nStages{-30, 60, 00, 20, 30, 30}, \/\/ 14 dB gain\nStages{60, 00, 00, 00, 30, 30}, \/\/ 15 dB gain\nStages{60, 00, 00, 10, 30, 30}, \/\/ 16 dB gain\nStages{60, 00, 00, 20, 30, 30}, \/\/ 17 dB gain\nStages{60, 30, 00, 00, 30, 30}, \/\/ 18 dB gain\nStages{60, 30, 00, 10, 30, 30}, \/\/ 19 dB gain\nStages{60, 30, 00, 20, 30, 30}, \/\/ 20 dB gain\nStages{60, 60, 00, 00, 30, 30}, \/\/ 21 dB gain\nStages{60, 60, 00, 10, 30, 30}, \/\/ 22 dB gain\nStages{60, 60, 00, 20, 30, 30}, \/\/ 23 dB gain\nStages{60, 90, 00, 00, 30, 30}, \/\/ 24 dB gain\nStages{60, 90, 00, 10, 30, 30}, \/\/ 25 dB gain\nStages{60, 90, 00, 20, 30, 30}, \/\/ 26 dB gain\nStages{60, 90, 30, 00, 30, 30}, \/\/ 27 dB gain\nStages{60, 90, 30, 10, 30, 30}, \/\/ 28 dB gain\nStages{60, 90, 30, 20, 30, 30}, \/\/ 29 dB gain\nStages{60, 90, 60, 00, 30, 30}, \/\/ 30 dB gain\nStages{60, 90, 60, 10, 30, 30}, \/\/ 31 dB gain\nStages{60, 90, 60, 20, 30, 30}, \/\/ 32 dB gain\nStages{60, 90, 90, 00, 30, 30}, \/\/ 33 dB gain\nStages{60, 90, 90, 10, 30, 30}, \/\/ 34 dB gain\nStages{60, 90, 90, 20, 30, 30}, \/\/ 35 dB gain\nStages{60, 90, 90, 00, 60, 30}, \/\/ 36 dB gain\nStages{60, 90, 90, 10, 60, 30}, \/\/ 37 dB gain\nStages{60, 90, 90, 20, 60, 30}, \/\/ 38 dB gain\nStages{60, 90, 90, 00, 90, 30}, \/\/ 39 dB gain\nStages{60, 90, 90, 10, 90, 30}, \/\/ 40 dB gain\nStages{60, 90, 90, 20, 90, 30}, \/\/ 41 dB gain\nStages{60, 90, 90, 00, 120, 30}, \/\/ 42 dB gain\nStages{60, 90, 90, 10, 120, 30}, \/\/ 43 dB gain\nStages{60, 90, 90, 20, 120, 30}, \/\/ 44 dB gain\nStages{60, 90, 90, 00, 150, 30}, \/\/ 45 dB gain\nStages{60, 90, 90, 10, 150, 30}, \/\/ 46 dB gain\nStages{60, 90, 90, 20, 150, 30}, \/\/ 47 dB gain\nStages{60, 90, 90, 00, 150, 60}, \/\/ 48 dB gain\nStages{60, 90, 90, 10, 150, 60}, \/\/ 49 dB gain\nStages{60, 90, 90, 20, 150, 60}, \/\/ 50 dB gain\nStages{60, 90, 90, 00, 150, 90}, \/\/ 51 dB gain\nStages{60, 90, 90, 10, 150, 90}, \/\/ 52 dB gain\nStages{60, 90, 90, 20, 150, 90}, \/\/ 53 dB gain\nStages{60, 90, 90, 00, 150, 120}, \/\/ 54 dB gain\nStages{60, 90, 90, 10, 150, 120}, \/\/ 55 dB gain\nStages{60, 90, 90, 20, 150, 120}, \/\/ 56 dB gain\nStages{60, 90, 90, 00, 150, 150}, \/\/ 57 dB gain\nStages{60, 90, 90, 10, 150, 150}, \/\/ 58 dB gain\nStages{60, 90, 90, 20, 150, 150}, \/\/ 59 dB gain\nStages{60, 90, 90, 30, 150, 150}, \/\/ 60 dB gain\n}\n<\/pre>\n<h3 id=\"linerarity-table\">Linerarity Table<\/h3>\n<pre>\nlinIFGains = []Stages{\nStages{-30, 00, 00, 00, 30, 30}, \/\/ 6 dB gain\nStages{-30, 00, 00, 10, 30, 30}, \/\/ 7 dB gain\nStages{-30, 00, 00, 20, 30, 30}, \/\/ 8 dB gain\nStages{-30, 00, 00, 00, 30, 60}, \/\/ 9 dB gain\nStages{-30, 00, 00, 10, 30, 60}, \/\/ 10 dB gain\nStages{-30, 00, 00, 20, 30, 60}, \/\/ 11 dB gain\nStages{-30, 00, 00, 00, 30, 90}, \/\/ 12 dB gain\nStages{-30, 00, 00, 10, 30, 90}, \/\/ 13 dB gain\nStages{-30, 00, 00, 20, 30, 90}, \/\/ 14 dB gain\nStages{-30, 00, 00, 00, 30, 120}, \/\/ 15 dB gain\nStages{-30, 00, 00, 10, 30, 120}, \/\/ 16 dB gain\nStages{-30, 00, 00, 20, 30, 120}, \/\/ 17 dB gain\nStages{-30, 00, 00, 00, 30, 150}, \/\/ 18 dB gain\nStages{-30, 00, 00, 10, 30, 150}, \/\/ 19 dB gain\nStages{-30, 00, 00, 20, 30, 150}, \/\/ 20 dB gain\nStages{-30, 00, 00, 00, 60, 150}, \/\/ 21 dB gain\nStages{-30, 00, 00, 10, 60, 150}, \/\/ 22 dB gain\nStages{-30, 00, 00, 20, 60, 150}, \/\/ 23 dB gain\nStages{-30, 00, 00, 00, 90, 150}, \/\/ 24 dB gain\nStages{-30, 00, 00, 10, 90, 150}, \/\/ 25 dB gain\nStages{-30, 00, 00, 20, 90, 150}, \/\/ 26 dB gain\nStages{-30, 00, 00, 00, 120, 150}, \/\/ 27 dB gain\nStages{-30, 00, 00, 10, 120, 150}, \/\/ 28 dB gain\nStages{-30, 00, 00, 20, 120, 150}, \/\/ 29 dB gain\nStages{-30, 00, 00, 00, 150, 150}, \/\/ 30 dB gain\nStages{-30, 00, 00, 10, 150, 150}, \/\/ 31 dB gain\nStages{-30, 00, 00, 20, 150, 150}, \/\/ 32 dB gain\nStages{-30, 00, 30, 00, 150, 150}, \/\/ 33 dB gain\nStages{-30, 00, 30, 10, 150, 150}, \/\/ 34 dB gain\nStages{-30, 00, 30, 20, 150, 150}, \/\/ 35 dB gain\nStages{-30, 00, 60, 00, 150, 150}, \/\/ 36 dB gain\nStages{-30, 00, 60, 10, 150, 150}, \/\/ 37 dB gain\nStages{-30, 00, 60, 20, 150, 150}, \/\/ 38 dB gain\nStages{-30, 00, 90, 00, 150, 150}, \/\/ 39 dB gain\nStages{-30, 00, 90, 10, 150, 150}, \/\/ 40 dB gain\nStages{-30, 00, 90, 20, 150, 150}, \/\/ 41 dB gain\nStages{-30, 30, 90, 00, 150, 150}, \/\/ 42 dB gain\nStages{-30, 30, 90, 10, 150, 150}, \/\/ 43 dB gain\nStages{-30, 30, 90, 20, 150, 150}, \/\/ 44 dB gain\nStages{-30, 60, 90, 00, 150, 150}, \/\/ 45 dB gain\nStages{-30, 60, 90, 10, 150, 150}, \/\/ 46 dB gain\nStages{-30, 60, 90, 20, 150, 150}, \/\/ 47 dB gain\nStages{60, 00, 90, 00, 150, 150}, \/\/ 48 dB gain\nStages{60, 00, 90, 10, 150, 150}, \/\/ 49 dB gain\nStages{60, 00, 90, 20, 150, 150}, \/\/ 50 dB gain\nStages{60, 30, 90, 00, 150, 150}, \/\/ 51 dB gain\nStages{60, 30, 90, 10, 150, 150}, \/\/ 52 dB gain\nStages{60, 30, 90, 20, 150, 150}, \/\/ 53 dB gain\nStages{60, 60, 90, 00, 150, 150}, \/\/ 54 dB gain\nStages{60, 60, 90, 10, 150, 150}, \/\/ 55 dB gain\nStages{60, 60, 90, 20, 150, 150}, \/\/ 56 dB gain\nStages{60, 90, 90, 00, 150, 150}, \/\/ 57 dB gain\nStages{60, 90, 90, 10, 150, 150}, \/\/ 58 dB gain\nStages{60, 90, 90, 20, 150, 150}, \/\/ 59 dB gain\nStages{60, 90, 90, 30, 150, 150}, \/\/ 60 dB gain\n}\n<\/pre>"},{"title":"Overview of the RFCAP format \ud83d\udcf8","link":"https:\/\/k3xec.com\/rfcap\/","pubDate":"Tue, 03 Nov 2020 09:47:48 -0500","guid":"https:\/\/k3xec.com\/rfcap\/","description":"<p>rfcap is a file format with extremely small ambitions. rfcap files\ncontain a fixed size header, and then a stream of raw IQ data. The\nrfcap header contains information about the IQ format type, and\ncapture metadata. The header is aligned to a 128 bit boundary, so\nmost iq formats can choose to ignore the header and throw out the\nfirst window, meaning existing tools like <code>gqrx<\/code> can read a subset\nof rfcap files in the right IQ sample format.<\/p>\n<div class=\"hz-alert-ok\">\nThis documentation is of a stable file format. Changes to this\nspec will result in a non-breaking and careful change, or a\nmajor version change. This format is safe to rely on.\n<\/div>\n<p>The biggest advantage of the rfcap scheme is that IQ data can be\npiped around without additional sample information such as\nIQ format, or sample rate, and the metadata remains attached to\nthe stream.<\/p>\n<h2 id=\"reference-implementation\">Reference implementation<\/h2>\n<p>The <code>hz.tools\/rfcap<\/code> package contains the reference\nimplementation of the rfcap spec, and will be maintained as the\nspec evolves. Go programmers are strongly advised to use this\npackage as a dependency for reading and writing SDR captures\nin Go.<\/p>\n<h2 id=\"format-description\">Format Description<\/h2>\n<p>Each rfcap file is comprised of a single 48 byte header, followed\nby a stream of IQ data, as described by the header.<\/p>\n<div class=\"hz-abi\">\n<div type=\"Header (48 bytes)\" class=\"hz-abi-yellow hz-abi-4b\">\n<a href=\"\">Header<\/a>\n<\/div>\n<div type=\"[]IQ\" class=\"hz-abi-yellow hz-abi-Nb\">\nSamples\n<\/div>\n<\/div>\n<p>Implementations are advised to store the header, in case samples\nneed to be written to disk, or piped to another application\noutside of the current process.<\/p>\n<h3 id=\"header\">Header<\/h3>\n<p>The Header is split up into fields containing metadata describing\nthe format and rate of the IQ samples to follow. At minimum,\nimplementations must be able to understand the\n<code>SampleRate<\/code> field, the <code>SampleFormat<\/code>\nfield, and if not only using uint8, the <code>Endianness<\/code>\nfield.<\/p>\n<p>The header itself is <em>always<\/em> encoded\n<span class=\"hz-highlight\">little endian<\/span>. This may make things a little\nconfusing when operating over the network, since you may be expecting network\norder &ndash; but it does make it significantly easier to consume on little endian\nsystems (which make up most of the target platforms), and makes things a little\neasier when the encoded data is also little endian floating point numbers (which\nis also usually the case on little endian systems).<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Field Name<\/td>\n<td class=\"hz-1-6th\">Type<\/td>\n<td class=\"hz-2-3rd\">Description<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td class=\"hz-1-6th\">Magic<\/td>\n<td class=\"hz-1-6th\">[6]byte<\/td>\n<td class=\"hz-2-3rd\">Currently always `RFCAP1` for rfcap v1<\/td>\n<\/tr>\n<tr>\n<td class=\"hz-1-6th\">Capture Time<\/td>\n<td class=\"hz-1-6th\">int64<\/td>\n<td class=\"hz-2-3rd\">\nNumber of nanoseconds since the Unix Epoch. Divide by\n`1e+9` to get a Unix Epoch in seconds, and\nperform a modulus to get the nanoseconds.\n<\/td>\n<\/tr>\n<tr>\n<td class=\"hz-1-6th\">Center Frequency<\/td>\n<td class=\"hz-1-6th\">float64<\/td>\n<td class=\"hz-2-3rd\">\nCenter Frequency of the capture, in Hz, as a floating\npoint 64 bit number.\n<\/td>\n<\/tr>\n<tr>\n<td class=\"hz-1-6th\">Sample Rate<\/td>\n<td class=\"hz-1-6th\">uint32<\/td>\n<td class=\"hz-2-3rd\">\nNumber of IQ samples per second this capture was taken at.\nEach IQ sample is comprised of the real and imaginary\nsample.\n<\/td>\n<\/tr>\n<tr>\n<td class=\"hz-1-6th\">Sample Format<\/td>\n<td class=\"hz-1-6th\">uint8<\/td>\n<td class=\"hz-2-3rd\">\nSample Format defined by the\n`hz.tools\/sdr.SampleFormat` enum.<br \/>\n<br \/>\n<ul>\n<li>1: Complex64 (interleaved 32 bit floats)<\/li>\n<li>2: uint8 (interleaved 8 bit uints)<\/li>\n<li>3: int16 (interleaved 16 bit ints)<\/li>\n<li>4: int8 (interleaved 8 bit ints)<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td class=\"hz-1-6th\">Endianness<\/td>\n<td class=\"hz-1-6th\">uint8<\/td>\n<td class=\"hz-2-3rd\">\nIn order to retain compatibility with an earlier rfcap\nversion, Little Endian files are denoted with a 0, rather\nthan a 1.<br \/>\n<br \/>\n<ul>\n<li>0: Little Endian<\/li>\n<li>1: Big Endian<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td class=\"hz-1-6th\">Reserved<\/td>\n<td class=\"hz-1-6th\">[20]byte<\/td>\n<td class=\"hz-2-3rd\">\nCurrently unused. Implementations must not rely on\nany information in this range, and when writing headers,\nthe data in these bytes must be all zeros.\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>"},{"title":"Overview of the RTL TCP Protocol \ud83d\udd0a","link":"https:\/\/k3xec.com\/rtl-tcp\/","pubDate":"Tue, 03 Nov 2020 09:47:48 -0500","guid":"https:\/\/k3xec.com\/rtl-tcp\/","description":"<p>The rtl_tcp program will allow a client to remotely receive\niq samples from the rtl sdr over a tcp connection. This document\ndescribes the mechanism by which the client and server communicate.<\/p>\n<div class=\"hz-alert-warning\">\nThis documentation is a work in progress, and a result of\nsource code spelunking or reverse engineering. It may contain\nerrors or outright lies. The names may not match the original\nname, but it's been documented on a best-effort basis to help\nfuture engineering efforts.\n<\/div>\n<p>All data sent to and from the server are encoded in\n<span class=\"hz-highlight\">network byte order<\/span> (big endian). This\ndoesn&rsquo;t matter for the actual <i>iq<\/i> data, since it&rsquo;s uint8 real\/imag\ninterleaved, but it will matter for the header.<\/p>\n<h2 id=\"tcp-client-stream\">TCP Client Stream<\/h2>\n<p>On connection to the rtl_tcp socket, the server will begin to stream\nIQ samples to the client. The first 12 bytes are part of the\nDongleInfo struct, but since it&rsquo;s 2 byte aligned, clients can\nchoose to ignore the DongleInfo struct if they so desire.<\/p>\n<div class=\"hz-abi\">\n<div type=\"DongleInfo\" class=\"hz-abi-yellow hz-abi-4b\">\n<a href=\"#dongle-info-struct\">DongleInfo<\/a>\n<\/div>\n<div type=\"[][2]uint8\" class=\"hz-abi-yellow hz-abi-Nb\">\nIQ\n<\/div>\n<\/div>\n<p>Following the DongleInfo struct is a stream of infinite length\nof interleaved IQ data, in the same format that the rtl-sdr library\nwill return data to the caller \u2014 a stream of interleaved\nreal and imaginary uint8 values, where 128 is 0, 255 is 1, and 0\nis -1.<\/p>\n<h2 id=\"dongle-info-struct\">Dongle Info struct<\/h2>\n<p>The first bytes sent to the client contain information about the\ntuner at the remote end of the rtl_tcp connection. This allows\nthe client to determine ranges for gain values, or if there&rsquo;s\nan intermediate frequency gain stage.<\/p>\n<div class=\"hz-abi\">\n<div type=\"[4]byte\" class=\"hz-abi-blue-alt hz-abi-4b\">Magic (RTL0)<\/div>\n<div type=\"uint32\" class=\"hz-abi-blue-alt hz-abi-4b\">Tuner Type<\/div>\n<div type=\"uint32\" class=\"hz-abi-blue-alt hz-abi-4b\">Tuner Gain Type<\/div>\n<\/div>\n<p>These bytes are aligned to 2 byte boundaries (each value is a\n4 byte uint32), so consumers need not care about this header if\nit&rsquo;s not using any information about the dongle.<\/p>\n<h2 id=\"request-struct\">Request struct<\/h2>\n<p>The client may, during the course of the connection, send a\nRequest to the server in order to adjust the settings of the\nremote device. Commonly, this is used to retune the device,\nchange the sample rate, or adjust gain settings.<\/p>\n<div class=\"hz-abi\">\n<div type=\"byte\" class=\"hz-abi-green hz-abi-1b\">Cmd<\/div>\n<div type=\"uint32\" class=\"hz-abi-green hz-abi-4b\">Argument<\/div>\n<\/div>\n<p>A full list of Commands, and the semantics of their Argument\nis detailed on the table below.<\/p>\n<table>\n<thead>\n<tr>\n<td class=\"hz-1-6th\">Command<\/td>\n<td class=\"hz-1-6th\">Definition<\/td>\n<td class=\"hz-2-3rd\">Argument<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>0x01<\/td>\n<td>Tune to a new center frequency<\/td>\n<td>Frequency, in Hz as a uint32<\/td>\n<\/tr>\n<tr>\n<td>0x02<\/td>\n<td>Set the rate at which iq sample pairs are sent<\/td>\n<td>Number of samples (real and imaginary) per second as a uint32<\/td>\n<\/tr>\n<tr>\n<td>0x03<\/td>\n<td>Set the tuner gain mode<\/td>\n<td>\n<ul>\n<li>0: automatic gain control<\/li>\n<li>1: manual gain control<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>0x04<\/td>\n<td>Set the tuner gain level<\/td>\n<td>Gain, in tenths of a dB<\/td>\n<\/tr>\n<tr>\n<td>0x05<\/td>\n<td>Set the tuner frequency correction<\/td>\n<td>Frequency correction, in PPM (parts per million)<\/td>\n<\/tr>\n<tr>\n<td>0x06<\/td>\n<td>Set the IF gain level<\/td>\n<td>\n<div class=\"hz-abi\">\n<div type=\"u16\" class=\"hz-abi-yellow hz-abi-2b\">Stage<\/div>\n<div type=\"u16\" class=\"hz-abi-yellow hz-abi-2b\">Gain<\/div>\n<\/div>\nTwo uint16 values, the least significant int16 (network byte order)\nis the gain value in tenths of a dB, and the most significant\nint16 is the gain stage.\n<\/td>\n<\/tr>\n<tr>\n<td>0x07<\/td>\n<td>Put the tuner into test mode<\/td>\n<td>\n<ul>\n<li>1: enable test mode<\/li>\n<li>0: disable test mode<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>0x08<\/td>\n<td>\nSet the automatic gain <i>correction<\/i>, a software\nstep to correct the incoming signal, this is <b>not<\/b>\nautomatic gain control on the hardware chip, that is\ncontrolled by tuner gain mode.\n<\/td>\n<td>\n<ul>\n<li>1: enable gain correction<\/li>\n<li>0: disable gain correction<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>0x09<\/td>\n<td>Set direct sampling<\/td>\n<td>\n<ul>\n<li>0: disable direct sampling<\/li>\n<li>1: I-ADC input enabled<\/li>\n<li>2: Q-ADC input enabled<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>0x0a<\/td>\n<td>Set offset tuning (TODO: EXPLAIN)<\/td>\n<td>\n<ul>\n<li>1: enable offset tuning<\/li>\n<li>0: disable offset tuning<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>0x0d<\/td>\n<td>Set tuner gain by the tuner's gain index<\/td>\n<td>\nEach tuner has a discrete set of supported gain values,\nwhich are returned in a sorted order via\nrtlsdr_get_tuner_gains. The argument here is\ntreated as an index into the tuner gains for the specific\ntuner on the remote end, and will set the gain by index,\nrather than in tenths of a dB.\n<\/td>\n<\/tr>\n<tr>\n<td>0x0e<\/td>\n<td>Set Bias Tee on GPIO pin 0<\/td>\n<td>\n<ul>\n<li>1: set pin high<\/li>\n<li>0: set pin low<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>"}]}}