hylobatidae.org

More things. On Sunday. You get the idea.

Lana Sator, post-Soviet urban explorer - she of rocket factory infiltration fame, somehow her rust-laden blog wasn't in my RSS reader. A situation now rectified! via rjp

Tracking a radiosonde (weather balloon) - listening in on various telemetry systems using hacked USB digital television adapters is definitely a thing. I need to investigate. via

Siding Spring Observatory fires - Australian bushfires come close to burning down a major observatory. (Spoiler: the expensive bits mostly survived. Luckily!) via proogs

Parting Moon Shots from NASA's GRAIL mission - dual-spacecraft gravitometric lunar mission with tiny educational outreach cameras attached, shooting gloriously atmospheric timelapses from orbit, shortly before colliding with the moon itself.

Bonus Calamitous Chemistry corner:

• Things I Won't Work With - you don't need to know anything about chemistry to enjoy the wonderful descriptions... 'In every case, the solution detonated spontaneously on standing. And by "spontaneously”, they mean "while standing undisturbed in the dark", so there’s really just no way to deal with this stuff. It’s probably a good thing they didn’t get crystals, because someone would have tried to isolate the hideous things. The cesium salt actually did give a few crystals, which they managed to pluck from the top of the solution and get X-ray data on. A few hours later the remaining batch suddenly exploded, though, which certainly must have been food for thought.'

So it would appear after the Radio-4-Matic escapades earlier this week, I gained some more readers!

Hello.

To you, I shall explain the Sunday Things - essentially, every Sunday I post some Things. Which are interesting. To me, at least. To you? Let us find out.

Kitchen Atom Bomb - after my own appearance on PM, here's quite possibly the greatest interview in the history of everything - with Richard Handl, a Swedish man who built a nuclear reactor in his kitchen.

Заброшенный сахарный завод - prised from the bookmarks, explorations of some sort of abandoned post-Soviet chemical factory? I could run it through an automated translator, but that's cheating!

The Basement - somewhere in Portland there's an old building with an old basement, with a room that holds a hyper-modern secret.

Bonus Cold War Transmissions corner:

• Inside the Russian Short Wave Radio Enigma - the mysterious UVB-76, producing curious signals from a rusting radio mast north of Moscow.
• Russian Woodpecker - vast, buzzing over-the-horizon radar, inconveniently close to Chernobyl...
• The Conet Project - recordings of atmospherically distorted numbers stations, CD1 CD2 CD3 CD4.

So, it seems someone at Radio 4 spotted this thing, and one thing led to another and I ended up being interviewed on PM. Yes, me on national radio! On a proper news programme! Blinking heck.

Not sure how long it'll stay up, so listen now. There's a mention at the beginning, then my bit's at the end.

It got edited down a fair bit (a mention of the Raspberry Pi itself got lost, along with a mention of non-Pi alternatives) - but it most definitely brings across the heart of why I did it. A couple of observations:

• Talking to a Radio 4 presenter is really weird. Perfect diction.
• Not helped by Skype being wired into the studio microphones. Incredibly clear.
• It's like talking to a radio. And the radio's talking back. Fantastic!

I promised 'Friday' for this third, code-containing article, and according to my radio it's still Friday (Today in Parliament's playing) - so here we are.

Before I even had a chance to fully document everything, this little time-travelling Roberts radio got spotted - first by the official Raspberry Pi news blog, then MediaUK, then by somebody else... Usually I'm more notable in the computer games world!

Edit 2013-01-12: that 'somebody else'? Radio 4!

Edit 2013-01-13: ... and now Hackaday!

It seems Raspberry Pi radios are definitely a thing - I'm glad to join the club.

Polyfuses

Firstly, a cheeky bait-and-switch with some entirely-not-software-related Raspberry Pi surgery, removing the pesky polyfuses which were messing with my WiFi. This procedure is to rectify a hardware problem that has long since been fixed on the Raspberry Pi - it is only necessary if you have an early Pi, wish to eradicate any remaining warranty and generally understand the implications of removing the current-limiting components from the USB subsystem. If you have any questions, don't.

So here goes. Some surgical pliers (no idea as to the correct terminology) proved ideal for grasping each polyfuse lengthwise, leaving just enough space either end to poke the tip of a soldering iron against the end of the fuse. Apply a small amount of angular force through the pliers so that when the solder joint melts, the end of the fuse pops up slightly, away from the solder pad. Otherwise, detaching both solder joints simultaneously is implausible.

Once the polyfuses are gone, trim some bits of wire to cross the joins. I used 'C' shapes of vaguely thick single-core wire, with bits of insulation left in the middles. Clamping the wire by the side of the board in place with a crocodile-clip was really helpful - the inner one was much trickier. Quickly solder each contact in place, taking care to neither cook the board nor use too much solder - as mentioned earlier, if you have any concerns regarding this procedure, please don't do it.

But still. Both my Pis (named gasbuggy and rulison, don't ask) made a rapid recovery and are now stable with Edimax EW-7811Un WiFi adapters. Yay!

Software!

... And after that diversion, now to the heart of the Radio-4-Matic - the dodgy scripting that metaphorically glues it all together. Implementation isn't exactly wonderful (I'm not much of a programmer), but these notes should hopefully help anyone wishing to improve on the concept, or is thinking of something else entirely. I kind of suspect there are UNIX utilities that would make something like this far easier, but I learned a lot while building this thing.

Anyway.

The first bash-script prototype had get_iplayer (and thus rtmpdump) piping the live Radio 4 straight into a file, with mplayer being started on that file after a crude sleep 8h;. The FLV-wrapped AAC audio straight from rtmpdump proved an issue, with mplayer claiming an end-of-file after playing up until the size of the file as it originally saw it, even thought it had been much-extended since then. Attempts to encourage it to do otherwise were futile.

Time to pipe that audio through ffmpeg, and wrap it in something more acceptable to mplayer's demands. Yes! It worked! For a bit.

The capture side, get_iplayer and rtmpdump, had a tendency to cut out - presumably whenever a seagull over the Atlantic farted, or if a grizzly bear in the Midwest ran out of toilet paper. Something to do with the tubes being blocked? Only one way to improve this - stick the capture utilities in a loop, respawning them whenever they fell over. This would introduce gaps in the recorded audio, resulting in an ever-increasing discrepancy against 'actual' time - and rtmpdump demonstrated a new tantrum by occasionally getting stuck, but without exiting. Sigh.

Then, even when things were actually streaming correctly, ffmpeg would occasionally choke with a 'non monotonically increasing dts to muxer in stream' error. Which was a bug which was apparently long-fixed in ffmpeg. Wait a minute...

ffmpeg

It turns out that the 'ffmpeg' packaged by Debian (and thus Raspbian) is actually libav, a forked version of the original project. Which helpfully provided an ffmpeg binary for 'backwards compatibility'.

Only one thing to do - compile a version of proper, full-strength unforked ffmpeg. Free software enthusiasts are no doubt crying out in agony at the thought of me ignoring whatever technical or political issues led to that fork...

Someone's written a guide to compiling ffmpeg on a Raspberry Pi (ignore stuff before 'Build and Install FFMPEG') - the configuration I used was as follows:

ffmpeg version N-45983-ge12cfd0 Copyright (c) 2000-2012 the FFmpeg developers
built on Oct 24 2012 14:28:08 with gcc 4.6 (Debian 4.6.3-8+rpi1)
configuration: --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-postproc --enable-libxvid --enable-pthreads --enable-libvorbis --enable-gpl --enable-nonfree

Edit 2013-01-12: Bob in the comments has compiled some more useful information on, erm, compiling ffmpeg.

Edit 2013-01-15: Bob's been finding issues with getting a working ffmpeg build on a Raspberry Pi, but he's managed to get one through cross-compiling on a larger machine. I shall investigate - my version got built ages ago. If anyone's afraid of this compiling lark, the actually-libav ffmpeg provided with Raspbian will work with the Radio-4-Matic, it'll just have streaming fail then restart more frequently...

So, after baking in the Raspberry Pi's hypothetical code-oven for some time, out popped a piping hot ffmpeg binary. Which rode straight over non-monotonical packets without issues, squashing one of the potential causes of death.

radio.php

Then came the first version of a hacked-together PHP script for monitoring (and potentially respawning) the various utilities. Then came the second. 'Spaghetti code' wasn't apt - no spaghetti would be quite so convoluted and branched. This thing was awful. The non-blocking IO was ... interesting to get my head around. (The central problem is with rtmpdump getting stuck but never exiting - there needs to be some intermediary checking that it's still returning data. And that intermediary also needs to check if any other subsection has fallen over, or if it's time to start mplayer on its next time-delayed buffer...)

Next came de-spaghettification. A couple of cheesy helper functions for dealing with processes were born, all with pitifully little error-checking apart from where it mattered most. Recording and playback were separated into, um, separate functions, each as a simple switch() statement executing a single segment each time the loop was traversed, switching to other segments as necessary. It's almost readable now. But still in PHP. Mwuhahahaha! A super-modular, object-oriented, multiple-streams-possible Python rewrite is pending.

Licensing: it's the good old MIT licence. In terms of open source, it's very open. More so than Linux.

Copyright (C) 2012-2013 Adam Foster - afoster@hylobatidae.org

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Download here:

• radio-4-matic-0.3.zip (5KiB ZIP)

One day, I may even set up a github account...

Requirements

• PHP - needs a PHP 5 command-line interpreter. Required package in Raspbian is php5-cli. I don't recall any extra sub-dependencies...
• get_iplayer - the Radio-4-Matic is running version 2.82, but that or anything more recent should do. Someone's already written a decent guide to getting it running on a Raspberry Pi.
• rtmpdump - available in Raspbian's repositories. Imaginatively called rtmpdump. See installation instructions for get_iplayer above.
• ffmpeg - see pre-baked requirements above. It'll work with the one from Raspbian, but it'll break. Monotonically.
• mplayer - installed by default? If not, install it!

Running

I've yet to add any particular automation to the current workings - I'm assuming you have SSH access enabled on your Pi, have working sound output and know some UNIX basics.

Edit the radio.php to point at the appropriate binaries (I've put all the necessary parameters in, but you may want to tweak them or change the paths depending on where you have them installed), set your playback timezone and adjust the recording time ranges to your liking. Start up a screen session and start Radio-4-Matic with php radio.php. Ctrl-A, D to detach, screen -r to reattach to the screen session - this way you can leave the system running without needing an SSH session open. Depending on where you live, wait something like eight hours then and BBC Radio 4 should start playing - adjusted to your location, of course. Peer at what one particular subsection is doing with a tail -f <logfilename>.log.

It'll probably run on a bigger desktop machine running Linux, or even Mac OS X or other UNIX systems, without modification - there's nothing Raspberry Pi-specific in there. Yet!

(Next tasks: have it running automatically on startup, check if WiFi connection is up, potentially twiddle said WiFi or reboot the Pi in an emergency, reuse any existing audio buffers, ignore You and Yours, integrate with Teasmade for the morning cuppa, and so on and so forth...)

Timing during playback isn't perfect, but for me it typically runs within about 20 seconds of 'proper', adjusted time - the pips are a particularly unfair way of judging its timeliness. I'll look into improving this, but for now it more than maintains the illusion. There's a fudge-factor in the code to adjust for streaming delay, but there's almost certainly a better way of doing this. The Pi's audio clock probably isn't a perfect 44.1kHz so there could be drift from that, too...

If streaming has failed, then on playback you'll get a (correctly timed) gap in the audio until recording resumed. A future feature could be some authentic radio noise until the 'signal' is regained.

Have fun - and if you happen to construct your own Radio-4-Matic, do let me know!

Command explanation

There's lots of arcane magic in the way the various utilities are being invoked, so here's a rough explanation. There are surely many unnecessary parameters in here - stuff that persisted through different iterations - but this is how I've got Radio-4-Matic running fairly reliably.

get_iplayer

The please-use-only-for-good-not-evil get_iplayer is a monster of a Perl script, which parses stuff from BBC iPlayer to create parameters for starting a Flash-streaming program such as rtmpdump or flvstreamer. The words 'dreadful' and 'hack' come to mind, but it can be incredibly useful for tasks such as this.

get_iplayer --liveradiomode flashaaclow --pid radio:bbc_radio_fourfm --stream --rtmp-liveradio-opts --live

Explanation: start get_iplayer, in live radio mode streaming low-bandwidth AAC audio over flash, programme ID 'radio:bbc_radio_fourfm', streaming to stdout, reassure rtmpdump that no-really-this-is-live.

ffmpeg

Audio from the above gets piped through ffmpeg, a Swiss-army-chainsaw audio/video conversion tool. We don't want to change the underlying AAC-compressed audio, instead just repackage it from its mplayer-confusing FLV container into a much-more-standard ADTS container.

ffmpeg -f flv -i pipe:0 -acodec copy -vn -f adts -strict -2 -

Explanation: start ffmpeg, input format FLV, input from pipe 0 (stdin), disable video recording, copy compressed audio without transcoding or recompressing, output format ADTS, standards compliance level -2, send to stdout.

mplayer

At the required moment, the script starts mplayer on one of the recorded audio files and lets it play until it reaches the end.

mplayer -novideo -nocache -ao alsa

Explanation: start mplayer, disable video output (simply to suppress a warning message, disable file cache (forget any idea of the file being of fixed length, it could still be increasing in size), audio output device 'alsa' (suppresses another warning message).

SSH tunnelling

I'm not going to go into much amount of detail on this, given that it's (a) optional, (b) much more involved to set up given the remote server requirements and (c) it's more than a bit dodgy. But anyway. It's possible, and the script above supports it. I found that streaming became a fair bit more reliable, but still not perfect. You need tsocks and an SSH tunnel. So there.

Continuing this series on the hacked-together, time-travelling Radio-4-Matic, time for some hardware. This was going to have been a guide to dismembering an old, non-functional radio and rebuilding it from the ground up, using GPIO to read various buttons and controls, but ... I couldn't. So instead, here's a guide to modernising a glorious vintage radio in a minimally invasive, fully-reversible manner...

The first prototype for the Radio-4-Matic consisted of the Raspberry Pi, a 5V power supply, USB keyboard, ethernet cable and a cheap monitor with HDMI input. My two-line proof-of-concept audio delay script worked surprisingly well - although it had a tendency to cut out whenever get_iplayer lost connection. Some extra electronics from SparkFun resulted in a more independent, bedside version of the beast - albeit one which suffered from serious mains hum, and which needed an empty cardboard box (formerly containing pasta) to improve the speaker's acoustics.

Such a botch-job could never be a permanent home for BBC Radio 4, obviously.

Some background: like many, I grew up in a house where there'd always be at least one radio switched on, permanently tuned to Radio 4. For us, it was a motley selection of Grundigs, Bushes and an ancient transistorised Cossor belonging to my dad. At some point in the past I'd acquired my grandfather's 1970s Roberts radio - a stylish design of chrome, wood, fake leather and glass. Using such an iconic British brand to surround some modern British electronics sounded like an eminently ridiculous idea - but transatlantic inquiries revealed nobody had the faintest idea where that particular radio had got to.

So, I resorted to eBay.

The plan was to remove the old electronics and drive the old speaker using the already-acquired modern amplifier. Plans brewed in my head, figuring out new GPIO idiocy to interface with the radio's newly liberated buttons and dials. HDMI, USB and ethernet ports on the back - a full-scale case-mod.

It turns out there's a surprising range of vintage radios on eBay, particularly the UK version. Many of which are selling for silly prices, or have sustained serious damage, or are modern DAB efforts that have been miscategorised by opportunistic sellers. Fortunately, I got a proper vintage Roberts R707 that (a) was cheap, (b) was in surprisingly good nick, and (c) had the seller offering transatlantic postage. Unfortunately, it was sold completely untested. But, given I was going to tear everything out and start again, that wouldn't matter, right?

Cue a pretty long pause as the radio winged its way across the Atlantic and the North American continent. The intertubes are way faster...

Arrival. One elderly radio, ready for disemboweling. Hang on, this thing looks in almost mint condition. Plenty of space inside to fit a Raspberry Pi without removing any existing systems. And it's got C-cell holders instead of cavernous voids for lantern batteries. A quick trip to Safeway for some new batteries surely won't hurt?

So, perhaps decades after it was last powered up, this vintage British radio took to the airwaves once again, this time in Seattle. VHF armed and ready, aerial extended to its impressively full length, we got tuning. Only to run slap bang into some dubstep.

Poor thing.

At which point I realised that, despite the awful noises the radio was making, it sounded fantastic. Almost terrifying, granny-humping bass - for something designed to live in a middle-class kitchen next to the Aga and relay The Archers, this sounded bloody good. It had to live. To tear out its transistors felt like sacrilege.

But how could it be subverted?

Opening the powered radio up for some exploratory surgery (this time tuned to some vintage Fallout 3 tunes courtesy of NPR), witnessing the frankly disgraceful soldering skills - I wondered about how best to sense its internal signals. Wait - that modern amplifier and speaker from earlier - I could quickly twist some extra wires on to its headphones connector, poke them into solder joints and perhaps listen to whatever was happening inside. That thick PCB trace with black wires leading into it, that's obviously ground - the volume control probably has some audio going through it ...

Music coming out of that tinny little speaker as well as the radio's more dignified effort. Bingo.

Tracing back across the PCBs, I soon located where the audio signals left the tuner circuitry and entered the switchboard on the top. By the looks of things, I'd be able to hijack at least one audio signal. But could I make the switches function?

At this point I reversed things - piping audio out from my laptop (Radiohead, appropriately enough) through some wires I could poke into the radio, tuned to a dead frequency. So, if I inject audio here... Radiohead coming out the radio's 1970s speaker, sounding startlingly good. Whatever the required specifications of this internal signal, the laptop's output was good enough.

I soon discovered that the AM tuner was separate from the FM tuner, with both feeding their audio signals into that switchboard on the top of the radio. While it appeared that the AM tuner would be powered down when the FM tuner was active (and various different circuits would get switched in and out for AM's LW, MW and SW), the switchboard and amplifier would work just fine with an always-on audio signal apparently coming from the AM tuner. So, given the paranoid racism on MW, the screeching interference on SW and lack of radio signals on LW, the AM wire would get the snip and a headphone jack added in its place.

Entertainingly, it seems to apply some sort of low-pass filter when the LW button is engaged. So a high-fidelity input signal ends up with some authentic LW muffling. Awesome.

Cue some quick surgery on a damaged headphones extension cable, extended with inappropriately thick wires and the join insulated with some off-brand Sellotape...

... And then to solder this mess into the radio! 3.5mm headphone cable, tip and centre of jack connected to the AM audio in, shroud connected to ground. I may have promptly stabbed the leather-effect Rexine with the soldering iron. Oops. Luckily it healed suspiciously well.

Some vigorous testing connected to the laptop proved the amplifier was more than adequate for the task (Radio 4 being simulated with pre-recorded Just a Minute and I'm Sorry I Haven't A Clue), then it was time to emplace the Raspberry Pi - held in place by the batteries, back panel and cabling...

Right now, there's a 5V power cable poking out the back - the photos above were taken when it was still connected to the intertubes via ethernet, but as of this evening the radio is properly a radio thanks to a handy little USB WiFi module. Setup was pretty easy, but I think I'm going to have to do some modding to this very-early Pi with the dodgy polyfuses. It's cut out a few times already...

Still, the all-important to-do list:

• Add 12V power supply, so the radio's not permanently running off batteries. The radio's battery life seems great, but I do need to characterise current and voltage requirements before buying any potentially underpowered kit.
• Attach DC-DC step-down power supply to 12V, for providing the Raspberry Pi with a nicely stable 5V. Component ordered!

Edit 2013-06-30: - I finally got round to making the whole thing run on mains power. Those batteries were finally deciding to get a bit faint...

When all this is done, I'll have a fully independent Raspberry Pi sealed inside a 1970s radio - the only clue to the radio's modified state being the Pi's activity LEDs gently visible through the air grille at the back. On VHF, it's still a conventional radio - but on any of the AM bands, it's a mysteriously intercontinental, time-travelling device. A fully-reversible hack - doing anything else to this magnificent old beast would have been cruel. It's hard to describe the audio quality, other than meaty - the tinny PWM-iness of the Pi's analogue-out has somehow combined with the chunky 1970s transistors to give something quite glorious. It's neither 'good' nor 'accurate', it's better than that...

Next up, the all-important software!