This page explores the idea of encoding URIs/IRIs in audio, eg. computed melodies or synthetic birdsong. It can be considered a form of audio steganography, or a kind of aesthetically pleasing but very slow modem. Unlike modems and much steganography, the payload is pretty small. A key use case could be addressed by finding an encoding of 'username' @ 'host' accounts and using XMPP/WebFinger/etc to discover everything else.
Contents |
Initial application scenario: URIs for home automation (Buttons)
The initial application idea for this was as an organically-inspired device pairing protocol for the DanBri/ButtonsAPI work, alongside a 'visual' option; in other words, a previously un-paired smart-phone / media-centre pair could 'meet' by virtue of the phone being able to 'see' (possibly using a UI like iphonearkit) a large barcode/qrcode that encodes an XMPP URI; or they could meet by virtue of the phone 'hearing' the mating song of the device. We don't limit this to phones and TVs; multi-touch tables, data projectors etc are all in scope.
The feel here is that we're going for a natural, organic, sensual approach than wifi/bluetooth; making use of people's intuitions about sights and sounds (and risks of them being overhead or seen). It would take some care to make sure the security properties of the technology mirror the intuitions that the approach encourage. We assume the devices are all connected to full generic Internet and once identified can exchange messages, via XMPP rather than HTTP since they might be behind nat/firewall.
danbri braindump scenario
Imagine you visit here with a new phone; i put my media centre into pairing mode. On screen you see a giant weirdo qr barcode; and from the speakers you hear a strange spooky woo-woo tinkle-tinkle chirrup sound, organicky and unsettling. This is the tv saying it is in the mood to pair. The phone then has two modalities for finding out about this device nearby, either it sees it, or hears it. We could go for a birdsong approach, or music, or a mix. Main thing to create is a spec and some opensource code.
Burglar bill has two opportunities to hijack things, but we go into the security stuff later. so audio, the phone records 3 seconds of the loop, extracts the URI, which might be xmpp:dantv@buttons.tv?subscribe;pin=3214
(this is all somewhat related to William Loughborough's IR-based Talking Signs system, although a different approach/app)
People have native appreciation for visible and audible. It's only sending an identifier, nothing more.
You might even have the basic core info-carrying melody loop on 5 seconds, and have a second line of sounds, deeper bass or other pleasant decorations, cycle on 20 second or other loops, to make it bearable to leave running a bit longer.
If the media centre was successfully contacted by the buttons device, an audio expression of that might be interesting; however the core design here should just be "how do we encode short (how short?) URIs/IRIs in audio so they can be reliably picked up by high-end fancyphones? (on the thinking that these devices will work their way into the mainstream soon enough...).
notes from dirk chat
"very common in some other area's up to about 1kBaud it is very doable to decode that. 300 baud is almost trivial (that gets you about 20 chars/second with error correction)"
Re 'q: can it be aesthetically pleasing', "a: ot an issue - make a song with 10-20 pleasing notes; and two parallel melody lines. Declaire one '1' and the other '0' and then take it from there."
danja initial thoughs
it *should* be possible to invisibly embed & retrieve a handful of chars in a short sound - you'd need a lot of redundancy (like FSK has), maybe subtly FM the audio?
Java playground
If you want to get started with a nice Java API for generating audio, Jfugue looks good. Try it!
Cornercase:tmp danbri$ svn co http://svn.foaf-project.org/foaftown/buttons/chirpchirp/java A java/lib A java/lib/jfugue-4.0.3.jar A java/build.sh A java/tv A java/tv/foaf A java/tv/foaf/chirp A java/tv/foaf/chirp/Chirp.java A java/tv/foaf/chirp/Chirp.class A java/readme.txt A java/run.sh Checked out revision 386. Cornercase:tmp danbri$ cd java/ Cornercase:java danbri$ ./build.sh Cornercase:java danbri$ ./run.sh
This will hopefully play a simple rhythm, as specified in the jfugue examples page.
Links
- project bluebell via jimh
- jfugue looks great for generating and playing music.
- Web Musician danbri/martin poulter collab from ages ago; bit broken now.
