Contents |
Buttons API brainstorming
This purpose of this page is to record ideas for a 'Buttons' API, designed in terms of message patterns for remote controls, media players, Web sites and other entities from the home automation and Social Web universe.
Buttons should
- assume as little as possible about UI and handset characteristics
- be usable with minimal up/down/left/right style dumb remotes (eg. apple remote)
- allow smarts to live in different places (media centre, wrapper code, iphone, ...)
- work with touch tables, alternate input devices
- work in basic way with low-resource media centres, eg. xbmc on a low cpu device (so maybe no sparql)
- allow richer services to be exposed (eg. sparql, full text search, auto-complete, recommendation)
- work with Websites (including opensocial/facebook apps), media centres, mobiles,
- allow generously for extensions and innovation
Media Centres
XBMC/Boxee
...
iTunes
...
Pairing and other XMPP issues
Button messages can be sent between any Jabber-addressable entities on the Internet. In general, the federation of XMPP servers to form the global Jabber network allows us to ignore a lot of awkward details, however the following points should be remembered:
- You can't rely on being able to send an XMPP message (chat or IQ) to something unless you're on its roster (ie. buddylist)
- large messages may not pass across the server-to-server links (this was an issue for initial JQbus SPARQL binding)
- some XMPP implementations (eg. Hyves.nl) haven't yet federated with others
Audio pairing scenario
See ChirpChirp...
Barcode pairing scenario
Imagine a large barcode displayed on the screen, encoding within it an XMPP URI and perhaps some (single use?) authentication token?
A smartphone app (if we're feeling fancy, even Augmented Reality clients) could spot this, decode and send a hello message to the media centre. The centre should probably auto-add anyone to its roster (or we insert some whitelist or socialgraph step here if spam is an issue). Once accepted, the Buttons remote has to earn some level of trust from the media centre. This could be through some routine (mumble mumble openid/oauth/foaf+ssl) to prove that the remote is controlled by a trusted party. Or it could be by passing in the on-screen barcode-encoded passcode. Once basic trust is achieved the remote can do certain things; perhaps flip channels but not record or browse private media. Who knows - lots to design here.
Questions to research:
- what level of on-screen display is needed before a modern (iphone/android/w3c-widget) device can decode a URI from on-screen barcode?
- which variant of barcode to use? (see danbri barcode bookmarks, esp re QR codes)
- would animation, missing regions of the QR (which are permitted, but hard to generate algorithmically?) degrade readability by device?
- what API is needed and available from media centres to make this auth scenario feasible? Example below - xbmc
- where does the hypnotoad fit in? How can we show the mediacentre is in receptive mode?
XMPP details:
- xmpp:romeo@montague.net?subscribe (from XEP-147
- can we pass extra parameters?
Experiment - here is xmpp:buttons@foaf.tv?subscribe;pin=1234 via qrcode.kaywa.com:
Code to play with - rqrcode. See also BBC-related experiments...
XBMC on screen barcode research
Can a wrapper script convinced XBMC to put stuff on screen like a generated barcode?
17:21 danbri: hi folks 17:22 danbri: if i wanted a script driving an xbmc box to display a giant full-screen barcode ... how could I go about that? closest i can see is ShowPicture(filename) 17:23 danbri: but then i have to get the generated barcode into the system, and it'll show up as a media object to the user 17:23 danbri: any chance data: URIs are supported? 17:23 danbri doesn't see mention of the possibility 17:24 danbri: so maybe FileUpload(filename, contents) ... ShowPicture(filename) then FileDelete(filename) ? will a delete remote the image from visibility directly
Brainstorm notes with Dirk
More scribbles to dig out, but rough idea...
Dirk-Willem van Gulik: 10:13:30 Yes - that is fine - but you need to do something prior/during the pairing. Dirk-Willem van Gulik: 10:13:42 Try sketch it out - it'll be fine. Dan Brickley: 10:18:13 OK, danbri visit's dirk's house. We use existing remote to put it into buttons-pairing mode. danbri's phone finds xmpp:dirktelly@webweaving.org?pin=1234 in the qrcode, sends a roster subscribe message there, which is autoaccepted. Sends a hello msg with a secret in it to the mediacentre , which generates a second fulls screen QR code that the iphone picks up. So the phone knows it has established a channel with the telly, and the telly can on screen say "I'm paired with someone, was it you or Burglar Bill?"; we use the original domestic remote to confirm. Afterwards, Dirk can manage access rules based on whim and xmpp-communicated proofs (eg. certs, openid/auth/etc); but the devices can at least communicate and use basic xmpp services like presence. Dan Brickley: 10:18:35 whether you choose to expose sparql db of all your mp3s is a decision at another layer :) Dan Brickley: 10:19:11 the secret and pin bit is handwavy, but the general narrative is what i'm aiming for Dirk-Willem van Gulik: 10:21:50 soudnds about right''''
Kinds of Remote APIs
Re 'smarter' remotes, there are two different designs that naturally present themselves, both are somehow inevitable. It's about where the 'smarts' live, and where the user looks for the UI.
- 1. dumb mode, handset (or touchtable...) essentially sends clicks, and the user looks to the main display to understand their role, success, and possible next actions. Essentially reproducing what infrared dump remotes do.
- advantages: we know it kinda works. is pretty universal. interfaces already exist.
- disadvantages: doesn't exploit rich ui, networked nature or extra knowledge of handset. each system works differently, so the remote reflects back the chaos (ok if the user likes that?). no feedback if you hit the top of a list or side of a grid.
- 2. smarter mode: more logic in handset, user can navigate without using main on-screen display, which might not even ack that something is being navigated.
- advantages: Allows multiple users to explore catalog immediately. Allows faster UI on phone (once the catalogue or parts is loaded), no network sluggishness or mediacentre delays. Allows personalised views...user can learn the ui once and then it'll be relatively portal across all encapsulated media centres.
- 2. disadvantages: handset would need to load up potentially large EPG dataset, and other info before it can do much. ... media centre APIs might not like having their ui bypassed.
From XBMC wiki:
Sends a raw Action ID (see [1]) to xbmc. Be careful with this one since it runs the action on the webserver thread and depending on the action could lock the GUI thread. The example sends ACTION_MOVE_DOWN: http://xbox/xbmcCmds/xbmcHttp?command=Action(4)
