Just before christmas Dan made a node.js xmpp bot that could appear in N-Screen and take commands via drag and drop (if you don’t care about this, the point is that it uses node.js with node-xmpp). Dan got it showing up in the N-Screen interface and reflecting back whatever you dropped on it. I thought it would be trivial to hook in one of the item-to-item recommenders I have working via a json http api. Basically to do http GETs in node.js you need something like this:

var http = require('http');

request.on('response', function (response) {
response.setEncoding('utf8');
var myStr = "";
response.on('data', function (chunk) {
console.log('BODY: ' + chunk);
myStr += chunk;
});
response.on('end', function (response) {
console.log(myStr);
}
});


Then you can parse myStr e.g.

var j = JSON.parse(myStr);


What we wanted to do was send that myStr as a string content of an xmpp message like this:

var cl = new xmpp.Client({
jid: jid + '/bot',
password: password
});

cl.send(new xmpp.Element('message', msg ).c('body').t(myStr) );
});


It really should have been simple, but in practice the myStr existed when printed out to console but was never sent. At first we thought it was something to do with Buffers – if you don’t do response.setEncoding(…) you get Buffers back and perhaps some stringification of those wasn’t working, but no. To add to the confusion we could create Json objects from the string but they still would not send, even if copied to new strings.

In retrospect it’s obvious it was the XMPP that was the issue, but it took line by line trial and error sending of some sample input before I found it: & characters in the input were causing silent failure when the message was sent – presumably due to an XML parsing problem that for some reason didn’t show up as an error (I know almost nothing about node.js so there could have been some user error there too). Escaping these & didn’t seem to work, so for now we’ve just removed them and it all works fine. We need to raise a bug report, but thought I’d blog for now in case it helps anyone else. The code is here.

All classes

“Agent”, 3.84 million
“Document”, 6.15 million
“Group”, 5.78 thousand
“Image”, 711.23 thousand
“OnlineAccount”, 15.47 thousand
“OnlineChatAccount”, found 324
“OnlineEcommerceAccount”, found 242
“OnlineGamingAccount”, found 240
“Organization”, 10.05 thousand
“Person”, 2.64 million
“PersonalProfileDocument”, 11.7 thousand
“Project”, found 726

All properties

“accountName”, 8.02 thousand
“accountServiceHomepage”, 7.24 thousand
“aimChatID”, 9.54 thousand
“based_near”, 7.35 thousand
“birthday”, 2.48 thousand
“currentProject”, found 648
“depiction”, 696.31 thousand
“depicts”, 617.16 thousand
“dnaChecksum”, found 65
“family_name”, 2.46 thousand
“firstName”, 4.2 thousand
“fundedBy”, found 237
“geekcode”, found 107
“gender”, 15.8 thousand
“givenname”, 24.17 thousand
“holdsAccount”, 9.88 thousand
“homepage”, 1.22 million
“icqChatID”, 22.8 thousand
“img”, 684.38 thousand
“interest”, 64.77 thousand
“isPrimaryTopicOf”, 1.54 million
“jabberID”, 2.98 thousand
“knows”, 1.08 million
“logo”, found 374
“made”, 1.97 million
“maker”, 1.97 million
“mbox”, 3.7 thousand
“mbox_sha1sum”, 43.9 thousand
“member”, 5.53 thousand
“membershipClass”, found 58
“msnChatID”, 7.68 thousand
“myersBriggs”, found 154
“name”, 1.77 million
“nick”, 96.7 thousand
“openid”, 80.24 thousand
“page”, 5.84 million
“pastProject”, found 179
“phone”, found 999
“plan”, found 139
“primaryTopic”, 278.11 thousand
“publications”, found 202
“schoolHomepage”, found 644
“sha1”, found 60
“surname”, 25.32 thousand
“theme”, found 282
“thumbnail”, 2.51 thousand
“tipjar”, found 73
“title”, 2.02 thousand
“topic”, 3.13 million
“topic_interest”, found 90
“weblog”, 300.06 thousand
“workInfoHomepage”, found 505
“workplaceHomepage”, 1.68 thousand
“yahooChatID”, 6.72 thousand

It’s trivial to move a bot from one to the other. In the source for CharBotGreen there’s a line

u = "http://twitter.com/statuses/update.json"


Using the Twitter-compatible Identica API you I can just replace that line with:

u = "http://identi.ca/api/statuses/update.json"


The only thing to watch for is that Identica stores names as lowercase and the authorisation fails if you don’t send it in lowercase.

Doesn’t work in Identi.ca:

req.basic_auth 'CharBotGreen', 'sekret'


works in Identi.ca:

req.basic_auth 'charbotgreen', 'sekret'


Thats it though – easy!

Last night though I found that my twitter bot, CharBotGreen had been suspended as a phisher, and tonight I find I’ve been suspended from twitter too. This is a bit of a blow, and the cause in both cases seems to be that I linked to my blog.

Using Google webmaster tools I discovered that several pages had links to viagra etc pages on them, invisible except in the source, with html inserted through the header php. Firefox and Safari made it difficult to find this out by inserting buggy ‘this is a phisher’ text (with broken links) over the source as well as the page itself.

I’ve now moved off Dreamhost completely – though it might have been simply that I had not updated WordPress enough. I’m on wordpress.com now, so I hope that’ll remove this riskiness.

The whole episode has made me rather depressed. Google has basically killed my online identity. I’m on various lists asking to be taken off, but there’s been no movement since last night, and I had no warning. It seems that there’s a blacklist being used in both cases, not competely sure what it is yet.

Anyway, if it happens to you, take it seriously and deal with it as soon as you can.

Update: I’m actually not on google’s suspended list any more. Hurrah! But still no Twitter. Guess it’s time to move to Identica with that and the madness of #fixreplies. Meh!

2nd Update: I got my Twitter account back this morning (2nd June, 3 days later). CharBotGreen is still suspended.

Useful links:

Google – My Site’s been hacked
Google webmaster tools
Google apps admin page: Google MX Records

I bought a Polaroid PoGo inkless bluetooth mini sticker printer having been entranced by psd’s one, but knowing it didn’t work with the iPhone and that I’d have to get my laptop out to print anything. The PoGo is a lovely toy but I was getting a bit irritated by this limitation. The problem was twofold:

•  iPhone bluetooth is crippled – you can only use bluetooth headphones, and not use it for file transfer. Annoying.
• iPhone stores pictures as (peculiar) pngs and PoGo only accepts jpegs (which I found by trial and error – I can’t find any PoGo docs on that at all)

The first issue was easily solved – I have a jailbroken 1st gen phone and I just installed iBluetooth with Cydia, which is a app installer based on .deb packages.

The second was more tricky. I looked at ImageMagick for iphone (it’s on Cydia) but didn’t get anywhere. I think I needed to install gcc which was a step too far. Instead I put ssh on it (pretty cool in itself), found some hints on the web, and found that iPhone actually creates jpgs as well as pngs (in /private/var/mobile/media/DCIM/100APPLE – the pngs are in /private/var/mobile/media/DCIM/999APPLE). Weird! Anyway, iBluetooth allows you to browse the filesystem, and send files you find there, and that worked.

So all you really need is iBluetooth as it turns out. Hope this is useful to someone.

I wanted to try out the Twitter API and since I was finding myself repeatedly going through the tedium of flipping browser tabs to see what was on Radio 4, I figured I’d make a bot that tweeted what was on Radio 4 instead. This had the added advantage that I could use some half-written code I’d started for a more complex event bot that was turning out to be too hard. I neglected to do a twitter search, however, which would have shown me that there were at least two similar services already working. Ah well. Here’s CharBotGreen

Thanks to: Damian for the name and technology suggestions, @psd for the picture, and Charlotte Green for being a great Radio 4 announcer (as are they all!)

Be warned – do not use my Ruby code as an example of good practice, as it most certainly is not.

What it does

Once a day – pulls down the Radio 4 programmes json (details – what an excellent service that is – beeb++) – and stores it in an H2 database like this, having wiped the database over night (sometime between 1am and 5.20am, when it’s on the world service and no detailed schedule is available anyway):


CREATE TABLE if not exists beeb(DT TIMESTAMP, PID VARCHAR(8), D DATE, T TIME, NAME VARCHAR(255));


So basically I start the Radio 4 day with an SQL representation of today’s schedule page. I started with PID as UNIQUE but then realised that the same PID could be broadcast twice a day.

Every 5 minutes – checks in the database for anything starting in the next 5 minutes and sends a tweet, either ‘starting now’ or ‘starting in a few minutes’ depending on the exactness of the match

SELECT * FROM beeb WHERE D = '#{d}' AND T >= '#{t}' AND T < '#{t1}';

where t is the current time and t1 is the time in 5 minutes (d is today’s date).

Technology

I use ruby and H2 over JDBC. You can see the every 5 minutes and daily scripts and the readme.txt. Why these technologies? Well, I wanted to learn Ruby and using Jruby means that you can use many ruby libraries but you can also access Java classes which is handy for using the H2 database. Why H2? well it’s a self contained, in-memory, SQL-compatible database written in pure Java, so I could keep everything in one directory. For something this lightweight there’s almost no point in using SQL but I wanted it for something a little more complex as well so it made sense (and makes it nice and easy). I use Json pure for the json parsing (it has to be pure to use it with Jruby). If you want to use Ruby rather than Jruby the SQL bit will take some fiddling with; the rest should be ok as is.

Hashtags

I jumped into a little chat on twitter about what hashtags to use and settled on #pid: and then the PID (such as b00h4r7x). I’m still not sure about this; I put the URL in as well.

It’s all super-simple

But good fun to do. Psd suggested that some Charlotte Green-style amusing incidents would be fun to put in there, though I’ve not worked out how to do that. Another improvement would be if it gave you a little more notice about what’s coming up as @bbcradio4live does.

uu is the expanded url. Is there a better way than this? Is it me or is Ruby documentation a bit thin on the ground? (Thanks to Damian for pointing out to me how tinyurls work – I’d never bothered to look before!)


setInterval ( "updateJson()", 60000 );

function lastW(obj){
var el=document.getElementById("lastW");
var id=obj.events[0].publicId;
var title=obj.events[0].title;
var thumb=obj.events[0].thumbnail;
if(el){
el.innerHTML="Libby just watched: <a href='http://www.joost.com/"+id+"'>"+title+"</a> on <a href='http://www.joost.com/'>Joost</a>";
}
};
</script>


<p>
<span id="lastW"></span>
</p>

I’m biased of course but I think this is pretty cool. Numerous (alpha, check the heath warning) APIs are at http://www.joost.com/doc/api. Allan and Toni_H have been playing around too.

Move along, nothing to see here. Yet.

–danbri