zaterdag 17 maart 2012

Hanselminutes rss podcast to playlist(.pls) file.

The script below can be used in a crontab on Linux or a scheduled job on Windows to always have the latest 50 hanselminutes podcasts in a playlist(.pls) file, e.g.:

 nodejs hanselminutesfeed2pls.js > /x/downloads/streams/hanselminutes.pls

It requires nodejs with the request and feedparser modules.

Motivation

It was written to scratch a personal itch. My mede8er media player cannot read podcasts. First I wanted to write a a script that updates my mediatomb's upnp server database. However as far as I know, Mede8er also cannot stream streams that do not originate from the upnp server. It can read .pls files both from smb shares and via upnp.

The code

 var MAXITEMS=50,//Mede8er does not like playlists longer than 50 ;) so take the last 50
     FeedParser = require('feedparser'),
     parser = new FeedParser(),
     fs = require('fs'),
     request = require('request');


function done(error,meta,articles){
  var s=0,
      l=Math.min(MAXITEMS,articles.length),
      i,
      j=0;
  console.log('[playlist]');
 
  for(i=s;i>l;i++){
    j++;
    console.log('File%s=%s',j,articles[i]['feedburner:origenclosurelink']['#']);
    //console.log('File%s=%s',j,articles[i].enclosures[1].url);//mede8er does not like the redirecting urls from feedburner
    console.log('Title%s=%s',j,articles[i].title);
    //console.log('Title'+j+'=','dummytitle');//to test whether there is a length limit or the title may contain odd data
    console.log('Length%s=%s',j,articles[i].enclosures[1].length);
  }
  console.log('numberofentries=%d',l-s);
  console.log('Version=2');
}

parser.parseUrl('http://feeds.feedburner.com/HanselminutesCompleteMP3',done);

The output of the script will be something like:

[playlist]
File1=http://s3.amazonaws.com/hanselminutes/hanselminutes_0310.mp3
Title1=Startup Series: Buying an Existing Small Company or Online Application
Length1=25249632
File2=http://s3.amazonaws.com/hanselminutes/hanselminutes_0309.mp3
Title2=Startup Series: Creating Alan Mendelevich's AdDuplex
Length2=27894768

[...]

File48=http://s3.amazonaws.com/hanselminutes/hanselminutes_0263.mp3
Title48=A C++ guy learns JavaScript - Chris Sells moves to the Web
Length48=23074547
File49=http://s3.amazonaws.com/hanselminutes/hanselminutes_0262.mp3
Title49=The Rise of the Micro-ORM with Sam Saffron and Rob Conery
Length49=80695131
File50=http://s3.amazonaws.com/hanselminutes/hanselminutes_0261.mp3
Title50=ASP.NET MVC 3 Tools Update with Phil Haack
Length50=25365528
numberofentries=50
Version=2

vrijdag 3 februari 2012

Javascript based "Koffiemanager" Part 2: UDP broadcast in javascript using NodeJS and Windows 8

A while ago I explored the idea of making a "Koffiemanager" clone in javascript using nodejs and or Windows 8. In this blogpost I present some experimental code written a few months ago.

Reverse engineering The Koffiemanager protocol was already done by Jeroen Oostrijck, http://jeroen.oostrijck.nl/de/index.html. Koffiemanager uses xml over udp broadcast on port 13473. When looking at the network traffic of the real koffiemanager I saw some weird binary prefix in the messages which still need to be sorted out if Iwant to use this protocol. It is NOT the unicode Byte Order Mark(BOM).

For testing I wrote some proof of concept code:

  • An UDP broadcast transmitter for NodeJs
  • An UDP broadcast receiver for NodeJs
  • An UDP broadcast receiver for Windows 8

UDP broadcasting client using NodeJS

The following sample code announces itself as a "Koffiemanager" client:

(function(){
  var dgram = require('dgram'),
      announce='<?xml version="1.0"?><de-coffee-message><header version="1" identifier=".Connect"></header><message GroupName="hutsefluts" MemberId="{D982458A-D439-4D31-87C1-A18336CF0432}" MemberName="kees"></message></de-coffee-message>',
      message = new Buffer(announce),
      client = dgram.createSocket("udp4"),
      baddr="255.255.255.255",
      port=13473;
      client.setBroadcast(true);

  client.send(message, 0, message.length, port, baddr);
  client.close();
})();

UDP broadcast echo service

The following code listens to broadcast messages on UDP port 13473 and echos them to the console. This code can be used to spy on a real "Koffiemanager" or to test a home made one. With this code I discovered the weird prefix in the messages from the real "Koffiemanager".

(function(){
var dgram=require('dgram'),
  socket=dgram.createSocket('udp4');
socket.bind(13473);
socket.setBroadcast(true);

socket.on("message", function (msg, rinfo) {
  console.log("server got: " + msg + " from " +
    rinfo.address + ":" + rinfo.port);
});

socket.on("listening", function () {
  var address = server.address();
  console.log("server listening " +
      address.address + ":" + address.port);
});

})()

Windows 8 Koffielistener

Just after the Windows 8 Developer Preview was available I wrote a Metro KoffieProtocol listener app in HTML/Javascript. The code below listens UDP port 13473 for broadcast messages and tries to strip the header sent by a real "Koffiemanager". The remaining data is presented on screen.

(function () {
    'use strict';
    // Uncomment the following line to enable first chance exceptions.
    // Debug.enableFirstChanceException(true);
     var DEPort = "13473",
         broadcastSocket;
         hostname = new Windows.Networking.HostName("localhost"),
         broadcast = new Windows.Networking.HostName("255.255.255.255"),
         textmessage;

     function broadcastBound(ev) {
         ev = ev;//breakpoint here
         textmessage= document.getElementById("message");
     }
     function broadcastBoundError(ev) {
         ev = ev;//breakpoint here
     }

     function onMessageReceived(ev) {
         var dr = ev.getDataReader(), s;
         try{
             s = dr.readString(dr.unconsumedBufferLength);
             s=encodeURIComponent(s);
           
             textmessage.innerHTML = s;
         } catch (ex) {
             var bytes = new Array(dr.unconsumedBufferLength),
                 sb = [],
                 i;

             dr.readBytes(bytes);
             //remove header
             while (bytes && bytes[0] !== 60) {
                 bytes.shift();
             }
            
             for (i = 0; i < bytes.length; i++) {
                 sb[i] = String.fromCharCode(bytes[i]);
             }
             s = sb.join('');
             s = encodeURIComponent(s);
             textmessage.innerHTML ="keepalive:"+ new Date().toTimeString()+"\n"+s;
         }
     }


     WinJS.Application.onmainwindowactivated = function (e) {
        var promise;
        if (e.detail.kind === Windows.ApplicationModel.Activation.ActivationKind.launch) {
             broadcastSocket = new Windows.Networking.Sockets.DatagramSocket();
           
            broadcastSocket.onmessagereceived = onMessageReceived;
            broadcastSocket.information.localServiceName = DEPort;
            
            promise = broadcastSocket.bindServiceNameAsync().then(broadcastBound, broadcastBoundError);
        }
    }
    
    WinJS.Application.start();
})();

Next Part3: Refocussing my Javascript based "Koffiemanager"

donderdag 15 september 2011

Javascript based "Koffiemanager" Part 1: Exploring

Me and my big mouth... In 2009 I said I could clone the "Douwe Egberts KoffieManager" in javascript/html. The "DE Koffiemanager" program is a coffee management program for office use. It is made available by Sara Lee's "Douwe Egberts" branche. Douwe Egberts or DE for short is a major coffee brand in the Netherlands. The coffee manager assists office workers in taking turns fetching coffee for their coworkers. It comes in the form of a desktop application that you install on the pc's of the participating coworkers.

How it works

You can use the coffee manager to form a "coffee" group with the people in the same room. Multiple groups may exist in the same office.

At regular interval's the coffee manager picks a person in a group and tells that person it is his turn for fetching coffee. All other people in the same group get a popup that allows them to select the drink they want this round.

The coffee manager collects all orders and present them to the person selected to fetch coffee.

If you are thirsty you can also offer to take turn in fetching coffee manually.

Technically every coffee manager can work as both a server and a client. One of the coffee managers in a group acts as the server. At startup of a coffeemanager it announces itself on the local network to see if there is a server present already. The first manager that is started becomes the manager. For this udp broadcast is used.

Pet program

One reason to clone the coffee manager is to have some scenario to test ideas and new technology on something that is simple but a bit more complex than Hello World' or Fibonacci sequences. The basic Coffee Manager is both simple and complex enough for making a totally overengineered esoteric implementation with lots of extravagant extras which add no extra functionality but are only there to please the Software Engineer. In this case me.

Cloning the coffee manager

Making a web based version of a coffeemanager using html and javascript could not be that hard was my first thought in 2009. It is not that different from a simple warehouse order picker or webstore program. However the original program has three unique technical features that are hard to implement in a web program:

  • tray icon
  • notifications using small popups in the bottom right corner of the screen
  • Websites use http for communication. Using another protocol such as the udp broadcast to elect a server in the original coffee manager does not really fit into this model. In a browser the only means of communication from javascript is HTTP-GET or HTTP-POST. Both are based on TCP. HTTP-POST can only be done to the host from where the page hosting the script originates.(Same origin policy). I cannot do a real broadcast/multicast in a classic web environment.

In 2009 I did a quick scan on how to solve these challenges. Now in 2011 I think a tray Icon is still not possible using web. I have seen some chrome extension that uses a tray icon but not a more general solution. Using a special features for pinned webpages in IE9 I can achieve both notification and tray icon like features. In Chrome a notification mechanism is available that allow you to popup those little notification windows in the corner of the screen.

In 2009 I experimented with the windows scripting host to run javascript on the server on windows(cscript script runner). It would allow me to start some program to do broadcast/unicast. To play by the rules that program should be written in Javascript too. For this I could compile javascript to a .NET assembly. However compiling javascript to .NET using Microsoft.NET or mono does not seem to be a really supported feature. I was not sure what other problems I would run into, e.g. javascript only has a limited set of datatypes. Socket programming is a bit low level. things as bytes exist in .NET and I am not sure how that translates to javascript in .NET.

Besides language problems another practical problem would be running the webserver part on every host. I cannot assume IIS is installed on every host or that every user knows how to install a webapp. An .exe that is a webserver by itself is what is required here!! This can be done in .net using http.sys. So that should not be much of an issue This same exe also should do the broadcasting stuff and start a webbrowser to itself. The .exe can also do the notification and trayicon things.

Node-js webserver

Using javascript on .NET is one possible solution but it would have been a lot of work. That is one of the reasons why I did not continue cloning the coffee manager in 2009.

Besides .NET or Windows Scripting hosts there are multiple other platforms to run javascript on the server. One of those which has become very popular is the Node-js server environment. Node-JS is a javascript runtime especially built for making javascript server using http but also raw TCP and UDP. Even broadcast and unicast is possible. Nodejs is a single .exe that is easy to deploy. Around it a large solid high quality ecosystem exists that delivers all kinds of frameworks , cloud services tools, and extensions. Most important for me programming node-js is fun. Sofar I think nodejs could be part of the solution in cloning the coffee manager.

Coffeescript

Although javascript is a nice language it is still very verbose. For experimenting succinctness is something high on my wishlist. Meet Coffeescript a better Javascript. Coffeescript is a language that takes the best of Ruby, Phyton, Haskell and other languages to be a a more succinct javascript. It has significant whitespace, no curly braces and does away lots of other verbose plumbing. It compiles straight to javascript. It is the preferred clientside scripting language for Ruby on Rails 3.

Coffeescript feels more like functional languages such as Haskell or Miranda. Like those languages It has pattern matching, list comprehension and destructuring assignment.

Since it has coffee in the name literally, I think it is a very suitable language to implement a coffee manager ;). (Yes I know Java is American slang for coffee).

Windows 8 ;)

Yesterday a developer preview of windows 8 was launched. Earlier Microsoft announced that the preferred means of making apps for this new OS would be HTML 5/javascript. This caused some noise... Later they said more classic development methods would also be available ;) .

Sofar I have not been able to run Windows 8 yet but I have studied some samples. I think Windows 8 presents an interesting new opportunity to create a clone of the coffee Manager that I should investigate.

Next post Multicast using Node-js