Openframeworks to Flash [MIDI]

I have been interested in using MIDI devices for different specific interfaces lately. I have been using mainly Openframeworks to play a little bit around with it and understand the possibilities of such hardware interfaces.

If you have followed my previous MIDI / OF tutorial, you’ll see how easy is to get your MIDI hardware up and running in OF, but what about using MIDI hardware in Flash? I was really interested on being able to use a MIDI controller within a Flash application, so I researched a little bit and found this helpful post.

Apparently, you can send data from OF (C++) to Flash via TCP using binary sockets in as3 and ofxTCPServer in OF.

The approach is simple:

1) Create a C++ TCP Server application using OF to send encapsulated MIDI messages.

2) Create an as3 client application that reads the MIDI messages and do whatever you want to do with them

TCP Server (OF)

Let’s start with the OF server side: you’ll need to use the ofxMidi and ofxTCPServer addons.

The first thing I did was to create a simple C++ class to encapsulate the necessary information of my MIDI messages to be sent:

/*
 *  SocketMessage.h
 *  MIDISocket
 *
 *  Created by Eduard Prats Molner on 04/10/2010.
 *  Copyright 2010 Jocabola. All rights reserved.
 *
 */

#ifndef SOCKET_MESSAGE
#define SOCKET_MESSAGE

class SocketMessage{

public:

	SocketMessage();
	~SocketMessage(void);

	int MIDI_val;
	int MIDI_ID;
	int MIDI_port;

};

#endif

I used the ofxTCPServer OF sample combined with my MIDI test class. This is how the TestApp header looks like:

#ifndef _TEST_APP
#define _TEST_APP

#include "ofMain.h"
#include "ofxMidi.h"
#include "ofxNetwork.h"
#include "SocketMessage.h"

class testApp : public ofBaseApp{

	public:
		void setup();
		void update();
		void draw();

		void keyPressed  (int key);
		void keyReleased(int key);
		void mouseMoved(int x, int y );
		void mouseDragged(int x, int y, int button);
		void mousePressed(int x, int y, int button);
		void mouseReleased(int x, int y, int button);
		void windowResized(int w, int h);

	void newMessage(ofxMidiEventArgs &args);

	ofxMidiIn midiIn;

	ofxTCPServer TCP;

	ofTrueTypeFont  mono;

	vector <string> storeText;

};

#endif

In the setup method we’ll need to initialise the MIDI and TCP Server as follows (setting up the TCP server at port 11999):

void testApp::setup(){
	midiIn.openPort();
	ofAddListener(midiIn.newMessageEvent, this, &testApp::newMessage);

	//load our type
	mono.loadFont("type/mono.ttf", 8);

	//setup the server to listen on 11999
	TCP.setup(11999);

	ofBackground(0, 0, 0);
}

Now, I’ll update the MIDI “new message” listener to be sending the desired MIDI arguments using the TCP server and encapsulating the MIDI port, id (which knob, button, toggle) and value into a SocketMessage class instance:

void testApp::newMessage(ofxMidiEventArgs &args){
	SocketMessage* sm = new SocketMessage();
	sm->MIDI_port = args.port;
	sm->MIDI_ID = args.byteOne;
	sm->MIDI_val = args.byteTwo;
	TCP.sendRawBytesToAll((char*) sm, sizeof(SocketMessage));
	storeText.push_back("Sending message: <port: " + ofToString(sm->MIDI_port) + "; id: " + ofToString(sm->MIDI_ID) + " value: " + ofToString(sm->MIDI_val) + ">");
}

Notice that I stored the messages into a storeText vector which I use for printing messages in the little server app. That’s how my update and draw methods look like:

void testApp::update(){
	if(storeText.size() > 28){
		storeText.clear();
	}
}

//--------------------------------------------------------------
void testApp::draw(){
	mono.drawString("TCP SERVER launched on port: " + ofToString(TCP.getPort()), 10, 20);

	for(int i=0;i<storeText.size();i++){
		mono.drawString(storeText[i], 10, 30 + 10*i);
	}
}

Client application (AS3)

Is time for grabbing the MIDI messages from Flash!

My Flash application is called MidiSocket and has a Socket class instance that acts as a client application:

//--------------------------------------
//  CONSTANTS & VARIABLES
//--------------------------------------

private var _client:Socket;

public function MidiSocket(){
	_client = new Socket();
	_client.addEventListener("ioError", retry);
	_client.addEventListener("connect", connectionEstablished);
	_client.addEventListener("socketData", socketData);

	_client.connect("localhost", 11999);
}

Now, before getting into the actual socket data parsing / fetching. Let’s make some memory about the two variables of the MIDI messages that are really important to us: byteOne and byteTwo.

As their names already tell us, each of these variables is represented by one byte (8 bits). byteOne is an unsigned integer that represents the ID of the MIDI parameter being changed (identifies a particular knob, a fader, a button, …). byteTwo is the actual value of it which is represented by an unsigned integer that goes from 0 to 127 (128 possible values).

When fetching the values in Flash, I faced the problem that the values being read where really long weird nonsense numbers. I suspected that had to do with the bit representation of the integers since Socket.readInt() returns a 32-bit integer, so I had to refresh “bit shifting”: Great explanation here.

Basically, I needed to shift right 24 bits to convert the values back to the original 8 bit representation. In other words, to be able to have nice integers from 0 to 128 (much easier to handle) once again.

private function socketData(evt:ProgressEvent):void{
	var buffer:ByteArray = new ByteArray();
	_client.readBytes(buffer, 0, _client.bytesAvailable);

	var val:uint = buffer.readInt() >> 24;
	var id:uint = buffer.readInt() >>  24;
	var port:uint = buffer.readInt();	

	trace("MESSAGE: <port: " + port + ", id: " + id + ", value: " + val + ">");
}

There are also two more important things to take into account. Make sure to read the whole buffer; otherwise you’ll be accumulating some weird data that will end with wrong values being read. So best practice is to always read the whole buffer: _client.readBytes(buffer, 0, _client.bytesAvailable);

Once you grab the bytes, remember to fetch the arguments in the same exact order as they are defined in the C++ SocketMessage class: For instance, first int will be the value, second the id and last one the port.

That’s pretty much it! Get the sources here and feel free to drop any questions as comments.

Considerations

1) I think it should be better to create a pure C++ command line tool for such a server app.

2) Notice that MIDI was just an example. Using this technique you can send any data from OF to Flash, and works very fast (complete real-time feeling with the MIDI controller).

3) I haven’t tried the buffer.readObject in depth. Would be great to serialise objects; Something like AFM / JSON and work with objects encapsulating all variables instead.

Have your say:

James says:

19/November/2010 at 17:13

Oh ya, it will be release as a commercial product soon.

James says:

19/November/2010 at 17:10

It is also possible in .Net, Java, C too. I have a workstation build with Java.

jocabola says:

17/November/2010 at 10:19

Thanks for sharing this Lawrie!
Your Flash Midi Server looks very interesting indeed!

Lawrie says:

16/November/2010 at 16:28

Cool! Thanks for posting this.
See also – http://www.abumarkub.net/abublog/?p=338
And my attempt at Midi in/out for Flash –
http://www.lawriecape.co.uk/theblog/index.php/archives/636