Thursday Jan 24th 2008 the Bay Area Computer Music Technology Group met for our Build Ruby Audio APIs Hack Session at Internaut Design. Thanks to David Lowenfels at Internaut for hosting!

Our focus was identifying existing Ruby Audio APIs for extension, and determining what would be the most useful approaches and protocols.

We identified four layers to explore and technologies that could be extended or bridged for each layer:
  • Raw DSP
  • Sound design
  • Composition layer
  • Inter-application communication

The first three layers essentially map to timescale and the fourth to inter-application protocols. The programatic structures would however be different for each of these time scales.

DSP Layer

For raw DSP there was an agreement that Ruby interpreters would not be directly fast enough to run serious DSP code and would likely need to rely on C libraries. We discussed generating csound orchestra files using erb and wrapping the csoundapi. I have been making a preliminary pass at writing a csound orchestra file writer that uses ERB that I will be posting to a public subversion repository in the near future.

David Lowenfels was interested in writing DSP code directly in Ruby and did some further inquiry into SND/Ruby that seemed very promising see the email below. The other alternative was directly accessing portaudio through ruby. A wrapper for this already exists but doing DSP this way seemed unlikely to run quickly or be a truly productive line of inquiry. There was some speculation on whether a Ruby interpreter would ever be fast enough to run DSP code but we came to no conclusion on this. Currently it looks like YARV is the fastest interpreter (needs verification).

SWIG was mentioned in a number of contexts as being an excellent way to bridge C libraries to Ruby (as well as an array of other languages).

Sound Design Layer

VST and Audio unit wrappers could presumably be easily written. Tim Thompson’s experience writing a Python API for VST was that working with VST would be succesful and strightforward. Audio units are probably well documented. We didn’t explore too far into this territory.

Using ruby in Max/MSP through jRuby and the JVM seems possible and straightforward if someone has the will to document the setup this could be fairly widely used. I am pretty sure it could be successful as I have used Jython with Max/MSP and it worked. There was also a suggestion to use jRuby and jMax, but extending max/msp elicited more enthusiasm and would potentially be of wider use due to installed user base.

Wrappers for csound would also be useful for sound design.

Composition Layer

Ruby is an excellent base language to extend for Domain Specific Languages (e.g. rails). A composition framework or DSL could be written on top of any of the technologies discussed for the DSP or Sound Design Layer. However, the most interesting approach (for me anyway) would be writing the composition layer to talk to OSC, thus providing the most portability across audio engines.

Inter-application communication

Making use of the Ruby OSC wrapper seemed like a very productive path. We discussed using OSC to expose the object space of a system. We also decided it would be useful to perform some stress testing for the existing Ruby OSC library (rosc) and optomize OSC library if necessary. OSC would provide interactions with the most platforms of interest and would allow the creation of a more high level API or domain specific language written in Ruby.

Future Events

At the end of the evening we reviewed the hack session itself. From this discussion it is likely that future hack sessions for the Bay Area Computer Music Technology Group will be organized around OSC and interactions with multiple languages including Ruby. It also seems that a more general Audio Open Source development hack session would be of interest. There may also be more Ruby focused hack sessions, or a Ruby focused sub group within a larger hack session.

Useful Links

Wrapping Csound: Using the Portaudio Ruby wrapper: Using OSC from Ruby to control audio applications: SNDs Ruby functionality:

Further Developments

David post meeting interchange about SND/Ruby:

seems like my intended work with ruby-clm is basically already done! :) I’ll report back when I have generated some audio examples.

-David

Begin forwarded message:

Hi, David! Thanks for your interest in Snd/Ruby.

I’m excited to see that I could generate audio with ruby. How does this fit in with Snd?

If you configure Snd with ./configure—with-ruby, you will end up with a new Snd program using extension language Ruby. You can type in Ruby/CLM/Snd commands in the Snd listener, eg:

x = 1 + 2
	os = make_oscil
	os.call(0, 0)
	os.frequency = 330

etc. There exists a bunch of *.rb files which are mainly translations of the corresponding Scheme files. Snd’s documentation tries to show the similarities between Scheme and Ruby Snd/CLM code.

Is it possible to run ruby-clm outside of snd?

Get the file ftp://ccrma-ftp.stanford.edu/pub/Lisp/sndlib.tar.gz and configure it with ./configure—with-ruby. If you install sndlib.so in ${RUBY_SITE_ARCHLIB} you will be able to require it like all other libraries with ‘require “sndlib”’. The contents of sndlib is nearly the same like Lisp CLM which is described in snd/sndclm.html. A few *.rb files are included in sndlib.tar.gz which may help a little.

If you have more question please don’t hesitate to ask them by email, or even through the clm mailing list.

Mike