Human interaction and GUI

This entry focuses on human interaction and the use of GUI in SuperCollider. Super Collider is an 'interactive' programming language meaning that new parts of the program can be written whilst the program is active and writing the program as you use it is therefor part of the program itself.
Here I am focusing more on user interaction with SuperCollider using things such as the mouse, the computer keyboard and midi controllers to create and change sound.

In previous posts I have demonstrated using the mouse as a controller using MouseX.kr and MouseY.kr. This was a quick and simple way for us to see how quickly we could start physically interacting with the program and changing the sounds in this way. MouseX and MouseY are both UGens with the arguments (minval, maxval, warp, lag). Minval and maxval are the screen ranges. Warp is a mapping curve which can either be linear (0) or exponential (1). lag is telling us how much lag there is to the cursor movement. I guess MouseX and Y act kind of like a knob would as a lot of values can be accessed quickly, it is just a quicker way of testing something rather than creating a knob GUI. MouseX and MouseY can be used to control different things at the same time such as amplitude and frequency. I will not focus to heavily on that now as I have explained this a bit in my first blog entry. We were shown a really interesting code example written by James McCarthy of a strummable guitar where you move the mouse to recreate the sound of strumming a guitar.

We were given example code in which you could use the computer keyboard to play notes. It uses doc = Document.current;
This means it calls back from the window that you are typing into which allows you to play sounds whilst you type into that window. Document is a class which “represents a text document within the context of your text editing environment” (help file).

I read about Laurie Spigel's 'Music Mouse' (http://retiary.org/ls/programs.html) in Computer Music by Nick Collins. Where “querty and mouse control sets in motion music sequences from single- to multi-note patterns”. This is a program that uses the computer keyboard and mouse as controllers as I have just described in terms of SuperCollider. I was interested by the fact it was created in 1986 as it shows this idea has been developed for a long time.

I found an interesting website online that shows how you can control SuperCollider using an android phone by sending OSC messages over the network:
https://github.com/glastonbridge/SuperCollider-Android/wiki/How-to-control-SC-Android-remotely

MIDI:

MIDI (musical instrument digital interface) are not actual sounds. MIDI is information about a sound that can only be heard once it has been synthesised. This is useful as sound can be synthesised to seem as though it is being played by a variety of instruments and also makes it fairly simple to edit information.

I tried connecting a midi keyboard using course material starting with:

MIDICLIENT.init which starts you off by giving a list of available MIDI devices.

MIDI notes range from 0-127 as there is usually a 7bit value range. This is converted into a 0.0 or 1.0 giving an appropriate amplitude control through the use of \vel.

We also looked at how a SynthDef can be created and then assigned to MIDIIn.noteOn. This means that you can define and shape the sound you want to create and then assign that to the MIDI information inputted.
I looked online for some inspiration as to some of the interesting outcomes of using MIDI with SuperCollider.

I found this video demonstrating a program written in SC called 'The Grid'. The MIDI is being controlled with a launch pad:



I also found this one:



His description of it: “Physically, it's a sheet of aluminium, suspended on rubber buffers in a
wooden frame, which uses piezo sensors to detect the position and velocity of strikes on the pad surface. The electronics is based on an arduino board, and there is some python code which runs on the host computer to map the raw incoming sensor data from the arduino into a stream of midi events, including the x and y coordinates of each strike.
These events are then picked up by supercollider and used to play sounds.”

GUI:
GUIs are user interfaces that allow users to interact with computers and other electronic devices through images (such as icons, scroll bars, windows) which are usually used in conjunction with the mouse click.

In SuperCollider the GUI class (which can be explored using [GUI] cmd+D) 'provides a means of writing cross platform GUI code'. We are given the power to code our own GUIs to create an interface for our SC projects. These can include things like sliders, knobs, buttons, drop down boxes, colour schemes and labels.
If you press shift + cmd+ N you can see a GUI which shows you the different widgets. You can click on the 'new window' button and then click on the different GUI options for example SCNumberBox and drag it into the window and see what that number box graphic looks like.

There are two different GUI implementations; OS X('cocoa') for mac and SwingOSC ('swing') which are java cross platform classes used with windows. The fact that the widgets are cross platform helps to give SuperCollider a distinctive look that is easy to recognise.

I spent a lot of time looking at code examples of different GUI code to get an idea of the possibilities for creating an interface especially when trying to create my own synthesiser. I found the widgets gave a lot of possibilities for different ways that someone could interact with something such as a synth. I found that there were a lot of colours that could be used for customising dials, backgrounds and sliders and that you could choose where about in the window you wanted to place different widgets and you could use SynthDefs to assign the sounds you want to be manipulated to the different GUI parts. I found it exciting seeing the code turn into something that could actually be manipulated by people using an interface but I also found it challenging especially placing things in the right place in my main window.
This is a print screen of my GUI:
\



The back ground changes colour. I liked the look of it but it still needs a lot of improvement. I didn't know how to add labels underneath the knobs and there is too much space at the bottom. I also wanted it to have a title and in general it needs to be more obvious what it does. The last 2 knobs on it don't work properly either. I am going to work on it and then repost it in a later entry.

I found the help files useful and think that I probably just need some more practice. I decided to look into why there is a fairly basic uniform GUI library.

I read the GUI chapter of David Cottle's ' Computer Music with examples in SuperCollider 3' and found that:

“When first building on an idea it is far more efficient for me to work with a non graphic (code based) synthesis program. For that reason (and also because I tend toward automation that excludes external control) GUIs are the last consideration for me. That's one of the reasons I prefer working with SC: GUIs are optional.”

It seems that he is quite a code purist and seemed quite negative when writing about GUIs. He does go on to give quite a few detailed GUI examples which are useful to see. He ends with “But I think often too much programming time is spent on user interface. Wouldn't you rather be working on the music?” which goes to prove that he doesn't want GUI to be too much of a focus.

I looked online to see if many people supported his view. I found a website www.creativeapplications.net and read “So far SuperCollider has a reputation for being primarily a text based language without a substantial GUI, and there have been many interesting works done that treat code as an art-form in itself and revel in the use of pure code as an interface.”

It made me think of a TOPLAP live code concert that I went to in which they projected the code they were writing live up onto a screen for everyone to see and it was really fascinating. By projecting it it showed that they wanted that process to be appreciated as well as the final outcome.

They had a link to a SuperCollider Flickr group: http://www.flickr.com/groups/supercollidergui/pool/

To this they were calling for people to send designs for standard elements in and then the online SC community would discuss the pros and cons. I clicked on the discussion link and couldn't see anything there so I am not sure if any of the designs were used but this may just be because I don't have a flickr account.

It seems a lot of Super Collider users do see their code an art form as well as the sounds that they output. I found a nice article on the internet in which the author writes:

“The thought process needed to imagine a desired result, being able to see beyond an existing solution and come up with something new, even the basic initial thought of "this is what I want the computer to do" is an entirely creative one. Inspiration is needed to want to create something. This has roots in creativity, something all the best programmers have, even require. What makes someone want to write a program? They want the computer to do something. That is pure creativity, often as a creative solution to a practical problem”

http://r.je/is-programming-art.html

I personally think it is important to have the option to use GUI as it increases the usability and makes makes interactivity more enjoyable for people that maybe have less interest in the code behind the sound but the person that wrote the code can still gain satisfaction that they have created something that someone else is able to use and gain enjoyment from.