MEA10732 miniproject (RhythmBox)

Introduction

In this miniproject we have chosen to present concepts from the course which have been used in our semester project which is entitled “RhythmBox”. This has resulted in a miniproject closely related to our semester project. Because of this, the title of the miniproject is the same as our semester project.

In this documentation, the miniproject will be described according to the following topics:

1. Concept design
2. Interface prototyping
3. System design
4. Conclusion

1. Concept design

Our semester project is based on a project proposal from Danmarks Radio called Musikariet. Initially the goal was to create a concept for teaching children about music. From the beginning we wanted to combine sound and graphics in the creation of such a concept.

The first concept that was developed was a mixture of  the classic computer game “Pocket Tanks” and a music game. The purpose of this game was to create a team-player based application. In the initial scenario the number of players were two. One of the players controls the angle of the gun on the tank by completing a rhythm related task. The precision of the player when solving the task is translated to a precision in the angle of the gun. The other player controls the power of the gun by completing a melody related task. See a concept sketch by clicking the link below.

Tank music game

A few iterations on this concept was created. After this, another concept was developed. The idea was that two users collaborate using a piano on the wall to play a melody. The concept was named PianoWall. See the demonstration video below.

The PianoWall concept was discarded because it turned out to be more difficult than first assumed. Another concept was developed, a collaborational rhythm game where the users claps their hans and taps their feet. See the video below.

Different patterns for the tapping pads are shown in the pictures below.

Rhythm game pad pattern 1

Rhythm game pad pattern 2

Rhythm game pad pattern 3

The collaborational rhythm game concept was discarded because we didn’t come up with a reasonable solution for adding visuals to the system. However, an iteration of this concept ended up developing into the concept used in the semester project. This concept was called RhythmBox and is a one-player rhythm game which used graphical and audible feedback. The user sees and listens to a rhythm, afterwards the rhythm is replicated only with visual support. Sketches of the graphical user interface of different iterations of the concept are shown below.

Graphical user interface sketch 1

Graphical user interface sketch 2

On the background of these sketches it was chosen to simplify the concept. The system can be used by one player at the time instead of two and the visual representation of the rhythm events is simplified. See the screenshot from the actual application below.

Screenshot from the actual application

2. Interface prototyping

RhythmBox was created by using Bill Verplank’s design framework. See the pictures below.

Bill Verplank's design framework 1/2

Bill Verplank's design framework 2/2

1. IDEA: Enhancing the process of rhythm teaching for the children aged 7-13.
2. METAPHOR: Visualizing and projecting the rhythm on a big screen.
3. ERROR: Usually the children don’t have any visual support in a teaching situation.
4. SCENARIO: The setup could be used as an interactive exhibition at Musikarium.
5. MODEL: Using a simple input; a glow with a piezo sensor and a CUI32 board. Getting a visual feedback in terms of graphical and audible representation of the rhythm.
6. DISPLAY: Using a big screen with graphical and audible representation of the rhythm.
7. TASK: First listen to the rhythm and then try to replicate the rhythm.
8. CONTROL: Using the clapping gesture to master the task.

3. System design

Based on the concept described above, an application was implemented in Java. The application is structured in three different modules, following the model-view-controller architecture. See the figure by clicking the link below.

RhythmBox system design diagram

1. Audio Module (Model): Manages the audio files and sampling.
2. Graphic Module (View): Manages the graphical part of the application.
3. Input Module (Controller): Manages sensor input using a CUI32.

The input module will be described here, as it uses concepts from the CDPT course. The module handles user input via a sensor glove. The clapping gesture was chosen to be the type of interaction for RhythmBox. Furthermore a film piezo sensor was used to detect the clapping input and sewn into the glove, in order to make it more intuitive  for the user. See the image below.

Glove with a piezo sensor sewn into the palm

The BASIC code used to get the input from the glove using the CUI32 is listed below:

10 dim a as pin an1 for analog input  
20 while 1 do  
30   if a>600 then  
40     print "1"  
50     sleep 250 ms  
60   endif 
70 endwhile

The Java code to handle input via USB serial communication is listed below:

public class InputManager implements SerialPortEventListener{

public void serialEvent(SerialPortEvent event){
     switch (event.getEventType()) {
	case SerialPortEvent.OUTPUT_BUFFER_EMPTY:		  
           break;
	case SerialPortEvent.DATA_AVAILABLE:		  
           // we get here if data has been received		  
           byte[] readBuffer = new byte[20];
		try {
		// read data
		while (inputStream.available() > 0) {
			int numBytes = inputStream.read(readBuffer);
		}
		//Parse the data and trigger events here.	                
	} catch (IOException e) {}
		break;
	}  
}
}

As we can see in the piece of code above, the javax.comm library provides a class called SerialPortListener, and uses a method call serialEvent, which listens on the serial port and checks its state. It runs a piece of code when new data is available.

4. Conclusion

Through the development of the concepts presented in this miniproject experience have been gained on how to develop concepts, and how to realize them. When developing a concept it is important to start off by evaluating a simple version of this concept, to avoid problems caused by involving too many variables.

Originally the concept included two players which should collaborate. When testing the prototypes internally in the group it became clear that the implementation of a collaborational concept would not be realistic in proportion to the other hypothesis stated in the semester project. The simplified concept was used to test two hypotheses in the semester project:

1. H1: The error rate when replicating a rhythm is lowered significantly when the first of two runs is enforced to be a listening run as compared to doing two runs of replicating.
2. H1: The error rate when replicating a rhythm is lowered significantly by using both audio and adding visual support.