- JPB-UG-1: Scrabble Referee: Word Recognition Component (Liam Brummit)
- JPB-UG-2: Scrabble Referee: Move Detection Component (Mitchell Whitehouse))
- JPB-UG-3: Acoustic Guitar Hero (Ryan Bibby)
- JPB-UG-4: Finger Mouse (Chris Lake)
- JPB-UG-5: CHiME Web-based Audio Browser and Annotation Tool (Ryan Temple)
JPB-UG-1: Scrabble Referee: Word Recognition Component
JPB-UG-2: Scrabble Referee: Move Detection Component
Description
Scrabble is a word game in which two players take it in turn to place letter tiles on a board to produce words. The game uses a scoring system that is based on the value of the letter tiles and the position they are placed.
The idea of these fun but challenging projects would be to create a Scrabble `referee’: a system which would employ a USB camera to oversee a game of Scrabble being played between two people on a physical board. The system would keep track of the game, automatically computing the score of each play and making sure that words are correctly spelt etc.
To make the project manageable within a year, the system has been split into two independent components: JB-1 will take a still image of a region of the scrabble board and use optical character recognition techniques to identify the word appearing in the image. JB-2 will take the streaming video sequence and use computer vision techniques to detect changes in the board indicating that a move has been played.
Requirements
- No previous computer vision experience required, but good Java skills or some C/C++ programming experience are essential. C/C++ programmers will find OpenCV useful, though the project can be equally well developed in Java
Initial reading
- Scrabble page on Wikipeida - here
- Jain (1989) “Fundamentals of digital image processing” Prentice Hall - useful text book, can be found in the library. *OpenCV computer vision library - see here.
- Bradski and Kaehler, (2008), “Learning OpenCV”, O’Reilly - details here.
- GOCR - Open-source character recognition software here.
JPB-UG-3: Acoustic Guitar Hero
Description
When learning to play an instrument it is usually very important to have a teacher who is able to tell you what you are doing wrong. Hearing your own mistakes is often difficult because so much attention is being spent on reading music and moving fingers. Further, without feedback, practise quickly becomes repetitious and dull. An interactive computer system is needed to make practise fun and to motivate the learner!
This project will build a system that uses acoustic analysis to perform the music teacher’s role. It will focus on fingerstyle acoustic guitar. Fingerstyle (or finger-picking) guitar is a style of playing that uses the fingertips to pluck the strings and involves elements of country, blues and ragtime. Although it can involve strumming chords, most early exercises focus on learning to coordinate the movement of the right-hand fingers which sequentially play patterns of individual strings. The difficulty for the beginner is in developing an even rhythm and getting a `clean’ and balanced sound from each string.
Although general music transcription and analysis is a challenging research area, this project should be able to achieve good results for a number of reasons: i) the system will not be transcribing an unknown piece of music, but instead, it will be comparing a performance against a known musical score; ii) practise exercises have a simple structure; iii) the exercise do no necessarily have to contain chords; iv) there is only one instrument playing and there will be little or no background noise.
The basic system might just check, i) the timing of the notes and score them according to how regular they are, ii) the pitch of the notes to make sure that the player is plucking strings in the correct order. More advanced version could also look at the amplitude of the notes and the sound quality to make sure that the notes are been played evenly and cleanly. The output could be in the form of a score presented at the end of the exercise, or better, a running indicator showing the quality as time progresses. The output could even be presented graphically and embedded into a game that would help motivate younger learners.
Requirements
- This project is suitable for ACS and HLT students or DataComms students with good programming skills. Experience from the Speech Processing and Speech Technology will be an advantage, but not required.
Initial reading
- Fingerstyle guitar page on Wikipeida - here .
- Klupuri (2003) “Automatic transcription of music”, Proc. Stockholm Music Acoustics Conference, SMAC03 – available here .
- Bello, Monti and Sandler (2000) “Techniques for automatic music transcription”, Proc. International Symposium of Music Information Retrieval – available here .
JPB-UG-4: Finger Mouse
Description
This project would attempt to implement a Finger Mouse System as described in Ko and Yang (1997). This system uses a video camera to track a user’s fingers allowing the user to interact with the computer through simple gestures.
The simplest interaction would be to simply use the finger as a pointer – similar to the basic operation of a computer mouse moving in 2D. A more sophisticated system, such as described by Ko and Yang, would recognise simple thumb and finger gestures as a mouse ‘click’.
The system could be demonstrated through the implementation of a simple drawing program that would be controlled entirely by the finger mouse. i.e. the user would use their finger to interact with interface controls that are drawn on a physical piece of paper.
Requirements
- No previous computer vision experience required, but sufficient C/C++ programming experience to allow the use of OpenCV would make the real-time aspects of the project easier to achieve.
Initial reading
- Ko and Yang - “Finger mouse and gesture recognition system as a new human computer interface”, Computer and Graphics, Vol 21, No 5, pp 555-461, 1997
- Jain (1989) “Fundamentals of digital image processing” Prentice Hall - useful text book, can be found in the library.
- OpenCV computer vision library - see here.
- Bradski and Kaehler, (2008), “Learning OpenCV”, O’Reilly - details here.
JPB-UG-5: CHiME Web-based Audio Browser and Annotation Tool
Description
A web tool to enable browsing and (possibly) annotation of a large corpus of audio recordings that have been collected as a part of the CHiME project.
If you are interested in this project please make an appointment to come and see me before selecting it.
Requirements
- Experience with Javascript and/or Flash.