- JPB-MSc-1: Data Visualisation for CHiME Domestic Audio Data [Group Project]
- JPB-MSc-2: Realtime Hearing Loss Simulation
- JPB-MSc-3: Serious Games - The Type-What-You-Hear Game
JPB-MSc-1: Data Visualisation for CHiME Domestic Audio Data [Group Project]
Description
As part of a funded research project we have recently collected 50-hours of audio data from a domestic family home. This data has mainly been used as background `noise’ for evaluating speech recognition algorithms, but it also provides an opportunity to learn about the characteristics of everyday listening environments. This project will employ unsupervised learning, clustering and data visualisation techniques in order to infer and represent interesting structure in the data. For example, is it possible to detect changes of state in the acoustic scene (e.g. a TV being turned on, a conversation starting)? Is it possible to detect repeating patterns in the data (e.g. a telephone ringing)? Is it possible to detect novel (i.e. previously unheard) acoustic events?
The project will experiment with traditional and `auditory’ acoustic feature extraction techniques and will start by applying conventional clustering and background modelling ideas. For an example of similar work see here.
This project is suitable for a team of 2 to 3 students. Students will be sharing the same data and some of the same analysis and statistical modelling tools but will be addressing different aspects of the problem. The exact direction of each individual’s project will be determined according to individual interests and experience.
Requirements
An interest in audio signal processing and statistical modelling.
Initial reading
- The CHiME project page
- Christensen, H., Barker, J., Ma, N., and Green, P. (2010) The CHiME corpus: a resource and a challenge for Computational Hearing in Multisource Environments. Interspeech’10, Makuhari, Japan, September 2010.
- Ntalampiras, S., Potamitis, I. and Fakotakis, N. (2011) Probabilistic novelty detection for acoustic surveillance under real-world conditions IEEE Trans. on Multimedia, 13(4), pp713-719, 2011</li>
JPB-MSc-2: Realtime Hearing Loss Simulation
Description
Hearing ability starts to decline from the age of about 40 onwards, and a large number of older people suffer from a degree of hearing loss that significantly impacts their quality of life. Hearing aids provide part of the solution to this problem but a more sustainable solution would be to engineer the environment to make it less challenging for hearing loss sufferers in the first place (e.g., reduce the amount of noise, provide better noise shielding, use more sound absorptive surface coverings, etc). A major barrier to this approach is that people with no hearing impairment have very little understanding of the difficulties faced by hearing loss sufferers, e.g. young architects, designers and planners do not pay sufficient attention to their needs.
This project aims to build a Realtime Hearing Loss Simulator. This will be a bit of software which can run on a mobile device and which would allow the user to experience the environment `through the ears’ of a hearing loss sufferer. The project will entail implementing some real-time signal processing which can filter the signal captured by the device’s microphone before relaying it to the listener. The device will also have an intuitive interface which allows the user to explore different degrees and types of hearing loss. As well as being educational, the device could potentially be used during the design and fitting of, say, work-place environments, helping designers to optimise the placement of acoustic screening etc.
Requirements
Some experience of mobile computing and signal processing would be helpful.
Initial reading
- Hearing loss demos at the Better Hearing Institute
- A Windows-based hearing loss simulator supplied by the National Institute for Occupational Safety and Health
- Moore, B.C.J, An Introduction to the Psychology of Hearing (5th Edition) Academic Press
JPB-MSc-3: Serious Games - The Type-What-You-Hear Game
Description
In order to understand human speech processing (i.e so that we can develop better hearing aids etc) it is useful to study how people mishear words in noisy situations. A typical procedure is simply to ask people to listen to noisy words over headphones and then to type what they hear. However, if you want to collect data from thousands of people there are obvious problems - most notably, how do you persuade people that they want to spend time listening to noisy words? One way of providing the motivation is to build the experiment into a simple web-based game. For example, imagine a game called `Type-What-You-Hear’ that is presented as a speed typing competition. You log into a web-site and are paired up with other competitors. You are all simultaneously presented a series of words over headphones and you score points by being the first person to type the words correctly. The game could keep a leader board and use other ‘gamification’ strategies to increase the engagement of listeners. The listeners are playing a fun online game, meanwhile valuable listening data is being collected for free.
The aim of this project is to build a client-server implementation of the Type-What-You-Hear game.
Requirements
Java programming and some client-side web programming skill e.g. HTML and Javascript