INTERNSHIP - Create a benchmarking tool for Speech-to-Text solutions

Topic

In most trains, conductors can relay new information to passengers using the audio systems. However, if a passenger misses this announcement, or can't understand it very well (e.g. they are wearing headphones, there's a language barrier, they are hard of hearing, ...), the passenger might miss crucial information. For this reason, Televic GSP is currently developing a speech-to-text engine, which automatically transcribes these passengers calls and shows them on the train displays.Since, this is still in active development, our model frequently changes. To determine the accuracy of each iteration, we would require a benchmark tool. This benchmark tool would allow us to plug in our model, and some labeled test-data, and then display useful metrics of the model. Useful metrics could be the word error rate (WER), the character error rate (CER), the accuracy on certain types of words (e.g. locations), the time it took to transcribe audio, etc... The goal of the student would be to:

* If this is taken as an internship:

** develop a benchmark tool, in which our model can be plugged in.

** create a representative test dataset from a set of recorded passenger files.

* If this is taken as a thesis (or the intern is working very quickly):

** investigate useful metrics to output

** allow third-party models to be plugged into the benchmark, so a useful comparison can be made.

If you are interested in this topic, please also register this on the Televic website at: https://www.televic.com/en/careers/internships-and-students so we can confirm the topic is still available.