Voice Comparison in LiveCode?

Paul Dupuis paul at researchware.com
Thu Dec 28 20:01:00 EST 2017


I have had an interest in getting a speech-to-text component created for
LiveCode, either as an old fashion External or, these days as a LC8/9 Widget

My skill at grabbing Sphinx and building it and figuring out how to wrap
and LCB interface around it is poor enough that it would take me a long
time to do.

Since this has come up, I'd like to pose two questions to the list:

1) is there someone out there with the expertise to create an LC8/9
Widget that wrapped Sphinx who would like to take this on?
AND
2) Are there others out there interested in having such a widget that
might co-fund this with me?

And I suppose a 3rd question to LiveCode, would this widget be a
candidate for the Feature Request crowd funding thing LiveCode has done
in the past?

Paul Dupuis
Researchware

On 12/28/2017 7:40 PM, Sean Cole (Pi) via use-livecode wrote:
> Hi Peter,
>
> I'm going to recommend to you this code source from CMUSphinx. Head for the
> main Github source and look at the Android build (link also below). Because
> this is in Java, you should be able to write something using the FFS in
> LCB. Ignore all the Gradle stuff. Just load in
> the pocketsphinx-android/src/main/java/)edu/cmu/pocketsphinx files and bind
> to the 'SpeechRecognizer' class and then the various handlers. Also check
> out the Sphinx4 project, particularly the allPhone stuff (
> https://github.com/cmusphinx/sphinx4/tree/master/sphinx4-core/src/main/java/edu/cmu/sphinx/linguist/allphone
> )
>
> https://cmusphinx.github.io/wiki/phonemerecognition/
>
> https://github.com/cmusphinx/pocketsphinx-android/
>
> Sean Cole
> *Pi Digital Productions Ltd*
>
>
>
>>> i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in)
>> that allows a user to practice speaking a set of words.  The app plays a
>> sample of a word and the user then tries to say the same word.  So far the
>> app can play sample words and capture the user's attempts for the same
>> words.  The sample words and user attempts are uncompressed WAV files.
>>> I'm trying to find the code to do the comparison of 2 WAV files.
>> Ideally, the code will be in the following formats (best first):
>>> 1. LiveCode
>>> 2. Pseudocode
>>> 3. Other code (Python, Java, C++ etc.)
>>> 4. Academic papers
>>>
>>> I'm considering 2 general methods:
>>>
>>> a. Compare 2 voice clips directly
>>> b. Convert 2 voice clips to text (using voice-to-text) and then compare
>> the words in text format
>>> Note that Ali Lloyd from the LiveCode team has developed various things
>> to help.  However I've hit problems as follows:
>>> a. Ali has wrapped a standard Android sound library that compares 2 WAV
>> files and gives a percentage match. However the comparison is either far
>> too forgiving or far too strict, i.e. highly unreliable.
>>> b. Ali has wrapped a standard Android voice-to-text library which works
>> well with the devices he's tried it on.  However, the Amazon tablets do not
>> support this Android library!
>>> Given the two developments from Ali both relied on preformed blackbox
>> code (Android Java libraries), i may have to implement a comparison
>> algorithm from scratch. A solution that's completely in LiveCode would have
>> several benefits:
>>> i. it may work!
>>> ii. it may work cross-platform
>>> iii. it may be understandable!
>>>
>>> General reading around this subject produces recommendations such as
>> using FFTs (Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral
>> Coefficient), etc. but I can't find anything that gives an end-to-end
>> method, from sound in to comparative score out!
>>> Any help with this would be gratefully received!
>>>
>>> Peter
>>> --
>>> Peter Reid
>>> Loughborough, UK
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>





More information about the use-livecode mailing list