Synchronisation of sound and vision

[Devin Asay]

Tore,

You can do audio recording on Mac now using the mergMicrophone library. It works great, and I believe is available in every edition of LC, including Community.

Devin

> On Feb 12, 2020, at 12:11 PM, Tore Nilsen via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Devin,
> I haven’t used callbacks much, and so far I haven’t run in to any problems. If missing callbacks is still an issue, then I agree with you that setting startTime and endTime is the best option. I use this method in a small application I have made for myself where I write comments to audio files handed in by my English students. They can then control playback of the segments I have commented on by clicking links in the field that shows the comments. The lack of audio recording capability on Mac has forced me to use written feedback where I otherwise would have preferred using two players and audio feedback.
> 
> Regards
> Tore
> 
>> 12. feb. 2020 kl. 19:57 skrev Devin Asay via use-livecode <use-livecode at lists.runrev.com>:
>> 
>> Tore,
>> 
>> I would agree if callbacks were 100% reliable. I have tried them in the past and found that in some cases they were missed. I never had any trouble when using time indices. But I should say that I haven’t needed to do this for several years, and the callbacks in the new player object might be completely reliable.
>> 
>> In other ways creating time indices makes your application more flexible, however. It’s dead simple, for instance, to set up an application where you can click on a line of text and play just that line. Set the startTime, set the endTime, set the playSelection to true, start playing. Done. That would be a little more challenging if all you had was callbacks.
>> 
>> One of the great things about LiveCode is that there is almost always more than one way to do what you want.
>> 
>> Regards,
>> 
>> Devin
>> 
>> 
>> On Feb 12, 2020, at 9:55 AM, Tore Nilsen via use-livecode <use-livecode at lists.runrev.com<mailto:use-livecode at lists.runrev.com>> wrote:
>> 
>> Using callbacks negate the need to fiddle with duration or  timescales and start or stop times. It uses the sampling intervals as is, regardless of time. In my opinion it is much easier than trying to calculate start and end times. You can easily handle large audio/video files using callbacks. I would recommend using one file per poem though, this simplifies the handling of the messages sent from the player. You can basically use the same message for all files, resetting a counter variable each time you load a new file to handle with line you would like to act upon.
>> 
>> You could also store the callbacks for each audio file in a text file and set the callbacks as a part of the handler used to load each audio file.
>> 
>> Regards
>> Tore
>> 
>> 12. feb. 2020 kl. 16:49 skrev Devin Asay via use-livecode <use-livecode at lists.runrev.com<mailto:use-livecode at lists.runrev.com>>:
>> 
>> Graham,
>> 
>> Take a look at the duration and the timeScale properties of player objects. By dividing duration by timeScale you get the length of the video in seconds.
>> 
>> 
>> put the duration of player  “foo” / the timescale of player  “foo” into totalSeconds
>> 
>> What you are contemplating is very doable, but you’ll have to do a fair amount of work to do to get the synching right. You can take one of several approaches:
>> 
>> - Calculate times as above to predict when to show/highlight the next line. Can be tricky with long video files and rounding errors.
>> 
>> - Check the currentTime property of the player to determine the startTime and endTime of each spoken line, and set the playSelection of the player to true. When the played segment ends, immediately load the following start and end times and play again. Something like this, from memory:
>> 
>> set the startTime of player “foo” to 444
>> set the endTime of player “foo” to 999
>> set the currentTime of player “foo” to the startTime of player “foo”
>> set the playerSelection of player “foo” to true
>> start player “foo"
>> - Break up the video or audio file into separate files, one line per file, then play each succeeding file when the previous one reaches its end. The playStopped message is your friend here.
>> 
>> Like I said, it’s doable, but takes a bit of thought and planning, creating segment indexes, that sort of thing.
>> 
>> Hope this helps.
>> 
>> Devin

Devin Asay
Director
Office of Digital Humanities
Brigham Young University