Autosaving stacks corrupted on network drives

[Richard Gaskin]

Brian Milby wrote:

 > The stack file isn’t a package.  If using an old format, then it may
 > have a resource fork.

Anything in the resource for was put there by the developer.  Having 
been born on Unix. MetaCard (the LC engine's original name) provided 
functions for using resources on Mac, but never relied on them in its 
engine or IDE.

As for Dropbox and other cloud storage services:

Even without multiple simultaneously users modifying the file, this is a 
pervasive problem not at all limited to LiveCode stack files.  Indeed, 
LC's saves are pretty sparse and rewrite the entire file, as opposed to 
more complex paging systems like HyperCard and SuperCard used.

One of the more complex paging data formats we LC folks use frequently 
is SQLite,

Tons of questions about that all over the Web:
<https://www.google.com/search?q=problems+with+SQLite+on+Dropbox>

One software vendor has gone so far as to explicitly advise their 
customers not to work on SQLite files directly in cloud-synced folders, 
recommending copying them there for backups only:

    Caution

    We strongly recommend that you do not work with a project inside
    the synchronized storage area, meaning that you never open a project
    file from right within the cloud folder (e.g. your Dropbox folder).
    Instead, we recommend to copy the project to a location that is not
    synchronized e.g. to your desktop and copy it back to your cloud
    service when you are finished working on it.

    Especially when working on a Mac computer we cannot guarantee that
    project files are damaged by your cloud service starting a
    synchronisation process while the MAXQDA project is opened, thus
    possibly corrupting the project.
<http://www.maxqda.com/faq/a-13-can-i-store-my-maxqda-projects-in-cloud-based-services-like-dropbox>


This is reinforced by a "superuser" in the Dropbox forums:

    "First, it's suggested that you not use database files within
     Dropbox. Conflicted copies are a likely result (as someone else
     posted about recently as well).
<https://www.dropboxforum.com/t5/Syncing-and-uploads/SQLite-file-sync/td-p/193051> 



On the SQLite list someone wrote:

   "If Dropbox decides to take a copy while SQLite is in the middle of
    processing a transaction, you will temporarily have a copy of the
    database with a partially-processed transaction on all the computers
    which have access to that shared area.

   "Dropbox doesn’t understand that the database file and the journal
    file go together, even if they’re in the same folder.  And in order
    to stop one user from hogging its servers there’s sometimes a short
    delay between when it updates its copy of one file and when it
    updates its copy of another file.  So it’s possible for one computer
    which has a copy of the database to have a newer database file than
    its journal file, or vice versa."
<http://sqlite.1065341.n5.nabble.com/Sqlite-Dropbox-td95173.html>



A user in a discussion on this problem in the Xojo forums noted succinctly:
"It does not need a Data corruption, it just needs 1 Network related 
delay of your sync and you loose Data."
<https://forum.xojo.com/34277-sqlite-database-on-dropbox/0>


Since LC writes stack files whole when saving, I'm not sure why this 
would be as much of a problem as it appears to be with paged formats.

I wonder if LC's pre-save step of making a backup copy of a file somehow 
throws off Dropbox's sync so that it's tardy in then syncing the actual 
new stack file.  Perhaps the extra network traffic may be enough to 
throw things off, per the Xojo user's point.


PS: FWIW with my own use syncing my LC Plugins folder and work files, 
I've seen far fewer "conflict files" since I switched to Nextcloud after 
having used Dropbox and Google Drive.  Maybe it's just luck, or maybe 
it's that most of the time I'm syncing over the local network (my 
Nextcloud box sits three feet from my main workstation).  Or maybe 
Nextcloud's use of OS-provided file-changed flags is just a bit more 
tidy than whatever monitoring method Dropbox and others use.

-- 
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  Ambassador at FourthWorld.com                http://www.FourthWorld.com