Data synchronization

Andre Garzia andre at andregarzia.com
Mon Sep 3 19:55:00 EDT 2007


Hello Alex,

I don't know much about Valentina but I decided to chop in this thread
(which I read and learned much from). If you make a litte server using
Valentina and an HTTP server on front, I'd advise you to work the following
way:

Define a clear HTTP based API for accessing your server. Try to use whats
being called "REST" which is a fancy name for using URLs and HTTP Methods to
simplify interfacing with a server, it's way easier to work with a RESTful
server than working with SOAP.

script your GET requests to acquire data from the DB. You can make your URLs
match your database design. for example, you're building a documentation
database:

  GET  /  (gets all records)
  GET  /category  (gets the categories available)
  GET /category/installing  (get records with category "installing")

The idea is to make sane URLs that are both human readable and machine
processable. If your entries have a timestamp marking the last modification,
you can create a URL such as:

  GET /recentchanges  (get the newest records)
  GET /modified/2007/10/06  (get records modified since 2007/10/06)

This will make easier to detect changes in the DB and replicate the changes
between clients. Use PUT calls to insert data onto your database, you can
put the content of the request as a XML with the fields and data for your
DB. If you're felling bold, you can use smart urls for insertion too.

  PUT /  (all content for the record is inside the request)
  PUT /category/installing  (the "category" for this request is obtained
from the URL)

The above snippet is not as useful as the GET requests, inserting data is
easy, querying data is what makes your system easy to extend and implement.
To modify a record use the POST method. Here it is a good idea to use fancy
urls such as:

  POST /id/1234 (changes record #1234)

Or you can insert all data into the request. The idea of URLs for modifying
data is to be able to query and modify batches of records with a single
request. For example suppose your documentation system has entries tagged as
"draft" or "publish", by switching this status setting you make some FAQ
available to the end user. You can have a request such as:

  POST /drafts/user/Soapdog  (will change all drafts by user "soapdog")

In this request you can simply add the XML to update the draft field to
publish. This can easily be done with SQL queries and in the end, this will
of course become a SQL query somewhere inside the system. The question you
must ask yourself is, do I want to put SQL in my HTTP Requests or should I
build my own XML/URL Dialect to make queries?

The second takes time but pays in the long run since you can tailor your
software to the specific needs of your system/workflow. This option will
make you have the better tool for doing your job since you created a DSL
(domain specific language) that is fine tunned to your job instead of using
the general purpose SQL.

Now getting back to the syncronization problem. Using the smart URLs you can
discover what's been changed in the database and fetch them easily. By
fetching just what you need, you reduce your bandwidth usage and you also
reduce the load on the RevOnRockets server which is single threaded anyway.
You can also make a little MD5 field on your records that you can use as a
locking sytem so that users don't overwrite each other. This MD5 record must
be sent with each POST request, if they don't match the one in the central
database, it means someone else altered the records in the meanwhile and you
should refetch that record and revise your changes.

does this help?
andre

>
>
>
>



More information about the use-livecode mailing list