Storing and Retrieving stack data?

[Sivakatirswami]

John Patten wrote:
> Hello All!
>
> I would like to save some user (student) input out of one rev project and have it accessible to another rev project (teacher) to generate reports. The users of the project could be on a school network (LAN/WAN) or at home on the Internet. What is the best way to save the data?  What is the most common practice for saving the data for this type of project? 
>
> Possible ideas...
>
> FTP user data as individual files to a ftp server? (Run through a process of reading back all the files in the directory to create report.)
>
> Send user data directly to mySQL database? (Require the mySQL database to be open to outside connections - direct connect to mySQL?)
>
> Send user data as a post process, using a CGI script to send data to mySQL database on server?
>
>
> The first idea seems the easiest, but I'm afraid could get to be pretty slow with a couple hundred files in the ftp directory. I could modify an existing file on the ftp server, however I run the risk of some data not being written to the file if simultaneous connections occur...?
>
> I have not done any work with Rev and mySQL, so if I could get away from the sql commands, all the better :-)  But, if it would be the best way to go, I'm willing to focus on that area. 
>
> Any ideas?
>
> Thank you!
>
>   

John: First, forget FTP many files, you don't need (or want) that

Assuming you don't need the relational tables side of MySQL or 
PostGreSQL, AND,

1) concurrent writes are not going to be an issue (the likelihood of two 
hits on the data container in a period < 1000 milliseconds, being 
virtually zero...)  then your very simplest option is to use a stack on 
the server with each card containing the student data. Then you 
architecture looks like

Desktop:

Student Stack
Teacher Stack

Server

CGI

Data stack

Use very simple POST and GET from in the desktop apps;
use generic functions calls in the CGI;
put all the logic in the data stack.

This will easily handle up to 3,000 students.

There are *lots* of advantages to this system: 
Most importantly, you can just copy the stack to your desktop and work 
on it, each card is a record, you view the layout in a visually rational 
way instead of having an abstract bunch of data in tab de-limited lines 
or custom props.

In our CGI you never us "go to card"; you always, name each card with an 
ID, then use "The card names of this stack" to get and index and read 
and write data to "fld x of card y"  It's very fast.


2) If concurrent writes are going to be an issues then externalize the 
data as individual .xml files, one per students. Again, use the ID as 
the file name or part of the file name.  then you do "set the 
defaultfolder to /home/public_html/student-records/ ; put the files into 
tRecords;

Your architecture then looks like

Desktop client --> cgi --> RecordsManager.rev (a stack) --> many XML files.

Again, resist the temptation to put any "logic" into the desktop client; 
let them just get and post data; resist the temptation to build out the 
CGI... just do "start using "recordsManager.rev" and use functions calls 
which are contained in the manager stack. Then the Manager stack has all 
the macho code to deal readng and writing to the individual XML files. 
If you are not a XML wizard, well, it's worth the learning curve and 
run, don't walk, to get Ken Ray's stack to help you with that and also 
you can use it on the server.

This model could actually scale to some pretty large numbers and is  
very,very fast.

CONS:
1) if you need dbase things like relational tables etc, then you need to 
go to SQL;
2) if you want to refactor the data you will need to process all the XML 
files in and then back out again to add or change nodes etc. But this is 
really not that difficult. (assumes again you have a handle on XSLT, 
with which you could refactor 10,000 XML files into new XML files in a 
second or two using simple shell like: put ("xsltproc -o " &tOutputFile 
&&tStyleSheet && tXMLfile) into tCmd
3) you *do* want to be writing good XML (use the same tag for the same 
class of data and attributes for differentiation, and don't proliferate 
many unique tags when all the data is really in the same class.. etc.)

 

This has an interesting advantage that you an also deliver the student 
records into web pages runtime using XSLT... And, what I myself  love 
about flat files systems with a clear nomenclature, if you want to get 
some info about

15368_trevor-johnston.xml

all you need to do is fetch and open the file.

Good luck!