externals

[Randall Lee Reetz]

Richard,

I have an aversion to solutions that are stack or developer specific.  
I will do them when all else fails.  But, if a functionality is  
useful to me it is most certainly useful to other developers... and  
users!  I don't measure performance so much by functionality-per- 
mips... instead, I look to how universal a solution is... how many  
developers will be able to do what I have done... because of what I  
have done.  That is the real measure of productivity.  That is the  
real value of a user-level development environment like Rev (and  
other xTalks).  That is the real value of a constant effort to  
generalize solutions introduced as concepts and one-offs and  
subsuming these generalities into the very fabric of the computer.   
What took scientists to do last year should be what it takes my  
mother to do this year.  Else we are all just making our monthly Saab  
payments (the priesthood that becomes of consulting).

So, of course I could (and have) written folder-delta monitoring code  
(though I am always astounded by how clean, sparse, and elegant is  
the code written by you and others with a deep programming  
history!).  But that doesn't really help others do the same.   
Programming is already an incredibly inefficient art.  When you add  
in the amount of development repetition happening around the world...  
well it is a wonder we are not in an even bigger economic crises.

It is charming when a modern western post-industrial person can learn  
how to weave a basket or make their own cheese, but the economy would  
collapse (to the level of pre-industry) if we all took up practices  
so quaint.  At a human level, we have a tendency to over-value the  
manual arts and undervalue the cold efficiency of automation.  The  
manual arts are, after all, so much more attractive and human!  Yet  
it is the west's concentration on automation that allows such quaint  
recreational forays into the manual arts.

This lesson needs to be learned in computing.  No Henry Ford has yet  
appeared in the development of soft logic.  Too often, when creating  
code, I am seduced by my own ego: "My god, look at what I have  
done... I have created this something from nothing!"  But what really  
matters is not how many people can use my code, but how many can do  
what I have done and create a software solution from scratch (and how  
much complexity that software can handle easily).

This is the rhetoric behind most of my posts... a drive towards  
greater and greater automation.  When we have the right tools, we can  
spend more of our time concentrating on more and more complex goals  
and less time trying to sharpening the chisel.

I noticed a thread starting this morning on charting tools... before  
I could upload a sample script, someone mentioned an external that  
simplified the process of generating charts, then another post hinted  
towards a future built-in affordance (part and parcel to Rev!  Thank  
you Rev!).  That is a perfect example of the evolution of automation  
of which I am speaking.  Of course there are times when such  
automated affordances don't do what I need them to do, and I am  
forced to go it alone, but for the vast majority of uses, the  
automation allows the developer to ignore details that are not  
specific to the larger goal driving their current project.

This community acts as a library of code... sort of... and that is  
part of the process.  People mention things here on this list, they  
seem esoteric at first... but the seed gets planted... the meme...  
and eventually down the road, it may become a blessed function or  
property or message and achieve a level of automation only possible  
when logic is imbedded directly into the development environment or  
OS itself.  There are many examples where full on software products  
(and even OS behaviors) can trace their start to XTalk hacks or  
prototypes.

Thanks, Randall





On Jan 21, 2009, at 11:28 AM, Richard Gaskin wrote:

> Randall wrote:
>
> > Richard wrote:
> >> I've posted the example to RevNet - in Rev, choose
> >> Development->Plugins->GoRevNet, and in RevNet look for
> >> "4W Folder Watcher" in the Stacks section.
> >>
> >> This modest example lets you pick a folder and it checks
> >> for changes to files (name, size, additions, deletions).
> >> It also provides a slider to govern the interval between
> >> checks so you can experiment to find the best balance
> >> between system overhead and responsiveness.
> >
> > Richard, how big is the folder you are monitoring?  How nested?
>
> Not nested at all, as I noted in my post.  As currently written, it  
> checks only for file changes in the target folder.
>
> > I have written similar code of course.
>
> Of course.
>
> > But it doesnt scale.
>
> True, but just about everything in computing involves tradeoffs.   
> The method that scales least efficiently at runtime may be the most  
> efficient one to develop, and the one which scales best may be cost- 
> prohibitive to develop.  The key is to find the most appropriate  
> balance based on the specific needs of a given application, and the  
> budget driving it.
>
> For the low cost of just a few minutes' time, polling in Rev will  
> give acceptable performance in some cases, and perhaps unacceptable  
> performance in others:
>
> On a folder with about 5000 files, it takes more than 400ms on a  
> 2.16GHz Mac. But with the 250 files in my Preferences folder it  
> takes only about four milliseconds, and with about a hundred files  
> it takes less than one millisecond.
>
> I have many folders with fewer than a hundred files, and very few  
> with more than a thousand; whether it's worth the orders-of- 
> magnitude greater cost to develop a solution that can more  
> gracefully account for unusually large sets of files can only be  
> determined by the specifics governing the project at hand, the age- 
> old tug of war between scope and budget.
>
>
> > And by definition it isnt real time.
>
> Define "real time".
>
> By definition all event messages will be sent after the fact since  
> they describe something that just happened, and many are derived  
> from polling (devices, processes, etc.) which makes them even  
> slower.  The only question is how long after the fact will be  
> acceptable for a given use.
>
> It's of course more efficient to poll as deep as possible, and in  
> the case of file changes it may be that digging into the kernal or  
> even the storage device driver with Assembler will yield the best  
> performance. But that sort of work can get quite expensive, so it  
> seemed at least worthwhile to find out how responsive this can be  
> in a very-high-level 4GL like Rev.
>
> Many file changes are the result of a user action, either dragging  
> a file into a folder, saving a file from within an application,  
> etc. Given that people are much slower than computers, anything  
> that happens in response to a user event will have a lot of leeway  
> in terms of perceived responsiveness.
>
> True, that doesn't account for all possible cases, such as  
> applications or processes that generate files on their own.  But it  
> may be useful for some folks anyway, so I posted it.
>
> The nice thing about doing this in Rev is that the engine is pretty  
> fast with string parsing, such as the result one gets back from  
> "the detailed files".  The bigger latency is obtaining that list  
> from the OS, which any polling method will be subject to (and hence  
> I appreciate your desire to move beyond polling, but which means  
> beyond Rev).
>
> In short, polling "the detailed files" may prove suitable for a  
> good many uses, if not yours perhaps for others here.
>
>
>
> Forgive me if you've already described this, but what exactly does  
> your application need to do that requires the shortest possible  
> notification time for file changes?
>
> And given that task, what would be the threshold of acceptable  
> performance to satisfy your project's goals?  Would the OS APIs  
> suffice, or will you need to go deeper?
>
>
>
> Innovation is rarely trivial to accomplish. Visionaries must  
> usually bind their vision with sweat to realize their goals.
>
> Jan presented a reasonable decision tree for your project's success:
>
>     As for having any suggestions? How about these:
>     - learn C and even some x86 assembler
>     - learn more about file systems
>     - learn more about operating systems
>     - research how to tap into the I/O parts of the kernel
>     OR
>     - try to find and hire someone with the above skills
>       to implement this for you
>     - have your checkbook ready and figure in a hefty
>       maintenance cost of the code when planning the project
>
> Keep us posted when you release.  Sounds like an interesting project.
>
> --
>  Richard Gaskin
>  Fourth World
>  Revolution training and consulting: http://www.fourthworld.com
>  Webzine for Rev developers: http://www.revjournal.com
>
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your  
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution
>