Randall Lee Reetz
randall at randallreetz.com
Wed Jan 21 14:31:13 CST 2009
I have an aversion to solutions that are stack or developer specific.
I will do them when all else fails. But, if a functionality is
useful to me it is most certainly useful to other developers... and
users! I don't measure performance so much by functionality-per-
mips... instead, I look to how universal a solution is... how many
developers will be able to do what I have done... because of what I
have done. That is the real measure of productivity. That is the
real value of a user-level development environment like Rev (and
other xTalks). That is the real value of a constant effort to
generalize solutions introduced as concepts and one-offs and
subsuming these generalities into the very fabric of the computer.
What took scientists to do last year should be what it takes my
mother to do this year. Else we are all just making our monthly Saab
payments (the priesthood that becomes of consulting).
So, of course I could (and have) written folder-delta monitoring code
(though I am always astounded by how clean, sparse, and elegant is
the code written by you and others with a deep programming
history!). But that doesn't really help others do the same.
Programming is already an incredibly inefficient art. When you add
in the amount of development repetition happening around the world...
well it is a wonder we are not in an even bigger economic crises.
It is charming when a modern western post-industrial person can learn
how to weave a basket or make their own cheese, but the economy would
collapse (to the level of pre-industry) if we all took up practices
so quaint. At a human level, we have a tendency to over-value the
manual arts and undervalue the cold efficiency of automation. The
manual arts are, after all, so much more attractive and human! Yet
it is the west's concentration on automation that allows such quaint
recreational forays into the manual arts.
This lesson needs to be learned in computing. No Henry Ford has yet
appeared in the development of soft logic. Too often, when creating
code, I am seduced by my own ego: "My god, look at what I have
done... I have created this something from nothing!" But what really
matters is not how many people can use my code, but how many can do
what I have done and create a software solution from scratch (and how
much complexity that software can handle easily).
This is the rhetoric behind most of my posts... a drive towards
greater and greater automation. When we have the right tools, we can
spend more of our time concentrating on more and more complex goals
and less time trying to sharpening the chisel.
I noticed a thread starting this morning on charting tools... before
I could upload a sample script, someone mentioned an external that
simplified the process of generating charts, then another post hinted
towards a future built-in affordance (part and parcel to Rev! Thank
you Rev!). That is a perfect example of the evolution of automation
of which I am speaking. Of course there are times when such
automated affordances don't do what I need them to do, and I am
forced to go it alone, but for the vast majority of uses, the
automation allows the developer to ignore details that are not
specific to the larger goal driving their current project.
This community acts as a library of code... sort of... and that is
part of the process. People mention things here on this list, they
seem esoteric at first... but the seed gets planted... the meme...
and eventually down the road, it may become a blessed function or
property or message and achieve a level of automation only possible
when logic is imbedded directly into the development environment or
OS itself. There are many examples where full on software products
(and even OS behaviors) can trace their start to XTalk hacks or
On Jan 21, 2009, at 11:28 AM, Richard Gaskin wrote:
> Randall wrote:
> > Richard wrote:
> >> I've posted the example to RevNet - in Rev, choose
> >> Development->Plugins->GoRevNet, and in RevNet look for
> >> "4W Folder Watcher" in the Stacks section.
> >> This modest example lets you pick a folder and it checks
> >> for changes to files (name, size, additions, deletions).
> >> It also provides a slider to govern the interval between
> >> checks so you can experiment to find the best balance
> >> between system overhead and responsiveness.
> > Richard, how big is the folder you are monitoring? How nested?
> Not nested at all, as I noted in my post. As currently written, it
> checks only for file changes in the target folder.
> > I have written similar code of course.
> Of course.
> > But it doesnt scale.
> True, but just about everything in computing involves tradeoffs.
> The method that scales least efficiently at runtime may be the most
> efficient one to develop, and the one which scales best may be cost-
> prohibitive to develop. The key is to find the most appropriate
> balance based on the specific needs of a given application, and the
> budget driving it.
> For the low cost of just a few minutes' time, polling in Rev will
> give acceptable performance in some cases, and perhaps unacceptable
> performance in others:
> On a folder with about 5000 files, it takes more than 400ms on a
> 2.16GHz Mac. But with the 250 files in my Preferences folder it
> takes only about four milliseconds, and with about a hundred files
> it takes less than one millisecond.
> I have many folders with fewer than a hundred files, and very few
> with more than a thousand; whether it's worth the orders-of-
> magnitude greater cost to develop a solution that can more
> gracefully account for unusually large sets of files can only be
> determined by the specifics governing the project at hand, the age-
> old tug of war between scope and budget.
> > And by definition it isnt real time.
> Define "real time".
> By definition all event messages will be sent after the fact since
> they describe something that just happened, and many are derived
> from polling (devices, processes, etc.) which makes them even
> slower. The only question is how long after the fact will be
> acceptable for a given use.
> It's of course more efficient to poll as deep as possible, and in
> the case of file changes it may be that digging into the kernal or
> even the storage device driver with Assembler will yield the best
> performance. But that sort of work can get quite expensive, so it
> seemed at least worthwhile to find out how responsive this can be
> in a very-high-level 4GL like Rev.
> Many file changes are the result of a user action, either dragging
> a file into a folder, saving a file from within an application,
> etc. Given that people are much slower than computers, anything
> that happens in response to a user event will have a lot of leeway
> in terms of perceived responsiveness.
> True, that doesn't account for all possible cases, such as
> applications or processes that generate files on their own. But it
> may be useful for some folks anyway, so I posted it.
> The nice thing about doing this in Rev is that the engine is pretty
> fast with string parsing, such as the result one gets back from
> "the detailed files". The bigger latency is obtaining that list
> from the OS, which any polling method will be subject to (and hence
> I appreciate your desire to move beyond polling, but which means
> beyond Rev).
> In short, polling "the detailed files" may prove suitable for a
> good many uses, if not yours perhaps for others here.
> Forgive me if you've already described this, but what exactly does
> your application need to do that requires the shortest possible
> notification time for file changes?
> And given that task, what would be the threshold of acceptable
> performance to satisfy your project's goals? Would the OS APIs
> suffice, or will you need to go deeper?
> Innovation is rarely trivial to accomplish. Visionaries must
> usually bind their vision with sweat to realize their goals.
> Jan presented a reasonable decision tree for your project's success:
> As for having any suggestions? How about these:
> - learn C and even some x86 assembler
> - learn more about file systems
> - learn more about operating systems
> - research how to tap into the I/O parts of the kernel
> - try to find and hire someone with the above skills
> to implement this for you
> - have your checkbook ready and figure in a hefty
> maintenance cost of the code when planning the project
> Keep us posted when you release. Sounds like an interesting project.
> Richard Gaskin
> Fourth World
> Revolution training and consulting: http://www.fourthworld.com
> Webzine for Rev developers: http://www.revjournal.com
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
More information about the use-livecode