Importing data into RevDB

[Andre Garzia]

David,

I've just imported 30 MB worth of text into a SQLite database over a remote
connection. What I tried using first was a CSV file as well and I could not
make it work. I am importing magazine articles, the comma count is all wrong
and unless I wrote a super dupper RegEx thing to cope with quote values that
can contain commas and quotes up to unlimited levels in a field then it
would never work. I ended up using XML as my data source format and all of a
sudden everything started falling into place.

To go from excel to XML is something I never tried. Maybe there's some
plugin or a CSV to XML tool that works but if you can get your data as XML
then it is easier to parse from Rev.

On Sun, Dec 6, 2009 at 12:33 AM, David Coker <davidocoker at gmail.com> wrote:

>
> >A note of caution when using this.
> >The chunking rules for Rev are that a line can contain items, but
> >items cannot contain lines.
> >Lines can contain items, items contain words.
> >   wrong =  line 6 of item 4, no matter the delimiter
> >
> >Not all functions obey a new setting.
> >Filter does not follow the linedelimiter, and always uses CR, as does
> >Sort.
> >
> >   wrong  = set the worddelimiter to comma
> >... since the Rev definition for a word delimiter is white space, not
> >a single character.
> >
> >I have found this useful on occasion when extracting data (such as CSV
> >and html) to handle 'embedded returns' in order to keep logical groups
> >of amorphous data together.  Another use is to demark field content as
> >lines such as
> >
> >write the contents of the fields of a card to a text file
> >
> >set the lineDel to "^"
> >put field "Title" into outputBuffer
> >put field "Description" into line 2 of outputBuffer
> >put field "Inventory" into line 3 of outputBuffer
> >put field "Prices" into line 12 of outputBuffer
> >set the lineDel to cr
> >put outputBuffer into url ("file:" & aPath & aFilename)
> >
> >Also as a utility function to make a run of characters.
> >put characterString("*", 25) into line 2 of field "studentNotes"
> >put characterString(space, 10) after line 6 of field "studentNotes"
> >
> >function characterString charToUse, stringLength
> >    set the lineDel to charToUse
> >    put charToUse into line stringLength of tempp
> >    return tempp
> >end characterString
> >    -- now that we are leaving this function
> >    -- the lineDel setting evaporates
> >    -- it only applies to this function
> >    -- the same way itemDel behaves
> >The same thing can be done with itemDel
> >
> >    set the itemDel to "^"
> >    put "^" into item 25 of tempp
> >    set the itemDel to comma
> >
> >Hope this helps,
> >
> >Jim Ault
> >Las Vegas
>
> Jim, Kay, Tom... all who have replied so far.
>
> I really appreciate all of the tips and suggestions and will probably enjoy
> taking some of them for a serious test run eventually. Unthinking, I started
> the update process on my previously mentioned database here on my primary
> development machine, which has been tied up for 34+ hours now.
>
> Performance has already degraded somewhat, so I've pretty much made this
> thing off limits until it has completed. (Assuming that it actually will at
> some point.)
>
> Much appreciated good folks! Hope you all have a great weekend. :-)
>
> David C.
>
>
>
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution
>



-- 
http://www.andregarzia.com All We Do Is Code.