Importing data into RevDB
andre at andregarzia.com
Mon Dec 7 12:17:46 EST 2009
I've just imported 30 MB worth of text into a SQLite database over a remote
connection. What I tried using first was a CSV file as well and I could not
make it work. I am importing magazine articles, the comma count is all wrong
and unless I wrote a super dupper RegEx thing to cope with quote values that
can contain commas and quotes up to unlimited levels in a field then it
would never work. I ended up using XML as my data source format and all of a
sudden everything started falling into place.
To go from excel to XML is something I never tried. Maybe there's some
plugin or a CSV to XML tool that works but if you can get your data as XML
then it is easier to parse from Rev.
On Sun, Dec 6, 2009 at 12:33 AM, David Coker <davidocoker at gmail.com> wrote:
> >A note of caution when using this.
> >The chunking rules for Rev are that a line can contain items, but
> >items cannot contain lines.
> >Lines can contain items, items contain words.
> > wrong = line 6 of item 4, no matter the delimiter
> >Not all functions obey a new setting.
> >Filter does not follow the linedelimiter, and always uses CR, as does
> > wrong = set the worddelimiter to comma
> >... since the Rev definition for a word delimiter is white space, not
> >a single character.
> >I have found this useful on occasion when extracting data (such as CSV
> >and html) to handle 'embedded returns' in order to keep logical groups
> >of amorphous data together. Another use is to demark field content as
> >lines such as
> >write the contents of the fields of a card to a text file
> >set the lineDel to "^"
> >put field "Title" into outputBuffer
> >put field "Description" into line 2 of outputBuffer
> >put field "Inventory" into line 3 of outputBuffer
> >put field "Prices" into line 12 of outputBuffer
> >set the lineDel to cr
> >put outputBuffer into url ("file:" & aPath & aFilename)
> >Also as a utility function to make a run of characters.
> >put characterString("*", 25) into line 2 of field "studentNotes"
> >put characterString(space, 10) after line 6 of field "studentNotes"
> >function characterString charToUse, stringLength
> > set the lineDel to charToUse
> > put charToUse into line stringLength of tempp
> > return tempp
> >end characterString
> > -- now that we are leaving this function
> > -- the lineDel setting evaporates
> > -- it only applies to this function
> > -- the same way itemDel behaves
> >The same thing can be done with itemDel
> > set the itemDel to "^"
> > put "^" into item 25 of tempp
> > set the itemDel to comma
> >Hope this helps,
> >Jim Ault
> >Las Vegas
> Jim, Kay, Tom... all who have replied so far.
> I really appreciate all of the tips and suggestions and will probably enjoy
> taking some of them for a serious test run eventually. Unthinking, I started
> the update process on my previously mentioned database here on my primary
> development machine, which has been tied up for 34+ hours now.
> Performance has already degraded somewhat, so I've pretty much made this
> thing off limits until it has completed. (Assuming that it actually will at
> some point.)
> Much appreciated good folks! Hope you all have a great weekend. :-)
> David C.
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
http://www.andregarzia.com All We Do Is Code.
More information about the Use-livecode