Should size be extended to variables?

[Richard Gaskin]

Mark Waddingham wrote:

 >> On 20 Aug 2016, at 20:05, Richard Gaskin wrote:
 >>
 >> As for byte sizes, my own use case is for establishing pointers to
 >> locations within a large file on disk.
 >>
 >> But as Richmond kindly reminded me, the byte chunk type supports
 >> that with "the number of bytes of...", so I'm set for now.
 >
 > This is why I asked 'for what purpose' :)

Of course, which is why I described mine. :)


 > For the case of optimising for memory footprint you need a measure
 > which allows to see the effect of your code changes on the amount
 > of memory used whilst running - and this is entirely a runtime
 > concern, so you need to take measurements which reflect active state.
 >
 > For the case of indexing an encoded format on disk you need to
 > measure the size of things as they are in your specific output format
 > - that will entirely depend on how you choose to encode the values
 > you are writing out and want to index.
 >
 > The encoded size of a string, for example, will depend on the text
 > encoding.
 >
 > A good pattern for your use-case is perhaps PDF. This format is a
 > serialised list of objects which has an index appended to the end.
 > The index is a mapping of thing you want to load to byte offset in
 > file. The index is generated as the file is written out (reading file
 > offsets as they are written out), then appended to the end of the
 > file. The final bit of the file is the byte offset to the start of
 > the index in the file.
 >
 > (Note that such an index could be generated as a post processing
 > step, and is perfectly viable as a separate file).

That's how I'm storing it now, as a separate "map" to the larger file.


 > In regards to 'number of bytes of', then this does an auto convert to
 > data relative to the native encoding (for strings) so probably isn't
 > what you want here if your text could be Unicode ( or needed in a
 > cross platform encoding).

These larger files can vary.  Sometimes I don't write them at all, 
merely read chunks of them to determine interesting locations, and then 
attempt to note the location in my "map" index file.

In other cases I have collections of LSON data (LiveCode encoded 
arrays), in which each encoded array has some other data of a knowable 
size prepended to it, and then appended to the collection.

In all cases, the data is read and written in binary mode.

If neither "number of bytes of" or "number of chars" will help, how can 
I know the true byte length of a chunk of data?

-- 
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  Ambassador at FourthWorld.com                http://www.FourthWorld.com