IdBuffer

TIdBuffer is a TObject descendant that implements a buffer used for input and output operations in the Indy library. TIdBuffer is used as a read and/or write buffer for the communication layer. TIdBuffer is optimized for adding data at the end of the buffer, and extracting data from the beginning of the buffer. 

TIdBuffer isolates the differences between memory allocation and pointer operations as implemented on the platforms supported by the Indy library. 

TIdBuffer implements properties that manage the size, growth, and encoding used for the buffer class instance, including: 

By default, TIdBuffer uses ANSI-encoding for string values added to the buffer using the Write methods. Set the value in Encoding to another TIdEncoding value to alter the encoding used in the Write methods. 

When additional internal storage is required for the buffer, the value in GrowthFactor is used to determine the number of additional bytes allocated. The default value for GrowthFactor is 2048. 

TIdBuffer implements methods that handle reading and writing various data types in the buffer. Extract methods are used to read and remove existing byte values in the buffer, including: 

TIdBuffer maintains an internal TIdBufferBytesRemoved event handler that is signalled when byte values are read and removed from the internal storage for the buffer. 

Use the overloaded constructors to create buffer instances and to perform assignment for the GrowthFactor or the TIdBufferBytesRemoved event handler used for the buffer. 

TIdBuffer implements overloaded Write methods that are used to convert various data types to an array of bytes as used in the internal storage for the buffer. Arguments passed to the overloaded Write methods generally indicate the value to be stored and the zero-based position in the internal storage for the buffer where the values will be stored. Other arguments may reflect additional information needed to convert the original data type to a sequence of byte values. 

TIdBuffer implements other properties and methods that provide access to the byte values stored in the buffer, including: 

TIdBuffer is used extensively in TIdIOHandler, both as a property and as an argument to methods that perform the read and write operation for the IO handler. 

The .Net platform forces us to perform copies from strings to Bytes so that it can do the proper unicode and other conversions. The copy is a separate issue and we considered several options. For .Net, we will always have to copy data to send or to receive to translate it to binary. 

For example if we have a string it must be converted to bytes. This conversion requires a copy. All strings are WideString and must be converted to single bytes by a convertor. This is not limited to strings. 

In VCL previously all strings were AnsiString so we used a pointer and just accessed the memory directly from the string. This avoided the overhead of a copy. 

We have come up with several ideas on how to allow the copy on .net, while avoiding the copy on VCL to keep the performance benefit. However we must do it in a single source manner and in a manner that does not impact the code negatively. 

For now for VCL we also do a copy. This has the advantage that Byte arrays are reference counted and automaticaly handled by Delphi. For example: 

  WriteBytes(StringToBytes(s));

The array returned by this function will automatically be freed by Delphi. 

There are other options that are nearly as transparent but have the additional overhead of requiring class creation. These classes can be used to copy for .net and proxy on VCL. It all works very nice and has low memory overhead. The objects can then be freed by default in methods that accept them. 

However after analysis, copy on VCL may not be that bad after all. The copy only really impacts strings. The overhead to copy strings is minimal and only used in commands etc. The big transfers come from files, streams, or other. Such transfers have to be mapped into memory in VCL anyways, and if we map directly into the byte array instead of the previous classes peformance should be fine. 

In short - copy under VCL should be acceptable if we watch for bottlenecks and fix them appropriately without having to create proxy classes. The only problem remains for transmitting large memory blocks. But if this is done against a fixed copy buffer the performance hit will be neglible and it is not a common task to transmit large memory blocks. 

For such transfers from streams, etc the user can declare a persistent array of bytes that is not freed between each call to WriteBytes. 

• Kudzu

TIdIOHandler