Top Tips for Descriptors

When externalizing data to a writable stream, or internalizing it from a readable stream, use the operators crafted for this purpose rather than 'rolling your own'. This code, which reads/writes the descriptor length using 4 bytes, then the data itself, is awkward and inefficient.

// Write the contents of aDes to aStream
void CMyClass::ExternalizeL(RWriteStream& aStream, const TDesC& aDes) const
{// Write the length of aDes
aStream.WriteUint32L(aDes.Length());
// Write the data from aDes
aStream.WriteL(aDes, aDes.Length());
}

// Read the contents of aStream and create an HBufC
HBufC* CMyClass::InternalizeL(RReadStream& aStream)
{// Read the length of aDes
TInt size=aStream.ReadUint32L();
// Allocate a buffer
HBufC* heapBuf = HBufC::NewL(size);
// Create a modifiable descriptor over heapBuffer
TPtr ptr(heapBuffer->Des());
// Read the descriptor data
aStream.ReadL(ptr,size);
return (heapBuf);
}

If you link against estor.lib, you can use the templated stream operators that Symbian OS provides, for much more efficient internalization and externalization. The length information of the descriptor is compressed to make descriptor storage as compact as possible.

You can use operator>> to internalize string data, if it was externalized using operator>>, but additionally, the stream data can be passed to HBufC::NewL(), as a read stream, to recreate the descriptor. Either of these are a significantly more efficient approach to that above:

// Write the contents of aDes to aStream
void CMyClass::ExternalizeL(RWriteStream& aStream, const TDesC& aDes) const
{
aStream << aDes;
}

// Read the contents of aStream and create an HBufC
HBufC* CMyClass::InternalizeL(RReadStream& aStream)
{// KMaxLength is defined elsewhere as the maximum length read from the stream
HBufC* heapBuf = HBufC::NewL(aStream, KMaxLength);
}

You’ll notice that the ExternalizeL() and InternalizeL functions above are leaving functions. This is because the templated streaming operators, operator>> and operator<<, may leave. They are not suffixed with L because they are operators, and can't be named according to the convention for leaving functions on Symbian OS. This makes it easy to forget that they leave, which isn't helped by the fact that the leave-checking tool LeaveScan doesn't flag them.

So, if you write code which uses them, you should make a special point of checking that it is leave-safe.

There's an interesting side effect to calling Des() on an HBufC to return a modifiable descriptor.

Recall that HBufC derives from TDesC, and thus an HBufC object doesn’t have a maximum length word, because it is non-modifiable and doesn’t need one. But when you call Des() to return a modifiable descriptor, a TPtr which references the descriptor data, the modifiable descriptor must have a maximum length value. So where does it come from?

Well, in fact, the maximum length of the heap cell in which the HBufC was allocated is used to set the maximum length of the returned TPtr. But beware! This may not necessarily be the length specified when the heap descriptor was allocated. There are two reasons why this can occur, the first of which is the most common:

1. If the length specified wasn't word-aligned (i.e. a multiple of 4 bytes) or
2. Because the amount of space left over in the free cell from which the heap cell was allocated was insufficent to create any other heap cells, so was left spare. The minimum size required for a heap cell is approximately 12 bytes so, if there are fewer than 12 bytes in the remaining cell, your descriptor will end up with the surplus tacked on the end.

The result in either case is that the maximum length of a TPtr returned from HBufC::Des() may not be what you expect, given the size specified when the heap descriptor was created - it could be longer. For example:

const TInt KMaxBufLength = 9;
HBufC8* buf = HBufC8::NewL(KMaxBufLength);
TPtr8 ptr(buf->Des());
TInt maxLength = ptr.MaxLength(); // maxLength > 9 but may not be 12

Don’t expect that it is simply rounded up to the next word-aligned value. In the example above, the maximum length is guaranteed to be at least three extra bytes but it may be more, depending on the remaining size of the heap cell from which it was allocated.

So, if you cannot guarantee that the value of MaxLength() is what you expect, it's safest to use the integer value used to first allocate the HBufC. If the length of the heap buffer was set to its maximum on creation, you can alternatively use that:

const TInt KMaxBufLength = 9;
HBufC8* buf = HBufC8::NewL(KMaxBufLength);
TPtr8 ptr(buf->Des());
TInt unpredictable = ptr.MaxLength(); // maxLength > 9 but may not be 12
TInt bufLen = buf->Length();

bufLen will be 0, because the buffer was not set to its maximum length on creation. This is done by calling NewMaxL():

HBufC8* buf = HBufC8::NewMaxL(KMaxBufLength);
TInt bufLen = buf->Length(); // This will be 9

So how does this matter?
Well, you can get caught out. Here's an example I used in my book. It illustrates the use of pointers to manipulate the contents of a descriptor and the use of an assertion statement to catch access beyond the descriptor length:

_LIT(KPanic, "TestPointer");
const TInt KBufferLength = 10;
void TestPointer()
{// Create a buffer with length KBufferLength = 10 bytes
HBufC8* myBuffer = HBufC8::NewMaxL(KBufferLength);
TPtr8 myPtr(myBuffer->Des());
myPtr.Fill('?'); // Fill with '?'

// Byte pointer to descriptor in memory
TUint8* ptr = (TUint8*)myPtr.Ptr();
TInt maxLength = myPtr.MaxLength();

for (TInt index = 0; index < maxLength; index++)
{// This fails at the end of the buffer (index = 10)
// because myPtr.MaxLength() > KBufferLength
__ASSERT_DEBUG(index<KBufferLength, User::Panic(KPanic, KErrOverflow));
(*ptr) = '!'; // Replace the contents with '!'
++ptr;
}

}

The base class TDesC defines both Size() and Length() methods, which are easy to confuse.

Size() returns the number of bytes the descriptor occupies.
Length() returns the number of characters the descriptor contains.

For 8-bit descriptors, where each character occupies a byte, this is the same thing. However, on all releases of Symbian OS since v5u, the native character width has been 16 bits; that is, each character occupies two bytes.

Thus, unless you're working with a very old SDK, you'll find that Size() always returns a value which is double that of Length().

In your APIs, use the descriptor base classes TDes and TDesC as parameters and return values. And remember to pass them by reference for efficiency, and never by value. Thus, descriptor parameters should be passed and returned either as const TDesC& for constant descriptors or TDes& when modifiable.

Callers of your API will not want to be constrained, say, to have to use a TBuf, just because your function requires it. And your functions shouldn't care what type of descriptor is passed to them, as long as any modifiable descriptor has a sufficient data area to extend into, as necessary.

One good reason for this 'agnosticism' is that, if you need to change the implementation later, and have exposed the descriptor type at the API level, you make it harder for yourself to change the code without affecting the API. You really don't want to have to ask your clients to change their code because you've broken source compatibility.

And anyway, unless you’re taking ownership, if you're implementing a function you shouldn’t need to know if an incoming parameter is stack- or heap-based. Its location and memory layout should be irrelevant. Likewise if you're calling a method, and receiving a descriptor as a return value, you don't need to know what type it is, unless it is a heap descriptor for which you become responsible for cleanup.

Thus, when defining functions you should always use the abstract base classes as parameters or return values. For example, class RFile defines straightforward file read and write methods as follows:

IMPORT_C TInt Write(const TDesC8& aDes);
IMPORT_C TInt Read(TDes8& aDes) const;

The descriptor to write to the file is a constant descriptor, while to read from the file into a descriptor requires the parameter to be modifiable (the maximum length of the modifiable descriptor determines how much file data can be read into it).

When writing a function which receives a modifiable descriptor you don’t have to know whether it has sufficient memory for an operation, such as Append(), since the descriptor methods themselves make the check and panic if the operation would cause an overflow. However, you may not want the descriptor methods to panic if the caller has not got a large enough descriptor, because sometimes they cannot know the maximum length required. (Remember, a panic is terminal and in effect causes a 'crash' which can be pretty ugly).

As an alternative, you could perform your own bounds checking before calling a method on the incoming descriptor, then return an error or leave when you need to indicate to the caller that the descriptor's maximum length is insufficient. In addition, you could write the required length into the 'too short' descriptor supplied (assuming it is at least 4 bytes in length) so the caller can use it to allocate a descriptor of the correct minimum length.

There are occasions when you pass and return heap descriptors, HBufC, by reference or pointer. I'll go into this in a separate post.

See also, this FAQ ("How do I use descriptors as parameters?") and this one, which covers returning them ("How do I use heap descriptors as return types?").

To read from a descriptor, you only need it to be non-modifiable, a TDesC. Class HBufC derives from TDesC, so it has access to all the non-modifiable functions implemented by TDesC. All you need to do is dereference the pointer.

_LIT(KBert, "Bert");
HBufC* bert = KBert.AllocL();
TPtrC halfOfBert = bert->Left(2);

One of the most common mistakes made when using descriptors is to call Des() on an HBufC* when you only need a constant descriptor.

_LIT(KBert, "Bert");
HBufC* bert = KBert().AllocL();
TPtrC halfOfBert = bert->Des().Left(2); // Unnecessary call to Des()

It won’t do any harm, but is totally unnecessary and wastes time, space and typing

An HBufC is derived from TDesC, and inherits non-modifiable descriptor functionality. Because it is not derived from TDes, it isn’t modifiable. So to write into it, you have to create a modifiable descriptor over the data area. This is done by calling HBufC::Des() which returns a TPtr and thus gives you access to all the modification functions such as Format(), Append() and Fill():

HBufC* robert = HBufC::NewL(4); // Read Only Bert
TPtr rwbert(robert->Des()); // Read Write Bert
_LIT(KBert, "Bert");
rwbert.Copy(KBert); // The data area of robert now contains Bert

Remember that that there must be space in the HBufC for the required change because descriptors don’t resize themselves automatically. If there isn’t, it will still compile - but you’ll get a panic at runtime, as described in Tip 2. This code will fail:

HBufC* robert = HBufC::NewL(2); // Read Only Bert
TPtr rwbert(robert->Des()); // Read Write Bert
_LIT(KBert, "Bert");
rwbert.Copy(KBert); // Panic! Not enough memory allocated to hold "Bert"

The reason why there is no HBuf class is discussed here.

Heap-based descriptors, HBufC, are allocated on the heap at run time. They can be used as local variables or as class members. As with all heap-based objects, memory leaks must be avoided through use of the cleanup stack - for local variables - or deleted by a destructor if ownership is through a member variable.

There's more about HBufC in the FAQ blog here.

These are suitable for when you know the required length at compile time. The recommended maximum length is 256 bytes or fewer (remember that’s 128 characters, a TBuf<128>, because each character occupies 2 bytes).

Larger stack buffers are not recommended. You should limit the size of stack based buffers by putting larger sized descriptor data on the heap using HBufC objects - or by making your larger TBuf objects member variables of C classes, which are always created on the heap. This is because stack space per process is limited on Symbian OS devices, typically to 8 or 12 KB. So if you use significantly larger buffers on the stack, you risk running out of space.

The Windows emulator will actually extend the program stack, so you may not realize there’s a problem until you deploy your code on an actual phone. How will you realize? Well, attempts to use the stack will result in access to memory which is not mapped into the address space of the process the code is running in. An access violation occurs, which generates an exception and a KERN-EXEC 3 panic. Which is fatal.

An attempt to access an area outside the area allocated for descriptor data will cause a panic in both debug and release builds, because the descriptor functions use __ASSERT_ALWAYS to check for out-of-bounds access. A panic causes your code will stop executing immediately, whether running in an application, server or test framework. So be absolutely certain that there is space in your target descriptor, if necessary, by doing a check first by using the Length() or MaxLength() methods.

_LIT8(KImageGif, “image/gif”); // 9 characters
TBuf8<8> mimeType8; // Space for only 8 characters
mimeType8.Copy(KImageGif); // Panic!

The default constructors of TDes and TDesC are declared private so the compiler won’t let you construct them directly. But there is no copy constructor declared for either class, so the compiler won’t complain if you make a copy of a valid descriptor (in fact, it will go as far as to help you, by invoking an implicitly generated copy constructor).

_LIT(KExample, "Fred");
TPtrC original(KExample); // a valid TDesC-derived descriptor
// Shallow copy the type and length into base class object
TDesC copy(original); // Uses the implicit copy constructor

Your code probably won’t crash if it has been written safely, but you will rarely have a valid reason for doing this. The code will fail to work anyway, because TDes and TDesC contain no string data, so in effect are abstract classes.

You can use the Search bar at the top of the page or the Google site search at the bottom.
Or simply use the "Find" capability of your browser.

Please note that the links in this blog to the sister blog (Descriptors FAQ) are currently broken. It's happened because I've been updating the FAQ blog in preparation for a new look, format and home.