Archive

Archive for February, 2009

Parameter checking

February 27th, 2009 alex 7 comments

One of the worst parts of Delphi’s RTL is the fact that parameters are not being checked for correctness in a function call. What this means is that the offending routine will not function as intended in certain circumstances. This usually leads to “side-effects”. Let’s take the Delete routine in System unit for example:

function TrimStringPart(const S: String;
    const C: Char): String;
var
  I: Integer;
begin
  Result := S;
  I := Length(Result);

  { Find the position of character C in the string }
  while (I > 0) and (Result[I] <> C) do Dec(I);

  { And remove anything after (+ it) }
  Delete(Result, I, Length(Result) - I + 1);
end;

This function will search for a given character in the string starting from the end. The part of the string starting with that character is deleted. This function works as expected in all cases, but a non-Delphi trained developer can spot a problem immediately: What happens if the character is not found in the string? I becomes 0 (zero) and Delete is called with invalid parameters. Well, lucky that Delete is all-forgiving and will simply exit when this happens. The logically correct function would look like this:

function TrimStringPart(const S: String;
    const C: Char): String;
var
  I: Integer;
begin
  Result := S;
  I := Length(Result);

  { Find the position of character C in the string }
  while (I > 0) and (Result[I] <> C) do Dec(I);

  { And remove anything after (+ it) }
  if I > 0 then  // <-- CHECK IF CHAR WAS FOUND
  Delete(Result, I, Length(Result) - I + 1);
end;

This code adds a check to see if the character was found prior to trying to delete parts of the string. It may be useless since Delete already does this, but it increases the readability of the code. Sadly I too rely on side-effects of Delete sometimes. I do not bother specifying how much to cut from a string if I want everything starting from element I to be removed. I just pass Length(S).

And that makes me wonder: how many applications would break if Delphi would throw exceptions when invalid parameters are passed to a RTL function?

Categories: Programming

Generics + System.Move = Kaboom!

February 10th, 2009 alex 6 comments

It’s probably not a news anymore that the Move method (in System unit) should not be used to move chunks of memory that contain references to managed type (like String, Interface of dynamic arrays). By moving only the reference to another memory block you’re not incrementing the reference count of that object which results in a big Kaboom later on.

A simple example of this would be:

type
  { Declare a data type which uses a ref counted
     object - String }
  TMyData = record
    FStr: String;
  end;

var
  A, B, C: TMyData;
begin
  { Initialize the initial string }
  A.FStr := 'Hello World!';

  { Move A to B (no compiler magic involved) }
  Move(A, B, SizeOf(A));

  { Move A to C (with compiler magic involved) }
  C := A;

  { Change the string stored in A and in C }
  A.FStr[1] := '_';
  C.FStr[1] := '+';

  WriteLn('A = ', A.FStr);
  WriteLn('B = ', B.FStr);
  WriteLn('C = ', C.FStr);

  ReadLn;
end.

The result is not surprising: A = _ello World!, B = +ello World! C = +ello World. While A and B have correct values, C certainly doesn’t. This happens for obvious reasons – C would either have a reference to B‘s or A‘s string, while not holding a reference count to them.

So what does this have to do with generics? Simple, while implementing generic collections you might be tempted to use the Move function to copy data from an internal array to some external one (an example would be ToArray() method of a generic list class). This is a good idea indeed, but only if your generic type is not a managed type! Moving an array of integer to another array of integers is safe, while moving an array of strings is not. This also means that you would have to use the most generic moving possible: copy element by element which would slow down the collection if the type is integer for example.

Below is a class designed to be as fast as possible depending on the actual type of a generic class:

type
  { Our mover class }
  TArrayMover<T> = class sealed
  private
   FIsManagedType: Boolean;

  public
    constructor Create();
    procedure Move(var Source, Dest: array of T;
      const SourceIndex, DestIndex, Count: Cardinal);
  end;

{ TMover<T> }
constructor TArrayMover<T>.Create;
const
  { Declare unsafe types which need
    element-by-element copy }
  UnsafeTypes = [tkLString, tkWString, tkUString,
    tkVariant, tkArray, tkInterface, tkRecord, tkDynArray];
var
  PInfo: PTypeInfo;
  I: Cardinal;
begin
  { Find out the type of the element }
  PInfo := PTypeInfo(TypeInfo(T));

  if (PInfo <> nil) and
    (PInfo^.Kind in UnsafeTypes) then
    FIsManagedType := true
  else
    FIsManagedType := false;
end;

procedure TArrayMover<T>.Move(var Source, Dest: array of T;
  const SourceIndex, DestIndex, Count: Cardinal);
var
  I: Cardinal;
begin
  { No range checking! }

  if FIsManagedType then
  begin
    for I := 0 to Count - 1 do
      Dest[I + DestIndex] := Source[I + SourceIndex];
  end else
    System.Move(Source[SourceIndex], Dest[DestIndex],
      Count * SizeOf(T));
end;

To use it, you first create an instance of TArrayMover. In it’s constructor it will decide if the data being operated on is unsafe to be copied directly. The Move method will then used that decision to select the appropriate copy method.

P.S. Have to fix my code now :)

Categories: Programming

How To: Creating a custom Variant type

February 9th, 2009 alex 3 comments

In this post I will detail on how to create a custom Variant for your data type. First of all, the help should be pretty useful in this case, but it if doesn’t help much, there is always the FmtBcd unit which creates a Variant for it’s BCD data type.

There are a few steps to be followed in order to create a custom Variant, and here they are:

  1. Create a public data type with all the functionality. This is the data type which you will wrap into a custom Variant. It is also a good practice to make that data type and all supporting functions public to consumers. In many cases people will not need a Variant wrapping your data type but rather the data type directly. This improves speed and readability of the code in many cases.
  2. A Variant is simply a record holding some data which is used by the RTL at run-time to decide which functions to invoke and in what case. So the next step would be to declare such a record. The basic structure of that record is the same except a single 4-byte field in which you will hold a reference to your data type (or a value if it fits directly).
  3. Step 3 involves creating a descendant class from TCustomVariantType which will act as a proxy between the variant manager (handled by the RTL) and your custom Variant.
  4. The last step is to “plug-in” your proxy class at unit initialization time; and unplug it at finalization.

In my case, I will make a custom Variant for BigCardinal data type declared HelperLib. First off, all the code I will add will reside in the same unit in which BigCardinal is implemented. I need that to avoid having more units in “uses” clause and secondly I need access to internals of BigCardinal. As the first step is already completed (I already have the data type created), I will start with the second one: Declaring a custom TVarData to hold my data:

type
  PBigCardinal = ^BigCardinal;

  { Mapping the BigCardinal into TVarData structure }
  TBigCardinalVarData = packed record
    { Var type, will be assigned at runtime }
    VType: TVarType;
    { Reserved stuff }
    Reserved1, Reserved2, Reserved3: Word;
    { A reference to the enclosed big cardinal }
    BigCardinalPtr: PBigCardinal;
    { Reserved stuff }
    Reserved4: LongWord;
  end;

In this structure, VType will hold an Id (which we will obtain at runtime) that will uniquely identify our variant data type. This Id will be used by the variant manager to call our proxy class for each operation we will make on the Variant. Reserved1, Reserved2, Reserved3 and Reserved4 should be ignored and not used. And finally BigCardinalPtr is a pointer to a BigCardinal structure.

The next step is to create our proxy class that will receive all requests to “work” on the Variants of our type. Here is the declaration:

{ Manager for our variant type }
  TBigCardinalVariantType = class(TCustomVariantType)
  private
    { Will create a big cardinal, or raise an error }
    function VarDataToBigCardinal(const
      Value: TVarData): BigCardinal;
    { Will create a variant from a BigCardinal }
    procedure BigCardinalToVarData(const Value: BigCardinal;
      var OutValue: TVarData);
  public
    procedure Clear(var V: TVarData); override;
    procedure Copy(var Dest: TVarData; const Source: TVarData;
      const Indirect: Boolean); override;
    procedure Cast(var Dest: TVarData;
      const Source: TVarData); override;
    procedure CastTo(var Dest: TVarData;
      const Source: TVarData;
      const AVarType: TVarType); override;
    procedure BinaryOp(var Left: TVarData;
      const Right: TVarData;
      const Operator: TVarOp); override;
    procedure UnaryOp(var Right: TVarData;
      const Operator: TVarOp); override;
    procedure Compare(const Left, Right: TVarData;
      var Relationship: TVarCompareResult); override;
  end;

Note that I have only overridden the methods in which I am interested in. If your data type doesn’t support comparison operations, simply do not override the Compare method. The two private methods declared in this class are used internally to easy the coding.

Well, we have created the proxy class, overridden the required methods, now let’s implement them:

function TBigCardinalVariantType.VarDataToBigCardinal
      (const Value: TVarData): BigCardinal;
begin
  { Check if the var data has a big cardinal inside }
  if Value.VType = VarType then
  begin
    { Copy the value to result }
    Exit(TBigCardinalVarData(Value).BigCardinalPtr^);
  end;

  { OK, try to convert the incoming var type to
    something useful }
  case Value.VType of
    varByte:
      Result := Value.VByte;
    varShortInt:
      Result := Value.VShortInt;
    varWord:
      Result := Value.VWord;
    varSmallint:
      Result := Value.VSmallInt;
    varInteger:
      Result := Value.VInteger;
    varLongWord:
      Result := Value.VLongWord;
    varUInt64:
      Result := Value.VUInt64;
    varInt64:
      Result := Value.VInt64;
    varString, varUString, varOleStr:
    begin
      { Be careful here, a string may not be a good number }
      try
        Result := StrToBigCardinal(VarDataToStr(Value));
      except
        on EConvertError do
          RaiseCastError;
      end;
    end;
  end;
end;

Note the fact that we check if the type (Id) of the Value is the same as the one assigned to us by the variant managed. In this case, we are sure that the Value contains a BigCardinal inside, in which case we simply return that value; otherwise we convert the incoming Variant to a BigCardinal.

procedure TBigCardinalVariantType.BigCardinalToVarData
      (const Value: BigCardinal; var OutValue: TVarData);
begin
  { Dispose of the old value. Check it it's ours first }
  if OutValue.VType = VarType then
    Clear(OutValue)
  else
    VarDataClear(OutValue);

  with TBigCardinalVarData(OutValue) do
  begin
    { Assign the new variant the var type that was
      allocated for us }
    VType := VarType;

    { Allocate space for our big cardinal pointer }
    New(BigCardinalPtr);

    { Copy self to this memory }
    BigCardinalPtr^ := Value;
  end;
end;

This function simply clears out the OutValue and then create an instance of our BigCardinal type into it. Again, note that if OutValue contains a BigCardinal inside we must clear it properly (to not leak references of BigCardinal).

procedure TBigCardinalVariantType.BinaryOp(var Left: TVarData;
    const Right: TVarData; const Operator: TVarOp);
begin
  { Select the appropriate operation }
  case Operator of
    opAdd:
      BigCardinalToVarData(VarDataToBigCardinal(Left) +
        VarDataToBigCardinal(Right), Left);
    opAnd:
      BigCardinalToVarData(VarDataToBigCardinal(Left) and
        VarDataToBigCardinal(Right), Left);
    opIntDivide:
      BigCardinalToVarData(VarDataToBigCardinal(Left) div
        VarDataToBigCardinal(Right), Left);
    opModulus:
      BigCardinalToVarData(VarDataToBigCardinal(Left) mod
        VarDataToBigCardinal(Right), Left);
    opMultiply:
      BigCardinalToVarData(VarDataToBigCardinal(Left) *
        VarDataToBigCardinal(Right), Left);
    opOr:
      BigCardinalToVarData(VarDataToBigCardinal(Left) or
        VarDataToBigCardinal(Right), Left);
    opShiftLeft:
      BigCardinalToVarData(VarDataToBigCardinal(Left) shl
        VarDataToBigCardinal(Right), Left);
    opShiftRight:
      BigCardinalToVarData(VarDataToBigCardinal(Left) shr
        VarDataToBigCardinal(Right), Left);
    opSubtract:
      BigCardinalToVarData(VarDataToBigCardinal(Left) -
        VarDataToBigCardinal(Right), Left);
    opXor:
      BigCardinalToVarData(VarDataToBigCardinal(Left) xor
        VarDataToBigCardinal(Right), Left);
    else
      RaiseInvalidOp;
  end;
end;

This one is pretty simple: for each type of operation we simply invoke the BigCardinal‘s operators. Note that we did not implement all possible operators but rather those that are supported by our BigCardinal data type.

procedure TBigCardinalVariantType.Cast(var Dest: TVarData;
    const Source: TVarData);
begin
  { Cast the source to our cardinal type }
  VarDataInit(Dest);
  BigCardinalToVarData(VarDataToBigCardinal(Source), Dest);
end;

The Cast method is invoked every time another Variant type needs to be converted into our Variant type. What we do is simply initialize the Dest parameter and then invoke our helper method.

procedure TBigCardinalVariantType.CastTo(var Dest: TVarData;
    const Source: TVarData; const AVarType: TVarType);
var
  Big: BigCardinal;
  Temp: TVarData;
begin
  if Source.VType = VarType then
  begin
    { Only continue if we're invoked for our data type }
    Big := TBigCardinalVarData(Source).BigCardinalPtr^;

    { Initilize the destination }
    VarDataInit(Dest);
    Dest.VType := AVarType;

    case AVarType of
      varByte:
        Dest.VByte := Big.ToByte();
      varShortInt:
        Dest.VShortInt := Big.ToShortInt();
      varWord:
        Dest.VWord := Big.ToWord();
      varSmallint:
        Dest.VSmallInt := Big.ToSmallInt();
      varInteger:
        Dest.VInteger := Big.ToInteger();
      varLongWord:
        Dest.VLongWord := Big.ToCardinal();
      varUInt64:
        Dest.VUInt64 := Big.ToUInt64();
      varInt64:
        Dest.VInt64 := Big.ToInt64();
      varOleStr:
        VarDataFromOleStr(Dest, UIntToStr(Big));
      varString, varUString:
        VarDataFromStr(Dest, UIntToStr(Big));
      else
      begin
        { No default convertion found! Trying to use the string }
        try
          VarDataInit(Temp);
          VarDataFromStr(Temp, UIntToStr(Big));
          VarDataCastTo(Dest, Temp, AVarType);
        finally
          { Dispose our variant }
          VarDataClear(Temp);
        end;
      end;
    end;
  end else
    inherited;
end;

This method is the inverse of the Cast one. In most cases we can do a direct cast, while in some we will first try to convert to a string and then from that string convert to the required Variant type (most custom variants implement conversion from ant to strings so that might help).

procedure TBigCardinalVariantType.Clear(var V: TVarData);
begin
  { Clear the variant type }
  V.VType := varEmpty;

  { And dispose the value }
  Dispose(TBigCardinalVarData(V).BigCardinalPtr);
  TBigCardinalVarData(V).BigCardinalPtr := nil;
end;

The Clear method is invoked every time a Variant that contains a BigCardinal is being cleared (either by compiler inserted calls or manually by calling VarClear() method). Our implementation will Dispose() the structure and then make the Variant empty. Note that FreeMem is not an option because it will not finalize the record — which is what we want because we have a reference to a dynamic array there that needs to be cleared out.

procedure TBigCardinalVariantType.Compare
    (const Left, Right: TVarData;
      var Relationship: TVarCompareResult);
var
  Res: Integer;
begin
  { Compare these values }
  Res := VarDataToBigCardinal(Left).CompareTo
    (VarDataToBigCardinal(Right));

  { Return the compare result }
  if Res < 0 then
    Relationship := crLessThan
  else if Res > 0 then
    Relationship := crGreaterThan
  else
    Relationship := crEqual;
end;

Compare is simple and doesn’t need any explanations. We simply “convert” the variants to BigCardinal and compare them.

procedure TBigCardinalVariantType.Copy(var Dest: TVarData;
    const Source: TVarData; const Indirect: Boolean);
begin
  if Indirect and VarDataIsByRef(Source) then
    VarDataCopyNoInd(Dest, Source)
  else
  begin
    with TBigCardinalVarData(Dest) do
    begin
      { Copy the variant type }
      VType := VarType;

      { Initialize the pointer }
      New(BigCardinalPtr);

      { Copy by value }
      BigCardinalPtr^ := TBigCardinalVarData(Source).
        BigCardinalPtr^;
    end;
  end;
end;

This method is invoked every time we assign a Variant to another Variant. See Help for more description on what Indirect and VarDataIsByRef mean in this context.

procedure TBigCardinalVariantType.UnaryOp
    (var Right: TVarData; const Operator: TVarOp);
begin
  { Select the appropriate operation }
  case Operator of
    opNegate:
      BigCardinalToVarData(VarDataToBigCardinal(Right), Right);
    else
      RaiseInvalidOp;
  end;
end;

We implement a single operator: opNegate. Even so, negation works on BigCardinals only is overflow checking is disabled.

Now that we have implemented our proxy, let’s plug it in! First you declare a global but implementation-only variable of your proxy type (in our case: TBigCardinalVariantType). You will probably declare that variable at the top of the implementation section so that other code can use it (later on this):

var
  { Our singleton that manages tour variant types }
  SgtBigCardinalVariantType: TBigCardinalVariantType;

… and then register it by using this code:

initialization
  { Register our custom variant type }
  SgtBigCardinalVariantType := TBigCardinalVariantType.Create();

finalization
  { Unregister our custom variant }
  FreeAndNil(SgtBigCardinalVariantType);

When SgtBigCardinalVariantType is created, it’s VarType member (which we used in our functions)  is automatically assigned a unique Id by the variant manager. That Id is used in our code to uniquely distinguish our custom Variant.

OK, we’ve go to the point where everything is ready to go with one little exception — there is no public method or routine that can create a Variant type from a BigCardinal. In this case I have decided to implement an “implicit” operator for BigCardinal that will do this creation automatically:

class operator BigCardinal.Implicit
      (const ANumber: BigCardinal): Variant;
begin
  { Clear out the result }
  VarClear(Result);

  with TBigCardinalVarData(Result) do
  begin
    { Assign the new variant the var type that was
      allocated for us }
    VType := SgtBigCardinalVariantType.VarType;

    { Allocate space for our big cardinal pointer }
    New(BigCardinalPtr);

    { Copy self to this memory }
    BigCardinalPtr^ := ANumber;
  end;
end;

That’s all! Now we can write code like this:

var
  X: Variant;
begin
  X := BigCardinal(1);
  X := X + '343267727663266525438020038463';
  WriteLn(X);
end;

In the next post I will create an invokable variant.

Categories: Programming

[Rant] Actions …

February 7th, 2009 alex 9 comments

Just a few words out of frustration: Do yourselves a favor and stay away from actions and ribbons in Delphi. At least for a while. If you really need fancy looking menus use JEDI.

P.S. This would be a follow-up of this post I’ve written last September.

Categories: Programming

TDelegatedEqualityComparer

February 5th, 2009 alex 4 comments

In this post I will try to exemplify the use of TDelegatedEqualityComparer class which can be found in Generics.Defaults unit in Delphi 2009. TDelegatedEqualityComparer is a simple class that requires two user provided routines at creation time. These routines are called each time the TDelegatedEqualityComparer must perform an equality check or hash-code generation. At first it may seem a pretty useless because you still have to declare two methods that will be passed to the TDelegatedEqualityComparer constructor thus minimizing its usefulness (you could simply create a descendant class from TEqualityComparer).  Now, in Delphi 2009 we have the “anonymous methods” that we can use “inline” when creating an instance of TDelegatedEqualityComparer class. This in what makes  TDelegatedEqualityComparer useful!

And now let’s see an example:

{ Declare a simple type. Using standard provided equality
  comparer will not work properly in this type }

type
  TMyRecord = record
  private
    FString: String;
  public
    constructor Create(const AStr: String);
  end;

{ TMyRecord }
constructor Create(const AStr: String);
begin
  FString := AStr;
end;

procedure Test();
var
  Dict: TDictionary<String, Integer>;
  Comparer: IEqualityComparer;
begin
  { Create a comparer we will use in the dictionary
     for our custom type }
  Comparer := TDelegatedEqualityComparer.Create(
    function(const Left, Right: TMyRecord): Boolean
    begin
      { Properly compare our type! }
      Result := Left.FString = Right.FString;
    end,
    function(const Value: TMyRecord): Integer
    begin
      { Generate a hash code }
      Result := BobJenkinsHash(Value.FString[1],
        Length(Value.FString)*2, 0);
    end);

  { Create a dictionary }
  Dict := TDictionary<String, Integer>.Create(Comparer);

  { Populate the dictionary }
  Dict.Add(TMyRecord.Create('One'), 1);
  Dict.Add(TMyRecord.Create('Two'), 2);
  Dict.Add(TMyRecord.Create('Three'), 3);

  { Check is we can find the values in the dictionary
    Our anonymous methods will be used for equality comparison
    Using the default comparer provided by CG will not work }

  ASSERT(Dict.ContainsKey(TMyRecord.Create('One')));
  ASSERT(Dict.ContainsKey(TMyRecord.Create('Two')));
  ASSERT(Dict.ContainsKey(TMyRecord.Create('Three')));

  Dict.Free();
end;

So why the default provided comparer will not work with our custom data type? Well that’s because the Generics.Defaults will create a “binary equality comparer”. This compares the memory occupied by the value, which is not good in our case because we have references to strings there. So two strings with the same value will have different addresses and thus the comparer would find those two different.

If you plan to use a certain data type multiple times in your application it is still better to create a TCustomComparer class and use it in your code everywhere you need.

Categories: Programming

Avoid using TDictionary

February 3rd, 2009 alex 3 comments

There are serious flaws in TDictionary in Delphi 2009. Try to avoid using it until a fixed version comes out. You can find a list of them in QC, but just to name two:

  1. TDictionary.Clear() is flawed. If you clear the dictionary and then try to add another item to it, you will get an AV.
  2. TDictionary.Add() is too slow. Adding 1000 elements in a Integer, Integer dictionary takes a few seconds.

There are probably more issues but these two are enough reason not to use TDictionary yet.

Categories: Programming