Posts tagged DeHL

DeHL, Delphi 2010 and Serialization

DeHLA few months have passed and I did not release a new version of DeHL yet. No, it’s not dead. I’ve just been busy with a delicate new feature — Serialization. This post will demonstrate the new capabilities of DeHL it’s advantages and and shortcomings.

But first — since the new releases will focus mostly on serialization and related stuff, I decided to drop Delphi 2009 support. It made no sense to support 2009 for future versions since no essential changes are made to the prior code. You can still use 0.7 release in Delphi 2009.

Back to serialization. The following list describes the changes that went into the new version:

  • In order to support serialization, DeHL’s type system was extended to support Serialize and Deserialize methods. Each type class (that describes a type in Delphi) now knows how to serialize values of the type it manages.
  • A new unit named DeHL.Serialization was added. It contains the base definitions of types used by the type system for serialization.
  • TPointerType, TRecordType<T>, TArrayType<T>, etc. were added for simplified type handling. The old method was a mix-up of Delphi 2009 and Delphi 2010 RTTI specifics (which have some essential differences in my case).
  • All classes can now implement ISerializable interface. The TClassType<T> detects whether this interface is implemented by the object and uses it for serialization (no, reference counting is not touched).
  • DeHL.Serialization.Abstract contains the semi-implementation of a “serializer” and it’s context. It is used by specific serializers.
  • DeHL.Serialization.XML defines the TXMLSerializer<T> which can be used to serialize/deserialize into XML nodes (uses TXMLDocument). Supports it’s own set of attributes (such as XmlRoot, XmlElement, etc.).
  • DeHL.Serialization.Ini defines the TIniSerializer<T> that you can use to serialize/deserialize type into Ini files or registry (through RTL’s TRegIniFile).
  • Most DeHL types (such as Nullable<T>TFixedArray<T>, BigInteger, etc.) provide their own serialization and deserialization methods.
  • All Enex collections (except a few that can’t actually) can be serialized and deserialized. They implement a custom serialization and deserialization technique through ISerializable.

Enough talk, a mandatory example:

type
  [XmlRoot('Testing', 'http://test.namespace.com')]
  TTest = class
    { Pointer to self }
    [XmlElement('PointerToSelf')]
    FSelf: TObject; 
    {A set of format settings }
    FFormatSettings: TFormatSettings;

    { And internal record }
    FInternal: record
      { Force the field to be an attribute of FInternal }
      [XmlAttribute('Value')]
      FOne: Integer;

      { Force this element to have same name but other namespace }
      [XmlElement('Value', 'http://other.namespace.com')]
      FTwo: String;
    end;

    FListOfDoubles: TList<Double>;
  end;
var
  LDocument: IXMLDocument;
  LXMLSerializer: TXMLSerializer<TTest>;
  LOutInst, LInInst: TTest;
begin
  CoInitializeEx(nil, 0);

  { Initialize the test object }
  LOutInst := TTest.Create;
  LOutInst.FSelf := LOutInst;
  GetLocaleFormatSettings(GetThreadLocale(), LOutInst.FFormatSettings);
  LOutInst.FInternal.FOne := 1;
  LOutInst.FInternal.FTwo := '2 - Two';
  LOutInst.FListOfDoubles := TList<Double>.Create();
  LOutInst.FListOfDoubles.Add(0.55);
  LOutInst.FListOfDoubles.Add(0.122);
  LOutInst.FListOfDoubles.Add(122.23);

  { Create the serializer and an XML document }
  LXMLSerializer := TXMLSerializer<TTest>.Create();
  LDocument := TXMLDocument.Create(nil);

  { Set the options }
  LDocument.Active := true;
  LDocument.Options := LDocument.Options + [doNodeAutoIndent];

  { Force fields to elements by default }
  LXMLSerializer.DefaultFieldsToTags := true;

  { Serialize the structure }
  LXMLSerializer.Serialize(LOutInst, LDocument.Node);

  { Serialize the structure }
  LXMLSerializer.Deserialize(LInInst, LDocument.Node);

  { Cleanup }
  LDocument.SaveToFile('c:\test.xml');
  LXMLSerializer.Free;
end.

The XML file generated by this code looks like this (INI looks uglier):

<Testing xmlns="http://test.namespace.com" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:DeHL="http://alex.ciobanu.org/DeHL.Serialization.XML" xmlns:NS1="http://other.namespace.com">
  <PointerToSelf DeHL:ref="Testing"/>
  <FFormatSettings>
    <CurrencyString>$</CurrencyString>
    <CurrencyFormat>0</CurrencyFormat>
    <CurrencyDecimals>2</CurrencyDecimals>
    <DateSeparator>/</DateSeparator>
    <TimeSeparator>:</TimeSeparator>
    <ListSeparator>,</ListSeparator>
    <ShortDateFormat>M/d/yyyy</ShortDateFormat>
    <LongDateFormat>dddd, MMMM dd, yyyy</LongDateFormat>
    <TimeAMString>AM</TimeAMString>
    <TimePMString>PM</TimePMString>
    <ShortTimeFormat>h:mm AMPM</ShortTimeFormat>
    <LongTimeFormat>h:mm:ss AMPM</LongTimeFormat>
    <ShortMonthNames>
      <string>Jan</string>
      <string>Feb</string>
      <string>Mar</string>
      <string>Apr</string>
      <string>May</string>
      <string>Jun</string>
      <string>Jul</string>
      <string>Aug</string>
      <string>Sep</string>
      <string>Oct</string>
      <string>Nov</string>
      <string>Dec</string>
    </ShortMonthNames>
    <LongMonthNames>
      <string>January</string>
      <string>February</string>
      <string>March</string>
      <string>April</string>
      <string>May</string>
      <string>June</string>
      <string>July</string>
      <string>August</string>
      <string>September</string>
      <string>October</string>
      <string>November</string>
      <string>December</string>
    </LongMonthNames>
    <ShortDayNames>
      <string>Sun</string>
      <string>Mon</string>
      <string>Tue</string>
      <string>Wed</string>
      <string>Thu</string>
      <string>Fri</string>
      <string>Sat</string>
    </ShortDayNames>
    <LongDayNames>
      <string>Sunday</string>
      <string>Monday</string>
      <string>Tuesday</string>
      <string>Wednesday</string>
      <string>Thursday</string>
      <string>Friday</string>
      <string>Saturday</string>
    </LongDayNames>
    <ThousandSeparator>,</ThousandSeparator>
    <DecimalSeparator>.</DecimalSeparator>
    <TwoDigitYearCenturyWindow>50</TwoDigitYearCenturyWindow>
    <NegCurrFormat>0</NegCurrFormat>
  </FFormatSettings>
  <FInternal Value="1">
    <NS1:Value>2 - Two</NS1:Value>
  </FInternal>
  <FListOfDoubles>
    <Elements>
      <Double>0.55</Double>
      <Double>0.122</Double>
      <Double>122.23</Double>
    </Elements>
  </FListOfDoubles>
</Testing>

On the first serialized/deserialized value, serializers build up a sort of an internal “object graph” and gathers all information about the data being serialized. The next uses of the same serializer instance yield an 10x performance gain since there is no need to rebuild all the information from scratch. I am still working on more optimizations that could give greater speed boost.

P.S. I can’t show the contents of the deserialized object here so you’ll have to take my word for it.

Note. This is just a preview of what is going on in the trunk. No version is released since I have to iron out the last problems and write the missing unit tests.



DeHL 0.7 is up

DeHL

After a few months of no releases, I finally decided to throw one out — so here it is, DeHL 0.7. This release is adding three more collection, new types and fixes some internal limitations of the library.

For the people that never tried DeHL – it is a collection of types and classes that use (and even abuse) the newer generation Delphi compilers into obtaining some features that were impossible in the past releases.

The newly added collection classes are:

  • TPriorityQueue<TPriority, TValue> which implements the priority queue.
  • TDistinctMultiMap<TKey, TValue> that is similar to TMultiMap but uses sets instead of lists to store the values. This ensures that all values for a key are distinct. As usual 2 more flavors of this class exist – TSortedDistinctMultiMap<TKey, TValue> and TDoubleSortedDistinctMultiMap<TKey, TValue>.
  • TBidiMap<TKey, TValue> implements the concept of bi-directional map in which there is a bi-directional relation between keys and values (both are actually keys). TSortedBidiMap<TKey, TValue> and TDoubleSortedBidiMap<TKey, TValue> are the other two flavors of this class.
  • TStringList<T> and TWideStringList<T> collections are actually generic variants of the normal TStringList and TWideStringList (which are used as base classes).

Other enhancements include:

  • Singleton<T> class. Which can be used to access the same instance of a class across all application.
  • TWideCharSet (this was inspired on other implementations out there but use dynamic arrays to save memory and speed in certain circumstances).
  • TType speed and reliability increases.
  • A global DeHL.Defines.inc files for all $IFDEF needs.
  • Types such as Nullable<T>, Scoped<T> and TKeyValuePair<TKey, TValue> now have their own type classes.

For a full list of changes see the changelog.

This release also makes use of some new improvements of the Delphi 2010 compiler (such as class constructors, or some fixed problems in the generics handling) which makes some features unavailable on Delphi 2009. I plan to keep Delphi 2009 supported for as long as possible, but many new planned features are not going to be available on that platform.

Have Fun!

DeHL 0.6 available

DeHL

Yes I know I have skipped 0.5. The reality is that 0.5 was due a long time ago, but I did not have enough free time on my hands to complete the unit testing for all new features added. I always try to add as many unit tests as possible to test all possible scenarios.

This release features a general reorganization of some key concepts, new type extension mechanics, boxing, a lot more collections, better support for associative collections through specific Enex extensions … and many more. I am not going to lay out all the changes here since there are quite a few of them. You can view the change log here.

One important note, check out the DeHL.Base.pas unit. There are a number of TOGGLE_XXXXX constants. Toggle them to true or false (if you are using Weaver FT builds) to enable or disable specific functionality. In Tiburon (Delphi 2009) all the features are disabled by default (to be on the safe side).

Get the latest build here. If you stumble upon some bugs please report them here.

Have Fun!

Arbitrary precision integers

I have decided to extract the BigInteger and BigCardinal data types from DeHL and make a bundle with them separately. Someone may be interested in these types and not so much in DeHL. The attached unit should compile on D2007 and D2006 (not sure for earlier versions).

What can you learn:

  • Shows you how to develop “value types” with predictable states in Delphi using records, operator overloading and “managed types” (in this case a dynamic array).
  • How to develop a proxy class that plugs your type into the Variant manager.
  • Demonstrates how to use operator overloading (and I have overloaded a lot of operators there).
  • The reliance of the predictable state of the managed types to check whether a value is initialized or not.
  • Shows how the use an in-place BCD-izer algorithm that helps converting a big number into a string. It took me a quite some time and tweaking to come up with it so it may be interesting to someone.

Where can you use it:

  • Everywhere, except FOR loops of course.
  • The types are transparent and should be used the exact way as normal integer do.
  • Use it in applications that require very big integer values.
  • … you’ll come up with something else.

A small example of course:

var
  a, b: BigInteger;
begin
  a := StrToBigInteger('8942374632463284632623837643468589' + 
        '26367423509458904389575645349857');
  b := a * a;
  WriteLn(IntToStr(b));
end.

… or as variants …

var
  a, b: Variant;
begin
  a := StrToBigInteger('8942374632463284632623837643468589' + 
        '26367423509458904389575645349857');
  b := a * a;
  WriteLn(b);
end.

Final note: the code is tested since it’s a part of DeHL (in which I try to test everything I can) so it’s safe to use. BUT if you find any bugs, please drop me a line.

Finally! The code! You can find it here: Arbitrary precision Integers (1686)

DeHL documentation

Since I had a few requests, I have started putting together some documentation on DeHL. It’s pretty basic and only includes some conceptual stuff, not API documentation. You can find it here.

Version 0.5 is shaping up quite well with tons of changes, and new stuff coming in — but that’s a another story.

Not much in this post,  Cheers!

What’s new in DeHL 0.4

In the previous post I have mostly talked about Enex (Enumerable Extensions) I have enabled for all DeHL’s collection classes. Now I will list the other changes that got into this release that may be of interest to you:

  • Enex (Enumerable Extensions) in DeHL.Collections.Enex. Support for chained queries, predicates and selectors.
  • THashTree (A tree that uses hashes to access the child nodes). in DeHL.Collections.HashTree.
  • TConverter can be created with given IType‘s as opposed from the default ones.
  • TSuppressedWrapperType which can be used to wrap another IType. It’s main function is to forward all calls to the enclosed IType except the Cleanup() and Management() ones. This way you can use temporary collections inside your collection without the “cleanup” problem.
  • Fixed 2 bugs in Resize() method of THashSet and TDictionary which resulted in corrupted heap in some cases.
  • TLinkedList properties called First and Last were renamed to FirstNode and LastNode to avoid naming conflicts with Enex’s First() and Last() methods.
  • Fixed a bug in TQueue‘s enumerator object. It enumerated the queue’s elements quite wrong.
  • Renamed TScopedPtr to a better name: Scoped.
  • Added a new type: Nullable. It can be used to represent a NULL value for non-object types such as Integer.
  • New life-time extensions for TRefCountedObject. Each ref-counted object can now “keep-alive” other ref-counted objects. This was an useful addition in order to support query-chains in Enex.

How to use the new Nullable<> type:

function GetTheSum(const AList: TList<Integer>): 
    Nullable<Integer>;
begin
  { Check if the list is non-null and has elements }
  if (AList = nil) or (AList.Count = 0) then
    Result.MakeNull();
  else
    Result := AList.Sum();
end;

var
  LSum: Nullable<Integer>;
begin
  { ... }
  LSum := GetTheSum(...);
  if not LSum.IsNull then
    WriteLn(LSum.Value);
end;

I agree, this is not the best example, but it is OK to get you started. Nullable values are useful when you function may return or even accept a normal (let’s say Integer) value or a NULL.

DeHL 0.4: Linq-ifying Delphi

Since I have started working on DeHL, one of the main purposes was to get to the point where I could use Linq-like extensions on collection classes. Of course I could not call the new functionality Linq since it’s not language-integrated, so I decided to give it another name: Enex (aka Enumerable Extensions). All collection classes in DeHL now support a wide set of methods that allow querying and selecting elements from the starting collection. A lot of work has been put into this release, ranging from extensions to type system, object life-time management, optimizations and bug-fixes and of course a lot of testing.

Now, back to Enex, let’s visit some cool examples of what the new functionality can do:

Let’s check a very simple example. Assuming you have a list of integers and you want to find the sum of all elements that are greater than the list average. In the “times before” you wold write something like this:

  { Calculate the average of the list }
  Sum := 0;

  for I := 0 to List.Count - 1 do
    Sum := Sum + List[I];

  Avg := Sum div List.Count;

  { Calculate the sum of the elements above average }
  Sum := 0;

  for I := 0 to List.Count - 1 do
    if List[I] > Avg then
      Sum := Sum + List[I];

… Now it is as simple as this:

  { ... }
  Sum := List.WhereGreater(List.Average()).Sum();
  { ... }

Now, let’s see another more complex example. Considering that you have two collections of integers. You have to determine the maximum value common to both collections. Check it out:

  { ... }
  Max := AColl1.Intersect(AColl2).Max();
  { ... }

Next: what is the sum of all odd numbers in interval [0..100]?

  Sum := TEnexEnumerable<Integer>
    .Interval(0, 100)
    .Where(function(Arg1: Integer): Boolean
      begin
        Exit(Odd(Arg1);
      end)
    .Sum();

The last one: You want to take all integers from a list with their absolute value and only once; and you want to write them on the screen directly:

  for I in ARandomList
    .Select(function(Arg1: Integer): Integer
      begin
        Exit(Abs(Arg1));
      end)
    .Distinct()
  do
    WriteLn(I);

What’s working and what’s not:

  • Some problems in the compiler prevent the use of generic function Select<TOut> and Cast<TOut> for now. These two could have been used to transform each element of the collection to some other type.
  • Everything is based in interfaces to make use of automatical garbage collection. the only restriction is that the first enumerable (from which the chain starts) should be either an interface or must be destroyed explicitly.
  • All collections have overriden versions of some functions. For example in a list, the First() method is implemeted directly so that you benefit from full speed whenever possible.
  • No lambdas … that makes writing selectors or predicates rather unpleasant.
  • Sum() and Average() are only supported for types registered as integers or reals (including BigCardinal and BigInteger).
  • All selector and predicate-based selection is done on-the fly so there is no additional memory used!

There are of course much more possible use cases for Enex-enabled collections, I simply do not have the time to list all of them.

You can download the latest version of DeHL here.

DeHL 0.3 is out

I’m proud to announce the 0.3 version of DeHL library. This release focused on bug-fixes and extensions to type support engine. This version should pretty much stabilize what I started and should leave me more time to extend other interesting parts of the library such as collections.

For those who are new, DeHL is a project of mine designed to fill in some holes in the Delphi’s RTL — most notably, some centralized and OOP-ish Date/Time support, more generic collection classes and “big integer” math support. DeHL also builds up a new layer of “type support” which is very useful to abstract all type management tasks in generic classes. This projects only supports Delphi 2009 (and above) since Generics, Anonymous methods, and Unicode are widely used whithin the library. So even if you are not interested in the library itself, it may prove a good read if you are interested in all new features Delphi 2009 has to offer.

New features:

  • Type support classes now expose methods to convert a type from an to a Variant.
  • Based on the newly added Variant conversion, a new type: TConverter<T1, T2> is present. You can use it to support “blind” conversion in a class.
  • Much improved type support system
    • Now more functionality is split between generic and non-generic variants of the type support classes (IType, IType<T>, TType, TType<T>).
    • TypeSupport has been renamed to TType and IType repectively.
    • IType/TType now export Name, Size, Management, TypeInfo and Family properties. You can use those properties to get more information about your generic type.
    • Custom type registration has been improved. A new cleaner API can be used to register you custom types into the DeHL’s type system.
    • TType<T>.Default is now an overloaded function. The first form is the usual one. The second form receives a set of “type families” that are expected to represent the generic type. This way you can effectively impose run-time type restrictions.
    • TClassType is now generic with T being class-restricted. This allows to avoid compile-time type incompatibilities.
  • All important standard types in Delphi are now supported. This support required the working custom types system, since these types can’t be handled “by default“:
    • Boolean, ByteBool, WordBool, LongBool
    • TDateTime, TTime, TDate
    • NativeInt, NativeUInt
    • Real
    • OleVariant
    • UCS4Char, UCS4String
    • UTF8String
    • RawByteString

Bug fixes:

  • Fixed a bug in BigCardinal and BigInteger variant types related to string conversion.
  • A few changes in TDictionary to avoid compiler problems.
  • In a multi-threaded environment, the custom type support would not unlock properly leading to a dead-lock.
  • Cleanup routines for TDictionary and THashSet were flawed since those used FillChar to clear the array. Managed types had problems with that.
  • Char type support class now acts properly.

Note: It may be possible that the Testing project will not compile due to a bug in the Delphi compiler. But that should not prevent the usage of the library itself.

The download section can be found here.

DeHL 0.2.3 released

Not many changes in this release:

  • Fixed a bug in BigCardinal related to zero-length numbers
  • BigCardinals embedded in variants now properly negate.
  • Support for the NOT operation for BigCardinal.
  • … As usual some tests.

0.2.3 is based on 0.2.2 and not on the trunk since trunk is seriously damaged. Downloads can be found here.

DeHL 2.2 Released

In this release I have changed the name of the library from DelphiHelperLib to DeHL to be shorter. The most important changes in this release are:

  • Renamed the library to DeHL in hopes to remove the length of the unit names.
  • Renamed all classes to use the T prefix rather than H. It was an unfamiliar mode of naming things.
  • BigCardinal and BigInteger can be used within a Variant type. An implicit operator is declared to allow seamlessly to convert from the BigInteger/Cardinal to Variants.
  • Type Support has been extended to offer better information about the clean-up practices of a type.
  • Added methods to move a number of elements from an array safely (for ref-counted Delphi types). Also fixed the code to use these methods instead of Move directly.
  • Added a new array type: TFixedArray. It is an immutable array which is used only for reading. TDynamicArray can be created off TFixedArrays.
  • Added work-around fixes for a few Delphi bugs blocking the compilation.
  • The testing module was cleaned up a bit. Now using a base class for all our test cases. Also tests for new scenarios have been added.
  • All container classes now cleanup their contained values (when required). This means it is now safe to use them with types that require cleanup.
  • Cleaned up the binary tree implementation a bit and fixed a bug.
  • Externalized the last remaining hard-coded strings in the code.

More information and downloads can be found here.