Just after the OutOfMemory monster raised it's head, was slayed and is now believed to have vanished in the test lab, Tess posted a link to a great graphical address space analysis tool that can actually visualize the cut that mscomctl.ocx and it's followers tear through my virtual memory blanket.

However, I wish there would be a better and standard way to integrate such great graphical tools with the debugger. A single picture says more than thousand lines of obscure command line responses.

I started something. There seemed to be a memory monster in my process and I wanted to find it.

To investigate who was pulling the blanket, I looked at the allocated virtual addresses with the wonderful windbg extension command "!address". However, this leads to information overflow and is generally of little use for this stage of my analysis.

But it indicates the problem could be rooted in the address space fragmentation caused by the loaded modules. I fed the output of "lm" into Excel to calculate the room between the load addresses. And there it was, my monster:

Monster, thy name is MSCOMCTL.OCX, summoned by an innocuous legacy (i.e. VB6) library. It breaks apart my pristine address space into pieces. I wonder why that is, let's dumpbin ...

Dump of file C:\Windows\syswow64\mscomctl.ocx

PE signature found

File Type: DLL

FILE HEADER VALUES
             14C machine (x86)
               4 number of sections
        3F2F1329 time date stamp Tue Aug 05 04:15:05 2003
               0 file pointer to symbol table
               0 number of symbols
              E0 size of optional header
            2302 characteristics
                   Executable
                   32 bit word machine
                   Debug information stripped
                   DLL

OPTIONAL HEADER VALUES
             10B magic # (PE32)
            5.02 linker version
           ACA00 size of code
           56C00 size of initialized data
               0 size of uninitialized data
            89D8 entry point (275889D8)
            1000 base of code
           A7000 base of data
        27580000 image base (27580000 to 27685FFF)

Somebody disobeyed the ancient proposals of rebasing Win32 DLLs. Maybe because in 2003 no machine with more than 512MB of RAM would be running VB6 code?

"Optimizing" module load addresses was a great performance skill back in the days, when CPU cycles were scarce and jumping around in mapped memory hurt even more. Today, nobody could probably care less.

Since Microsoft owns and signs MSCOMCTL.OCX, I cannot do the old rebase trick and let the app modify the file itself, which would be utterly stupid anyway. But, there is no version of LoadLibrary, that would allow me to "unforce" the image base address.

So what can I do?

SYMPTOM:

Premature fragmentation of the address space.

CAUSE:

Some modules are linked with arbitrary, inappropriate load addresses.

RESOLUTION:

Block the respective address ranges as early as possible, but only temporarily using VirtualAlloc(...,...,MEM_RESERVE, ...) and VirtualFree(..., MEM_RELEASE). This works only for modules that are loaded late in the game, most likely through delay-loading, JIT compilation or COM activation.

Let's see where this will leaves us ...

If you're interested in the equivalence of Stephen King's "It" in the world of software programming, have a look at ReedR's Slaying the Virtual Memory Monster post. It's really scary.

This week I'm starting to face a similar monster and that is Large Object HeapManaged Address Space Fragmentation, as aptly described in Andrew Hunter's The Dangers of the Large Object Heap.

However, just like "It" lurking in sewers and dark waters, my monster hides behind the 3rd-party curtain, in the disguise of the Microsoft Report Viewer Redistributable controls.

SYMPTOM:

After displaying a couple of large server reports, where "large" means many Custom Report Items, i.e. bitmaps, rendering fails with "Exception of type System.OutOfMemoryException was thrown."

Investigation shows the exception occurs on the client:

0:017> !CLRStack -p
OS Thread Id: 0x1bc (17)
ESP       EIP     
0f71ea54 7c90e514 [HelperMethodFrame: 0f71ea54] 
0f71eac0 792ec8d0 System.IO.MemoryStream.set_Capacity(Int32)
    PARAMETERS:
        this = 0x11b50308
        value = 0x10384000

0f71ead4 792ec868 System.IO.MemoryStream.EnsureCapacity(Int32)
    PARAMETERS:
        this = <no data>
        value = <no data>

0f71eae0 792ec7b6 System.IO.MemoryStream.Write(Byte[], Int32, Int32)
    PARAMETERS:
        this = 0x11b50308
        buffer = 0x11b6d45c
        offset = 0x00000000
        count = 0x0000fe53

0f71eb10 09c4a8d3 Microsoft.Reporting.WinForms.ServerReport.ServerUrlRequest(Boolean, System.String, System.IO.Stream, System.String ByRef, System.String ByRef)
    PARAMETERS:
        this = <no data>
        isAbortable = <no data>
        url = <no data>
        outputStream = 0x11b50308
        mimeType = <no data>
        fileNameExtension = <no data>

0f71eb54 09c4a72a Microsoft.Reporting.WinForms.ServerReport.InternalRender(Boolean, System.String, System.String, System.Collections.Specialized.NameValueCollection, System.IO.Stream, System.String ByRef, System.String ByRef)
    PARAMETERS:
        this = 0x0d7a956c
        isAbortable = <no data>
        format = <no data>
        deviceInfo = <no data>
        urlAccessParameters = <no data>
        reportStream = <no data>
        mimeType = <no data>
        fileNameExtension = <no data>

CAUSE:

The process' address space is fragmented to the point that a large object (byte[272121856]) cannot be allocated because the longest contiguous block is just little over 120 MB.

0:017> !address -summary

-------------------- Usage SUMMARY --------------------------
    TotSize (      KB)   Pct(Tots) Pct(Busy)   Usage
   37942000 (  910600) : 43.42%    80.68%    : RegionUsageIsVAD
   3b1b8000 (  968416) : 46.18%    00.00%    : RegionUsageFree
    ab10000 (  175168) : 08.35%    15.52%    : RegionUsageImage
    1300000 (   19456) : 00.93%    01.72%    : RegionUsageStack
      13000 (      76) : 00.00%    00.01%    : RegionUsageTeb
    16d0000 (   23360) : 01.11%    02.07%    : RegionUsageHeap
          0 (       0) : 00.00%    00.00%    : RegionUsagePageHeap
       1000 (       4) : 00.00%    00.00%    : RegionUsagePeb
       1000 (       4) : 00.00%    00.00%    : RegionUsageProcessParametrs
       1000 (       4) : 00.00%    00.00%    : RegionUsageEnvironmentBlock
       Tot: 7fff0000 (2097088 KB) Busy: 44e38000 (1128672 KB)

-------------------- Type SUMMARY --------------------------
    TotSize (      KB)   Pct(Tots)  Usage
   3b1b8000 (  968416) : 46.18%   : <free>
    cda4000 (  210576) : 10.04%   : MEM_IMAGE
    13e0000 (   20352) : 00.97%   : MEM_MAPPED
   36cb4000 (  897744) : 42.81%   : MEM_PRIVATE

-------------------- State SUMMARY --------------------------
    TotSize (      KB)   Pct(Tots)  Usage
   3cf34000 (  998608) : 47.62%   : MEM_COMMIT
   3b1b8000 (  968416) : 46.18%   : MEM_FREE
    7f04000 (  130064) : 06.20%   : MEM_RESERVE

Largest free region: Base 45fd0000 - Size 07e20000 (129152 KB)

RESOLUTION:

I'm not in control of the many byte arrays used to read the various streams that make up a rendered report. However, I shall endeavor a thorough investigation into all the chunks that stain my memory blanket to see how I would possibly be able to contain the same.

To be continued ...

If you’re one of the two people using Windows CardSpace with myOpenID and recently updated to IE8, you may have to switch to “Compatibility View” on the myOpenID login page to be able to log in.

Google has it, other services using Information Card, such as Windows Azure have had similar problems.