High CPU in ASP.NET - Finding Large DataTables with the Public SOS

This is yet another one strictly for the dump analysis crowd – what can I say, it’s been a long week! Last episode, we highlighted the lack of support for many useful windbg commands. Well, chief among these was the DumpDataTables command that provides a nice summary of the largest tables and their column counts, typically a nice starting point for high-memory/high-CPU issues in traditional application. So how do we fend for ourselves and reproduce this with the SOS that we have?

Some very helpful support engineers that I’ve been working with over the last few weeks offered the following as a potential substitute:

   1: $$ Run as $><C:\Extensions\DumpLargeTables.txt

   2:  

   3: .printf "\n===== Printing Large datatables with 1000+ rows with their column names =====\n\n"

   4:  

   5: .foreach(m_table {!dumpheap -type System.Data.DataTable -short})

   6: {

   7:     .foreach (m_Count {!dumpfield -field nextRowID m_table})

   8:     {

   9:          $$ Change the value of row count here

  10:         .if( ${m_Count} > 1000)

  11:         {

  12:             .printf "Table Name : "; .printf /D "<?dml?><exec cmd=\"!do ${m_table}\">${m_table}</exec> - "

  13:             !dumpfield -field tableName -string ${m_table} 

  14:             .printf " Row Count : ${m_Count}"

  15:             .printf "\t|---Columns\n";

  16:  

  17:             .foreach(m_col {!dumpheap -type System.Data.DataColumn -short})

  18:             {

  19:                 r @$t2 = poi(${m_col}+0x3c)

  20:                 .if(@$t2 == ${m_table})

  21:                 {

  22:                     .printf /D "\t\t|-- <?dml?><exec cmd=\"!do ${m_col}\">${m_col}</exec> - ";

  23:                     !dumpfield -field _columnName -string m_col

  24:                 }

  25:             }

  26:             .printf "\n";

  27:         }

  28:     }

  29: }

Unfortunately, this too relies on a command (!dumpfield) that is also not available in the public SOS.

Through some collaborative trial-and-error, we then came up with the following work-around for accessing the nextRowID field using its offset, in a manor supported by the public SOS:

   1: .foreach (DT {!dumpheap -mt 65242d0c -short}){.echo ${DT};?poi(${DT}+0xd0)}

…which gives you something to work with if you drop it in excel and sort it (original addresses removed):

   1: Evaluate expression: 14 = 0000000e

   2: Evaluate expression: 3 = 00000003

   3: Evaluate expression: 20 = 00000014

   4: Evaluate expression: 29 = 0000001d

   5: Evaluate expression: 11 = 0000000b

   6: Evaluate expression: 28 = 0000001c

   7: Evaluate expression: 1 = 00000001

   8: Evaluate expression: 1 = 00000001

To be version/Method-Table agnostic, I tried the following, which should be comparable to the one above:

   1: .foreach (DT {!dumpheap –type System.Data.DataTable -short}){?poi(${DT}+0xd0)}

…but I see some strange results with the latter, so find your method tables and work from there.

How to read this (we’re simply using the POI function to read the address + offset for nextRowId - d0):

   1: 0:026> !do 0x02c671cc 

   2: Name: System.Data.DataTable

   3: MethodTable: 65242d0c

   4: EEClass: 6515da18

   5: Size: 296(0x128) bytes

   6: GC Generation: 2

   7: (C:\WINDOWS\assembly\GAC_32\System.Data\2.0.0.0__b77a5c561934e089\System.Data.dll)

   8: Fields:

   9:       MT    Field   Offset                 Type VT     Attr    Value Name

  10: 7a5d2a58  40009a3        4 ...ponentModel.ISite  0 instance 00000000 site

  11: 7a5ecd2c  40009a4        8 ....EventHandlerList  0 instance 00000000 events

  12: 7933061c  40009a2      1e4        System.Object  0   shared   static EventDisposed

  13: 65244194  40007a5        c  System.Data.DataSet  0 instance 00000000 dataSet

  14: 6524534c  40007a6       10 System.Data.DataView  0 instance 02c682d0 defaultView

  15: 79332c4c  40007a7       d0         System.Int32  1 instance        3 nextRowID

  16: 65245798  40007a8       14 ...DataRowCollection  0 instance 02c674d8 rowCollection

  17: 652456b8  40007a9       18 ...aColumnCollection  0 instance 02c67388 columnCollection

  18: 65245728  40007aa       1c ...straintCollection  0 instance 02c674a0 constraintCollection

  19: 79332c4c  40007ab       d4         System.Int32  1 instance        2 elementColumnCount

  20: 65244b08  40007ac       20 ...elationCollection  0 instance 02c6845c parentRelationsCollection

  21: 65244b08  40007ad       24 ...elationCollection  0 instance 02c683f8 childRelationsCollection

References:

[1] –
http://blogs.msdn.com/vijaysk/archive/2008/01/15/windbg-scripting-dump-data-column-names-from-a-table.aspx

[2] –
http://blogs.msdn.com/mvstanton/archive/2005/10/11/479861.aspx

[3] –
http://mcfunley.com/205/using-windbg-to-log-exceptions-part-2