w3wp.exe going to 100%

Hello, everyone.

I have an aspx page that queries a OLAP cube via ADO (not ADO.NET).

The page is run at various times during the day on an ad-hoc basis, and is also run 90 times each morning to make canned reports.

This had been running fine for a month or two; recently, however, the first time the page is run each morning, the system spikes to 100% cpu utilization, and the entire web application becomes unresponsive.

When I first started seeing the problem, the 100% was distributed between lsass.exe and w3wp.exe (mostly lsass.exe). When this occured, if I rebooted the system, then re-ran the canned reports, everything would be OK. Until the next day.

Yesterday, I applied the lsass.exe hotfix from KB article 821008. This morning, lsass.exe was fine, but w3wp.exe had spiked to 100% utilization, and the web app was again unresponsive. I recycled w3wp.exe, and was able to re-run the canned reports.

I took a IISState log this morning (see below). Not sure how to interpret it, but it seems like the MSOLAP OLE DB driver may be blocking the other threads? I appreciate any help...

Jonathan

------------8<-------------8<--------------------

Opened log file 'C:\iisstate\output\IISState-1684.log'

***********************
Starting new log output
IISState version 3.3.1

Wed Jul 07 10:29:16 2004

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 1684

Note: Thread times are formatted as HH:MM:SS.ms

***********************




Thread ID: 0
System Thread ID: d58
Kernel Time: 0:0:0.62
User Time: 0:0:0.46
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fe58 77f23741 SharedUserData!SystemCallStub+0x4
01 0006fe5c 77de1817 ntdll!ZwWaitForSingleObject+0xc
02 0006fecc 77de168f kernel32!WaitForSingleObjectEx+0xac
03 0006fedc 589435ae kernel32!WaitForSingleObject+0xf
04 0006feec 58944d7d w3dt!WP_CONTEXT::RunMainThreadLoop+0xe
05 0006fef4 58981d98 w3dt!UlAtqStartListen+0x2d
06 0006ff00 58985d54 w3core!W3_SERVER::StartListen+0x8e
07 0006ff0c 010017af w3core!UlW3Start+0xea
08 0006ff44 0100195e w3wp!wmain+0x236
09 0006ffc0 77def38c w3wp!wmainCRTStartup+0x12f
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23




Thread ID: 1
System Thread ID: d9c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 00a1ff9c 77f2262b SharedUserData!SystemCallStub+0x4
01 00a1ffa0 77f4b5b2 ntdll!NtDelayExecution+0xc
02 00a1ffb8 77dea990 ntdll!RtlpTimerThread+0x45
03 00a1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 2
System Thread ID: ebc
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff28 77f230c7 SharedUserData!SystemCallStub+0x4
01 00ccff2c 77de30bc ntdll!ZwRemoveIoCompletion+0xc
02 00ccff58 588e2360 kernel32!GetQueuedCompletionStatus+0x27
03 00ccff90 588e2531 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x31
04 00ccffa4 588e1d27 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x21
05 00ccffb8 77dea990 W3TP!THREAD_MANAGER::ThreadManagerThread+0x35
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 3
System Thread ID: d0c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff28 77f230c7 SharedUserData!SystemCallStub+0x4
01 00d0ff2c 77de30bc ntdll!ZwRemoveIoCompletion+0xc
02 00d0ff58 588e2360 kernel32!GetQueuedCompletionStatus+0x27
03 00d0ff90 588e2531 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x31
04 00d0ffa4 588e1d27 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x21
05 00d0ffb8 77dea990 W3TP!THREAD_MANAGER::ThreadManagerThread+0x35
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 4
System Thread ID: 3f8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff28 77f230c7 SharedUserData!SystemCallStub+0x4
01 00d4ff2c 77de30bc ntdll!ZwRemoveIoCompletion+0xc
02 00d4ff58 588e2360 kernel32!GetQueuedCompletionStatus+0x27
03 00d4ff90 588e2531 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x31
04 00d4ffa4 588e1d27 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x21
05 00d4ffb8 77dea990 W3TP!THREAD_MANAGER::ThreadManagerThread+0x35
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 5
System Thread ID: 928
Kernel Time: 0:0:0.156
User Time: 0:0:0.640
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d8ff28 77f230c7 SharedUserData!SystemCallStub+0x4
01 00d8ff2c 77de30bc ntdll!ZwRemoveIoCompletion+0xc
02 00d8ff58 588e2360 kernel32!GetQueuedCompletionStatus+0x27
03 00d8ff90 588e2531 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x31
04 00d8ffa4 588e1d27 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x21
05 00d8ffb8 77dea990 W3TP!THREAD_MANAGER::ThreadManagerThread+0x35
06 00d8ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 6
System Thread ID: df4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e4fcec 77f2372d SharedUserData!SystemCallStub+0x4
01 00e4fcf0 77f55297 ntdll!NtWaitForMultipleObjects+0xc
02 00e4ffb8 77dea990 ntdll!RtlpWaitThread+0x158
03 00e4ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 7
System Thread ID: eb4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0170ff14 77f23741 SharedUserData!SystemCallStub+0x4
01 0170ff18 77de1817 ntdll!ZwWaitForSingleObject+0xc
02 0170ff88 77de168f kernel32!WaitForSingleObjectEx+0xac
03 0170ff98 589851f7 kernel32!WaitForSingleObject+0xf
04 0170ffb8 77dea990 w3core!HTTP_COMPRESSION::CompressionThread+0x85
05 0170ffc4 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 8
System Thread ID: d0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.

No remote call being made

# ChildEBP RetAddr
00 0174fe20 77f2313f SharedUserData!SystemCallStub+0x4
01 0174fe24 77bf7b85 ntdll!NtReplyWaitReceivePortEx+0xc
02 0174ff8c 77c00829 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x193
03 0174ff90 77c00771 RPCRT4!RecvLotsaCallsWrapper+0x9
04 0174ffb0 77c00857 RPCRT4!BaseCachedThreadRoutine+0x9c
05 0174ffb8 77dea990 RPCRT4!ThreadStartRoutine+0x17
06 0174ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 9
System Thread ID: f10
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0178ff8c 77f2262b SharedUserData!SystemCallStub+0x4
01 0178ff90 77f4f0fc ntdll!NtDelayExecution+0xc
02 0178ffb8 77dea990 ntdll!RtlpIOWorkerThread+0x3d
03 0178ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 10
System Thread ID: 1c8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
succeeded
Loaded Son of Strike data table version 5 from "C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
Thread 10
Not a managed thread.


Begin System Thread Information

# ChildEBP RetAddr
00 0188fee4 77f2262b SharedUserData!SystemCallStub+0x4
01 0188fee8 77de18ea ntdll!NtDelayExecution+0xc
02 0188ff50 77de16ee kernel32!SleepEx+0x68
03 0188ff5c 792e9ebe kernel32!Sleep+0xb
04 0188ffb8 77dea990 mscorwks!ThreadpoolMgr::GateThreadStart+0x54
05 0188ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 11
System Thread ID: c70
Kernel Time: 0:0:4.953
User Time: 0:1:28.250
*** ERROR: Symbol file could not be found. Defaulted to export symbols for -
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 018cf538 77f23741 SharedUserData!SystemCallStub+0x4
01 018cf53c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 018cf578 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 018cf580 7ad603a9 ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 018cf590 7ad6b384 msolap80!Ordinal98+0x203a9
05 018cf5c8 084a1344 msolap80!PrivateConnectionDialogSetDB+0x4867
06 51e30a5c 0844cb2c 0x84a1344
07 00000000 00000000 0x844cb2c




Thread ID: 12
System Thread ID: f20
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 12
Not a managed thread.


Begin System Thread Information

# ChildEBP RetAddr
00 019cfe74 77f2372d SharedUserData!SystemCallStub+0x4
01 019cfe78 77de1bfa ntdll!NtWaitForMultipleObjects+0xc
02 019cff20 77deb0e4 kernel32!WaitForMultipleObjectsEx+0x11a
03 019cff38 791d25d5 kernel32!WaitForMultipleObjects+0x17
04 019cffa0 791d254c mscorwks!DebuggerRCThread::MainLoop+0x90
05 019cffb0 791d4d50 mscorwks!DebuggerRCThread::ThreadProc+0x68
06 019cffb8 77dea990 mscorwks!DebuggerRCThread::ThreadProcStatic+0xb
07 019cffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 13
System Thread ID: 738
Kernel Time: 0:0:0.15
User Time: 0:0:2.609
Thread Status: Thread is in a WAIT state.
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 13
ESP EIP


Begin System Thread Information

# ChildEBP RetAddr
00 01aafe90 77f2372d SharedUserData!SystemCallStub+0x4
01 01aafe94 77de1bfa ntdll!NtWaitForMultipleObjects+0xc
02 01aaff3c 77deb0e4 kernel32!WaitForMultipleObjectsEx+0x11a
03 01aaff54 79262acd kernel32!WaitForMultipleObjects+0x17
04 01aaff74 791bbea4 mscorwks!WaitForFinalizerEvent+0xc8
05 01aaffb8 77dea990 mscorwks!GCHeap::FinalizerThreadStart+0x96
06 01aaffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 14
System Thread ID: c90
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 14
Not a managed thread.


Begin System Thread Information

# ChildEBP RetAddr
00 01e5ff38 77f2262b SharedUserData!SystemCallStub+0x4
01 01e5ff3c 77de18ea ntdll!NtDelayExecution+0xc
02 01e5ffa4 792ea483 kernel32!SleepEx+0x68
03 01e5ffb8 77dea990 mscorwks!ThreadpoolMgr::TimerThreadStart+0x30
04 01e5ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 15
System Thread ID: 948
Kernel Time: 0:0:0.31
User Time: 0:0:0.0
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 15
ESP EIP


Begin System Thread Information

# ChildEBP RetAddr
00 6e90ff54 77f230c7 SharedUserData!SystemCallStub+0x4
01 6e90ff58 77de30bc ntdll!ZwRemoveIoCompletion+0xc
02 6e90ff84 792e9b2f kernel32!GetQueuedCompletionStatus+0x27
03 6e90ffb8 77dea990 mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0x49
04 6e90ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 16
System Thread ID: 70c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 7a41f348 77f23741 SharedUserData!SystemCallStub+0x4
01 7a41f34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 7a41f388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 7a41f390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 7a41f5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 7a41f634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 7a41f678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 7a41f684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 7a41f6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 7a41f724 08471d35 0x84a0f0c
0a 7a41f76c 08471abe 0x8471d35
0b 51e3d0ac 696d7265 0x8471abe
0c 50797469 00000000 0x696d7265




Thread ID: 17
System Thread ID: ccc
Kernel Time: 0:0:0.15
User Time: 1:15:34.359
Thread Type: Other
# ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
00 7afdfe5c 7ae26dd4 msolap80!DllRegisterServer+0xb7bcc
01 7afdfe6c 7ae26aed msolap80!DllRegisterServer+0xb77bc
02 7afdfe84 7adacc13 msolap80!DllRegisterServer+0xb74d5
03 7afdfea4 7adaccb3 msolap80!DllRegisterServer+0x3d5fb
04 7afdfec8 7ad69ed5 msolap80!DllRegisterServer+0x3d69b
05 7afdfef8 7ad7dd78 msolap80!PrivateConnectionDialogSetDB+0x33b8
06 7afdff20 7ad7dc7b msolap80!DllRegisterServer+0xe760
07 7afdff7c 7ad7dba9 msolap80!DllRegisterServer+0xe663
08 7afdff84 77b690a2 msolap80!DllRegisterServer+0xe591
09 7afdffb8 77dea990 msvcrt!_endthread+0xa6
0a 7afdffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 18
System Thread ID: 8ec
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 7a38f348 77f23741 SharedUserData!SystemCallStub+0x4
01 7a38f34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 7a38f388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 7a38f390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 7a38f5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 7a38f634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 7a38f678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 7a38f684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 7a38f6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 7a38f724 08471d35 0x84a0f0c
0a 7a38f76c 08471abe 0x8471d35
0b 51e4ff18 00000000 0x8471abe




Thread ID: 19
System Thread ID: e00
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 7f5af348 77f23741 SharedUserData!SystemCallStub+0x4
01 7f5af34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 7f5af388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 7f5af390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 7f5af5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 7f5af634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 7f5af678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 7f5af684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 7f5af6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 7f5af724 08471d35 0x84a0f0c
0a 7f5af76c 08471abe 0x8471d35
0b 51e62d64 00000000 0x8471abe




Thread ID: 20
System Thread ID: ea4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
*** WARNING: Unable to verify checksum for c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net files\ranking\20142246\2d848f07\assembly\dl2\d2e63100\1e99732b_0160c401\dotnetcharting.dll
*** ERROR: Module load completed but symbols could not be loaded for c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net files\ranking\20142246\2d848f07\assembly\dl2\d2e63100\1e99732b_0160c401\dotnetcharting.dll
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 01f1f348 77f23741 SharedUserData!SystemCallStub+0x4
01 01f1f34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 01f1f388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 01f1f390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 01f1f5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 01f1f634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 01f1f678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 01f1f684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 01f1f6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 01f1f724 08471d35 0x84a0f0c
0a 01f1f76c 08471abe 0x8471d35
0b 51e75a88 11090000 0x8471abe
0c 00100000 000005b1 dotnetcharting+0x90000




Thread ID: 21
System Thread ID: 878
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 64ebf348 77f23741 SharedUserData!SystemCallStub+0x4
01 64ebf34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 64ebf388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 64ebf390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 64ebf5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 64ebf634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 64ebf678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 64ebf684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 64ebf6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 64ebf724 08471d35 0x84a0f0c
0a 64ebf76c 08471abe 0x8471d35
0b 51e8890c 00180662 0x8471abe
0c 75707465 00000000 0x180662




Thread ID: 22
System Thread ID: a8c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 7c33f348 77f23741 SharedUserData!SystemCallStub+0x4
01 7c33f34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 7c33f388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 7c33f390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 7c33f5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 7c33f634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 7c33f678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 7c33f684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 7c33f6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 7c33f724 08471d35 0x84a0f0c
0a 7c33f76c 08471abe 0x8471d35
0b 51e9b7a0 00000000 0x8471abe




Thread ID: 23
System Thread ID: dc8
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 7f62f348 77f23741 SharedUserData!SystemCallStub+0x4
01 7f62f34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 7f62f388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 7f62f390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 7f62f5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 7f62f634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 7f62f678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 7f62f684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 7f62f6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 7f62f724 08471d35 0x84a0f0c
0a 7f62f76c 08471abe 0x8471d35
0b 51eae4d0 64657361 0x8471abe
0c 426e656b 00000000 0x64657361




Thread ID: 24
System Thread ID: 288
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
*** WARNING: Unable to verify checksum for c:\windows\assembly\nativeimages1_v1.1.4322\mscorlib\1.0.5000.0__b77a5c561934e089_eb18131b\mscorlib.dll
*** ERROR: Module load completed but symbols could not be loaded for c:\windows\assembly\nativeimages1_v1.1.4322\mscorlib\1.0.5000.0__b77a5c561934e089_eb18131b\mscorlib.dll
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 24
ESP EIP
7f56f6e8 7ffe0304 [FRAME: ComPlusMethodFrameStandalone] [DEFAULT] [hasThis] I4 System.Data.Common.UnsafeNativeMethods/IDBInitialize.Initialize()
7f56f6f8 0888ed63 [DEFAULT] [hasThis] Void System.Data.OleDb.OleDbConnection.InitializeProvider()
7f56f700 0888e83a [DEFAULT] [hasThis] Void System.Data.OleDb.OleDbConnection.Open()
7f56f72c 08471d35 [DEFAULT] Void System.Data.Common.DbDataAdapter.QuietOpen(Class System.Data.IDbConnection,ByRef ValueClass System.Data.ConnectionState)
7f56f738 08471b9c [DEFAULT] [hasThis] I4 System.Data.Common.DbDataAdapter.FillFromCommand(Object,I4,I4,String,Class System.Data.IDbCommand,ValueClass System.Data.CommandBehavior)
7f56f788 08471abe [DEFAULT] [hasThis] I4 System.Data.Common.DbDataAdapter.Fill(Class System.Data.DataSet,I4,I4,String,Class System.Data.IDbCommand,ValueClass System.Data.CommandBehavior)
7f56f7b0 08471a0f [DEFAULT] [hasThis] I4 System.Data.Common.DbDataAdapter.Fill(Class System.Data.DataSet)
7f56f7bc 7a4418c4 [DEFAULT] [hasThis] Void Ranking.RankingChart.Render(Class System.Web.UI.HtmlTextWriter)
at [+0x77c] [+0x2b3] c:\inetpub\wwwroot\Ranking\RankingChart.aspx.vb:123
PARAM: this: 0x51bfbc5c (ASP.RankingChart_aspx)
PARAM: int8 writer: 1371536656
7f56f950 0854d3d2 [DEFAULT] [hasThis] Void System.Web.UI.Control.RenderControl(Class System.Web.UI.HtmlTextWriter)
7f56f968 083fe3a5 [DEFAULT] [hasThis] Void System.Web.UI.Page.ProcessRequestMain()
7f56f9ec 083fb576 [DEFAULT] [hasThis] Void System.Web.UI.Page.ProcessRequest()
7f56fa28 083fafe3 [DEFAULT] [hasThis] Void System.Web.UI.Page.ProcessRequest(Class System.Web.HttpContext)
7f56fa30 083fafbc [DEFAULT] [hasThis] Void System.Web.HttpApplication/CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute()
7f56fa40 08407468 [DEFAULT] [hasThis] Class System.Exception System.Web.HttpApplication.ExecuteStep(Class IExecutionStep,ByRef Boolean)
7f56fa88 08406fc3 [DEFAULT] [hasThis] Void System.Web.HttpApplication.ResumeSteps(Class System.Exception)
7f56fad0 081e8d02 [DEFAULT] [hasThis] Class System.IAsyncResult System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(Class System.Web.HttpContext,Class System.AsyncCallback,Object)
7f56faec 01bf678f [DEFAULT] [hasThis] Void System.Web.HttpRuntime.ProcessRequestInternal(Class System.Web.HttpWorkerRequest)
7f56fb28 01bf6340 [DEFAULT] Void System.Web.HttpRuntime.ProcessRequest(Class System.Web.HttpWorkerRequest)
7f56fb34 01bf2f5d [DEFAULT] [hasThis] I4 System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
7f56fbec 79206069 [FRAME: ContextTransitionFrame]
7f56fcc8 79206069 [FRAME: ComMethodFrame]


Begin System Thread Information

# ChildEBP RetAddr
00 7f56f348 77f23741 SharedUserData!SystemCallStub+0x4
01 7f56f34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 7f56f388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 7f56f390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 7f56f5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 7f56f634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 7f56f678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 7f56f684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 7f56f6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 7f56f724 08471d35 0x84a0f0c
0a 7f56f76c 08471abe 0x8471d35
0b 51ec139c 79bbf5e4 0x8471abe
0c 00000000 00000000 mscorlib_79980000+0x23f5e4




Thread ID: 25
System Thread ID: 8e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 7a3cf348 77f23741 SharedUserData!SystemCallStub+0x4
01 7a3cf34c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 7a3cf388 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 7a3cf390 7ad6803f ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may be wrong.
04 7a3cf5e0 7ad679d8 msolap80!PrivateConnectionDialogSetDB+0x1522
05 7a3cf634 7ad67c15 msolap80!PrivateConnectionDialogSetDB+0xebb
06 7a3cf678 7ad675ac msolap80!PrivateConnectionDialogSetDB+0x10f8
07 7a3cf684 7ad6bdc0 msolap80!PrivateConnectionDialogSetDB+0xa8f
08 7a3cf6c0 084a0f0c msolap80!PrivateConnectionDialogSetDB+0x52a3
09 7a3cf724 08471d35 0x84a0f0c
0a 7a3cf76c 08471abe 0x8471d35
0b 51ed4248 00000000 0x8471abe




Thread ID: 26
System Thread ID: 7d0
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Other
# ChildEBP RetAddr
00 7f6aff70 77f230c7 SharedUserData!SystemCallStub+0x4
01 7f6aff74 77f5e6ae ntdll!ZwRemoveIoCompletion+0xc
02 7f6affb8 77dea990 ntdll!RtlpWorkerThread+0x3b
03 7f6affec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 27
System Thread ID: 918
Kernel Time: 0:0:0.62
User Time: 0:0:0.656
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 27
ESP EIP


Begin System Thread Information

# ChildEBP RetAddr
00 7f82ff10 77f23741 SharedUserData!SystemCallStub+0x4
01 7f82ff14 77de1817 ntdll!ZwWaitForSingleObject+0xc
02 7f82ff84 77de168f kernel32!WaitForSingleObjectEx+0xac
03 7f82ff94 792e91e0 kernel32!WaitForSingleObject+0xf
04 7f82ffb8 77dea990 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x3a
05 7f82ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 28
System Thread ID: 940
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 28
Not a managed thread.


Begin System Thread Information

# ChildEBP RetAddr
00 7facff24 77f2262b SharedUserData!SystemCallStub+0x4
01 7facff28 77de18ea ntdll!NtDelayExecution+0xc
02 7facff90 792ea315 kernel32!SleepEx+0x68
03 7facffb8 77dea990 mscorwks!ThreadpoolMgr::WaitThreadStart+0x22
04 7facffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 29
System Thread ID: f00
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 084eff70 77f230c7 SharedUserData!SystemCallStub+0x4
01 084eff74 77f5e6ae ntdll!ZwRemoveIoCompletion+0xc
02 084effb8 77dea990 ntdll!RtlpWorkerThread+0x3b
03 084effec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 30
System Thread ID: 84
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0852ff70 77f230c7 SharedUserData!SystemCallStub+0x4
01 0852ff74 77f5e6ae ntdll!ZwRemoveIoCompletion+0xc
02 0852ffb8 77dea990 ntdll!RtlpWorkerThread+0x3b
03 0852ffec 00000000 kernel32!BaseThreadStart+0x34

Closing open log file C:\iisstate\output\IISState-1684.log

Hi Jonathan,

Thread ID: 11
System Thread ID: c70
Kernel Time: 0:0:4.953
User Time: 0:1:28.250
*** ERROR: Symbol file could not be found. Defaulted to export
symbols for -
Other information: Thread is waiting for a lock to be released.
Looking for lock owner.
Owning thread System ID: ccc
Thread Type: Other
# ChildEBP RetAddr
00 018cf538 77f23741 SharedUserData!SystemCallStub+0x4
01 018cf53c 77f3d64e ntdll!ZwWaitForSingleObject+0xc
02 018cf578 77f22044 ntdll!RtlpWaitForCriticalSection+0x126
03 018cf580 7ad603a9 ntdll!RtlEnterCriticalSection+0x46
WARNING: Stack unwind information not available. Following frames may
be wrong.
04 018cf590 7ad6b384 msolap80!Ordinal98+0x203a9
05 018cf5c8 084a1344 msolap80!PrivateConnectionDialogSetDB+0x4867
06 51e30a5c 0844cb2c 0x84a1344
07 00000000 00000000 0x844cb2c
...

Thread ID: 17
System Thread ID: ccc
Kernel Time: 0:0:0.15
User Time: 1:15:34.359
Thread Type: Other
# ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may
be wrong.
00 7afdfe5c 7ae26dd4 msolap80!DllRegisterServer+0xb7bcc
01 7afdfe6c 7ae26aed msolap80!DllRegisterServer+0xb77bc
02 7afdfe84 7adacc13 msolap80!DllRegisterServer+0xb74d5
03 7afdfea4 7adaccb3 msolap80!DllRegisterServer+0x3d5fb
04 7afdfec8 7ad69ed5 msolap80!DllRegisterServer+0x3d69b
05 7afdfef8 7ad7dd78 msolap80!PrivateConnectionDialogSetDB+0x33b8
06 7afdff20 7ad7dc7b msolap80!DllRegisterServer+0xe760
07 7afdff7c 7ad7dba9 msolap80!DllRegisterServer+0xe663
08 7afdff84 77b690a2 msolap80!DllRegisterServer+0xe591
09 7afdffb8 77dea990 msvcrt!_endthread+0xa6
0a 7afdffec 00000000 kernel32!BaseThreadStart+0x34

The dealock of msolap80.dll is obvious, there are a batch of threads
waiting for it to release the lock. However, the DllRegisterServer
function call in the log is masked and useless. This is because
Symbols doesn't match and so iisstate use the exposed function name
DllRegisterServer and an offset to represent the actual function
call. Therefore from the log we cannot determine what the thread was
doing at that time.

I saw an exactly similar SQL2K/OLAP/ASP case with this kind of call
stack is currently opened in PSS. I recommend you also report an
incident to PSS for debugging. A SQL engineer who has the internal
Symbols of msolap80.dll can help you find out the root cause. Also,
if the issue is finally confirmed being caused by a bug, the case
charge will be refunded or waived. You can find your local PSS phone
number from the following link:

http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS

Best regards,

WenJun Zhang
Microsoft Online Support
This posting is provided "AS IS" with no warranties, and confers no
rights.
Get Secure! - www.microsoft.com/security