SQL Server Cluster resource doesn’t come online | Service control: stop before startup

It was Monday morning. With great difficulty I was getting ready to work (as usual). To halt all my activities I get a call from Office stating that one of the SQL Server 2005 resource on a Windows 2003 Cluster was not coming online. I logged on to the server and checked the logs.

The System Event Log did not have much information except for this error.

I had posted about this error sometime back, when one of the DR instances was not coming starting. I did not find any value in increasing the timeout value on this instance since there was hardly any activity on this instance and the resource utilization was minimum. The SQL Server Error Log had the following messages.

Using dynamic lock allocation. Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.
This is an informational message only. No user action is required.
Attempting to initialize Microsoft Distributed Transaction Coordinator (MS DTC).
This is an informational message only. No user action is required.
Service control: stop before startup

This error message was not much useful in finding out the root cause. Then I checked the Cluster Log (c:\windows\cluster\cluster.log)

ERR SQL Server : [sqsrvres] OnlineThread: ResUtilsStartResourceService failed (status 5b4)
ERR SQL Server : [sqsrvres] OnlineThread: Error 5b4 bringing resource online.

I had posted here on how to get meaningful information from the hexadecimal error codes in the Cluster Log. Error 5b4 means this operation returned because the timeout period expired. Since this error message was also related to timeout, it was not much useful.

After a while I saw this error message in the System Event Log.

Severity: 16 Error:2, OS: 2 [Microsoft][SQL Native Client]Named Pipes Provider: Could not open a connection to SQL Server [2]

This made it clear that the Cluster Manager was not able to connect to the SQL Server instance using Named pipes while trying to start it. Quickly checked if the Named Pipes protocol was disabled in SQL Server Configuration Manager. It was not.

Then I started SQL Server from the command prompt using sqlservr.exe, there were no errors reported and all the databases came online. With the sqlservr.exe running in another window, I was able to connect to the SQL Server instance through SSMS using both the Instance Name and Pipes. This confirmed that there were no issues with SQL Server, only the Operating System was not able to connect to the Instance.

By this time, this issue was already raised to a High Severity incident. Enter the Windows Administrators. They informed on the bridge that they had a Change Control to apply a Security patch. The Security patch was installed successfully (surprisingly there was no entry in the Event Log) but it was supposed to be rebooted. By that time, the Maintenance Window was over and they were reaching out the concerned for the approvals for the ad hoc reboot. Before they could *reach out*, the scene was already created.

Once they got the approvals, both nodes of the cluster were rebooted. Like always, the reboot did the trick and the SQL Server resource came online! The DBA could do nothing much here but this incident reiterated the fact that communication is important in all walks of life.

2 thoughts on “SQL Server Cluster resource doesn’t come online | Service control: stop before startup

  1. Pingback: @pradeepadiga posts SQL Server Cluster resource doesn’t come online | Service control: stop before startup | sqlmashup

  2. Pingback: SQL Server Cluster resource fails with “Data source name not found and no default driver specified” error - SQL Server - SQL Server - Toad World

Leave a Reply