beta_lockinfo

An SQL text by Erland Sommarskog, SQL Server MVP.

If you ever tried to track down a blocking situation by running sp_who, DBCC INPUTBUFFER and the various DMV:s while the users are screaming, beta_lockinfo is the thing for you. Note that beta_lockinfo requires SQL 2005 SP2. For previous versions of SQL Server (down to SQL 6.5), use my older aba_lockinfo instead.

What it is

beta_lockinfo is a stored procedure that provides information about processes and the locks they hold as well their active transactions. beta_lockinfo is designed to gather as much information about a blocking situation as possible, so that you can instantly find the culprit, and kill the blocking process if the situation is desperate. Then you can sit back and analyse the output from beta_lockinfo to understand how the blocking situation arose and figure out what actions to take to prevent the situation from reoccurring. The output from beta_lockinfo shows all active process as well as passive processes with locks, which objects they lock, the command they last submitted and which statement they are executing. You also get the query plans for the current statements.

Download

Get the source to beta_lockinfo here.  (SourceSafe version 12, checked in 2009-06-26 23:20.)

You need to have SQL 2005 SP2 or later to run beta_lockinfo. If you would like to run beta_lockinfo on SQL 2005 RTM or SP1, see at end of the article, which changes you need to make to the code.

You can put the procedure in any database you like. However, the database must be in compatibility mode 90 or later.

Notes for old aba_lockinfo users

If you have used my old aba_lockinfo in the past, you will find that beta_lockinfo is the same in essence, but there are nevertheless several changes. The parameter list is new, and I've abolished the alternative to display all locks. You can only get them aggregated. The parameter @fancy is now called @textmode. By default, process data is only displayed for the first row(s) in grid mode. Some columns in the old result set are replaced with more interesting columns from the DMV:s in SQL 2005. The column memusage is gone entirely. The columns have also been rearranged a bit.

I hope you will appreciate the changes.

I want your input!

I would certainly appreciate to get any feedback on problems you face, be that error messages or incorrect results. Please appreciate that it may be difficult to recreate error situations, as they may relate to conditions at your site. The more information you can include, the better.

I am also interested in hearing from you, if you think that anything should be added to or dropped from the output, or if you have opinions on the output format as such. Given the wealth of information available in the DMV:s in SQL 2005, it has been a delicate affair to select what to include or not.

As for known issues, please see the end of the document.

You reach me at esquel@sommarskog.se.

The way it works

beta_lockinfo first compiles information about all locks in the system from the DMV sys.dm_tran_locks. In order to make this information more easily digestible, beta_lockinfo aggregates on a number of columns, so if a process holds 10000 locks a big table, you will only see 2-3 lines in the output. beta_lockinfo then unwinds the blocking chain from the DMV sys.dm_os_waiting_tasks.

beta_lockinfo then proceeds to find information about active transactions from sys.dm_tran_session_transactions and sys.dm_tran_active_transactions.

Next beta_lockinfo retrieves process information from a number of places: sys.dm_exec_sessions, sys.sysprocesses (yes, some information is available here only), sys.dm_exec_requests, sys.dm_os_tasks, sys.dm_exec_sql_text and sys.dm_exec_text_query_plan. By default, beta_lockinfo only looks at "interesting" processes, as explained below.

beta_lockinfo then goes on to retrieve the most recently submitted command for each process. The final task before returning the result set is to translate the ids from to sys.dm_tran_locks to object and index names through the new catalog views.

Note that since beta_lockinfo reads the various types of information at different points in time, the information may be out of sync. For instance, a process that held a lock could have exited before beta_lockinfo retrieves the process information. It could also have moved on to a new statement, so the lock information does not match the current statement.

Parameters

beta_lockinfo takes four parameters, @allprocesses, @textmode, @procdata and @debug, detailed in the table below:

Interesting processes

An interesting processes is a processing which is doing something interesting:

• Holding or waiting for a lock (beyond a single Sch-S lock on database level which about all processes in hold).
• Being blocked or blocking some other process.
• Has a task in sys.dm_os_tasks, and is not a system processes. (See note below.)
• Has an open transaction.
• Is not the process running beta_lockinfo.
• Does not exist in sys.dm_exec_sessions (see below).

Note that since activation procedures for Service Broker runs as system processes (sys.dm_exec_sessions.is_user_process = 0), such a process will not be listed if it holds no other lock than the standard schema-stability lock on the database. Had I not included the condition on is_user_process, beta_lockinfo would also list various background tasks which would be too much noise.

Why would a process be missing from sys.dm_exec_sessions? There are a couple of reasons why this could happen.

• Since beta_lockinfo retrieves locking and process information separately, a process that held a lock may have gone when beta_lockinfo reads sys.dm_exec_sessions.
• The topic for sys.dm_tran_locks in Books Online, lists two cases where the column request_session_id can be negative to indicate locks without a session.
• The session is gone, but the locks are left orphaned. This would be a bug in SQL Server.

Sessions, execution contexts and requests

Up to SQL 6.5, life was very easy. A process was a connection was a session. With SQL 2005 there are now a whole bunch of different concepts: sessions, execution contexts, requests, connections and tasks. At least. This section describes how beta_lockinfo uses these concepts.

Let's start with a connection, because it's the easiest in this context: beta_lockinfo completely ignores the DMV sys.dm_exec_connections, right or wrong.

The main concept is the session, as listed in sys.dm_exec_sessions. A session is identified with a session_id, or what old-time users of SQL Server knows as a spid.

A session may be idle, or it may be executing one or more requests. A request is a batch of SQL commands or an RPC call to a stored procedure. In the very most cases, a session runs only one at request at a time, but SQL 2005 introduced a new feature, Multiple Active Result Sets, known as MARS. When MARS is in effect, a session may submit a new command batch, even though the previous batch has not completed, and the batches are executed in an interleaved fashion. Requests appear in the view sys.dm_exec_requests. A request is identified by the session_id and the request_id. As long as MARS is not in effect, request_id is usually 0. When MARS is in effect the requests are numbered 1, 2 etc. Beside MARS, I have noticed one more case when request_id is non-zero: requests executed by sessions connected through HTTP endpoints.

An SQL command can be executed single-threaded or by parallel threads. Each such thread is known as an execution context, identified by the exec_context_id. (You may recall that sysprocesses in SQL 2000 has a column with the cryptic name ecid. That's the same thing.) The main thread for the request has exec_context_id = 0, and any non-zero number for a exec_context_id indicates parallel execution. There is no view as such that defines the execution contexts, but all execution contexts have a row in sys.dm_os_tasks. Now, here is a funny thing: if a request with a non-zero request_id requires a parallel plan, the request_id for the threads with a non-zero exec_context_id is nevertheless zero, meaning that if a session has multiple requests running, you cannot tell which request the non-zero execution contexts belong to. (I assume, though, that all contexts belong to the same request, since multiple requests are interleaved, and not truly multi-process.)

A task is something that is listed in sys.dm_os_tasks, and is identified with a varbinary(8) value. All execution contexts, and thus all requests, have a row in sys.dm_os_tasks, but there are also non-session entries in sys.dm_os_tasks. However, these are completely neglected by beta_lockinfo.

In the output, beta_lockinfo includes session_id, exec_context_id and request_id. But instead of one column for each, beta_lockinfo merges them into one string with the ids separated by slashes, for instance 56/0/2 (request 2 for session 56, the main thread) or 65/7/0 (execution context 7 for session 65, unknown for which request). In the very common case that the both exec_context_id and the request_id is 0, beta_lockinfo suppresses these, and list only the session_id. Thus, you will not see 64/0/0, only 64.

In the output for a process, any non-zero requests appear first, with one line per request. Next follow one or more lines with the session_id proper. There is always at least one such line for a session, even if there is no such row for the session in sys.dm_os_tasks. Finally, any execution contexts with a non-zero exec_context_id for the session are listed in order. Occasionally, you may see the same exec_context_id listed twice. This is because two tasks can have the same exec_context_id. (Why it is so, I don't know. Maybe it's a bug. But I've seen it happen.)

Locks are displayed per session. In sys.dm_tran_locks they are listed per execution context and request, but I've opted to collapse them on session level. My experience from aba_lockinfo, which displays locking information for per execution context, is that this generate lots of lines in the output for a massively parallel query, but this mainly serves to make the output difficult to read. And since it's not possible to (easily) determine to which request an execution context belongs to, it was logical to collapse the requests as well. After all, multiple requests should not be that common. (Then again, that may reflect my attitude against MARS, a feature that I find to be of dubious value.)

Finally, there is process, which is not really an SQL Server concept, but which I use in this document somewhat loosely. Sometimes I refer to entire session, sometimes to the unique combination of session, execution context and request.

How to find the blocker and the blockee

The key column in the result set is the blklvl column. If this column has double exclamation marks !! this process is a lead blocker blocking one of more other processes (which could be another execution context of the same thread) without being blocked itself. If there is a number, the process is blocked. 1 indicates that the process is blocked by a lead blocker, 2 indicates that the process is blocked by a process which in its turn is blocked by a lead blocker and so on. A process is not involved in blocking has a blank in this column.

The values in blklvl may be in parentheses. That is, instead of !! you see (!!). This happens when you have a parallel query, where the various threads are blocking each other, but they are not blocked by any other session, nor blocking any other session. Thus, if you are investigating a blocking issue, you can ignore sessions where blklvl values are in parentheses.

The column blkby shows which process the process is directly blocked by. The blkby column is formed as a string with session_id, exec_context_id and request_id concatenated with slashes, and with the latter two suppressed if both are zero. Sometimes you may see a string like 54/-1/-1. This means that when beta_lockinfo read dm_os_waiting_tasks there was a blocking task, but when beta_lockinfo read dm_exec_sessions, dm_os_tasks etc, the task had exited. You may also see something like (+4) after the process string. This means that the process is in fact blocked by four more processes but the one listed.. (Which one that is listed when there are several blockers is fairly arbitrary.)

There are two other possible values for blklvl: DD and ??. DD means that the process is involved in an on-going deadlock. If you look at blkby, and then go that spid, look at its blkby and so on, you can expect to come back where you started sooner or later. If a process is blocked by a deadlocked process without being involved in the deadlock itself, blklvl displays 1, 2 etc as with regular blocking. The value ?? means that the process was listed in sys.dm_tran_locks as waiting for a lock, but it was not listed in sys.dm_os_waiting_tasks. This could indicate that by the time beta_lockinfo read sys.dm_os_waiting_tasks, the process had been granted the lock. 

To see what a process is waiting for, look at the lstatus and waittype columns. In most cases, a process is waiting for a lock, which is indicated by WAIT or CONVERT in the lstatus column. All non-granted locks appear first in the output for a processes. In the case of parallel execution, the main thread or a sub-thread may be waiting for another thread to complete. In this case waittype will be CXPACKET.

Transaction Information

The output from beta_lockinfo includes a couple of columns related to active transactions. Of these, opntrn is taken from sysprocesses and mirrors the current value of @@trancount for the process.

The columns trn_start and trn_since are also straightforward. They reflect when the current transaction started. The first column gives the absolute point in time, while the second tells you for how many seconds the the transaction has been running. This value can in fact be negative, which indicates that the transaction started after beta_lockinfo started running. You can use these columns to determine whether a transaction has been orphaned. This is particularly likely if trn_start is before last_batch, but keep in mind that it's perfectly legal to start a transaction from client level.

The column trninfo holds information from various columns in the transaction DMVs that I preferred to collapse into a single column to make it easier to read. The column can have up to five fields separated by hyphens:

When using MARS, a session may have multiple active transactions. In that case, beta_lockinfo displays information about the oldest transaction.

Result set

As outlined above, the result is organised so that there is essentially one row per group of locks. Locks are aggregated on the following columns in sys.dm_tran_locks: resource_type, resource_subtype, resource_database_id, resource_description (application locks only) resource_associated_entity_id (for the resource types it applies to), request_mode, request_status and request_owner_type.

To this come rows for processes with non-zero request ids, listed before the locks, and processes with non-zero execution context ids, listed after the locks. When the parameter @procdata is 'F', process-data is listed only for the first lock row, and it is also listed for any request rows listed before the locks.

In several cases, NULLs and other values that means "nothing" are displayed with blank space to make the output easier to read. I've retained NULL in the cases where I think a NULL has something to tell.

The data is sorted by spid (with non-zero request ids first, non-zero execution-context ids last), lstatus (with CONVERT and WAIT first), object and rsctype.

How much resources does beta_lockinfo take?

Unfortunately, beta_lockinfo is not as lean as I would like it to be. The chief reason for this is that access to sys.dm_tran_locks is nowhere cheap when there are plenty of locks on the system. I had a situation with 2.3 million locks, and a mere SELECT COUNT(*) took 20 seconds to run. The SELECT COUNT(*) with GROUP BY that beta_lockinfo performs took 70 seconds. This was on a workstation, not a real server. On the other hand, this was on an idle machine and this was a convoluted test case. Normally when you have 2.3 million locks, your server will be anything but idle. An execution time of five minutes is unfortunately not to be unexpected in such a case. Again, I like to stress that the decisive factor is the number of locks.

Of the other parameters @textmode = 1, will take some more resources that you may want to avoid on a swamped server.

If your system is really hogged, and you just want to know who is creating all this mess, try this:

SELECT request_session_id, COUNT(*)
FROM   sys.dm_tran_locks
GROUP  BY request_session_id

It appears that as long as you do not include any character columns in the output, the GROUP BY operation does not take that much more time than a plain SELECT COUNT(*). (For which 20 seconds is still far too much for my taste.)

The version of beta_lockinfo I have posted here, does not run on SQL 2005 RTM and SP1. If you still want to run beta_lockinfo on these versions of SQL Server, this section details the changes you need to make to get it work.

First find these two lines:

coalesce(nullif(es.original_login_name, ''),
         suser_sname(es.original_security_id)),

Replace with:

coalesce(nullif(es.login_name, ''), suser_sname(es.security_id)),

(This is the same as the line above.) If there has been impersonation, you will only see the currently impersonated login in the column login; the original login is not available.

Next find these two lines:

OUTER  APPLY sys.dm_exec_text_query_plan(er.plan_handle,
             er.statement_start_offset, er.statement_end_offset) AS etqp

Here you have two choices. Either delete this line entirely, and in such case you should change the line reading

convert(xml, etqp.query_plan),

To simply read "NULL,". You can also change the OUTER APPLY to read

 OUTER APPLY sys.dm_exec_query_plan(er.plan_handle)AS etqp

If you do this, current_plan will still have a value, but for a stored procedure that will be the plans for all statements in the procedure.

I did not want to put out a proper version of beta_lockinfo for RTM and SP1, since I've not performed any serious testing on these versions.

Contact info

If you have questions or comments and not the least suggestions for improvements just mail me, Erland Sommarskog, on esquel@sommarskog.se.

Revision history