라이프로그


SQL 서버와 연결장애 발생... DB이야기

7월2일 Ranking 을 산정관련 데이터를 저장하는 RankingDB 서버와 어플리케이션간에

연결장애가 발생 했다..

장애가 발생 하였을때 난 자고 있어서 (연속되는 야근으로 인해..무진장 피곤했음 -_-; 엄청난 부재중 전화가...)

여하튼.. 출근후 모든 로그를 검사해본 결과 내 결론은 '알수 없다' 였다. 

인터넷을 뒤지다가 Microsoft 고객지원쪽에서 뜻밖의 문건을 발견 했다.

http://support.microsoft.com/kb/942861


뭐 대충 내용은 Windows 2003 에서 지원하는 TCP 뭐시기 기능과 Broadcom 5708 chipset 을 장착한 랜카드간에

문제가 있다는 것이고.. 해결하려면  최신버전의 드라이버를 설치하거나..

TCP 뭐시기 기능을 끄(레지스트리 수정해야 한다.) 라는 것인데..

그러면 성능상 문제가 발생 할 수 있고...

TCP 뭐시기를 끄고 나서도 문제가 발생하면 랜카드를 교체하란건데..


여하튼 급히 내려가 RankingDB 서버 랜카드를 확인해보니 Broadcom 5708 chipset 이였다.. -_-;

아.. ㅅㅂ 수백 종류나 되는 랜카드 중에 하필 이놈이.. 그것도 가장 바쁜 RankingDB 서버에 꼽혀 있는거야..

여기저기 문제발생의 요인이 이것인것 같다라고 메일을 보내고..

Broadcom
홈피 가보니 2008년2월 버전의 드라이버가 올라와있긴한데..

이문제에 대한 명확한 해답이 없다..

그래서 다음주 정기점검 시간에 랜카드를 교체하기로 했는데..

오늘.. (토요일이다.. -_-) 지난 어떤 날보다 더많은 연결장애가 발생하는 것이 아닌가..

결국 긴급점검 올리고 랜카드를 교체했다..

교체후 현재까지 장애가 발생하지 않는 것으로 보아 .. 문제는 역시 랜카드 였지 싶다..

그런데 만약 또 발생한다면? ..... 에잉~



거의 1시간 가까이 인터넷을 뒤져서 찾은건데.. 자주찾아가는 김정선님 블로그에 해석한 글이 있더라... -_-;

도데체 나는 글을 어디로 읽는거야.. -_-;;~

똥구녁으로 읽는건지.원..


-----------------------------------------------------------------------------------------------------------------

SYMPTOMS

loadTOCNode(1, 'symptoms');
Consider the following scenario. A server is running Windows Server 2003 and Microsoft SQL Server.
You have an application that connects to SQL Server by using TCP/IP.
In this scenario, you may intermittently receive one of the following error messages when the application connects to SQL Server:
Error message 1
[Microsoft][ODBC SQL Server Driver][DBNETLIB] General Network error. Check your network documentation
Error message 2
ERROR [08S01] [Microsoft][SQL Native Client]Communication link failure
Error message 3
System.Data.SqlClient.SqlException: A transport-level error has occurred when sending the request to the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.)
You may also receive one of these error messages when the network load on SQL Server is high. For example, you may receive one of these error messages when you replicate databases in SQL Server. Or, you may receive one of these error messages when a multiple-user application accesses databases in SQL Server.

 

CAUSE

loadTOCNode(1, 'cause');
This problem occurs because the TCP Chimney Offload feature is enabled on the server. The TCP Chimney Offload feature is enabled by the Windows Server 2003 Scalable Networking Pack.

Typically, this problem occurs when the network adapter uses the Broadcom 5708 chipset. For example, the following network adapters use the Broadcom 5708 chipset:
Broadcom NetXtreme II
Hewlett-Packard NC373i Multifunction Gigabit Server Adapter
This problem may also occur when the network adapter uses other chipsets.

 

RESOLUTION

loadTOCNode(1, 'resolution');
To resolve this problem, ask the hardware vendor if the hardware vendor has the following updates:
The latest basic input/output system (BIOS) update for the server
The latest firmware update for the network adapter
The latest driver update for the network adapter
If the hardware vendor has no updates, see the "Workaround" section to work around this problem.

 

WORKAROUND

loadTOCNode(1, 'workaround');
Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:
322756 (http://support.microsoft.com/kb/322756/) How to back up and restore the registry in Windows


To work around this problem, disable the TCP Chimney Offload feature. To do this, follow these steps:
1.Click Start, click Run, type cmd, and then press ENTER.
2.At the command prompt, type the following command, and then press ENTER:
Netsh int ip set chimney DISABLED
Note You do not have to restart the server after you run this command.
If the performance of Windows Server 2003 decreases after you disable the TCP Chimney Offload feature, follow these additional steps:
1.Click Start, click Run, type Regedit, and then click OK.
2.Locate the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
3.Double-click the EnableTCPChimney registry entry.
4.In the Edit DWORD Value dialog box, type 0 in the Value data box, and then click OK.
5.Double-click the EnableRSS registry entry.
6.In the Edit DWORD Value dialog box, type 0 in the Value data box, and then click OK.
7.Double-click the EnableTCPA registry entry.
8.In the Edit DWORD Value dialog box, type 0 in the Value data box, and then click OK.
9.Restart the server.

 

STATUS

loadTOCNode(1, 'status');
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

 

MORE INFORMATION

loadTOCNode(1, 'moreinformation');
The Windows Server 2003 Scalable Networking Pack provides the following features:
TCP Chimney Offload
Receive-Side Scaling (RSS)
Network Direct Memory Access (NetDMA)
These features optimize the performance of Windows Server 2003 when Windows Server 2003 is processing network traffic.

To obtain the Windows Server 2003 Scalable Networking Pack, you must install Windows Server 2003 Service Pack 2 (SP2). Or, you must install the update that is described in Microsoft Knowledge Base article 912222. For more information, click the following article number to view the article in the Microsoft Knowledge Base:
912222 (http://support.microsoft.com/kb/912222/) The Microsoft Windows Server 2003 Scalable Networking Pack release
To enable the TCP Chimney Offload feature, the server must have a network adapter that supports offload technologies.

 

How to use a network trace to determine whether the TCP Chimney Offload feature is enabled

loadTOCNode(2, 'moreinformation');When you directly capture a network trace on the server, the network trace contains only TCP three-way handshakes and UDP traffic if the TCP Chimney Offload feature is enabled. The network trace does not contain other traffic because the TCP Chimney Offload feature lets the network traffic bypass that specific part of the TCP/IP stack. The driver that captures packets resides in that specific part of the TCP/IP stack.

The information and the solution in this document represents the current view of Microsoft Corporation on these issues as of the date of publication. This solution is available through Microsoft or through a third-party provider. Microsoft does not specifically recommend any third-party provider or third-party solution that this article might describe. There might also be other third-party providers or third-party solutions that this article does not describe. Because Microsoft must respond to changing market conditions, this information should not be interpreted to be a commitment by Microsoft. Microsoft cannot guarantee or endorse the accuracy of any information or of any solution that is presented by Microsoft or by any mentioned third-party provider.

Microsoft makes no warranties and excludes all representations, warranties, and conditions whether express, implied, or statutory. These include but are not limited to representations, warranties, or conditions of title, non-infringement, satisfactory condition, merchantability, and fitness for a particular purpose, with regard to any service, solution, product, or any other materials or information. In no event will Microsoft be liable for any third-party solution that this article mentions.

-----------------------------------------------------------------------------------------------------------------

덧글

  • 도2008 2008/07/05 22:57 # 답글

    너무 고생이 많습니다. 꾸벅~
  • 이재학 2008/07/07 10:00 # 삭제 답글

    ㅎㅎㅎ 잼있당..
  • 양DB 2008/07/07 14:57 # 삭제 답글

    뭐가 재밌다는 거여~ 이사람은.. 남은 똥출탓구먼 -_-
  • 도전천곡 2008/07/13 22:02 # 삭제 답글

    번역판을 올리시오~~
  • somma 2008/07/14 10:38 # 삭제 답글

    음.... 졸 재밌었겠구만..ㅋㅋㅋㅋ
  • 2010/02/08 11:39 # 삭제 답글 비공개

    비공개 덧글입니다.
댓글 입력 영역