Tenable research discovered a DoS vulnerability in IBM Spectrum Protect (ISP) 8.1.9.300. The flaw exists in the adsmdll.dll and dstadll.dll files due to improper validation of user-supplied data when processing a CertQryResp verb message sent to default TCP port 1500.
An unauthenticated, remote attacker can exploit the issue, via a series of specially crafted messages, to terminate the dsmsvc and dstasvc (Storage Agent) processes.
The CertQryResp verb message has the following format:
// be = big endian
//
// Mandatory verb header
struct hdr
{
be16 verbLen; // 16-bit total len of the verb; 0 if verbType is 8
int8 verbType; // 8-bit verb type
// if verbType == 8, it's an extended verb and
// hdr_ex follows this header
int8 magic; // must be 0xCA
};
// Optional extended verb header
// Used for verbTypes that are greater than 0x100
struct hdr_ex
{
be32 verbType; // 32-bit verb type
be32 verbLen; // 32-bit total len of the verb
};
struct CertQryResp
{
// Generic headers
hdr hdr; // hdr.verbType = 8
hdr_ex hdr2; // hdr2.verbType = 0x00031900
// verb-specific header starts
int8 version;
be16 dataOffset; // offset to the msg where the verb data starts
be16 opRC;
be16 certFormat;
be16 certOffset; // offset to the verb data where the certificate starts
be16 certLength;
// verb data starts
byte cert[certLength];
};
All ISP verbs are validated in the SmIsValidVerbEx() function in adsmdll.dll/dstadll.dll. However, the function only checks to make sure CertQryResp.dataOffset falls within the message boundary (i.e., sizeof(CertQryResp)):
.text:000000018099AD2F lea rcx, [rdi+verbCertQryResp.dataOffset]
.text:000000018099AD33 call GetTwo
.text:000000018099AD38 movzx eax, ax
.text:000000018099AD3B cmp eax, r15d ; r15d = MsgSize
.text:000000018099AD3E jbe ok_1809A331
It does not check whether or not CertQryResp.dataOffset + CertQryResp.certOffset is within the message boundary. When the CertQryResp verb message is processed, the pointer to the certificate is calculated as: pCert = pCertQryResp + pCertQryResp->dataOffset + pCertQryResp->certOffset:
.text:0000000180AE8466 lea rcx, [rsi+verbCertQryResp.certOffset]
.text:0000000180AE846A call GetTwo
.text:0000000180AE846F lea rcx, [rsi+verbCertQryResp.dataOffset]
.text:0000000180AE8473 movzx ebx, ax
.text:0000000180AE8476 call GetTwo
.text:0000000180AE847B lea rcx, [rsi+verbCertQryResp.certLength]
.text:0000000180AE847F movzx r15d, ax
.text:0000000180AE8483 add r15, rbx ; dataOffset + certOffset
.text:0000000180AE8486 add r15, rsi
.text:0000000180AE8489 call GetTwo
.text:0000000180AE848E movzx r14d, ax
This can cause an Out-of-bounds read on the certificate if CertQryResp.certOffset contains a value (i.e., 0xffff) that is greater than the message size. The Out-of-bounds read can cause an access violation (i.e., pCert lands in an inaccessible page), resulting in process termination:
(d6c.74c): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
MSVCR120!MoveSmall+0x1aa:
00007fff`4877c64a 4c8b540af8 mov r10,qword ptr [rdx+rcx-8] ds:000001d3`2f477fff=????????????????
0:414> k
# Child-SP RetAddr Call Site
00 00000028`b01fe198 00007fff`42755475 MSVCR120!MoveSmall+0x1aa [f:\dd\vctools\crt\crtw32\string\amd64\memcpy.asm @ 362]
01 00000028`b01fe1a0 00007fff`427d3324 gsk8cms_64!GSKASNUtility::asncpy+0x2a5
02 00000028`b01fe1d0 00007fff`40f9e642 gsk8cms_64!GSKString::GSKString+0x64
03 00000028`b01fe210 00007fff`40f766d4 gsk8km_64!GSKKM_strerror+0x254b2
04 00000028`b01fe720 00007fff`37d24ed7 gsk8km_64!GSKKM_OpenKeyDbData+0xa4
05 00000028`b01fe780 00007fff`37e984eb adsmdll!tlsValidateKeyDbPw+0xb7
06 00000028`b01fe850 00007fff`37e99286 adsmdll!HandleCertQryResp+0x1cb
07 00000028`b01fe8d0 00007fff`37e96b7a adsmdll!HandleNegotiate+0x756
08 00000028`b01fea30 00007fff`37d75c46 adsmdll!SmV2AuthProcess+0x28a
09 00000028`b01feab0 00007fff`380d00da adsmdll!smExecuteSession+0x3a6
0a 00000028`b01ff7f0 00007fff`373bc413 adsmdll!SessionThread+0x43a
0b 00000028`b01ff8b0 00007fff`48764f7f adsmdll!startThread+0x153
0c 00000028`b01ff900 00007fff`48765126 MSVCR120!_callthreadstartex+0x17 [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 376]
0d 00000028`b01ff930 00007fff`546b84d4 MSVCR120!_threadstartex+0x102 [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 354]
0e 00000028`b01ff960 00007fff`5711e871 KERNEL32!BaseThreadInitThunk+0x14
0f 00000028`b01ff990 00000000`00000000 ntdll!RtlUserThreadStart+0x21
Proof of Concept
Attached is a PoC to terminate dsmsvc.exe. The PoC can be used as follows:
python3 ibm_spectrum_protect_CertQryResp_dos_CVE-2020-4559.py -t <target_host> -p 1500
...
connection 00000300: certSize 00001500, certOffset 0000f000
connection 00000301: certSize 00001600, certOffset 00001000
connection 00000302: certSize 00001600, certOffset 00002000
connection 00000303: certSize 00001600, certOffset 00003000
connection 00000304: certSize 00001600, certOffset 00004000
connection 00000305: certSize 00001600, certOffset 00005000
Traceback (most recent call last):
File "ibm_spectrum_protect_CertQryResp_dos.py", line 110, in <module>
r = recv_verb(s)
File "ibm_spectrum_protect_CertQryResp_dos.py", line 47, in recv_verb
hdr1 = s.recv(4)
ConnectionResetError: [Errno 104] Connection reset by peer
Note that we tested the PoC on a Windows Server 2016 virtual machine with 2 CPUs and 16 GB of memory. The script may or may not terminate dsmsvc.exe depending on the memory page where pCert lands. To produce a reliable crash, full page heap can be enabled on dsmsvc.exe:
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64>gflags /p /enable dsmsvc.exe /full
This puts the CertQryResp verb message at the end of a page followed by a non-accessible guard page.