Giving Back - Securing Open Source IoT Projects

Ori Hollander  November 07, 2018

For the past several months, the security research teams at Vdoo have been undertaking broad-scale security research of leading IoT products, from the fields of safety and security. In most cases, the research was carried out together with the device vendors for the sake of efficiency and transparency.

The research goal is to contribute knowledge and tools to mitigate risks, as well as encourage the devices' manufacturers to implement the right security for their products. We believe that an appropriate implementation of the security essentials will dramatically decrease the chances of exploiting vulnerabilities on the device.

Open-source projects are implemented in many connected devices. In order to provide the highest security level for those devices as well, the research focuses on some of the most common projects. The findings are then implemented in all of our automated IoT security solutions for the widest risk mitigation coverage.

As part of this research, our researchers discovered zero-day vulnerabilities in several known open-source projects. In this article, we will discuss vulnerabilities found in 3 different projects - in the popular Lighttpd web server, the Live555 Media Library and a Linux driver for the Realtek's RTL8189ES Wi-Fi chip.

Live555 Media Library

The Live555 Media library forms a set of C++ libraries for multimedia streaming, using open standard protocols (RTP/RTCP, RTSP, SIP). These libraries - which can be compiled for Unix (including Linux and Mac OS X), Windows, and QNX (and other POSIX-compliant systems) - can be used to build streaming applications. The library is in use in different projects, some that are very well-known, and it is in use in several IoT devices.

While going over the source code of the library, we found an integer overflow vulnerability, which can cause denial of service to an application using its RTSP Server functionality. The vulnerability was found in version 2018.08.05 of the liveMedia library compiled for 32-bit architectures, and it was fixed in version 2018.08.26, as seen in the change log.

Technical Deep-Dive

In this part of the article, all code references will be to the files in the latest vulnerable version available online that can be found here.

One of the protocols that are implemented in the library is the RTSP protocol. This protocol is pretty similar to HTTP. When implementing an RTSP server application, the developer can use the RTSPServer class implemented in liveMedia/RTSPServer.cpp.

Eventually RTSP requests reach the function RTSPServer::RTSPClientConnection::handleRequestBytes (line 607 of liveMedia/RTSPServer.cpp). In the function, there is a loop intended to parse the request when it becomes a valid full request. As part of the parsing, the Content-Length header is parsed (if exists) to check if all the data came in (or in rare cases, if too much data was read).

The Content-Length header is parsed into the variable contentLength (line 204 of liveMedia/RTSPCommon.cpp). The code then checks if the entire request data was received. As shown in Figure 1, it does so by comparing the amount of bytes already received (represented by the pointer reached by adding the number of bytes read altogether to the pointer that indicates the beginning of the request buffer) to the amount of bytes expected (represented by the pointer reached by adding the contentLength to tmpPtr, a pointer that indicates the end of the request headers portion). If not enough data was received yet, the handler loop breaks to allow retrieving more data before trying to parse the request's body. This is the first place that an integer overflow could happen. If the Content-Length header is a number that is big enough, when added to tmpPtr an overflow can occur, and the pointer retrieved won't be the correct pointer, meaning that the loop might not break. In that case, the code will immediately proceed to the parsing logic, even though it shouldn't.

OS projects livemedia RTSPCommon

Figure 1 - line 717 of liveMedia/RTSPCommon.cpp - If the expected content length is shorter than the number of bytes read by this point, the loop breaks to allow reading the remaining bytes. In a case that contentLength is large enough, there will be an overflow and the result of "tmpPtr + 2 + contentLength" will wrap around, making the "if" condition false, and the loop won't stop.

After passing the test, the actual RTSP parsing logic begins. Towards the end of the loop the contentLength variable is used again to determine if there was too much data read (line 878 in Figure 2). To understand what happens, let's have a look at the source code:

OS projects livemedia RTSPCommon 2

Figure 2 - line 878 of liveMedia/RTSPServer.cpp - The contentLength variable is added to a pointer representing the beginning of the content portion. If the result shows that the buffer holds extra bytes, they are moved to the beginning of the buffer.

As we can see, the code calculates the requestSize variable by adding the contentLength variable to the headers portion size (indicated by fLastCRLF+4-fRequestBuffer). Afterwards, the number of bytes left to read is calculated by subtracting requestSize from the amount of bytes already seen. Let's see an example request:

Content length

Let's assume that the request was read as a whole to the buffer and it is all parsed at one time. In this example, fRequestBuffer (from Figure 2) points to the beginning of the request, fLastCRLF points to the end of the headers, and contentLength is 12. The requestSize variable will be calculated, and the result will be the full length of the request. Since fRequsetBytesAlreadySeen is the length of the packet too, numBytesRemaining will be 0. If the result would be more than 0, that would mean that we read a few bytes from the next request, and they need to be moved to the beginning of the buffer. That's what happens in line 883.

Now let's examine another example:

Content length 2

Note that there is no request body here. In this case, the contentLength variable is very big, and it will cause the integer overflow at the beginning of the loop that we discussed earlier, meaning that the code in Figure 2 will be reached, even though it shouldn't have. Now let's check the calculations: The requestSize variable will be calculated, but this time the result will not be as expected. Since all the variables are of 32-bit length, adding 4294957237 to the headers portion length will result in an overflow, causing the requestSize to be smaller than the headers portion length. So, if in our example the length of the headers portion is 59 (counting every newline as two characters, '\r\n'), requestSize will be 4294957237+59, which is 4294957296. This number, when looking at it as a 32-bit signed integer is -10000. The fact that requestSize is unsigned does not matter, since the arithmetic operations act the same as long as all variables are of same length (32 bit in this case). Now numBytesRemaining will be calculated as fRequsetBytesAlreadySeen-(-10000). Since the number of bytes seen so far is 59, the result of the calculation will be 10059, which is more than 0, so the if expression on line 882 is true, causing the memory movement to happen. Let's look at the memmove arguments. The destination buffer is fRequestBuffer, and the source buffer is &fRequestBuffer[requestSize], and as you can recall, requestSize is now -10000, meaning that memmove function will start copying a number of numBytesRemaining bytes (which is 10059) from 10000 bytes before the beginning of fRequestBuffer to the beginning of fRequestBuffer.

If we look carefully, we could see that the number we gave in the Content-Length header (4294957237) is the unsigned representation of -10059. We can see that numBytesRemaining will be equal to negating contentLength.

Since the length of fRequestBuffer is REQUESTBUFFERSIZE, if contentLength will be equivalent to something smaller than minus REQUESTBUFFERSIZE, the result will be copying more than REQUESTBUFFERSIZE bytes to the buffer, which results in overwriting the memory that is right after fRequestBuffer, which holds important variables for the continuation of the flow, which can result in a crash.

As said before, the first overflow could rarely happen in a 64-bit system since the variable is still 32-bit wide (this is compiler dependent, but is true for at least one checked case), and adding it to a 64-bit pointer using unsigned operators will mostly not result in an overflow (unless the pointer starts in an extremely high memory address, which is not likely to happen), and the loop will stop at that point. The same is true for cases that the compiler compiled the variables as 16-bit variables on a 32-bit system, but that should not happen (and in the cases tested by us that didn't happen). Since the first overflow won't happen in 64-bit systems, the second piece of code we examined will not be reached at all, and it will not be exploitable.

Wi-Fi "Death Packet"

While researching multiple different devices we've noticed that sometimes mysterious Wi-Fi packets crash the device. Looking into the crash, we discovered a serious bug in the Wi-Fi driver. The research involved trying to find the exact place of the crash and trying to understand where the code came from. We found out that large pieces of the Wi-fi driver's code are similar to different pieces of code we found online, meaning that the vulnerability may be out there in different devices too. Narrowing down the problem, we understood that the bug affects specific drivers for the Realtek RTL8189ES Wi-Fi chip, and may also reside in drivers for other Realtek chips.

It is important to note that in most devices running modern Linux the bug does not exist anymore since all network drivers were combined in such a way that there is a main driver with extensions for each device, and there are no more drivers all meant for specific devices, but the bug still exists in older versions of Linux or in specific devices that have the old driver code in them.

Technical Deep-Dive

Some embedded devices have a feature to enable a Wi-Fi Access Point (AP). When sending a raw Wi-Fi packet to the AP, if the source mac address of the packet is the same as the AP's mac address (which is retrievable easily by sniffing the beacon packets), the RTL8189ES driver crashes. In the next lines we'll explain why, by using snippets of code that we reversed from the driver. Reversing the code was made simpler by finding snippets of similar code online, and the function and variable names we used are mostly from these snippets.

When the device starts, it allocates macids for stations. It starts with itself, and as can be seen in this function, a mac address equal to the adapter's mac address gets the id of NUM_STA (which is 32, it is actually meant to indicate that this is a special id, not to be really used):

macid code

Upon receiving a packet (except for beacon packets), the code in the updaterecvframephyinfo_88e function below gets the macid for the source address and saves it as StationID for the next functions to handle (in the function "sa" stands for source address):

macid code 2

One of the functions that are called during the process is odmProcessRSSIForDM. As seen at the beginning of the function, it takes a pEntry from an array of "station info"s, using the StationID as the index into the array. After taking the pEntry, the function checks if it's valid, but it does that only by checking that it isn't NULL.

macid code 3

At the definition of pODM_StaInfo, the size of the array is 32. This fact can be seen in many sources online, and that can also explain why the NUM_STA constant is 32.

All this means that when sending a packet that has the source address of the AP itself, the function tries to reach index 32 of the dmodm->pODMStaInfo array. Since the array's length is 32, the last index in the array is 31, thus index 32 is out of the array's bounds. When the code tries to reach to index 32 of the dmodm->pODMStaInfo array, it actually takes the value of whatever is after the array in the dm_odm struct, which is usually not NULL thus considered valid by the code, and when trying to dereference it as a valid pEntry the driver crashes.

Lighttpd Use-After-Free Bugs

Lighttpd is a very popular open-source web-server, used among many IoT devices. While auditing lighttpd 1.4.50, we discovered 3 use-after-free bugs in the HTTP header parsing code. These security issues have been fixed by this commit and are incorporated in the lighttpd version 1.4.51 release.

Technical Deep-Dive

The httprequestparse function handles the initial parsing of incoming HTTP requests. The parsing is done line by line. During header parsing stage, when encountering any one of the "If-Modified-Since", "If-None-Match" and "Content-Type" headers, the pointer to the header value's string is stored in a con->request struct member (con->request holds various info regarding the current HTTP request). We'll use the "If-Modified-Since" header as an example: lighttpd code

Figure 1 -

The code adds the header's key-value pair into the con->request.headers array. lighttpd code 2

Figure 2 -

Note that ds->value->ptr is now the same value as Figure 1's con->request.httpifmodified_since

When parsing the next line of the request, if it starts with a space or tab, it is treated as the continuation of the last header's value (due to "Line Folding" feature - as header field values can be folded onto multiple lines). In such a case, the current line's string is appended to the last header's value: lighttpd code 3

Figure 3 -

Inside bufferappendstring() call, realloc will be called on ds->value->ptr as part of this sequence:

buffer append string

Depending on the values' strings lengths, and the heap state and algorithms, the realloc may allocate a new buffer at a new location, freeing the original one.

As a result, con->request.httpifmodified_since that was set to the old pointer will become a dangling pointer, pointing to a freed memory location.

Later accesses to these dangling pointers can then be reached, and potentially crash the server (Denial of Service), in case the original pointer's page has been freed.

About Vdoo

Vdoo is a technology-driven company that strives to change the reality of unprotected connected devices by building a products line that supports device manufacturers in embedding security into their connected devices during the development stage. In addition to developing products and services, significant efforts are invested in a wide scope research of connected devices and its supply chain.


Ori Hollander, Senior Security Researcher, Vdoo

Or Peles, Vulnerability Research Team Leader, Vdoo

Share this post
Ori Hollander-Security Researcher

Ori Hollander

Security Researcher

Ori is a Security Researcher at Vdoo where he uncovers new vulnerabilities in connected products. Before joining Vdoo, he was part of the Israeli Defense Forces’ technology unit in roles focused on the analysis and development of embedded devices.

Our latest updates