Tuesday 1 December 2020

Filtering and Collecting the Results from the Network Scanner

 by Vaggelis Atlasis 

Previously, I added a function that sends a TCP packet to every IP address in the subnet in an attempt to initiate hand-checking. Using a dictionary with common TCP ports, the packet was sent to each and every one of the ports. If there was a response, it was classified as as one of types of response: the host listening in the port, the host rejection connection in the port or the host prohibiting communication on the port. 

The next step, was filtering the useful from the two packets (ping, TCP) we send to each address in the subnet and save them in afile. 

Setting Up the Collection of Our Results

The information we are mainly interested from each subnet is the addresses that responded, their hostnames, their reachability and the TCP ports we receive replies from. I decide that the best way to collect the information was through a dictionary, making the IP address the key and the rest of the information the value, as a list. That way it would be easy to assign the gathered information to the corresponding IP address without worrying about each address being assigned pieces of information separately, or replacing the existing information like it could in a list. 

Therefore, in the function send_ping, I intialised the dictionary results, where all the results would be collected. The reason results was intialised inside the function was that this was the function were we obtained all the details for each address (including the TCP related information). Next, we create the key for each address in the subnet with the line results[str(address)] = "", making the address the key of the dictionary and leaving its value empty for now. We turn it into a string for convenience and to avoid potential errors. 

Formatting the Information Obtained from the TCP Function

In order to make the results from the send_tcp function available to store in a variable in the send_ping function, we need to return them. However, as there may be multiple ports that respond for a given address, it is efficient and convenient to store them in a list, which we call tcp_open_ports. Hence, every time we get a response with a TCP layer from the host, we run the line tcp_open_ports.append(p) where p is the port that responded from the TCP common ports list. At the end of the function, the list is returned with the line return tcp_open ports

Collecting the Results

With the return of TCP ports now in place, we can now run the function whenever a host reponds to the ping packet and is reachable. In that case, we store the returned information from the send_TCP function to a variable named tcp_open_ports. With the line results[str(address))] = [host_name, "Reachable", tcp_open_ports], we make the value of the given address all of the wanted information in the form of a list. The host name and reachability (given by the send_ping function) and the TCP ports (given by the send_tcp function). 

In the case that communication is prohibited by the host, we set the value of the given address as results[str(address))] = [host_name, "ICMP Uncreachable"]. There is no need to run the send_tcp function as we will receive no reply. We also specify that it was the ICMP protocol (part of the ping packet) that is unreachable to avoid confusion. 

Finally, if there is no response by the address, the value of the IP address is simply given as results[str(address))] = [host_name, "No response"]

The dictionary results with all the collected information is then returned by the function. 

The updated version of the two functions mentioned above. New bits are placed inside the red boxes (click to enlarge. Note: there should a tcp_open_ports.append(p) in all of the selection loops in the send_tcp function - except the last one). 



Saving the Results in a Text File

The easiest way to save our results in a text file is using the csv (comma-separated value) library. Therefore, at the top of the program we import csv and later on define the function saveResults, with results as a parameter, of course. 

Next we use the lines you see in the picture below to open the file in order to write to it using the csv writer. We start by writing the row Ip Address, Host Name, Reachability, TCP Open Ports as subtitles for the file. Then, I used a nested for loop; the first one to write the keys and their respective values in a different line and the second one to count the length of the value of each key, which is a list. If the length of the list is 3, we just write each item in the list separately (for formatting pouproses). If not, the length will be 2. In that case, an empty pair of square brackets is printed to indicate that there was no TCP response. 

The code for the saveResults function which saves the results in a csv file. 



Finalising Everything 

As results is being returned from the send_ping function, it needs to be stored in a variable so that we can access it. Therefore, I went to where the function is called (process.map(send_ping, targets) and simply declared it as a variable. That way, the returned information can be stored and accessed at will. 

Finally, with a few lines of code almost identical to the saveResults function above, the results are printed in a user-friendly display. 

The new code in the main function (top line in not new but I added it for reference).


The Code 

from scapy.all import IP, ICMP, TCP, sr1, UDP, DNS, RandShort, DNSQR
from ipaddress import IPv4Network, ip_address
from multiprocessing import Pool
from argparse import ArgumentParser
import csv

dns_server=None

tcp_common_ports = {
    21: "ftp",
    22: "ssh",
    23: "telnet",
    25: "smtp",
    53: "domain name system",
    80: "http",
    110: "pop3",
    111: "rpcbind",
    135: "msrpc",
    139: "netbios-ssn",
    143: "imap",
    443: "https",
    445: "microsoft-ds",
    993: "imaps",
    995: "pop3s",
    1723: "pptp",
    3306: "mysql",
    3389: "ms-wbt-server",
    5900: "vnc",
    8080: "http-proxy" }

#the following function will to resolve an IPv4 address to its host name
def resolve_IP(ip_address_to_resolve):
    global dns_server
    reverse_ip_address = ip_address(ip_address_to_resolve).reverse_pointer
    #print("The reverse pointer of" , ip_address_to_resolve , "is" , reverse_ip_address)
    packet = IP(dst = dns_server)/UDP(dport = 53, sport = RandShort())/DNS(rd = 1, qd = DNSQR(qname = reverse_ip_address, qtype = "PTR"))
    response = sr1(packet, verbose = 0, timeout = 2)
    host_name = "not resolved"
    if response:
        if int(response.getlayer(UDP).sport) == 53:
            try:
                #print("The host name of" , ip_address_to_resolve , "is" , response.an.rdata.decode('utf-8'))
                host_name = response.an.rdata.decode('utf-8')
            except Exception as e:
                pass
    return host_name

       
def send_ping(address):
    results = {}
    results[str(address)]=[]
    packet = IP(dst = str(address))/ICMP(type = 8, code = 0)
    response = sr1(packet, verbose = 0, timeout = 2)
    host_name = resolve_IP(address)
    if response:
        if int(response.getlayer(ICMP).type) == 0:
            print("Host" , address , "is reachable. Its host name is" , host_name)
            tcp_open_ports = send_tcp(address)
            results[str(address)]=[host_name,"Reachable", tcp_open_ports]
        elif int(response.getlayer(ICMP).type) == 3:
            print("Destination" , address , "is unreachable")
            results[str(address)]=[host_name,"ICMP Unreachable"]
    else:
        results[str(address)]=[host_name,"No response"]
    return results
 
   
def send_tcp(address):
    ports = list(tcp_common_ports.keys()) ##returns a list of all keys (aka port numbers) of tcp_common_ports dictionary
    tcp_open_ports = []
    for p in ports:
        packet = IP(dst = str(address))/TCP(sport = RandShort(),dport = p, flags = "S")
        response = sr1(packet, verbose = 0, timeout = 2)
        if response:
            if response.haslayer(TCP):
                if str(response.getlayer(TCP).flags) == "SA":
                    print("Host ", address , "is listening in port",p,tcp_common_ports[p])
                    tcp_open_ports.append(p)
                elif str(response.getlayer(TCP.flags)) == "R":
                    print("Host ", address , "is rejecting conenction on port",p,tcp_common_ports[p])
            else:
                if int(response.getlayer(ICMP).type) == 3 and int(response.getlayer(ICMP).code) == 13:
                    print("Host " , address, ": communication prohibited on port",p,tcp_common_ports[p])
                else:
                    print(response.summary())
    return tcp_open_ports

def saveResults(results):
    with open ("scanner_results.csv", "w") as file:
        write = csv.writer(file)
        write.writerow(["IP Address", "Host Name", "Reachability",  "TCP Open ports"])
        for r in results:
            for k in r:
                if len(r[k]) == 3:
                    write.writerow([k, r[k][0], r[k][1], r[k][2]])
                else:
                    write.writerow([k, r[k][0], r[k][1], "[]"])
       

def main():
    parser = ArgumentParser()
    parser.add_argument("-p", "--processes" , help = "the number of processes to run in parallel (must be integer)"  , type = int, default = 10)
    parser.add_argument("-dns", "--dns_server" , help = "the DNS server to be used for our querries"  , type = str, default = "8.8.8.8")
    values = parser.parse_args()
   
    global dns_server
    dns_server = values.dns_server
   
    network = input("Please enter the IPv4 network you'd like to scan (e.g. 192.168.1.0/24) or just an IPv4 address: ")
    valid_target = False
    while valid_target == False:
        try:
            targets = IPv4Network(network)
            valid_target = True
        except Exception as e:
            print(e)
            network = input("Please enter a valid IPv4 subnet: ")
           
    with Pool(values.processes) as process:
        results = process.map(send_ping, targets)
   
    print()
    print("IP Address \t Host Name \t Reachability \t TCP Open ports")
    print("---------------------------------------------------------------------")
    for r in results:
        for k in r:
            if len(r[k]) == 3:
                print(k, "\t", r[k][0],"\t", r[k][1], "\t", r[k][2])
            else:
                print(k, "\t", r[k][0],"\t", r[k][1], "\t", "[]")
    saveResults(results)

if __name__ == "__main__":
    main()

For Next-Time

By the next blog post, I will have uploaded the code to GitHub, where it will be available and accessible more easily. I will also look to improve and develop the code further.