Home arrow Latest Tutorials arrow Network Troubleshooting Tutorial for a Small LAN
Network Troubleshooting Tutorial for a Small LAN PDF
Sunday, 10 August 2008

This tutorial shows you how to troubleshoot a LAN network and restore internet connection for desktop or mobile computers from this LAN.

We asume that you have a FreeBSD router connected to the Internet on a network card (WAN network card) and to LAN on another network card (LAN network card).

The network layout will be like the on in the next picture:

network troubleshooting for a simple lan with a freebsd router

 A simple LAN network behind a FreeBSD router with NAT (Network Address Translation)

The FreeBSD router from our example has two network cards: em0 and em1. em0 is WAN (Wide Area Network) card connected to the Internet, and em1 is LAN network card connected to our LAN network via an ethernet switch.

Router Network Configuration
-----------------------------------------

WAN Network Card (WAN NIC)
IP: 80.x.y.z
Netmask: 255.255.255.248
Gateway: 80.x.y.w
DNS1: 80.a.b.c
DNS2: 80.a.b.d

LAN Network Card (LAN NIC)
IP: 192.168.0.1
Netmask: 255.255.255.0

Note1: WAN NIC settings are given by your ISP, you cannot use any value you want for WAN Network Card setup!
Note2: FreeBSD router must be configured to work with NAT (using either IPFW + natd or PF, OpenBSD's packet filter) How to configure a router with NAT will not be described here, you can find tutorials regarding this issue on our website.


Configuration of a Desktop Computer From LAN
------------------------------------------------------------------

IP: 192.168.0.2
Netmask: 255.255.255.0
Gateway: 192.168.0.1
DNS1: 80.a.b.c
DNS2: 80.a.b.d

Note: You can choose a private subnet class for your LAN computers (including LAN network card of the router). All IPs of all computers from LAN must be on the same subnet (must have IPs from the same subnet).


Network Troubleshooting
-----------------------------------

Internet is not working from a LAN computer

This is usualy a repot by a user that "Internet is not working" for him.


Step 1. We ping the router from a desktop computer on LAN
----------------------------------------------------------------------------------
In this case from a command line shell from desktop computer we ping IP on LAN card of the router (Start / Run / cmd on windows):

ping 192.168.0.1

If we have a reply then we know the following informations:
- the network card of the desktop computer works ok (it is not hardware broken, the drivers of network card are installed ok);
- the physical link is working with the FreeBSD router (Ethernet cable from desktop computer to switch is ok, ethernet cable from switch to FreeBSD LAN network cards is ok, LAN switch is working properly).

We can try to ping an Internet host from our desktop computer:

ping www.google.com

In most cases (but not all cases) it will not work (since the user reported that his "Internet is not working").

At this point, before starting more troubleshootings, we can think of a DNS problem. When pinging google.com, if we do not have a reply is good to have written down an Internet IP address (valid and working on the Internet) and we can try to ping that address. If we have a reply then the problem is from misconfiguration of DNS server on LAN desktop or from a broken DNS server from our ISP.

So by pinging router LAN IP which is working and pinging google which is not working (also pinging an IP on a working host on the Internet, we know that the problem is not on our desktop station or LAN network but on our router.

If we do not have a reply to our FreeBSD Router, then the problem could be one of the following:
- network card is not working on our desktop computer;
- network card is not properly configured: drivers are not installed or IP, Gateway, netmask and DNS are not properly configured;
- ethernet cable from desktop computer to LAN ethernet switchis broken;
- ethernet cable from Ethernet switch to FreeBSD router is broken;
- switch is not working (or one or more switch ports), or because of a power shock the switch is frozen, and must be reseted.


Step 2. We ping a host on the Internet to check Internet connectivity
---------------------------------------------------------------------------------------------

ping www.google.com

If we have a reply then, we have the following informations:
- we have Internet connection on our FreeBSD router
- we have LAN connection from our desktop computer to FreeBSD router

In this case the problem coud be one of the following:
- Routing and NAT are not properly set on FreeBSD router
- Gateway IP is not set on desktop computer

If we do not have a reply when pinging an Internet host from our FreeBSD router:
In this case either:
a. - we do not have properly setup the WAN IP, netmask and gateway;
b. - there is a problem with Internet line/connection from our ISP.


Step 3. We will verify if we have properly setup our network card,
and we have right ethernet cables connected to WAN and LAN
-----------------------------------------------------------------------------------------

a. To verify if we have setup properly network card use the following commands:
Disconnect your LAN cable, and type:

ifconfig

With LAN network cable disconnected, you must see (when typing ifconfig) only your WAN network card as "active". This is useful in case if by mistake you switched cables and connected WAN ethernet cable into LAN network card. After you make sure cable is connected to the right internet port you may reconnect LAN network cable.

If IP is properly setup for WAN and LAN network cards, don't forget to also check your subnet of your WAN network card.

Then we can check if we have a correct default route (Gateway IP from your ISP).To do that use the following command:
netstat -nr

You shoud get something like this:

# netstat -nr
Routing tables
Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            80.x.y.w           UGS         0 20386343   em0

80.x.y.w is IP of your uplink Router from your ISP. If you do not have this "default" router, then you must add it in order for your FreeBSD router to have internet connection. You can do that with :

route add default 80.x.y.w         #(from command line)

and to make this setting permanent,
edit /etc/rc.conf and add:

defaultrouter="80.x.y.w"

We've checked, we have right connection to Internet lines and LAN ethernet cable, on right network cards from our FreeBSD router, we have proper IP setup on both cards, still pinging an Internet host (domain name) is not working. In this case we still have some things to check (see next step).


Step 4. Get more informations about the problem with Internet line/connection from our ISP
------------------------------------------------------------------------------------------------------------------------------
After we make sure our FreeBSD router is properly configured we know the problem is from our ISP:
- either from our connection to our ISP;
- or from ISP to outside connection of our ISP.

Note: If you are a beginner with FreeBSD and you are not 100% sure that the problem is with your Internet connection from your ISP you can configure a desktop windows computer with IP, subnet, netmask and DNS from your ISP and test your Internet connection that way. If it will work then review the previous steps from your tutorials because probably did not properly configured your FreeBSD router.

If the Internet Line/Connection to our ISP is not working we must contact our ISP technical support to solve the problem. Still we can get more informations about the problem to tell, and this information will also be useful for our ISP technical support people.

We will check to see if our FreeBSD router can "see" our uplink router from our ISP by pinging our gateway (default router) IP from our ISP:

ping 80.x.y.w

If is working then we can try to ping an IP from the Internet (an host IP not host domain name):

ping 91.a.b.c

Note:
91.a.b.c must be a valid IP of a server on the Internet, (for example an IP of google), You must know one such IP before your Internet connection  will be down, you can easy find IP of a google host by pinging www.google.com (and writting down the returned IP) .

If ping works (returns reply) when pinging an IP on the Internet, then the problem is our DNS server from our ISP.

If ping is working to our gateway (ISP router from our uplink) but ping to an IP from the internet is not working (for example IP of a host on google.com) then the problem is with Internet connection from our ISP to outside.


Advanced Network Troubleshooting
--------------------------------------------------
Here are some usefull tips when you troubleshoot your network connection:

Tip1.  When you use ping tool, if you get replies, don't forget to look for reply times. If you got replies but times are not low, then still you might have a problem with your connection.

Tip2. See if you have errors/collisions on your FreeBSD router network cards. Type:
netstat -ni
Look at values for: Ierrs, Oerrs and Coll.
You can also refresh this info every few seconds (-w 2 for two seconds):
netstat -w 2 -I em0

Tip3. See your network routes on FreeBSD router:
netstat -nr

Tip4. See active connections to your FreeBSD router:
netstat -p tcp

Tip 5. See more statistics about your network interfaces:
netstat -s
netstat -s -p


Tip 6. Display network memory buffer size
netstat -m

Tip 7.
Check if you have hardware link (conectivity) with desktop computer from FreeBSD router
arp -a
(try this if you've tried to ping desktop computer and ping is not working. Also note when you try to ping desktop computer that
it is possible to not have a reply because of a firewall installed on desktop computer, by default on, on windows. Disable that firewall on desktop
computer to be able to test. After you've restored network/internet connectivity don't forget to reenable firewall).

Tip 8. Use arping to see if an IP is already asignet on a computer on your subnet
Arping will querry the subnet and return IP / MAC - hardware address of that machine.
arping 192.168.0.2

Tip 9. Be carefull when you configure netmask for Router LAN network interface card, and for desktop computers from LAN. Keep in mind that any machine that will try to access another computer with IP belonging to the same subnet (defined by your netmask) will not go to default route but will search that computer in your LAN, by asking your ethernet switch for a MAC hardware address. If your LAN setup use multiple private subnets, some of them routed through a LAN router (for example you have another LAN connected to your LAN via a router) computers from your second LAN will not search for your router if your desktop computers from your second LAN have configured to see computers from first LAN using your netmask. So when you have multiple small LANS connected to each other, if you want them separated, use small separate subnets, and route IPs using a router.

Tip. 10. Look for errors in /var/log/messages:
tail -f /var/log/messages
or:
less +G /var/log/messages

Tip.11. Use top to see CPU usage.
top
CPU states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle

If you have a low idle value, look for application that consumes CPU.
Also if there is no high consuming CPU app, look for interrupt value. High interrupt value means the router has a problem with networking stack (routing).


Network Troubleshooting Tools and examples
----------------------------------------------------------------
ping 192.168.0.2
                          # ping a host
netstat -nr                                    #show routing table without resolving domain names
netstat -ni                                    # show informations about errors/collisions
traceroute www.example.com  # find route to example.com
tcpdraceroute www.example.com  # find route to example.com using tcptraceroute (which uses tcp to discover path).

netstat -i -I em0                          # show statistics about your network card
sockstat -c                                 # show connected sockets
sockstat -l                                  # show listening sockets and processes

arp -na 
                                      # show arp table
arp -d 192.168.0.2                     # delete a record from arp table
arp -s 192.168.0.2  00:10:b5:99:bf:c4       # add a static record in arp table

tcpdump -tlni em0
                     # listen on em0 network interface and sniff packets that pass via em0
tcpdump -tlni em0 -n icmp       # listen for ICMP traffic on em0 network interface
tcpdump -i nfe0 -c1 -n -s0 -vvvv icmp    # capture one ICMP packet and decode it


FreeBSD Sysctl variables that could affect network performance
----------------------------------------------------------------------------------------
net.inet.tcp.sendspace
net.inet.tcp.recvspace
net.inet.udp.recvspace
kern.ipc.nmbclusters



Last Updated ( Tuesday, 12 August 2008 )
 
< Prev   Next >

Other BSD Systems

OpenBSD

Misc

Solaris

Polls

Best BSD firewall?