Inter-site connectivity and last mile technologies guide

Edward Wincott, Janet

May 2005

Overview

This guide introduces the technologies available in the UK for the connection of computer networks between multiple sites to create WAN (Wide Area Networks). The size range of networks to be linked considered in this guide extends from single remote users and small workgroup sites requiring connectivity to a central main network or the Internet, to large LAN (Local Area Network) inter-connections requiring multi-megabit bandwidths. This reflects the potential needs of small to medium enterprises and the general network requirements of academic institutions. Solutions for the very high inter-site data transfer needs required by, for example, research organisations for special projects, are outside the scope of this paper.

The aim of this guide is to help in the decision making process as to which technology should be adopted or investigated for a particular application. The target audience is IT managers and technicians who have a good understanding of LANs and some knowledge of WANs and who have been tasked with establishing, upgrading or reviewing inter-site data communications. Such readers may benefit from the technology comparison and characteristics tables, presented in quick reference form, for the various solutions which could be considered. Other readers who may benefit are non-technical decision makers for whom this guide will provide an insight into the issues and options available in selecting inter-site data communications solutions.

Introduction

There are many decisions to be made in selecting a WAN (Wide Area Network) solution. The considerations include:

Medium: cable (copper or optical); wireless (optical, radio or microwave).
Provision: buy outright; lease circuits; opt for contended service.
Management: self-procure and maintain; managed service.
Bandwidth: what is needed; supported by chosen technology; affordability.
Distance between sites.
Reliability and service availability.
Security.
Availability of technology in required localities.
Packaging and marketing of technologies; recognisable products.
Use of technologies to build a suitable network to satisfy the organisation’s needs.

Factors in Selecting an Inter-site Connection Technology

What is the link to be used for? As is said, ‘You don’t buy a Ferrari if what you need is just something for the shopping’. More fundamental is the need the network is to satisfy. For example, it may be a stunning achievement to put in place a 10 or 100Mbit/s WAN which enables all users to communicate with each other and use a bandwidth-hungry system. But if the need is only for access to the Internet and college learning materials, then a much simpler Internet-only connection to access an Extranet may be a better solution than a high performance, meshed WAN.

Having decided the link’s purpose, the all-important question of the required bandwidth can be considered. This often leads to the chicken and egg situation of which applications can be supported over the affordable bandwidth. If more bandwidth was available, a greater range of more bandwidth hungry applications would be considered, e.g. videoconferencing. (A high quality H.323 IP videoconferencing session can consume more than 2Mbit/s.)

Factors influencing the required bandwidth are:

numbers of users to be supported over the link
intensity of usage
applications
Internet access requirements
thin client systems
operating system
location of authentication and application servers

As an indication of bandwidth needs, 128kbit/s is suitable for a small group of up to say five users for e-mail, web access and online text based conferencing, and is just about usable for Voice over IP. Greater bandwidth becomes necessary when the number of users requiring simultaneous access increases, for example, for accessing a bandwidth-hungry database or videoconferencing over IP.

For linking LANs of any real size, e.g. a central office or campus to remote campuses or large offices, where a degree of real-time interaction is required, 2Mbit/s links are really the minimum. (Having said this, WAN connections to smaller offices can in many cases be satisfactorily be effected with sub-1Mbit/s links). These links are generally implemented in the form of terrestrial circuits. A 2Mbit/s link naturally complements an office/campus LAN operating at the old perfectly respectable 10Mbit/s (up to 100Mbit/s) standard which is fine for modest file and print networks. A 2Mbit/s bandwidth will of course always represent a bottleneck on today’s networks which generally operate at 100Mbit/s. In this scenario, bandwidth management has a role to play. Advice on aspects of bandwidth management can be found at www.ja.net/bmas.

Typically many organisations begin with a single 2Mbit/s circuit link between sites, increasing this to 2 x 2Mbit/s and beyond as needs grow. Where cost effective, 10Mbit/s fibre Ethernet is increasingly the defacto standard and where possible, up to 25Mbit/s wireless or 100Mbit/s laser is widely deployed.

In practice most organisations base their decision over the bandwidth required on their current experience of the perceived performance of the links and performance indicators such as round trip times, packet loss figures and percentage bandwidth utilisation.

Having selected the ideal bandwidth, the technologies available in the particular area can be considered. The capabilities of the telecommunications providers operating in the locality will have a major bearing on which technologies are available and at what cost. A further key consideration will be the geography of the land between the various sites and the distance between the sites to be connected.

Reliability and service availability will also be of vital importance. WAN links generally represent a considerable investment by an organisation and the ongoing annual rental costs can be significant. It is rare that an organisation can afford duplication, fault-tolerance and resilience. The paradox is that the main links of the WAN are usually absolutely mission critical, but are often installed at the limit of available funding. It is essential that the organisation recognises the importance of the communication links and allocates budget accordingly to ensure that >99.8% reliability is intrinsic in the technology selected. Otherwise fail-over systems must be put in place.

Selecting an Inter-site Connection Technology

The next two tables match available technology solutions to different bandwidth requirements and geographical distances. They are:

General Guide to Inter-site Connection Technology
Technology Suitability Table

The remaining tables give more detailed information about bandwidth ranges, installation and running costs, and applications supported by each technology.

Local Building to Building Campus Solutions
Small Office / Remote Outreach Centre / Single Power Home-user
Short Distance 1km-4km, Medium – High Bandwidth Connections
Medium Distance 4km-25km, Medium – High Bandwidth Connections
Longer Range 25km+, Medium – High Bandwidth Connections

Table 1. General Guide to Inter-site Connection Technology

B a n d w i d t h R e q u i r e m e n t s *

GEOGRAPHY

Low (0-2Mbit/s)

Medium (2-50Mbit/s)

High (up to 100Mbit/s)

Very high (100Mbit/s +)

Local building to building

1. Analogue private wire

2. Analogue leased circuits (EPS 8)

1. Wireless 802.11

2. Private fibre optic cable

1. Infrared laser

2. Private fibre optic cable

Private fibre optic cable

Small office, home user, remote outreach centre

1. ISDN

2. ADSL Broadband VPN

3. Cable VPN

4. Satellite VPN

1. G.SHDSL Broadband

2. Broadband Fixed Wireless Access and Public Access 802.11 Wireless Hotzones

Not generally required

Short range: 1km to 4km

1. Analogue leased circuits (EPS 8)

2. Broadband (xDSL, fixed wireless, hotzone) VPN

3. Cable VPN

1. Wireless 802.11

2. Leased line digital circuits

3. Short Haul Data Services (LES 10/ p2p)

1. Infrared laser

2. Private fibre optic cable

3. Short Haul Data Services (LES 100/ p2p)

4. Leased line digital circuits

1. Private fibre optic cable

2. Short Haul Data Services (LES 100/ p2p)

Medium range: 4km to 25km

1. Broadband (xDSL, fixed wireless, hotzone) VPN

2. Cable VPN

3. Frame relay (rarely used)

1. Wireless 802.11

2. Leased line digital circuits

3. Short Haul Data Services (LES 10/ p2p)

1. Licensed microwave

2. Short Haul Data Services (LES 100/ p2p)

3. Leased line digital circuits

1. Leased Line Digital Circuits

2. Short Haul Data Services (LES 100/ p2p)

Long range: 25km+

1. Broadband (xDSL, fixed wireless, hotzone) VPN

2. Cable VPN

3. Frame relay (rarely used)

1. Leased line digital circuits

2. Wireless 802.11

3. Licensed microwave

1. Licensed microwave

2. Leased line digital circuits

1. Leased line digital circuits

2. ATM/SMDS

*Example users/uses of:

Low bandwidth: e-mail, Internet access and thin client supported applications
Medium bandwidth: e-mail, Internet access and network applications, including high bandwidth applications such as videoconferencing
High bandwidth: large organisations and regional network operators, with high inter-site traffic, supporting most network applications

Note: This table is intended as a visual guide to the technology options and is not prescriptive. It is valid at the date of publication (May 2005).

Table 2: Technology Suitability

Local building – building campus solutions

Private Fibre Optic Cable Connections

Wireless 802.11

Infrared Laser

Analogue Leased Circuits

Small office/remote outreach centre

Single power home-user

Digital Subscriber Line – Broadband

Public Access 802.11 Wireless Hotzones

Broadband Fixed Wireless Access

ISDN (Integrated Services Digital Network)

Cable

Satellite Broadband

Short Distance 0km – 4km, medium – high bandwidth connections

Private Fibre Optic Cable Connections

Analogue Leased Circuits – EPS (Engineering Performance Specifications) 8

Wireless 802.11

Infrared Laser

Medium distance 4km – 25km, medium – high bandwidth connections

Short Haul Data Services – LES (Local Area Network Extension Service) or p2p (peer-to-peer)

Leased Line Digital Circuits

Wireless 802.11 (up to 14km)

Longer range 25km+ medium – high bandwidth connections

Leased Line Digital Circuits

Frame Relay

SMDS (Switched Multi-megabit Data Service)

Licensed Microwave

Technologies for the future

Free Space Optics – AirFibre

IP over Powerlines

802.16 Wireless – WiMAX

UMTS (Universal Mobile Telecommunication System)

Table 3: Building – building campus solutions

Technology

Bandwidth

Range

Cost

Application

Install

Annual

Private Fibre Optic Cable Connections

Very high

Limited only by equipment connected

Full duplex

Limited by connecting equipment and protocol

Physical barrier and rights of way may be an issue

High

£25/metre

Low

Main use; inter-building, on-campus links

Installed in underground / overground ducting or via overhead catenaries

Inter-site links feasible, but as distance increases cost becomes prohibitive

Wireless 802.11

802.11

802.11b 2.4GHz (WiFi)

802.11g 2.4GHz

802.11a 5GHz (WiFi5)

Medium

Half duplex

Nominal - 2Mbit/s

Realistic - 1Mbit/s

Nominal - 11Mbit/s

Realistic - 5.5Mbit/s

Nominal - 54Mbit/s

Realistic - 25Mbit/s

Nominal - 54Mbit/s Realistic - 25Mbit/s

< 12km direct single hop, greater distances possible with multiple hops

Increasingly widespread use of 802.11b 2.4GHz channels is resulting in higher levels of interference in some locations, necessitating reduction in bandwidth

Medium

£3k - £5k depending upon bandwidth

Low

Inter-building and inter-campus links of medium bandwidth where line of sight exists and where it is impractical to lay direct fibre optic cable

Reliable links can be established with careful setup of equipment

Most network applications can be supported (network authentication, e-mail, file transfer, Internet access) apart from extremely bandwidth hungry applications

Subject to interference from domestic appliances etc.

Infrared Laser

High

< 100 Mbit/s

Full duplex

< 4km direct

Affected by fog/adverse weather

High

c.£10k

- £20k

Low

Inter-building and inter-campus links of high bandwidth where line of sight exists and where it is impractical to lay direct fibre optic cable

Precise installation is vital

Subject to disruption by adverse weather, so some form of backup link is highly recommended

Analogue Leased Circuits (EPS8)

Medium < 2Mbit/s

Line quality/distance dependent

Full duplex

4km circuit

Sites must share same BT exchange

Low

£1,120

Low

£770pa

Ideal for inter-connection of sites situated in close proximity where cost of private fibre or LES circuit cannot be justified and where 2Mbit/s is acceptable

Provides 2Mbit/s Megastream type bandwidth at a fraction of the cost

Table 4: Remote outreach centre / single remote user solutions

Technology	Bandwidth	Range	Cost		Application
			Install	Annual
Digital Subscriber Line ‘Broadband’	Moderate, generally 2Mbit/s down, 256kbit/s up Subject to contention Full duplex Fast service 8Mbit/s down, 400kbit/s up	Limited by Internet boundary and availability of DSL enabled exchange Reach is 3.5km from exchange for full bandwidth or 5.5km for 250K Fast broadband reach up to 2km. Limited availability	Low £0 - £260 £50	Low As increasing upload and download performance is provided, costs increase 1Mbit/s/256kbit/s £780pa (1) 2Mbit/s/256kbit/s £1200pa 8Mbit/s £480pa	Always-on Internet access Single user to central site and workgroup inter-site links via VPN (Virtual Private Network) - not always 100% satisfactory 10:1 contention 33:1 contention
PAWH (Public Access 802.11 Wireless Hotzones) AND BFWA (Broadband Fixed Wireless Access)	Moderate Subject to contention Half duplex	Limited only by Internet boundary and availability of WISP (wireless ISP) coverage	Low	Low	Always-on Internet access PAWH - single user to central site via VPN/extranet. BFWA - workgroup inter-site links via VPN - not always 100% satisfactory
ISDN	Moderate 2 x 64kbit/s 30 x 64kbit/s Full duplex	Limited only by availability of ISDN at A and B ends	Low	Relatively high Charged on call time basis	Occasional Internet access Single user to central site and workgroup inter-site links via RAS (Remote Access Server)
Cable	Moderate Subject to contention Full duplex	Limited only by Internet boundary and availability of fibre cable in street	Low	Low As increasing upload and download bandwidth is provided, costs increase	Always-on Internet access Single user to central site and workgroup inter-site links via VPN - not always 100% satisfactory
Satellite	Moderate Typically 1Mbit/s down 384kbits/s up 2Mbit/s down 1Mbit/s up available Subject to contention Full duplex	Limited only by Internet boundary and line of sight to satellite	Moderate £300 -£1000 If kit must be purchased cost can be up to £1050	Low - moderate 512kbit/s/128kbit/s £708pa 1Mbit/s/256kbit/s £1668 - £4260pa 2Mbit/s/384kbit/s £6120pa (2) 2Mbit/s/1Mbit/s £12600pa	Always-on Internet access Single user to central site and workgroup inter-site links via VPN Up to 750 millisecond latency may be an issue for some applications

(1) Prices quoted are generally those for ‘business-type’ multi-user suitable services.

(2) Some providers offer special deals for schools and educational organisations.

Table 5. Short Distance 0km – 4km, Medium – High Bandwidth Connections

Technology

Bandwidth

Range

Cost

Application

Install

Annual

Private Fibre Optic Cable Connections

Very high

Limited only by equipment connected

Full duplex

Limited by equipment and protocol

High

£25/metre

Low

Medium

£8-10/m

Inter-building, on-campus links

Inter-site links feasible, but as distance increases cost becomes prohibitive

Analogue Leased Circuits (EPS8)

Medium < 2Mbit/s

Line quality/distance dependent

Full duplex

4km

Sites must share same BT exchange

Low

£1,120

Low

£770pa

Ideal for inter-connection of sites situated in close proximity where cost of private fibre or LES circuit cannot be justified and where 2Mbit/s is acceptable

Provides 2Mbit/s Megastream type bandwidth at a fraction of the cost (although this may be matched by a fraction of the service when there are problems with EPS circuits)

Wireless 802.11

802.11

802.11b 2.4GHz (WiFi)

802.11g 2.4GHz

802.11a 5GHz (WiFi5)

Medium

Half duplex

Nominal - 2Mbit/s

Realistic - 1Mbit/s

Nominal - 11Mbit/s

Realistic - 5.5Mbit/s

Nominal - 54Mbit/s

Realistic - 25Mbit/s

Nominal - 54Mbit/s Realistic - 25Mbit/s

< 12km direct single hop, greater distances possible with multiple hops

Increasingly widespread use of 802.11b 2.4GHz channels is resulting in higher levels of interference in some locations, necessitating reduction in bandwidth

Medium

£3k - £5k depending upon bandwidth

Low

Inter-building and inter-campus links of medium bandwidth where line of sight exists and where it is impractical to lay direct fibre optic cable

Reliable links can be established with careful setup of equipment

Most network applications can be supported (network authentication, e-mail, file transfer, Internet access) apart from extremely bandwidth hungry applications

Subject to interference from domestic appliances etc.

Infrared Laser

High

< 100 Mbit/s

Full duplex

< 4km direct

Affected by fog/adverse weather

High

c.£10k

- £20k

Low

Inter-building and inter-campus links of high bandwidth where line of sight exists and where it is impractical to lay direct fibre optic cable

Precise installation is vital

Subject to disruption by adverse weather, so some form of backup

Table 6. Medium Distance 4km – 25km, Medium – High Bandwidth Connections

Technology	Bandwidth	Range	Cost		Application
			Install	Annual
Short Haul Data Services (LES/p2p)	10, 100, 1000Mbit/s	< 25km	High - very high	Moderate Good value	Inter-connection of sites up to 25km apart, but much better suited to connection of sites relatively close together. Depending on the supplier, it can be difficult to diagnose faults when the WAN to which they are connected fails.
Leased Line Digital Circuits	‘Kilostream’ 64kbit/s upwards in 64K steps to 2Mbit/s ‘Megastream’ 2,4,6,8,10 upto 155 and 622Mbit/s ‘Megastream Ethernet’ 10 - 1000Mbit/s	No limit	High	High	Inter-connection of remote sites Depending upon bandwidth, suitable for e-mail, network authentication, remote access to file servers, high-bandwidth applications (videoconferencing) Marketed as; Kilostream, Megastream, LearningStream Products which may be more suitable for higher bandwidth/longer range; Megastream Ethernet, LearningStream Ethernet
Wireless 802.11 802.11 802.11b 2.4GHz (WiFi) 802.11g 2.4GHz 802.11a 5GHz (WiFi5)	Medium Half duplex Nominal - 2Mbit/s Realistic - 1Mbit/s Nominal - 11Mbit/s Realistic - 5.5Mbit/s Nominal - 54Mbit/s Realistic - 25Mbit/s Nominal - 54Mbit/s Realistic - 25Mbit/s	< 12km direct single hop, greater distances possible with multiple hops Increasingly widespread use of 802.11b 2.4GHz channels is resulting in higher levels of interference in some locations, necessitating reduction in bandwidth Limited range at higher bandwidths	Medium £3k - £5k depending upon bandwidth	Low	Inter-building and inter-campus links of medium bandwidth where line of sight exists and where it is impractical to lay direct fibre optic cable Reliable links can be established with careful setup of equipment Most network applications can be supported, network authentication, e-mail, file transfer, Internet access, apart from extremely bandwidth hungry applications Subject to interference from domestic appliances etc.

Table 7. Longer Range 25km+, Medium – High Bandwidth Connections

Technology	Bandwidth	Range	Cost		Application
			Install	Annual
Leased Line Digital Circuits	‘Kilostream’ 64kbit/s upwards in 64K steps to 2Mbit/s ‘Megastream’ 2,4,6,8,10 upto 155 and 622Mbit/s ‘Megastream Ethernet’ 10 - 1000Mbit/s ‘LearningStream’ 2Mbit/s ‘LearningStream Ethernet’ 10Mbit/s	No limit No limit No limit No limit	High Distance dependent £4250 - £9000 for 2Mbit/s £600/site	High Local ends plus fixed main link charge, so longer length circuits become better value eg. £8000pa Less than Megastream	Inter-connection of remote sites Depending upon bandwidth, suitable for e-mail, network authentication, remote access to file servers, high-bandwidth applications (videoconferencing) Marketed as; Kilostream, Megastream, LearningStream Products which may be more suitable for higher bandwidth/longer range; Megastream Ethernet, LearningStream Ethernet
Frame Relay	< 2Mbit/s	No limit	High	Moderate Distance independent	Inter-connection of widely dispersed remote sites where bandwidth needs are moderate Technology supports bursts of higher than average traffic density Very rarely deployed today
ATM / SMDS (Switched Multi-megabit Data Service)	Medium - high 2Mbit/s (E1) 34Mbit/s (E3) 155Mbit/s (STM-1) 622Mbit/s (STM-4)	No limit	High	High	Technology for large organisations or regional network providers/ISPs with high inter-site traffic needs
Licenced Microwave	High 100Mbit/s	No limit	£14-16,000 hardware for single hop 100Mbit/s	£600 for 38GHz licence	Technology for large organisations or regional network providers/ISPs with high inter-site traffic needs