Edward Wincott, Janet

Anthony Morgan, Cardiff University

Tom Franklin, Franklin Consulting

Overview

This case study describes the implementation of an innovative dark fibre solution to provide inter-connectivity for three remote sites across the centre of Cardiff. The technology employed is a rapid deployment, slot-cut conduit system through which fibre optic cables are blown. The solution satisfied the need for an unlimited bandwidth inter-connection with high security, moderate capital cost and low recurrent costs. The motivation for the project, the planning, the equipment and its installation are considered. Post installation issues such as the performance and benefits are also assessed.

Executive Summary

Main Points:

• Needed to connect the main campus of the Cardiff University to the campus of the recently merged University of Wales College of Medicine, 3.5 km away.
• Two separate slot cut fibre routes were cut to provide resilience.
• Total cost including circuit and equipment and five years maintenance was £300,000.
• One cable has been severed by utility companies on ten occasions, but there has as yet been no network outage due to the fault tolerant duplicate routing.
• The main benefit is a comparatively inexpensive inter-connection solution providing virtually unlimited bandwidth (with up to 48 dark fibres per cable).

Introduction

Cardiff University is a medium sized university with 22,000 students, 5,000 staff and an annual turnover of £300 million. It has 28 schools and almost 5,000 student residences, most of which are within walking distance of the main campus.

The University merged with University of Wales College of Medicine (UWCM) in December 2004 and connectivity between the previously separate organisations was required, in particular between the University and a hospital, a teaching block and a hall of residence some distance from the main campus. The perceived benefits of connecting the remote sites included access to resources on the University network, electronic communications and access to JANET.

Service Requirements and Constraints

Cardiff University’s network is configured as three separate VLANs (Virtual Local Area Networks) – one for students, one for staff and one for management. Where possible these logical networks are also kept physically separate to maximise security, so a connectivity solution was sought which would support multiple physically separate networks. The solution also had to be affordable in terms of both capital and recurrent cost when viewed over a five year period. 

An infrastructure solution that supported an extension of the network to more sites in the future was preferable. The initial sites connected in this project were the main Cathys Park Campus and the Heath Hospital site some 3.5 km away. Since the completion of the initial project the University has also connected a hall of residence and will be connecting a new building which is currently under construction.

The original budget for the project was provided by a £200,000 grant from HEFCW (the Higher Education Funding Council for Wales). This proved to be insufficient and the University added a further £100,000 to the budget. There was no budget for recurrent costs so a solution which capitalised the costs was considered advantageous.

Identification of Possible Solutions

The University considered laser and wireless, deep trench and ‘microduct’ fibre optic cable and leased line solutions.

Laser/Wireless

Wireless systems were considered but at least two separate laser or wireless links between sites would have been required to enable the physical separation of the student and administration networks. However, where wireless or laser installations are mounted too close together there is a danger of interference between systems. 

In the case of laser links, there is a small but significant divergence of the laser beam. When two systems are mounted in parallel the spreading of the laser beams can result in transmissions from one unit being picked up at the receiver of the second system, resulting in interference and data loss. To avoid this, transceivers must be mounted sufficiently far apart or the beams angled to prevent beams overlapping at the receivers. For parallel beam alignment, the minimum recommended separation is 1.2°, ie. 20 metres for each kilometre.

Beam divergence is much greater with microwave wireless systems than with laser systems. To prevent interference a very large separation of antennae must be achieved. Where antennae cannot be widely separated, a different technique must be employed in which completely different channels or frequency bands have to be used to avoid interference and data loss.

For the inter-site links required at the University, the distances involved and the limited mounting points for the equipment meant that the necessary transceiver separation for laser links could not be achieved. This ruled out a laser solution. A potential wireless solution was also ruled out due to the cost and complexity of achieving a high bandwidth, multiple channel system, given the geography of the sites.

Fibre Optic Cable

The next option was a fibre optic cable-based connection. This would provide high bandwidth as well as multiple physically separate data paths. The traditional method of implementing a fibre optic connection is for deep trenches to be dug, wide bore ducting installed and multi-core fibre optic cable pulled through. This is ideal when connecting buildings on a campus where distances are short or where ducting is already in place to support other services. Deep trench ducting becomes more expensive and problematic when distances become great, the land it must cross is not owned or the public highway has to be crossed.

The traditional method of installing fibre optic cabling was preferred by the University because the depth of the trenching provides security from damage by contractors during any subsequent work. Although capital costs can be very high, there is no on-going annual cost and the cabling system can support network needs for many years into the future. However, during the research into a possible solution, a novel system for installing fibre optic links was identified which promised greatly reduced installation costs.

FibreSpan offers a low cost, dark fibre solution for inter-site connectivity through their patented method of laying cables in a shallow slot cut in the road or pavement, to a depth of 280‑300 mm. FibreSpan's microduct is laid in a 12 mm wide slot which is cut by a large cutting wheel. Access chambers are installed and fibres (up to 72 for multimode and 96 for single mode) are blown along the microduct. The fibres are terminated at each end and can then be connected to switches or routers as for standard fibre optic links.

FibreSpan holds a DTI/OFTEL Telecommunications Licence and has code powers which permit it to dig trenches along the public highway for the installation of communications cabling.

The cost model for FibreSpan microduct based systems comprises an initial installation charge, in common with deep trench/ducted solutions, plus an annual maintenance cost. The maintenance service is essential as this covers damage to the fibre due to third party contractor excavation. The key point to note is that there is no bandwidth related cost and the FibreSpan system is therefore one of unrestricted bandwidth. This contrasts sharply with the solutions generally offered by most mainstream fibre companies. The usual fibre optic connection solution is on a leased line basis using client end links to the company’s core network infrastructure. Telcos typically sell leased line bandwidth as a commodity in Mbit/s or Gbit/s service packages at varying costs depending upon bandwidth. It is uncommon for any telco to lease their private dark fibre circuits at a fixed cost and where this is offered, the cost is very high.

With traditional multi-core fibre cable connection, one pair of fibres would be used for the student network and a second pair for the staff network. Normal practice would be for 8, 16 or more multi-core cable to be installed. This would leave spare fibres for use if one fibre became damaged or if the organisation wished to expand the network. This provides a degree of resilience and upgradeability.

With the relatively low cost of the FibreSpan microduct solution, it became possible within the University’s budget to install two completely separate microduct links between the two sites. With 48 core fibre cable blown through between sites, the solution developed consisted of two fibre channels per microduct route. Both student and staff networks could be supported in each of the independent conduits, so providing essential fault tolerance.

Leased Line

The University considered the possibility of using leased lines (such as LAN Extension Service as these were considered to be the most reliable), but the flexibility afforded by dark fibre which allows management of the network to meet evolving needs was preferred.

Cost/Technical Information

Fibre optic connection via microduct was only available from FibreSpan so a direct cost / technology comparison with other potential suppliers of similar systems was not possible. The comparison that could be made was with fibre optic cable run in traditional deep trench ducting.

The five year cost for a fibre optic cable laid in a traditional deep trench would have been around £600,000 as opposed to the estimate of £300,000 for two slot cut fibre cable ducts each with 48 fibres over five years. The specification for two independent slot cut fibres was considered highly desirable in order to incorporate fault tolerance into the link. These prices covered both the capital and recurrent costs.

Considering both capital and recurrent costs over five years, the FibreSpan microduct solution was seen as highly attractive as it was half the cost of digging a deep trench.

Parameter	Technology
	Deep trench traditional fibre optic cable	FibreSpan slot cut  microduct
Installation cost	For private circuits – expensive to install	For private circuits – a quarter the cost of traditional method to install
	For leased circuits – cost rapidly increases with bandwidth	Compared to low bandwidth leased circuits – more expensive Compared to high bandwidth leased circuits – fixed cost results in increasing cost attractiveness
Annual cost	For leased circuits, cost escalates with bandwidth	Fixed maintenance cost, independent of bandwidth
Installation speed	Time consuming to install	Very rapid to cut slot, insert microduct and blow fibre cable
Vulnerability to contractor excavation	Limited vulnerability	Vulnerable due to shallowness of installation
Resilience	Only provision is via spare cores in fibre cable	As result of low cost, multiple microduct routes can be installed to provide resilience
Maintenance contract availability	Not usually covered	Maintenance contract available

Table 1: Comparison of traditional fibre optic cabling and FibreSpan microduct

Although deep trench installed fibre which is less vulnerable to being severed would have been preferred, the University chose FibreSpan’s microduct system based on the excellent value for money and total project cost.

The FibreSpan Solution

There are three elements to the service offered by FibreSpan:

• Planning. Network design, including a full site survey and liaison with local authorities and landlords.
• Deployment. Rapid, unobtrusive and discreet installation of the ‘all fibre’ network.
• Support. All networks are fully maintained and supported.

There is an installation fee and an all-inclusive fixed annual fee for dark fibre connectivity which includes comprehensive maintenance.

Fibre Cable to be Installed

The cable is G652 single mode fibres to specification MHT 1400, set in a buffer layer to provide dimensional and thermal stability. The outer thermoplastic layer is designed to provide protection and facilitate installation. The units are designed for blowing into microducts and tube bundles. The fibres are dry, not coated with gel, thus permitting fast and contamination-free connections, and have the following specification:

Fibre type:	Single mode to G652 (ITU-T ) and MHT 1400
Attenuation typical:	0.4dB/km max at 1310nm (at room temp) 0.3dB/km max at 1550nm (at room temp)

NB. dB is the difference (or ratio) between two signal levels and is used to describe the effect of system devices on signal strength.

	2 fibre unit	4 fibre unit	8 fibre unit	12 fibre unit
Outer diameter (nom)	1.1mm	1.1mm	1.5mm	1.6mm
Mass ‘w’ (nominal) g/m = kg/km	1.0 g/m	1.0 g/m	1.8 g/m	2.2 g/m
Min bend radius (MBR) when deployed	50mm	50mm	80mm	80mm
Fibre Colours	Blue, orange	Blue, orange, green, red	Blue, orange, green, red,  grey, yellow, brown, violet	Blue, orange, green, red, grey, yellow, brown, violet, black, aqua, pink, white

Table 2: FibreSpan fibre bundle specifications

Feasibility and Planning

To determine the feasibility of a microduct solution it is necessary to work out a route which will not cut other service cables or pipes. However, as the microduct is installed in  slots there is little real risk of cutting through other services. The corollary is the risk that utility companies will cut the microduct fibre when they dig up their own cables or pipes. FibreSpan takes responsibility for checking with the local planning authority where existing cables are, and for negotiating with them the exact route for the cable.

Cardiff University carried out the initial design of the fibre routes and then discussed these with a site engineer from FibreSpan. A key element of the plan for the link was that there should be two independent cable routes in order to build resilience into the fibre network.

There were some minor modifications proposed to the routes. It was then FibreSpan’s responsibility to negotiate the precise route with the local council. The University owns some service ducts in the campus area and so it was planned to use these where possible. This would reduce the cost of the installation (as there would be less route to cut) and was technically sensible since the University ducts were considerably deeper than the FibreSpan slots would have been. FibreSpan would then run microduct from the University ducts to the destinations.

Procurement

FibreSpan has a patent on the technology used for their method of laying cable, and as such are the sole suppliers of the equipment. Cardiff University IT Services therefore had to certify to the University's purchasing department that there was only a single supplier for the solution and consequently it was not possible to go out to tender. However, a sufficient market research had been carried out to ensure that the selected solution was the most economically advantageous.

Implementation

Installation of the first fibre was straightforward with no problems encountered and was completed in Spring 2003. However, the second route to the hospital crossed some land owned by the NHS. It took many months of negotiation before this short stretch of grass could be crossed so the second line was not completed until June 2004.

Inspection chambers are required every 450 metres along the microduct route and a total of 15 were required for this installation along the two routes.

The fibre optic cables installed by FibreSpan were terminated on fibre patch panels in the normal way, presented as SC connectors. The fibre cables were then simply patched to the network switches and operate as another piece of the network.

 

Figure 2: Picture of the slot cutting machine in action

Figure 3: Picture of the microduct laying operation

Project Timescales

It took six months from the provision of funding by HEFCW through the identification of a solution and the securing of additional funding to the signing of contracts. From then it took FibreSpan five weeks to complete the first route, including getting the necessary permissions from the Council to lay the cables and laying the microduct and fibre cables. The second cable took considerably longer to complete due to the negotiations with the NHS.

Operational Performance and Reliability

As with any other part of the network the fibre is managed and, apart from when it has been cut by contractors digging up the roads, it has given no problems.

The main problem has been repeated road works on one stretch of the cable route which have resulted in it being cut ten times in the last two and a half years. This section is at a busy roundabout and the local Council will only allow that stretch of road to be dug up on a Sunday, and then only if there is not a major event (such as the cup final) going on. This has on several occasions led to significant delays in the cables being repaired. Since the link comprised two cables which followed different routes, the University has not, as yet, lost service by having both cables cut or other faults developing at once. Both cables are in service all the time, with automatic switching when there is an outage on one of the cables to ensure fault tolerance.

Benefits of Project

FibreSpan’s microduct based slot-cut fibre system provided a cost effective solution to high-bandwidth inter-site connectivity between the University’s sites.

Since it was possible to install two separate microduct routes between the two sites the University achieved a resilient connection between the sites. With 48 core fibre cable in each microduct, it was possible to maintain physical separation of the VLANs between sites along each fibre optic cable.

Lessons Learned

The need to select a cable route where the roads are unlikely to be dug up is the most important lesson learned. Whether utilities companies check a route for other services before digging it up is unclear, but where a number of ducts, pipes etc. are concentrated, the chances of a microduct cable being cut are very high.  FibreSpan cable is so near to the surface that other services will invariably be below the fibre and when these are excavated the microduct is highly likely to be damaged.

Thought should also be given to the time when access to the cable will be granted by the local authority should faults occur, as this can cause significant delays in effecting repairs. It is therefore worth checking what restrictions exist on undertaking repairs that require roads to be dug up.

Unfortunately the Council has considerable control over the routes that the cables may take, and in this case insisted that the cable was routed through the interchange which led to the problems with the cable being cut and delays in its repair.

To build resilience into a slot cut fibre link it is worth considering laying two microducts along two different routes where this is affordable.

To simplify the procurement process the University has also moved to having preferred suppliers for both their CAT-5 cable and fibre optic cables, which means that they do not have to go to full open tender or OJEC (Official Journal of the European Communities) procurement for each project.

Summary

The implementation of a FibreSpan microduct connection between the University’s main campus at Cathys Park and Heath Hospital using two separate duct routes has been highly successful and cost effective. Overall the solution has been put in place at a lower cost than a traditional deep trench method with the added benefit that fault tolerance has been introduced. Minimal annual costs are being incurred for a fibre circuit which is supporting current bandwidth needs and which will support unrestricted future bandwidth when needed at no additional circuit cost.