Migrating to an Ethernet-centric Infrastructure

DWDM is alive and well! We have seen an increase in demand for DWDM systems, all driven by an increased demand for bandwidth in metro, regional and long-haul networks. While DWDM systems are generally deployed by communications service providers, one of the most interesting recent developments is the growing interest in building private long-haul DWDM networks. For many years private DWDM networks have been built in metro and regional areas. Private long-haul networks are built by governments, the research and education community, and large enterprises, such as financial institutions. The reasons for building such dedicated high capacity private networks are several. It all starts with a bandwidth need of course, but there must also be a bandwidth supply that is affordable. A number of carriers offer fiber lease arrangements, provide space to house equipment, and offer maintenance and repair services that provides users a chance to separate the procurement of fiber from the equipment that “lights” the fiber. DWDM equipment has continued to decrease in price and is now affordable for high capacity private long-haul applications. Another motivation is assurance the network will meet future capacity needs. When the network is privately controlled it is possible to add capacity by adding additional DWDM transceivers. On the other hand if individual wavelengths are purchased from a carrier there can be a concern that the carrier’s system may fill up, causing substantial delays in adding capacity.

Demand continues to grow for capacity on carrier DWDM networks. Carriers that deployed DWDM systems in the late 90’s may still have available capacity on those systems, but the incremental cost to activate that capacity may be high. Carriers have to decide if there is enough future demand to make the up-front investment to deploy modern systems, or if it is more economic to add capacity to existing systems. Modern DWDM systems have ultra-long optical reach (the distance signals can travel before electrical regeneration is required), high capacity, and much lower cost than the systems from the last millennium. In short, the combination of evolving traffic demands and continued focus on capex budgets has carriers scrutinizing the decision about whether to procure new systems or continue to add capacity to old systems. Increasingly, however, we expect they will opt to move toward new systems..

Another motivation for choosing to deploy new systems is the need to support new signal types. Many of today’s core IP networks are being interconnected using a 10Gigabit Ethernet LAN interface. Ciena’s systems carry this signal by enclosing it in a digital wrapper based on international standards. This approach allows the network to evolve to support inexpensive Ethernet interfaces while providing SONET-like management capabilities. Many of the old DWDM systems are SONET-oriented, and cannot easily carry these high speed Ethernet signals. New systems also support the introduction of yet higher speed channels, such as 40G. There is not yet widespread deployment of 40G because the costs of 40G transmission are higher per bit than 10G channels. With time the economics will improve, and all new systems must support 40G, while many old systems cannot.

So far we’ve spoken about economic motivations and service motivations, but have not directly discussed the need for growing system capacity. While your question mentions research publications on 1000 channel DWDM systems, such systems have not proved to be commercially viable. Ciena currently offers the industry’s highest capacity commercially deployed DWDM system, with a capacity of 1.92 Tb/s. We’ve found flexibility to be more important than continued increases in capacity. System flexibility allows the trade-off of ultimate capacity and optical reach for lowest network cost in a particular network design. Today’s multi-reach systems provide cost-optimized solutions for metro, regional and long-haul networks.

But what about fiber, are we running out? While there are always places in the network that may not have fiber deployed or available, in general there is plenty of fiber in the most popular traffic routes. In cities there is generally a great deal of fiber between the large central offices of local operators, both owned by those operators as well as by others. In the metro most of the fiber is not even using DWDM at present. As you suggested when fiber does run out, there will be plenty of opportunity to take currently lit, but inefficiently utilized fiber, and upgrade it with DWDM equipment with high capacity. To cite an example that demonstrates the relative unconcern of carriers today about running out of fiber in long-haul networks, some network operators have chosen to light long-distance fiber networks with equipment that only supports 40 10Gbit/sec wavelengths. The majority, however, are using systems with 80 to 192 channel capacity.

Operations Support Systems (OSS) are large and varied so there is no single answer, and certainly no simple answer, to this question. However, there are some fundamental challenges that apply to a broad set of situations. We look at these challenges from two perspectives. First, what functionality do we need in optical Ethernet to make it manageable by operations support systems? Second, what changes are required to adapt current OSS implementations so they can manage Optical Ethernet networks and services?

Many optical Ethernet services today are supported by network platforms developed for LAN environments, and lack the functionality needed for performance monitoring on a service or customer basis, and also lack the ability to identify and localize faults in a large scale carrier application. Ethernet management standards are currently under development to address this gap in functionality, both in the IEEE and in ITU-T. It will take some time to complete standards, and to introduce equipment implementing the standards to carrier networks. While standards are progressing, Ciena provides solutions to allow optical Ethernet transport solutions to be managed like a carrier’s traditional SONET/SDH transport network, with full fault and performance management. For example, the ITU G.709 standard, OTN (Optical Transport Network), is ideally suited for carrying both TDM & Ethernet. The 10Gb Ethernet LAN PHY signal is now often used as a high speed router interface, but neither SONET nor SDH can carry this signal because the circuit bandwidth is too large. However, OSS systems can easily manage Optical Ethernet services with the benefits of SONET/SDH like management for OTN networks with little or no change in their systems or operational procedures. Avoiding a change to operational procedures is a large advantage to service providers.

What about the adaptation of OSS implementations to support optical Ethernet? That depends on the role the OSS in the carrier’s network and the flexibility of the OSS implementation. Of course a prerequisite is adequate management functionality in the network itself, so the OSS is not blind or powerless. Assuming information is available, a foundation for the management of optical Ethernet is the information model, a representation of what optical Ethernet looks like and how it should behave that must be integrated into the OSS. The Telemanagement Forum has work underway developing standardized information models and interfaces to element management systems. This is a very complex area since OSS are so varied, but let’s just say some OSS will have an easier time adapting than others.

SONET and Optical Transport Network (OTN) are both framing formats designed for optical networks. They both consist of an information payload and overhead information. The overhead information of both standards provides a rich set of management functions, including performance management, fault notification, fault localization, and management communications. Both include a multiplexing hierarchy, although the SONET hierarchy has a finer granularity than the OTN hierarchy. The rates specified by the OTN were chosen to allow it to carry specific SONET signals as a payload. These are the 2.5G, 10G and 40G SONET signals. When OTN carries SONET it does so transparently, without changing any of the SONET overhead information. This is particularly useful in wholesale network applications, where one carrier operating a SONET network procures a transmission segment from another carrier. Here transparency to management information and timing are critical to avoid SONET interoperability problems. There are three OTN rate signals defined so far, each somewhat higher than the corresponding 2.5G, 10G and 40G SONET signals. Additionally the OTN framing format includes Forward Error Correction, a feature that helps improve reach and reliability of optical transmission at high rates -- rates of 10G and beyond.

While the OTN conveniently accommodates SONET, it also can be used to carry other signals, including Ethernet and storage to name a few. The increasing interest in OTN today is driven by an application where it is used to carry a native 10 Gigabit Ethernet signal. The use of OTN to encapsulate 10 GE results in a transmission solution with the management benefits of SONET at the price points of Ethernet.

Further, though your question is about SONET, an ANSI (North American) standard, it is important to note that OTN is a global standard that has been accepted by both ANSI and ETSI (European standard). This means that no longer will SONET and SDH networks compete and require translation when the circuits cross the ocean. Instead, a circuit that is terminated in Europe and Asia will be the same format as North & South America -- a feature that will benefit everyone.

Many telecom operators have an extensive investment in SONET/SDH transmission equipment, along with automated operations support systems. Any service that can be carried over SONET, by adding a card to one of their existing pieces of equipment, is easy to provide to virtually all their customers served by SONET equipment. An additional benefit of a SONET platform is the excellent management visibility it provides, and the fast protection mechanisms it supports. The service provider can be sure the service is meeting service level agreements.

Where SONET falls short is in the provisioning of high bandwidth Ethernet services. For services such as Gigabit Ethernet, the bandwidth is a large fraction of the bandwidth of a SONET platform in most deployed networks, and it is less likely that existing equipment will have the free capacity to support it. If one has to install a new piece of SONET equipment to carry a Gigabit Ethernet service, the equipment cost is quite high. For high bandwidth Ethernet services, a combined OTN/DWDM platform is much more cost effective. The OTN is the ITU standard aimed at Optical Transport Networks. It is simpler to implement than SONET, and as part of a DWDM system, is much more scalable than the typical SONET add/drop multiplexers found in carrier networks. OTN has the same powerful management functionality as SONET, and the same protection switching capabilities.

One can use DWDM without OTN as well. What you would have to be careful of is that the equipment provided sufficient management visibility to serve as a satisfactory service delivery platform. Ciena has products that meet this need, even without OTN. The IEEE has formulated standards for link management of native Ethernet links, but these are not yet widely deployed.

Thanks for touching base again, Michael.

The topic of ROADM and ASON is an interesting one. The ASON architecture includes both an internal and external network node interface definition (I-NNI and E-NNI, respectively). The E-NNI ASON interface is compatible with optical networks that contain ROADMs, as the E-NNI is able to make use of an abstract view of a network domain. In a sense, the E-NNI never needs to know there is a ROADM inside the domain; it just needs to learn what connectivity the domain can establish.

In the case of an I-NNI, the status is quite different for ASON standards concerning a network domain including network nodes that are ROADMs. Although the ASON architecture encompasses photonic network elements and all-optical network reconfiguration, no standards yet exist for a photonic layer I-NNI, and I am not aware of any standards under active development. This is also the case in the I-ETF, in the GMPLS drafts.

There are significant technical difficulties in obtaining and interpreting necessary analog parameters for a photonic layer path computation. Additionally, vendors of optical networking equipment use different technologies, with significantly different propagation performance. Therefore, the practical application of multi-vendor photonic layer interworking is not near, at least not for the general networking case. There will always be specific situations where distances are short and nonlinear effects small, where photonic interworking will be achievable without a great deal of difficulty.

In summary, the emphasis of standards work in the ITU is on developing an E-NNI, and this seems to be a good solution for the electronic layer as well as the photonic layer, where ROADMs are included.