Description of tests of USB-connected echo-cancelling loudspeaker/microphone systems for video-conferencing applications

Document Authors

Ronnie Gibb and Stephen Lawrence at Glasgow University

28th April 2011

This series of test has been conducted on USB-connected echo-cancelling loudspeaker/microphone systems for video-conferencing applications.

These would be used in place of a headset/microphone combination in desktop or small room software-based video-conferencing systems. It would allow natural hands-free two-way conversations, and would also be required when more than one attendee is taking part in the video-conference. You can read a summary of the results as well as the more detailed reports.

Overview

The Video Technology Advisory Service (VTAS) undertakes a number of technical studies including the evaluation of videoconferencing equipment. The main goal of this evaluation is to provide objective advice for higher and further education and research organisations so that they may make informed choices when purchasing videoconferencing equipment for use over IP, ISDN and other networks.

This test is one of a number of similar technical appraisal tests conducted by VTAS, and conducts tests on a basic Audio Input-Output device known as an Echo-cancelling Loudspeaker. This is for use in PC-based video-conferencing systems, and provides a very effective audio resource for allowing these simpler systems to connect with their larger compatible systems.

Echo-cancelling Loudspeaker/microphone equipment is only one of the components necessary for videoconferencing. The effective and efficient use of videoconferencing is also dependent on the room environment, the network, booking procedures, etc. For further information see other material available on the VTAS site.

http://www.ja.net/services/video/vtas/

Need for echo-cancellation.

Acoustic echo arises when sound from a loudspeaker - for example, the incoming sound loudspeaker in a video-conference system - is picked up by the outgoing audio microphone in the same room. It may normally be sited on a table near the participants 1 metre or so from that loudspeaker. The problem exists in any communications scenario where there is a loudspeaker and a microphone.

Examples of acoustic echo are found in everyday surroundings such as:

• A standard telephone or mobile phone in speakerphone or hands-free mode
• Hands-free car phone systems
• Dedicated standalone "conference phones"
• Video-conference systems

In most of these cases, direct incoming sound from the loudspeaker enters the outgoing microphone. This is called direct acoustic path echo. The difficulties in cancelling acoustic echo stem from the alteration of the original incoming sound by the ambient space. This colours the sound that re-enters the microphone. These changes can include certain frequencies being absorbed by soft furnishings, and reflection of different frequencies at varying strength. These secondary reflections are not strictly referred to as echo, but rather are "reverb".

Acoustic echo is heard by the far end talkers in a conversation. So if a person in Room A talks, they will hear their voice bounce around in Room B, then be picked up by the microphone in room B, and fed back to them, with a delay, and added room acoustics from Room B. The echo-canceller used in Room B benefits the speaker sited in Room A, and vice-versa. In multisite links, your local echo-canceller benefits everyone else connected in the meeting.

This returned sound needs to be cancelled, so it will not be sent back to its origin. Due to the round-trip transmission delay, this acoustic echo is very distracting. It gets worse with longer round-trip delays e.g. long distance video-conferences.

Acoustic echo cancellation is essential for hands free, natural videoconferencing, i.e. without the use of headsets or headphones/microphones. The ability for both sites to converse simultaneously is referred to as double talk. This is the natural way to confer, and efficient echo cancellation is needed in order to enable double talk.

Echo cancellers are now an integral part of most good quality videoconferencing systems and are now almost entirely automatic in operation, requiring no setup.

For small groups (two to six people) a single microphone/loudspeaker will be adequate. The systems tested here are all self-contained single-unit microphone and loudspeaker combinations, except one which uses the PC speakers, or connects to an external loudspeaker.

For this series of tests, all connect to the PC by USB connection. It is recommended by many of the manufacturers that a USB sound system should not be connected through a USB hub; or it should be powered externally if a hub is used.

These are the functions of the items tested here.

The Equipment Test Procedure

The evaluations were conducted between two internal University of Glasgow Venues.

One is a permanent JVCS-registered video-conference venue, equipped with a Tandberg C90 H323 video-conference unit. This venue has been designed as an ideal video-conferencing location, with appropriate audio and video facilities. The room has also been acoustically treated for best results for conferencing.

The other (test) venue is equipped with a standard Windows-based Dual-Core Dell 740 PC, and large-screen 40inch display monitor, fed by the VGA output of the PC. It was located in a multi-purpose meeting room, approximately 4.7m x 2.7m, with no special acoustic treatment or sound-deadening.

For compatible H323 video-conferencing, the software application used in that PC is PolyCom PVX Desktop Video-conferencing application, version 8.0.4. A standard £30 webcam provided the video source for this application. Only H323 application video-conferencing was tested, although other video-conferencing applications could be possible.

The test venue audio provision was by means of the various Echo-cancelling Microphone / loudspeakers on test in this procedure. These would replace a Headset / microphone, which is the most basic sound facility required for incoming and outgoing sound for personal/desktop video-conferencing.

Video Tests

No video tests were carried out for this project, apart from using the video content to confirm correct connection etc for the various test links.

Audio Tests.

No Objective tests were carried out, as all test items have no separate in/out facilities for measuring voltage levels etc. They are all intended as self-contained human interface input/output devices.

Also, they have no built-in accessible adjustments to set mic levels etc, and depend on the settings in the connected PC and Software codec.

However, the PC and PolyCom PVX (Software H323 Codec) incoming audio levels were set up to their optimum as specified in their operating and setup instructions for best practice. The microphone AGC was switched off in the PolyCom PVX software, to allow for best comparisons of the connected systems.

This allowed adjustment for best modulation into the PC, for best microphone level in use with the video-conferencing software, without perceived distortion

Software Tests

No tests were carried out on any software supplied for optional installation with each product. This varied between each manufacturer, so each may have been incompatible trying to install on another manufacturer’s product.

Test Procedure

The test procedure can be summarised as follows. The completed test schedule is set out in Appendix A.

As recommended, the devices were connected by USB cable direct to the PC USB ports.

In all cases bar one (PolyCom Communicator), each device Loudspeaker system was automatically recognised by the PC, which configured and made it the “preferred” playback sound device, replacing the internal sound card and connected loudspeaker system – they were now muted. The PolyCom Communicator required (Windows-only) drivers installation, and updating from web-sourced files as well.

The device Microphone was also recognised as the “preferred” input device, and was made available for use in the PVX software. Normally it was auto-detected in the PVX, when the PVX application was started AFTER connecting the test USB device.

While audio quality and echo-cancelling efficiency are important for the user, ease of setting up and operating the equipment is also important, particularly for non-technical users.

All tests were conducted by two testers, checking results at both sites for incoming and outgoing sound from the test systems.

Tests were carried out at IP 768 to give best quality sound, using 64kbits/s audio rate at G722 coding. This will “work” the test systems harder, as it gives them a higher-quality audio-signal to work with, requiring more processing.

The audio signals included:

• In-PC audio files, played back into the test devices to test the loudspeaker’s sound quality.
• Off-air TV test signals from a connected video-conference codec at 768 Kbits/sec to check video-conference audio playback quality from the PC-connected device.
• Both testers speaking and listening at both locations for the incoming (Loudspeaker) quality, and for the out-going microphone quality, & echo-cancelling efficiency as a far-end device.

Tests

The tests were carried out considering the various headings listed below.

A.        Hardware Description

• Dimensions    
• Looks 
• Connectors    
• Power
• Cables Supplied         
• Software supplied      
• Paperwork Supplied  

B.        Setup Procedure

• Procedure + problems           
• Documentation quality          
• Time                     

C.        Connectivity

• Cables                  
• Connectors          
• Comments           

D.        Ease of Installation

E.         Audio tests

• Frequency response  
• Headroom           
• Levels                   
• Echo Cancellation
• Adjustments?        
• Comments           

F.         Echo Cancellation quality

G.        Value for money

Summary Points

A subjective summary lists the various headings, to give an overall figure for how well the item does its task. It should be borne in mind that all reviews are subjective, and these results are the views and observations of the two reviewers at the time.

Each summary was marked as follows

• Very good = 5;    Good = 4;    Fair = 3;     Poor = 2;    Very Poor = 1.

Function

Hardware Description

Setup Procedure

Connectivity

Ease of Installation

Audio Tests

Echo Cancellation Quality

Value for Money

Total

Summary

Most systems operated well as intended – they did exactly “what it says on the box”. However, see the individual reviews for detailed comments.

When testing, all systems took time to “relearn” the room acoustics if they were physically moved while in a link. This could be in the order of 1-2 seconds, depending on how much movement, and handling noise was transmitted to the system while moving. Mic movement during a link is to be avoided where possible

All systems tested (bar PolyCom) were easy to connect to the PC, and were all recognised by the PC as USB Audio Hardware, and by the H323 Video-conference software used for this test.

There is no reason to suppose that they would not work well with other compatible software e.g. Skype, WebEx, Adobe Connect etc, but these were not tested in this trial.

Conclusions

These products show that the digital technology applied in these various systems is now so efficient that highly-effective real-time echo-cancelling is possible in small-form reasonably-priced (cheap?) hardware.

A few years ago this would only have been possible with a rack-load of equipment!

All users who wish the advantage and ease of not using cumbersome headset-mics should very seriously consider purchasing one of these systems for any video-conferencing or collaboration application.

Notes

Input Devices - Microphones, video cameras, white boards, etc. that provide the sound, vision and data input signals.

Output Devices - Loudspeakers, television picture monitors or data screens, white boards, etc. that generate the sound, vision and data output information at each site.

The list of manufacturers represented here is not exhaustive. VTAS would be pleased to consider suggestions for future products and suppliers although the in-depth evaluation of equipment is a time consuming process and VTAS is currently only able to test a limited number of products each year. The presence of a product in this list indicates only that the product has been evaluated by the VTAS product evaluation team and should not be taken as a recommendation. Similarly, the absence of a popular product from the list should not be seen to reflect negatively on that product.

The evaluation process produces a snapshot of the features and performance of a product at a specific moment in time. It is worth noting that a significant period of time may have elapsed between the date of testing and the time of reading this report, and during this period some products may have been updated by the manufacturer or even superseded. Full technical specifications of all the products evaluated (together with many others) are included in the manufacturers' web sites. It is strongly recommended that these sites be read in conjunction with the test reports and their associated comments, so that a more complete assessment of a product is obtained.