Human Factors and… Sound Design

/0 Comments/in /by

Today we start a new series of interviews where we talk with different specialists associated with Human Factors and User Experience Design. The idea is to find out how HF Specialists are perceived by other disciplines, where our interests overlap, and how we can work better together to improve the design of products and services for all.

Our first interview is with Gareth Worthy a Sound Designer working in audio post-production and music and sound design for video and film. Gareth has advised us on a number of projects where sound and the auditory environment were important user experience considerations. We caught up with Gareth to get his views on sound design, in particular on the Human Factors of audible alarms and using sound to enhance the user experience in public places.

Some of our readers might not have realised that sound design is a profession! Could you describe what a sound designer does?

A sound designer is the person responsible for all (non musical) sound creation for specific projects. I generally work in audio for broadcast. My role as a sound designer would involve creating the non musical sounds that you would see on screen. That could be anything from making animated characters come to life, robotic movements, adding ambiences or foley sounds, such as footsteps and other natural sounds.
Within the industrial world a sound designer would be responsible for sound creation of a wide range of applications, it could be the “ping” of an elevator door opening, an alarm sound or the confirmation sounds of a self service checkout.

Do you think that the role of sound design in public places like stations and airports might change post COVID-19?

I think there is a possibility for this to happen. My thoughts are that some tasks that are normally performed by humans could become much more automated. Any type of task such as an airport check-in may become fully automated. Whilst using the systems the user would encounter a wide range of sounds such as human voices in the form of instructions, confirmation sounds when tasks are completed, error sounds when processes are incorrect and many more. There are so many changes happening across the world at the moment I think it’s quite difficult to predict how this will change.

What sort of sound design mistakes do you see in public places, like public address systems or door opening chimes, for example?

From my perspective the worst and most common would be the use of public address systems. I’m sure everyone can relate to being on the underground and the PA is so badly distorted you have no idea what it is saying. It’s certainly what I notice the most and always falls into three categories, too loud, too quiet or distorted/generally poor quality and inaudible.

From a sound design point of view, what makes a good audible alarm?

We have learned experiences of what alarms sound like (car alarm, fire alarm, alarm clock), so we need to work within an existing framework for it to be understood as an alarm.
There are two main aspects to this, pitch and amplitude. Firstly pitch or tone. An alarm needs to alert someone enough to trigger a specific response, such as complete a task. Generally speaking we all know what a fire alarm sounds like, if we changed the sound of the alarm to a calming harp sound we can all agree that it wouldn’t have the same effect. The same can be said for amplitude. It must be loud enough to be heard and prompt a response, but not so loud as to completely disorientate the subject. A balance of these is required.

How could people achieve this? (make a good audible alarm)

I think there are two aspects to consider, one is the actual alarm sound production, the other is the delivery system. The first steps must be to do the correct research on human response to frequency and amplitude, once you have this work alongside a sound designer to ensure that the source material is of a high quality. The second aspect is quite self explanatory. The delivery system (PA, Speaker etc) must be able to reproduce the alarm in a sufficient quality to be effective.

Why do you think that people ignore sounds or audible alarms sometimes? (I see that happen a lot)

We have already spoken about poor delivery systems so I won’t speak about that again but it’s certainly an aspect. An alarm that is not “alarming enough”, so I would class that as poor design. I think repetition would play a part in people ignoring alarms. If an alarm was sounding many times I think it would become ignored. Finally I think that if the alarm is not deemed important or dangerous enough by the listener, it would also lead to being ignored.

For anyone that has an audio aspect to their project, would you have one piece of advice for them?

It would be not to overlook the audio aspect of a project or leave it to the last minute. In my experience sound design is often left until the last moment. Audio can provide such an important element to any project and giving the professionals who work with it enough time to really produce great results should be important to any company.

Use of Augmented Reality (AR) to assist wayfinding in built environments, Part 3: AR technologies and associated user experiences

/0 Comments/in /by

Nigel Scard

Principal Human Factors Specialist

In our previous blog post we discussed the wider context of emerging digital technologies, and how these technologies will contribute to the development of ‘Zero UI’ based interaction. In this post we’ll look more closely at the role AR will play in this, some of the associated hardware technologies and the differences in the resultant user experience.

At present, AR is most commonly being introduced for both enterprise and consumer application based on using smart phone or tablet devices. Such devices, because of their widespread use and technology maturity are the obvious choice for hosting AR software and apps. Indeed, Liv Systems’ current internal project looking at the use of AR as a tool to aid navigation in built environments is based on a prototype app to be hosted on smart phones or tablets. However, whilst these devices may be able to host AR reasonably effectively, it is likely that the most powerful and effective AR experiences will be delivered by smart glasses-based systems.

AR using smart glasses is starting to be trialled and used by many enterprises for a variety of applications. These include collaborative working on designs, presented as virtual objects placed in the real world, and maintenance activities in which digital information is overlaid onto real world objects. For instance, the wiring within an electrical cabinet could be overlaid onto the users’ view of the cabinet with the cabinet still closed, or perhaps the wiring could have digital information overlaid onto it, providing digital labelling and explanatory information.

Perhaps the most widely used AR smart glasses for enterprise use is currently the Microsoft HoloLens which supports the Microsoft Mixed Reality platform.

The Microsoft HoloLens

 

Example use case for the HoloLens

Whilst smart glasses such as the HoloLens are fine for certain types of enterprise use and also potentially for home based use, in the long term for people to make use of smart glasses in their daily lives, we need to be moving more towards the type of smart glasses which are unobtrusive and look almost exactly like conventional glasses.

Google attempted to introduce smart glasses to the consumer market a few years ago, but they were not very successful due to privacy concerns associated with the built in camera and furthermore they were not a very elegant design, so again, regardless of privacy concerns they would have been unlikely to appeal to consumers outside of a very niche market. Over the last few years though there have been efforts to design smart glasses which could appeal to a wider market and in the last year a US and Canada based company ‘North’ has developed a viable consumer product in their ‘Focals’ smart glasses.

These smart glasses feature a laser projected AR display, a built-in mic and speaker and a device known as the ‘loop’ which is worn as a ring with a tiny joystick functioning as an input device. The app support for the glasses can display message and appointment notifications, calendar, directional and weather information, texts and be used to order Ubers. They are coupled with Alexa and texting is supported by voice-to-text.

 

‘Focals by North’

Interestingly AR based wayfinding assistance is one of the features already developed for the Focals smart glasses, providing further evidence that there is considerable interest in this as one of the most useful application areas for general consumers.

 

Directional information AR provided by ’Focals by North’

Smart glasses technology is still in its early days but as these technologies mature and provided their cost and aesthetic appeal proves acceptable to the public, it’s not unreasonable to expect the uptake of such devices could become widespread, providing the basis for the widespread use of AR based software and apps by the general public.

AR hosted on smart glasses will demonstrate a number of user experience benefits in comparison to AR provided by smart phones or tablets:

  • It will be hands free
  • It will provide a far greater field of view
  • It will be able to exploit stereoscopic vision and hence make use of depth of field
  • Digital information would be overlaid onto our view of the world in such a way that we would not need to look at a specific device to get the AR view we want; hence the AR information would be integrated more seamlessly with our visual sensory input.
  • It will be more immersive and hence offer a more compelling user experience
  • When combined with natural language-based user interaction, it will enable a user experience which is closer to the concept of ‘Zero UI’ .

Until such time as smart glasses are widely available, it is nevertheless worthwhile to explore the potential of AR using smart phone or tablet devices. This is because many people may continue to prefer such devices, smart phones still offer the potential to provide interesting and effective AR experiences and furthermore much of what we learn from developing AR for smart phones will be applicable to AR provided by smart glasses.

In our next blog post we’ll go into detail regarding our own research project on the use of AR for wayfinding in built environments, including exploring different approaches to providing wayfinding assistance using AR and the potential benefits to managers of built environment infrastructure which may be provided by the use of AR by their customers/users.

As always, please feel free to get in touch as we’re keen to discuss these topics with industry professionals and academics who are as excited about the potential of these technologies as we are.

Use of Augmented Reality (AR) to assist wayfinding in built environments, Part 2: AR in the wider context of emerging technologies and ‘Zero UI’

/0 Comments/in , /by

In our previous blog post we introduced some of the reasons Liv Systems are interested in exploring the use of AR to assist wayfinding in built environments and gave a brief overview of our own internal research project on this topic.

In this post we’ll discuss how AR aligns with some of the wider developments in emerging digital technologies and the types of user experience they enable.

We are currently living in exciting times in terms of the developments in a number of emerging digital technologies, in particular AR, VR and AI.

These technologies have had their ups and downs over the years and both VR and AI suffered from some false starts, possibly due to being over-hyped, especially in the case of AI, together with earlier limitations in computer processing power. Yet even during the ‘AI winter’ research on AI progressed and thanks to greatly improved computing power over the last 2 decades, together with the new approach to AI in the form of neural networks, AI is now having an enormous impact.  VR is also showing far more promise than before, thanks to the advances in computer graphics made possible by improvements in graphics processor units (GPUs) which have been largely driven by the computer gaming industry. AR has as yet to have an equally big impact, yet the technology driving it is mostly the same as that required for VR and AR is starting to be used for a number of enterprise applications as well as to a more limited extent in the consumer market (for instance with the huge popularity of ‘Pokémon Go’).

These technologies should be of great interest to human factors and UX (user experience) professionals, as they are set to fundamentally transform how we interact with the digital world and in the case of AR, even transform how we interact with the real world.

They will achieve this transformation through greatly increasing the bandwidth of the human-system interface. This will be accomplished by providing the basis for a move towards what has become known as ‘Zero UI’. A better phrase might be ‘invisible UI’ as the concept behind Zero UI is that the user interacts directly with digital information and only has minimal awareness of the graphical user interface (GUI) and physical devices required to achieve that interaction. The user interface therefore becomes in essence, invisible.

A well-known example of Zero UI from popular culture is ‘Jarvis’ from the film Iron Man. Jarvis is Tony Stark’s Virtual Personal Assistant (VPA) and AR based system.

Tony Stark with ‘Jarvis’ from the film ‘Iron Man’

Of course Jarvis packs a lot of cinematic wow factor and real world Zero UI is not likely to be as spectacular, however Jarvis does present a vision of how interaction with information systems can be provided in the future by interacting directly with digital information which is both presented as visually overlaid onto the real world view and through natural language based interaction with an AI driven virtual personal assistant.

Natural language UIs are already in widespread use, with the popularity of Alexa, Siri and Google Assistant, in addition to the increasing use of ‘Chatbots’.  These systems have been made possible thanks to a combination of greatly improved natural language processing (enabled by improvements in AI based pattern recognition) together with the neural network based AI which has enabled these systems to get better and better at responding appropriately to our verbal requests.

Whilst people may have at first felt a bit weird about talking to their computers or smart phones, the enormous popularity of stand-alone devices, such as the Amazon Echo and the Google Home demonstrates that many of us do like talking to our computers after all!

When natural language systems perform effectively, they can perform a task, such as playing a song we want to hear, or provide us with some transport related information such as a journey plan, based on a simple verbalised enquiry which may take just a few effortless seconds. Using a more traditional GUI based interaction, to achieve the same results may require multiple clicks, gestures and text inputs on a screen-based system, which is generally more time consuming and requires greater cognitive effort. In contrast merely talking to a computer is more intuitive, less effortful and requires virtually no computer literacy. Hence ironically the more advanced UIs become and the more we move towards achieving Zero UI systems, the more accessible such systems may become to the less tech savvy.

Another big advantage of Zero UI is it will allow us to escape from our screens whilst retaining the rapid access to the digital world we have become accustomed to. Optimists may hope this will enable us to engage more with the real world around us again. Such optimism could prove unfounded, but at least we may be less likely to carelessly step out in front of traffic or walk into lamp posts or each other! Also, it will give our eyes a much needed rest.

In our next blog post we’ll discuss what AR’s contributions are likely to be within this wider context of emerging technologies, the technology hardware used for AR and some of the possible application areas of AR.

We’d be delighted to hear from you if you’d like to discuss any of these topics with us, share ideas or find out more about our work.