Shopping enhanced by augmented reality
We teamed up with Whole Foods to create an experience for SXSW 2018 to demonstrate how augmented reality can enhance and compliment the grocery shopping experience. Our vision was to surface connections between products and lifestyles to reveal serendipities. The experience connects people to products they love based on what it knows about their shopping and dietary preferences. It strikes a balance between convenience and discovery by leveraging smart data to surface the right products at the right time and even when to reach outside of the stores offerings to find what users are looking for.
Working closely with Whole Foods 365 brand we set out to utilize AR and specifically iOS ARkit to demonstrate that AR can be much more than a gimmick and can be utilized to enhance the experience of shopping. By looking at interactions at various distances, 25ft, 10ft, and 1ft were able to orient the user to the space and serve personalized content based on their profile and shopping list, and their proximity to products in the store.
To a lot of the team members on both the design team and client side, augmented reality was not a clearly understood concept. From a technology and practical use standpoint we needed to quickly familiarize ourselves with what it is, how it's used, and more importantly how it's implemented. One of the first things we did was conduct a competitive analysis. While there were no direct competitors in the space, several products were using AR in various ways, some better than others.
Looking at a dozen augmented reality mobile applications we were able to play with the technology personally and communicate the practical uses to the greater team. Amazon uses it to place objects in your physical space to see what it will look like, Gap lets you try on clothes in a virtual dressing room and Pokemon Go lets you capture virtual Pokemon in the environment around you. This was extremely helpful to get an idea of the ways we could use the technology and how to design for it. This sparked ideas for concepts like wayfinding, displaying product information and discovering new products that relate to your personal tastes.
After conducting several "shop-alongs" with various types of shoppers and a survey of a larger sampling size we determined the content that is important for the different mindsets at key decision points during the shopping experience. From entering the store, to shopping for individual items to the items themselves. A core feature of the experience was which food items match to the customers dietary preferences. From a distance they can quickly identify which items on the shelf meets their requirements. When up close they can then specific nutrients highlighted that they identify as important.
We also interviewed employees to see if there were more ways to leverage the technology than just assisting the customers while they shop. We wanted to empower the employees with technology that places the store inventory within reach no matter where they were in the store.
Whole Foods 365 has an established brand that integrates into the greater Whole Foods brand but has its own qualities that communicate the proposition of their unique value. One would think you can just take an established identity and apply that to new designs but the augmented reality experience requires specific styling to be visible in the physical space and to increase accessibility to those with other needs.
A critical step in the design process after the discovery and definition of the project is the visual design. In order to align on the visual design and brand extension several moodboards and concepts were presented to stakeholders. Deciding on a visual language helped swiftly move from wireframes to high fidelity visual designs in order to for 3d modelers and animators to start on 3d assets for the augmented environment.
From the greatest distance the user gets a type of birds eye view of the store. This shows the location of the store departments and how many items from their list are within each. This is also an opportunity to show the locations of what they call "friends" or other vendors they team up with within their stores. The design was kept very much on brand using their fun font and bright color palette. Moments of interaction utlilizes 3d iconography in the AR space to bring the icons to life and delight with subtle animations.
At a reduced distance of less than 25ft the user is served content that is more assisting, helping them locate items placement within aisles and on shelves and to push them personalized promotions based on their list. In this example they are shopping for wine and a path appears on the ground. Following this path they are lead to a wine display that has several specials that match their list item. They can also be nudged toward other products that compliment each other, like a pasta or some cheese. Its these sort of moments that bring fun and discovery to something most grocery shoppers don't entirely enjoy.
In order to reinforce the Whole Foods mission and brand promise we wanted to utilize the closest proximity or "product proximity" to help users make smart choices inline with their goals and buying habits. At the product level we used image recognition to scan product barcodes and reveal personalized information in the AR space based on their profile. We quickly show how the product meets the users dietary preferences letting them know if something is inilne with their dietary preferences, or if they might want to avoid it. We also rendered 3d models of the product to reinforce the scanned product and used animation to bring it to life. They can then tap on a category icon and learn more about the products nutritional info or find related products and recipes.