Tap to Interact: Towards Dynamically Remixing the Internet of Things

The number of networked Smart Devices available in everyday environments is rapidly increasing; however, many current devices adopt mutually incompatible networks, protocols, and application programming interfaces. As such, creating mobile applications that dynamically discover and integrate ambient functionality across multiple vertical markets remains challenging. In this paper, we introduce a novel integration technique that enables commodity mobile devices (e.g., mobile phones) to mediate control messaging between incompatible Smart Devices situated in the user’s environment. The approach enables a variety of control capabilities and protocol translation services to be dynamically installed into a user’s mobile device on-demand using plug-ins. The approach features an intuitive “Tap to Interact” workflow that allows a user to tap nearby Smart Devices with a smartphone to install required interaction plug-ins and automatically “wire” them together in interesting and potentially unforeseen ways. In our demonstration, we show how this approach enables a Sphero Robotic Ball to be utilized as a physical interface for controlling media playback on an Apple TV, interacting with networked-enabled lighting equipment, and flying a Parrot AR Drone helicopter – by leveraging a commodity smartphone as a plug-and-play Smart Gateway between mutually incompatible devices.