Guidelines for the Accurate Design of Empirical Studies in Wireless Networks

Traditionally, wireless protocol proposals have been often tested and validated using only analytical and simulation models. However, as the wireless environment is very complex to model accurately, and since the cost of wireless cards has decreased in an exponential way, today more and more research papers include evaluation of new proposals using experimentation on real devices. Indeed, experimentation is a mandatory step before possible deployment of new network protocols with real users. However, wireless experimentation is much more complex to set up and run than simulation, and it is important to avoid many pitfalls that can occur during experimentation. The objectives of this paper are twofold. First, we describe typical problems currently encountered in wireless-based experimentation, and we present simple guidelines to avoid them. Second, we propose an experimental methodology where the detection of anomalies, calibration of the measurement setup, and clear definition of the scenario (among others) make easier the repeatability of results. Finally, we showcase an implementation of the proposed methodology with an experimentation scenario whose objective is to analyze the stability of the wireless channel.