Experience in using MTurk for Network Measurement

Author
Huz, Gokay
Bauer, Steven
Beverly, Robert
claffy, c.k.
Date
2015Metadata
Show full item recordAbstract
Conducting sound measurement studies of the global Internet is inherently difficult. The collected data significantly depends on vantage point(s), sampling strategies, security policies, or measurement populations -
and conclusions drawn from the data can be sensitive to these biases. Crowdsourcing is a promising approach to address these challenges, although the epistemological implications have not yet received substantial attention
by the research community. We share our findings from leveraging Amazon's Mechanical Turk (MTurk) system
for three distinct network measurement tasks. We describe our failure to outsource to MTurk an execution
of a security measurement tool, our subsequent successful integration of a simple yet meaningful measurement
within a HIT, and finally the successful use of MTurk to quickly provide focused small sample sets that could not be obtained easily via alternate means. Finally, we discuss the implications of our experiences for other
crowdsourced measurement research.
Description
The article of record as published may be located at http://dx.doi.org/
10.1145/2787394.2787399
Includes article and presentation.
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.Collections
Related items
Showing items related by title, author, creator and subject.
-
An automated Ada physical source line counter
Walsh, Kevin J. (Monterey, California. Naval Postgraduate School, 1993-09);Tools to count lines of code have not been standardized or automated in a flexible fashion. This lack of flexibility can lead to ambiguous interpretations of the size of software modules, especially when the person performing ... -
Principles for measurability in protocol design
Allman, Mark; Beverly, Robert; Trammell, Brian (ACM, 2017-04);Measurement has become fundamental to the operation of networks and at-scale services - whether for management, security, diagnostics, optimization, or simply enhancing our collective understanding of the Internet as a ... -
Nonlinear estimation with sparse temporal measurements
Frontera, Paul J. (Monterey, California: Naval Postgraduate School, 2016-09);Nonlinear estimators based on the Kalman filter, the extended Kalman filter (EKF) and unscented Kalman filter (UKF) are commonly used in practical application. The Kalman filter is an optimal estimator for linear systems; ...