Boffins laugh in the face of Twitter's API limits. Now they can slurp info to their hearts' content
While politicians and the public demand Facebook dam its indiscriminate dispensation of data, academics want to open the social network info-spigot wider still.
In a paper popped onto ArXiv this week, boffins from the Instituto Politécnico Nacional's ESIME Culhuacan in Mexico, and the University of Warwick in the UK, describe a technique for getting around Twitter's API rate-of-access limitations to harvest data from the social network more efficiently.
The paper doesn't mince words about its intent to flout Twitter's rules for the sake of science. It's titled "a web scraping methodology for bypassing Twitter API restrictions."
To test and train of data science algorithms, eggheads must have something to work with, the researchers – A. Hernandez-Suarez, G. Sanchez-Perez, K. Toscano-Medina, V. Martinez-Hernandez, V. Sanchez and H. Perez-Meana – declare.
"Gathering information from Online Social Networks is a primordial step in many data science fields allowing researchers to work with different and more detailed datasets," they said. "Although an important proportion of the scientific community uses the Twitter streaming API for collecting data, a limitation occurs when queries exceed rating intervals and time ranges."
Twitter, they claim, has become the preferred social network for data collection, because of its usability, reach, and varied types of data. Its real-time and historical data have proven useful for research on rumor propagation, tracking people geographically, spam and botnet detection, and disaster response, they stated.
US presidential historians and prosecutors no doubt can find noteworthy tweets, too.
http://www.theregister.co.uk/2018/03/29/twitter_api_bypass/
= new reply since forum marked as read
