- Streaming APIs
- Signing Up for Twitter
- Using Twitter's API with Node.js
- Extracting Meaning from the Data
- Pushing Data to the Browser
- Creating a Real-Time Lovehateometer
- Summary
- Q&A
- Workshop
- Exercises
This chapter is from the book
This chapter is from the book
This chapter is from the book
Extracting Meaning from the Data
So far, you created a way to retrieve data in real-time from Twitter, and you saw a terminal window move very fast with a lot of data. This is good, but in terms of being able to understand the data, you are not able to answer the question set. To work toward this, you need to be able to parse the tweets received and extract information. Twitter provides data in JSON, a subset of JavaScript, and this is great news for using it with Node.js. For each response, you can simply use dot notation to retrieve the data that you are interested in. So, if you wanted to view the screen name of the user along with the tweet, this can be easily achieved:
twit.stream('statuses/filter', { track: ['love', 'hate'] }, function(stream) {
stream.on('data', function (data) {
console.log(data.user.screen_name + ': ' + data.text);
});
});
Full documentation on the structure of the data received from Twitter is available on the documentation for the status element. This can be viewed online at https://dev.twitter.com/docs/api/1/get/statuses/show/%3Aid. Under the section “Example Request,” you can see the data structure for a status response. Using dot notation on the data object returned from Twitter, you are able to access any of these data points. For example, if you want the URL for the user, you can use data.user.url. Here is the full data available for the user who posted the tweet:
"user": {
"profile_sidebar_border_color": "eeeeee",
"profile_background_tile": true,
"profile_sidebar_fill_color": "efefef",
"name": "Eoin McMillan ",
"profile_image_url": "http://a1.twimg.com/profile_images/1380912173/
Screen_shot_2011-06-03_at_7.35.36_PM_normal.png",
"created_at": "Mon May 16 20:07:59 +0000 2011",
"location": "Twitter",
"profile_link_color": "009999",
"follow_request_sent": null,
"is_translator": false,
"id_str": "299862462",
"favourites_count": 0,
"default_profile": false,
"url": "http://www.eoin.me",
"contributors_enabled": false,
"id": 299862462,
"utc_offset": null,
"profile_image_url_https": "https://si0.twimg.com/profile_images/1380912173/
Screen_shot_2011-06-03_at_7.35.36_PM_normal.png",
"profile_use_background_image": true,
"listed_count": 0,
"followers_count": 9,
"lang": "en",
"profile_text_color": "333333",
"protected": false,
"profile_background_image_url_https": "https://si0.twimg.com/images/themes/theme14/bg.gif",
"description": "Eoin's photography account. See @mceoin for tweets.",
"geo_enabled": false,
"verified": false,
"profile_background_color": "131516",
"time_zone": null,
"notifications": null,
"statuses_count": 255,
"friends_count": 0,
"default_profile_image": false,
"profile_background_image_url": "http://a1.twimg.com/images/themes/theme14/bg.gif",
"screen_name": "imeoin",
"following": null,
"show_all_inline_media": false
}
There is much more information available with each response including geographic coordinates, whether the tweet was retweeted, and more.
Try It Yourself
If you have downloaded the code examples for this book, this code is hour14/example02.
To parse data from Twitter, follow these steps:
- Create a new folder called parsing_twitter_data.
Within the parsing_twitter_data folder, create a new file called package.json and add the following content to declare ntwitter and Express as dependencies:
{ "name":"socket.io-twitter-example", "version":"0.0.1", "private":true, "dependencies":{ "express":"2.5.4", "ntwitter":"0.2.10" } }Within the express_twitter folder, create a new file called app.js with the following content. Remember to replace the keys and secrets with your own:
var app = require('express').createServer(), twitter = require('ntwitter'); app.listen(3000); var twit = new twitter({ consumer_key: 'YOUR_CONSUMER_KEY', consumer_secret: 'YOUR_CONSUMER_SECRET', access_token_key: 'YOUR_ACCESS_TOKEN_KEY', access_token_secret: 'YOUR_ACCESS_TOKEN_KEY' }); twit.stream('statuses/filter', { track: ['love', 'hate'] }, function(stream) { stream.on('data', function (data) { console.log(data.user.screen_name + ': ' + data.text); }); });Install the dependencies by running the following from a terminal:
npm install
Start the server by running the following from a terminal:
node app.js
Watch the terminal; you should see that now only the screen name of the user and the tweet are displayed (see Figure 14.5).
Figure 14.5. Parsing data received from Twitter
- Kill the server by pressing Ctrl+C in the terminal.
)