EDIT: This is now way easier to do from within TouchDesigner using the Threaded Web Comp here: http://derivative.ca/Forum/viewtopic.php?f=22&t=9164 . Tutorial to come soon.
OH BOY! This one is going to be exciting!
Somewhere in my meanderings of Touchdesigner, I decided it was a good idea to try this. Assuming my mantra of “How hard can it be?”, I set out to give it a shot.
I recently posted this project that uses the method I am going to show you.
http://www.design.ianshelanskey.com/uncategorized/touchdesigner-twitter-live-feed-asu-emerge-hud/
Once the information is in Touchdesigner you can use it for just about anything, it’s getting it there that becomes a headache. That’s the part of the problem I will focus on in this tutorial.
First things first: head over to GitHub and download Python Twitter Tools(https://github.com/sixohsix/twitter). This is a python wrapper for Twitter’s Application Programming Interface(API), which handles how software interacts with Twitter and can do stuff like post tweets, search for hashtags, and pull tweets from other users timelines.
Extract the files from the ZIP to your working folder. Mine is on my desktop called twitterTest.
I like to work in commandline, but most of this stuff should be able to be done in GUI.
In CMD navigate to the project folder and find a file called setup.py:
chdir Desktop/twitterTest chdir twitter-master
Once you have found it, install it using this command:
python setup.py install
You should see a bunch of text scrolling by saying “Installing….”, once this has completed test to make sure the module has installed by typing into CMD:
python import twitter
If you don’t get an error, you’ve installed the python Twitter API! Now the real fun begins.
I’m sorry to say, but you will need a Twitter account in order to use this. So start there. Once you have made a new account you need to become a Twitter Developer. This is super easy. Head over this link https://apps.twitter.com/, sign in, and click create app. You should be met by this screen:
Fill out the form and agree to the terms.
The next page will have information we need in order to access twitter. Keep it open.
Open up you favorite text editor and lets make some Twitter magic happen.
In the Text editor paste the following:
from twitter import * import os
MY_TWITTER_CREDS = os.path.expanduser('~/.my_app_credentials') if not os.path.exists(MY_TWITTER_CREDS):
oauth_dance("$$NAME_OF_APP$$", '>$$OAUTH_KEY$$', '$$OAUTH_SECRET$$', MY_TWITTER_CREDS)
oauth_token, oauth_secret = read_token_file(MY_TWITTER_CREDS)
twitter = Twitter(auth=OAuth(oauth_token, oauth_secret, '$$OAUTH_KEY$$', '$$OAUTH_SECRET$$'))
t = twitter.statuses.home_timeline() log = open('tweets.txt','w')
log.write(str(t[0]))
log.close()
Replace the bold text with the information on the Twitter Developer page and we should be all set to be authenticated by Twitter.
Save the programs as ‘.py’ and jump back into commandline, find it and run:
python twitterTest.py
A small script will run, and then a window will pop up asking you to sign into Twitter again. Do so and then copy the pin they give you into the command line, where it is asking for the pin. Hit enter and you should be all authenticated. If you look in your project folder, there will be a a new .txt file called ‘tweets’. Open it and you will see the mess you’ve gotten yourself into.
This is one Tweet. It’s one big dictionary that holds data and several smaller dictionaries inside it. I went ahead and parsed through it making it a bit easier to read:
Parsed Tweet
Here you can see the structure behind what is happening. Each piece of data has a key associated with it. In order to find the text of the tweet for instance, you need to change code in the program to this:
from twitter import * import os
MY_TWITTER_CREDS = os.path.expanduser('~/.my_app_credentials') if not os.path.exists(MY_TWITTER_CREDS):
oauth_dance("$$NAME_OF_APP$$", '$$OAUTH_KEY$$', '$$OAUTH_SECRET$$', MY_TWITTER_CREDS)
oauth_token, oauth_secret = read_token_file(MY_TWITTER_CREDS)
twitter = Twitter(auth=OAuth(oauth_token, oauth_secret, '$$OAUTH_KEY$$', '$$OAUTH_SECRET$$'))
t = twitter.statuses.home_timeline() log = open('tweets.txt','w')
log.write(str(t[0]['text']))
log.close()
This will create a dictionary out of the first tweet (index 0) in your timeline, then look for the data associated with the key ‘text’. If you run this program you should get just the text from the last tweet on your timeline. You can run a for loop that iterates through the last 100 tweets by changing the program to this:
from twitter import * import os
MY_TWITTER_CREDS = os.path.expanduser(‘~/.my_app_credentials’) if not os.path.exists(MY_TWITTER_CREDS):
oauth_dance(“$$NAME_OF_APP$$“, ‘$$OAUTH_KEY$$‘, ‘$$OAUTH_SECRET$$‘, MY_TWITTER_CREDS)
oauth_token, oauth_secret = read_token_file(MY_TWITTER_CREDS)
twitter = Twitter(auth=OAuth(oauth_token, oauth_secret, ‘$$OAUTH_KEY$$‘, ‘$$OAUTH_SECRET$$‘))
t = twitter.statuses.home_timeline() log = open(‘tweets.txt’,’w’)
for i in range(len(t)):
log.write(str(t[i][‘text’]))
log.close()
There is some information in the tweet is stored as another dictionary within the larger dictionary. Hashtags work that way.The hashtag dictionary lives in another dictionary called ‘entities’ which also holds the keys ‘user_mentions’ and ‘urls’. If a tweet has multiple hashtags and you want the text from all of them you must write a ‘for’ loop that iterates through all of them.
hashtags = t[0][‘entities’][‘hashtags’]
for i in range(len(hashtags)): print(hashtags[i][‘text’])
Run the script, and you should see all of the hashtags print out in console.
This is great, but how do I make cool visuals out of them in Touchdesigner?!
As of the date of this post, I have had issues with how Touchdesigner communicates with the network socket. It seems to glitch and drop frames for a brief moment while it cooks, then loads the stuff you wanted. To get around this, I ran a python program in command-line to write the information I needed into a .TXT file then imported it into Touch. Here is the documentation for how to do that: https://docs.python.org/2/tutorial/inputoutput.html
In the parameters of that text DAT, turn on refresh, and now whenever that file saves Touch will refresh it.
Awesome, now you have a separate program that is pulling new data from the internet for you while Touch remains chugging along at its normal pace. Now all that you need to do is structure the data in such a way that its easily manageable in touch.
If your data is a 1d array, or 1 piece of data per index, the best way to do this is the just separate each line you write in your .TXT file with ‘\n’. ‘\n’ is an End of Line character(EoL) commonly use in text files and data structures. If you add it to the end of your ‘.write()’ statement, it will create a newline that you can read in Touch.
log.write(str(t[0][‘text’])+’\n’)
The file loads up to look something like this.
From here you can convert to a table and replicate, clone, and texture instance to your hearts content. 
But suppose you had more than a 1d array. Like maybe you want a username, the date, and the tweet per index. In order to do this we are going to take advantage of a parameter in the convert DAT: “split cells at”. This parameter allows you to look for a string in a text DAT convert the text DAT into a table where each cell is split at the string. Generally when I use this technique, I use ‘|’ because you don’t normally see that character in data. So in the python twitter program, your ‘.write()’ statement looks like this:
log.write(str(t[i][‘created_at’])+’|’+str(t[i][‘favorite_count’])+|+str(t[i][‘text’])+’n’)
(Protip: use the same idea with the Substitute DAT in Touch to have scripts adapt to new information without the hassle of figuring out where the quotes go, or converting data types. I use this technique in my GOTO Cue example here: http://www.design.ianshelanskey.com/uncategorized/building-tools-to-make-programming-easier-goto-cue )
After passing the ‘filein’ to a convert with the ‘split at’ parameter set to ‘|’ we get this:
Now each column is a different piece of data. This makes it super easy to have instances and replicated objects have different parameters.
From here, the world is your oyster. You can use this data in tons of different ways. Here I made a data visualization of NASA’s last 100 tweets and ordered them by date. The size of the circle refers to the number of Favorites the post got and the color refers to the time it was posted.
I used this exact same method of retrieving information that Touch would stall over for this Voice Recognition project:
http://www.design.ianshelanskey.com/technology/topology-of-words-fall-14-ame-530/
English