Wrangling Twitter, Facebook and Blog updates

I've got a number of blogs (why?) and I wanted to hook them up such that new posts got fed automatically to my Twitter account and Facebook status. Here a few tools I've found useful to do this:

• The Twitter app in Facebook updates your Facebook status from Twitter.
• Even better, Selective Twitter Status, lets you feed your tweets selectively to your Facebook status by ending them with #FB (kudos to this post for the tip).
  
• Yahoo Pipes allows you to combine multiple RSS feeds into a single feed, which is usefule if you are using something like the next item.
• To feed a single RSS feed into Facebook, under your Profile page click on Settings, then select Blog/RSS and give it the RSS url. This is nice because it will posts things as Noteson your Wall instead of in your status, but it's limited to a single feed (hence Yahoo Pipes is useful to use with this).
  
• TwitterFeed can post any number of RSS feeds to Twitter. It even supports OAuth, meaning you don't have to share your twitter username and password (plus you authenticate to TwitterFeed with OpenId, which is cool).

So, what I actually do? The following:

1. I used Selective Twitter Status to update my Facebook status via selective feeds.
   
2. I used TwitterFeed to feed my blogs posts into Twitter. Since they don't have the "#fb" tage, Selective Twitter Status won't carry them over into Facebook.
3. I used Yahoo Pipes to combine by blogs RSS feeds and then the Facebook RSS function (under Profile/Settings) to display the blog postings on my Facebook Wall.
4. To update my Twitter status I the Ubiquity built in Twitter command in FireFox (nice post on using this).
5. I also have started using TweetDeck to monitor Twitter and may start using it to update Twitter.
6. I also use Bookmarklets for Sharing in Facebook and Noting in Google Reader for sharing random stuff I run into on the net. I'm finding the Sharing in Facebook to be better since the result looks nicer with a thumbnail and quoted text.

Installing TweetDeck on my Mac and "Error #2038"

After installing TweetDeck on my Ubuntu system I went to install it on my Mac (I love cross-platform apps), but the install kept failing with a "Error# 2038" error (see right).

So I had installed Air some time ago for something else (hmmm, can't remember what), so I thought perhaps it needed an update. So I wandered over to Adobe and installed a new version.

This represented an upgrade from "1.1.0.5790" to "1.5.1.8210" (see left).

And that did it - after that I went back to Tweetdeck and re-tried the install and it worked.

DISPLAY causes vi (vim) to hang...

Ran into a problem the other day that whenever I SSH'ed from hoe into a system at work and ran vi (OK, it's actually vim, but I'm old school and know it as "vi"), vi just hung. It took a good old ^C interrupt to get it to respond.

I ran "strace vi file" and saw it was hanging on the network.

Huh? It wasn't a network filesystem, so why is vi hanging on the network?

Well, on a hunch, I unset my DISPLAY environment variable:

% env | grep DISPLAY
DISPLAY=c-XXX.hsd1.il.comcast.net:0.0
% unsetenv DISPLAY

And then things worked fine.

So vi was trying to do some X11 thing and was getting blocked by my ISP or my NAT router or something between me and work. What the heck vi (vim) is trying to do X11 I don't know....

Creating a local virtual host for testing

I have a modest little website. It's nothing too fancy, but has some php and css that make it non-trivial, enough that I'm uncomfortable making changes to it live, so I wanted to set up a secondary test server to try things out on.

Sounds easy enough, but where? A virtual host on the same server as my website seems logical enough, but I don't own the box it runs on, I get the space thanks to some generous friends and I didn't want to bug them for another favor to set up a virtual host.

Thinking about it I realized I was already running Apache on my laptop, so in theory could set up a virtual host on it. I could use /etc/hosts to create a "test hostname" pointing at my localhost I could use in URLs. So that's what I went about doing.

First, I added an entry to /etc/hosts:

$grep test.vwelch.com /etc/hosts
127.0.0.1 test.vwelch.com

Then I added the virtual host entry for test.vwelch.com to my Apache configuration so that it would recognize URLs using it and serve content from a separate location that it normally would. This is on a Mac, so the details here might vary based on your OS, but if you search for "virtual host configuration" and the name of your OS I suspect you'll find help.

$ cat /etc/apache2/other/vwelch.conf
NameVirtualHost test.vwelch.com


ServerName test.vwelch.com
DocumentRoot /usr/local/www/



Options FollowSymLinks ExecCGI
AllowOverride All
Order allow,deny
Allow from all

Then I checked out a copy of my website from my git repository into /usr/local/www, restarted Apache ("apachectl restart") and pointed my web browser at "http://test.vwelch.com" and I had a complete copy of my website in a local playground that I could develop on.

Creating a OWASP WebGoat Ubuntu-based VM

I wanted to play with OWASP's WebGoat. So I created a Ubuntu Desktop VM, and installed Java and WebGoat. You also need a web proxy to do the lessons, so I installed WebScarab along with the FoxyProxy extension for FireFox to make using it easy. This post details all the steps I went through.

I did find these directions for doing this similar. They are outdated, but kudos to them for some basic concepts I followed.

[Update 11/30/2011: And there is now apparently a new version of my directions.]

I started with a fresh copy of a Ubuntu 8.0.4 Desktop image (I made my own, but you should be able to use an existing image). Fire up the image, login in, fire up a terminal (Applications, Accessories, Terminal) and become root ("sudo -i").

The first step was to install java:

# apt-get-install sun-java6-bin sun-java6-jdk

Note that tomcat is now included with WebGoat, so there is no need to install it separately. The next step is to download WebGoat:

# wget http://webgoat.googlecode.com/files/WebGoat-OWASP_Standard-5.2.zip
# unzip WebGoat-OWASP_Standard-5.2.zip
# mv WebGoat-5.2/ /usr/local
# cd /usr/local/WebGoat-5.2/

You will need to edit the webgoat.sh file to set JAVA_HOME and change the java version from 1.5 to 1.6. I use 'vi', you can use your favorite editor.

# chmod 755 webgoat.sh
# cp webgoat.sh webgoat.sh.orig
# vi webgoat.sh

Add "export JAVA_HOME=/usr/lib/jvm/java-6-sun" at the start of file, after the line with CATALINA_HOME. And then change the "1.5" on lines 17, 19, and 23 to "1.6".

Next step was to install WebScarab, which is a proxy you'll need for WebGoat lessons. Download and install as follows:

# cd /tmp
# wget http://dawes.za.net/rogan/webscarab/webscarab-current.zip
# unzip webscarab-current.zip
# mv webscarab-* /usr/local/webscarab

Now, to make it easy to use WebScarab from FireFox, I installed FoxyProxy into FireFox. Visit the FoxyProxy web page, install it and restart FireFox.

When FireFox starts back up, configure FireProxy as follows:

1. Click on FoxyProxy icon in lower right and select Add New Proxy. The FireProxy window should appear.
2. On the General tab, under Proxy Name enter "WebScarab".
3. On the Proxy Details tab, select Manual Proxy Configuration.
4. For Host or IP Address enter "127.0.0.1"
5. For Port enter "8008"
6. On the URL Patterns tab select Add New Pattern
7. For Pattern Name enter "WebGoat"
8. For Pattern URL enter "http://*127.0.0.1/WebGoat/*" (The first "*" here matches the "guest:guest" string for authentication.)
   
9. Click OK
10. Close Proxy Settings
11. Under Mode select Use proxies based on their pre-defined patterns and priorites.
12. Close FoxyProxy Options
    

Now you probably want to open a couple of terminal windows (or tabs) one for WebScarab and one for WebGoat. In the first window fire up WebScarab:

# java -jar /usr/local/webscarab/webscarab.jar

You should see a WebScarab window open after a moment.

Then, in the other window, fire up WebGoat:

# cd /usr/local/WebGoat-5.2
# ./webgoat.sh start80

WebGoat will churn for a little bit, but then you should eventually see a message like "INFO: Server startup in 3546 ms". At this point you are ready to go by pointing your browser at http://guest:guest@127.0.0.1/WebGoat/attack

At this point you should see not only WebGoat appear in your browser, but WebScarab should start displaying bunch of requests. Click on the image at the top of this post for what things should look like.

That's as far as I've gotten. Next step will be to dive into WebGoat.