Shabana's guest post on Disability.gov

My sister (Shabana Wollin) was invited as a guest blogger at Disability.gov.  She wrote a post on negotiating in a tough job market.  I can tell you from personal experience, Shabana has mastered the art of understanding a client's technical problem space, figuring out if she's the right solution fit, and then agressively negotiating an arrangement that benefits both parties.  Check out the post and comment!

A quick note about Hu Moments (re: DARPA Shredder Challenge)

Kat Scott from the SimpleCV/Ingenuitas team was gracious enough to share some feedback with me on my use of the SimpleCV package and approach to the Shredder Challenge.  Not surprisingly, she and her colleagues have had their own thoughts on how to tackle the puzzles, and I for one am eagerly looking forward to their attempts.

Kat (very diplomatically!) confirmed my comment that shape matching would not work, and then very kindly shared some specific technical information on why.   

"Your approach is a good first start, but unfortunately the Hu moment is not going to work in this situation. The Hu moment is a measure of the image moment, which is kinda like a moment in physics. Basically a moment describes how much work you have to do to get something to spin. Round things have less of a moment and spin easily, long things are a lot more difficult to spin. Moments are actually a good way to match things that are similar shapes, but not to align edges that are a like puzzel pieces."

Kat also forwarded a video of a simplified visual explanation of the math behind the Hu moments that illustrates how robust the calculations are in identifying a particular image, regardless of transformations applied to it:  http://www.youtube.com/watch?v=O-hCEXi3ymU 

Personally, I found that while the video definitely drove the point home, Kat's simple, straightforward explanation was key for me in understanding the nature of this feature.

I hope this little bit sets off a light bulb or two for others, as it did for me.  At the moment, my thinking on the first puzzle has turned to collision detection; the type you might be familiar with if you've played any old-school platform scroller arcade game.  These games determined when two polygons intersected each other, and I think the principle should apply to determine whether two puzzle pieces in the shredder challenge are a good fit.

My latest homework is to understand the Separating Axis Theorem and see if I can develop an approach based on this concept.

DARPA Shredder Challenge - Part Deux

As promised, here is a bit of code that follows my preliminary line of thinking on Puzzle1: 

#!/usr/bin/python
from SimpleCV import *
from optparse import OptionParser
import random

cmdOptParser = OptionParser()
cmdOptParser.add_option("-i","--input",dest="inputFilename",help="Filename of the image to operate on")
cmdOptParser.add_option("-o","--output",dest="outputFilename",help="Filename of the image to save on")

(options,args) = cmdOptParser.parse_args()
puzzleImg = Image(options.inputFilename)
#  The source images for the challenges are flippin huge. So let's up the recursion limit to support the search for many blobs
sys.setrecursionlimit(10000)
#  Kick things off by creating a binarized() version of the source image
binarizedSrc = puzzleImg.binarize()
#  Now lets find the blobs. Lets set some mins to make sure we don't worry about the itty bitty scraps
binBlobs = binarizedSrc.findBlobs(-1,2000)
#  We have a set of identified blobs, so we can do some basic analysis. Let's just stupidly try to match up blobs. This won't really work for us because we are not trying to find duplicates etc, we need matching tiles. But anyway.
blobBestMatches = {}
for i,blob in enumerate(binBlobs):
  blobMatchSet = {}
  for c,blobCmp in enumerate(binBlobs,1):
    blobMatch = blob.match(blobCmp)
    if (blobMatch > 0):          #blobMatch at 0 means it is itself
      blobMatchSet[c] = blobMatch
  blobBestMatches[i] = min(blobMatchSet,key=blobMatchSet.get)    #this clever little bit identifies the index of the dict that contains the lowest possible match value.
for k,v in blobBestMatches.iteritems():
  randomColor = (random.randint(0,255),random.randint(0,255),random.randint(0,255))
  print randomColor
  binBlobs[k].draw(randomColor,-1,-1,binarizedSrc.dl())
  binBlobs[v].draw(randomColor,-1,-1,binarizedSrc.dl())
  binarizedSrc.save(options.outputFilename)

Forgive me if the code formatting creates issues for you.  The script is dirt simple, and should be easy to follow.  It is only a proof of concept that (usefully) gets as far as identifying the distinct blobs.  From there, I've asked it to perform a simple match and identify the pairs by coloring each of the matches the same.  Running the script produces something similar to this:

Doesn't that look a lot easier now?

This last bit is not at all useful in solving the puzzle because we are not attempting to identify duplicates in the image.  Rather we need to match each shred to a "fit match".  I have not yet figured out the math to express the "fit match", but at least I'm a step closer in this marathon.

DARPA Shredder Challenge - Preliminary (quarter-baked) thoughts...

DARPA opened its Shredder Challenge late last week, and in typical DARPA style threw down a fantastic, seemingly impossible gauntlet:  extract meaningful information (intel) from a series of damaged and destroyed documents.

I am struck by two things about this challenge:

1. DARPA is asking "How would you...", and not "Can you...".  That is, DARPA believes it is wholly possible to complete the challenge, and it is only a matter of your ingenuity to find any particular path to a solution.  This is a striking position considering that the simplest of the puzzles looks like this:  (reproduced unwisely without any permission from DARPA. )And it's worth 2 points!
2. Anyone can win this challenge.  I mean that it occurs to me that there are not (likely) a cadre of paper shredding experts with degrees in advanced document reconstruction that are shoe-ins for this contest.  Anyone with a methodical mind should be able to design a solution approach, test it and stand a chance to win the purse.

I have often admired DARPA's innovative challenges (like their Network Challenge -or "Red Balloon"- problem), and I decided that I would enjoy at least participating in this one.  I am thoroughly confident that I won't score even a point, but I hope to share some ideas that may prove helpful to others.

Some thoughts on tools...

I am interested in machine vision and image processing, so I decided to take this opportunity to learn more about OpenCV and apply its capabilities to Puzzle 1.   OpenCV is a very popular image-processing library with several interfaces including Python and Processing.  I develop primarily on a (aging) Mac and I found getting started with OpenCV on OSX something of a Sisyphean ordeal.  Here are some key notes I learned through a couple of discouraging hours of headaches:

• Use homebrew to install OpenCV, rather than Cmake and compiling the library from source.  Surely more awesome ninjas can solve the myriad dependency and python version compatibility issues that crop up in compiling the OpenCV libraries from source, but for the restofus: 
  • brew install git 
  • brew update 
  • brew install opencv
• You'll need to update your environment's path variables to allow python to locate the appropriate bindings.  This site does an excellent job walking you through the necessary shell magic.
• Forget OpenCV entirely and use SimpleCV.  I found that in the interest of getting productive quickly (especially on a mac), SimpleCV has nailed down the process of setting up your environment easily and correctly.

SimpleCV:  Computer Vision for the Restofus.

I will be using SimpleCV for the bulk of my exploration of the DARPA challenge, and I would encourage anyone interested in trying their hand at machine vision to do the same.  In addition to providing a simple, one-and-done installation package for most environments, SimpleCV also simplifies the native OpenCV python interface.  You can access Camera streams and Kinect devices, implement common image processing algos etc. in fewer, more readable lines of code.

I love the snip on their homepage that captures a frame from your webcam in 3 lines of code.

A few notes I learned that may help you:

• Use the appropriate single "super" package installer to get yourself the required libraries and the SimpleCV stable code base
• Then download the latest Git commit and re-install SimpleCV.  The latest Git revision has Blob detection algorithms baked into the SimpleCV interface, and allows you to by-pass dealing with the installation of the cvblob-python project.
• Give the SimpleCV Interactive Shell a try.  It might get in the way of more experienced developers, but for newbies its a great way to quickly experiment with key features.
• Browse the source.  The documentation has gaps, and I found myself understanding how to operate the image processing capabilities much more clearly as I started looking at the classes directly.
• Read the OpenCV docs.  Remember that SimpleCV is a wrapper to OpenCV (and some other libs as well).  So it will help you to understand the native libraries, while taking advantage of the scaffolding that SimpleCV can provide.

It took me mere minutes to get my SimpleCV environment setup to a point that I could quickly and intuitively load in an image and begin to run some basic operations.  

The following is my line of thinking on the first puzzle.

Puzzle 1:  "What is the appropriate title being referenced?":

The challenge image looks basically like a jigsaw puzzle (albeit one where the edges on the pieces are more subtle).  My main objective is to achieve just enough reconstruction to extract the intel.  No extra points for the completeness of the reconstruction, so I am not concerned about the tiny little scraps at the bottom right.

My first thought about this puzzle is to get a hold of my pieces.  That is, I'd like to be able to run match algorithms against a particular scrap, so I need to be able to identify, extract and manipulate each bit of the paper.  Enter Blob Detection.  The complicated mathematical description aside, I am interested in clumping pixels together that look like they are part of the same entity.  Given that the background is a nice distinctive Pepto pink, this ought to be a doable excerise.  And thanks to SimpleCV's findBlobs(), you get the following image pretty quickly:

 Detected blobs are highlighted here in white. Note that the source had the bits of scrap in yellow, and thie image shows that we can locate them all.

Neato.  Well, now that we have the blobs, what about running some matches.  I was nearly floored to learn that SimpleCV has a terrific implementation (or rather re-interpretation) of OpenCV's matchShapes().  I am struggling to understand the technical details, but from what I can gather, this method compares two contours and reports back a value that indicates the stregth of the match.  The method is based on comparing Hu Moments which express information about a contour and is not sensitive to rotation, scale or reflection!!!!  

And so that's my approach to Puzzle 1:  loop through each "puzzle shred" and compare it to every other shred in order to find its match.  Then perform the same loop with each pair and so on until we reduce a couple of hundred shreds into a handful.

Of course this is easier said than done, but at least I can break down the approach.

1. Extract the shreds as distinct, contiguous blobs
   • So, something you will notice if you simply run findBlobs() using SimpleCV is that it is fairly indiscriminate.  That is, while the image above looks like each shred is highlighted individually, in fact, each is fact composed of several "sub blobs".
   • We (I) first need to clean the image with a few passes of a "morphological operation".  Probably a morphClose() which should basically (hopefully) give us more robust blobs.
2. Adapt the blob match to our needs
   • In its current format, simply running the blob match between two contours would really only tell you whether (or rather to what degree) they are identical.  We are after something close, but not quite an exact match.  I'll need a more clever math wiz than I to adapt the algorithm to make that happen.
3. Profit!!!

So with that.  Back to more thinking.

Paramiko: easy, native ssh in Python...

I just discovered Paramiko (docs here)!  This module allows SSH interaction natively (meaning directly through code, and not through system calls), and supports a pretty comprehensive scope of operations over ssh.  The key items that caught my attention are:

• Intutive syntax and application
  • mysshclient = paramiko.SSHClient() mysshclient.connect('hostadd',username='user',password='pass')
  • Addressed the missing host key issue we are all familiar with directly on the shell
    mysshclient = paramiko.SSHClient() mysshclient.set_missing_host_key_policy(paramiko.AutoAddPolicy())
• Key based authentication
  • Terrific example here
  • mysshclient = paramiko.SSHClient() mysshclient.set_missing_host_key_policy(paramiko.AutoAddPolicy()) privateKeyPath = '/path/to/your/key' privateKey = paramiko.RSAKey.from_private_key_file(privateKeyPath) mysshclient.connect('hostaddr',username='user',pkey=privateKey)
• Send commands and interact with output on the remote machine
  • mysshclient = paramiko.SSHClient() mysshclient.set_missing_host_key_policy(paramiko.AutoAddPolicy()) mysshclient.connect('hostaddr',username='user',password='pass') stderr, stdout, stdin = myssh.exec_command('ls -latrh') stdout.readlines()

Brilliant.  Will be applying Paramiko to a teeny automation script that will start an Amazon EC2 instance, run a series of commands on the server (neat scraping and analysis stuff I hope to write more about), and then process some post shutdown ops.

Mint.com let me down today...

I have been a loyal Mint.com user for over 2 years, and today I experienced a level of dissatisfaction with the service that surprised me.

 A little background:  At around 6:45am CST today (10/13/2010), I received an email with no message body or subject line from "Mint.com <stage-mini@mint.com>".  The message contained an single 1x1 pixel image that "phoned home" to mkt41.net - an email marketing firm that presumably tracks visits and message load statistics.  After a little Googling, I learned that Mint.com apparently sent a "flood of emails" the previous morning due to a "simple" misconfiguration of a test server "with no customer data".  Stephen Mann (Mint.com's Director - Customer Advocacy) assures us (other loyal users like me no doubt) that measures were being put in place to ensure this never happens again.

So when I received a second blank email - identical to the first - about 6 hours later, I was upset.  I then learned that many others had received a half dozen or more emails well after Stephen Mann's rather inadequate message.  As I re-read that message that Mann posted early on 10/13, I grew more upset as I saw clearly both the incompetence and the utter lack of professionalism represented in the content.  

Here are my two main issues with Mint.com/Stephen Mann's message:

• Stephen Mann's choice of language in posting "sorry for being annoying" and "We also apologize if we woke you up from your sleep! " indicates that he thinks he's managing the social presence for a brand new shiny web2.0 side project.  Mint.com manages the personal financial information for hundreds of thousands (millions?) of users.  Stephen Mann should take this more seriously.  Moreover, a $30million sale to Intuit should firmly signal (at least to Mint.com) that Mint.com has a greater responsibility to its userbase.  We should not be subject to amateurish mistakes like allowing a test server to communicate with the outside world.
• I believe Stephen Mann is either too incompetent to have correctly determined the technical details of the problem, or outright lied in this message.  If the mistake occurred on a test-system with "no customer data", how was the email delivered to my email address?  If the test system has access to my email address, what other detail exists in this environment?

My final concern is that not many people appear to be aware of this problem!  To my own tiny part, I have hooked the RSS feed for this utterly shameful feedback thread into twitterfeed.  Any updates to the forum will be pumped into my embarrassingly lonely twitter stream.

For shame Mint.com.  Cereal.

Track the Forum Here:  http://gsfn.us/t/1n67r

On becoming a positive deviant...

I just finished Atul Gawande's "Better" and found his closing thought particularly inspiring. Gawande is a gifted author, and masterfully guides his readers through a series of case studies that illustrate instances of remarkable achievement, how these were accomplished (and then replicated), and what he learned about the people that dedicated themselves to the relentless pursuit of "better".

Gawande's closing centers around a lecture he once gave to medical students that he hoped would help guide them in their own efforts to distinguish themselves in their careers (and likely their lives in general). He offers his students 5 ideas that he feels would condition them into a pattern of behavior that would set them in the right direction:

• Ask an unscripted question: he encourages stepping outside the routine to establish a more human connection; not just with patients, but also with collegues or friends.
• Count something: in one of my favorite lines in the book, Gawande suggests that everyone should be a scientist in their world.   Counting something - as he did the number of instances sponges or tools were left behind in the body after surgeries - conditions the mind to ask the types of probing questions that invariably lead to innovation and change.
• Embrace change: he is careful to caution against blind adoption. Rather he encourages the willingness to learn, critically evaluate, and adopt new ideas if they bear merit.
• Don't complain: Gawande suggests (from his own experience) that nothing is as demoralizing in healthcare than hearing doctors complain.   I'm inclined to think that a variation of that sentiment applies equivalently to virtually any situation.
• Write something: perhaps this last suggestion was the most compelling as it motivated me to write this short blurb. I find that writing (particularly about things that are meaningful to me personally) naturally forces me to think more critically and creatively.   At the very least it has forced me to put the iPad keyboard through it's paces! (this post was authored on my spanking new iPad using SquareSpace's iPhone app.  Thanks Mona!)

I highly recommend Better, as well as Gawande's other texts (including The Checklist Manifesto, and his very insightful contributions in the New Yorker).

Trusting Trust...

This brief talk by Ken Thompson is an elegant treatise on computing trust.  I've read it 3 times now, and feel like it is only just starting to sink in.

Panasonic BL-C111A rocks. Cereal.

A few weeks ago, I purchased a Linksys WVC54GCA Wireless IP-Camera and found myself somewhat frustrated.  The has been dutifully monitoring the breakfast nook in the kitchen, and sure enough, the pups spend minutes out of the day in there.  That's right, minutes!

Since Romy and Mothi have decided that the living room is more their style these days, I thought it was a good opportunity to revist the IP-Camera project, and (hopefully) correct some of my first-time missteps.

 
What's over there?

PTZ.  Pan. Tilt. Zoom.  One of the issues I had with my Linksys fixed camera was that it monitored only a fairly narrow portion of the room.  Now in the living room, I knew I need my camera to be able to cover a larger area, and the ablity to move the camera around was a necessity.  The Pannie sports a 180 degree pan, and a 150 degree tilt.  Its digital zoom isn't going to win any awards, but the range of movement easily compensates.

But it'll cost ya!

PTZ adds an easy $100 to the standard fixed versions.  If I wanted to also keep wireless, I'm looking at $250.  Yipes.  Enter Powerline Adaptors.  I'd like to write a more detailed post on these suckers alone, but in a nutshell, Powerline Adaptors convert the electrical wiring in your home into network cabling.  So anywhere you have an outlet, you have network at abour 85mbps.  A fantastic compromise which put my camera investment back down to $160.

Network Printing should be this easy...

The BL-C111A.  Box to streaming video in less than 6 minutes. I timed it.  Fine, I cheated a little.  I only run Macs at home, so using the included setup CD wasn't going to fly.  I found this fantastic tip that pointed me to the camera's default address (192.168.0.254). 

By     Jefferson Harkins (FL, VI, NS) - See all my reviews
(REAL NAME)  
Panasonic doesn't support it, but this camera, and all Panasonic Netcams, work fine with a Mac. Later versions of Safari, Netscape and Firefox allow you to view full motion video. No add-ons or active-x or other such nonsense is needed with. The video just works.

As for setup, you will have to set this (and other Panasonic Netcams) up with a broswer, not with the automatic setup CD that works with windows. It's not difficult. Configure your Mac to use the built in Ethernet connection only, configured manually to IP address 192.168.0.5, subnet 255.255.255.0. Connect an ethernet cable between your Mac and the Camera (with a modern Mac any cable will do, with older ones you'll need a crossover cable.) Then access the camera at the address specified deep in the documentation (for this camera it's 192.168.0.253). The first time in you'll be asked to select a username and password. Do that and then you'll have full access to the setup utilities where you can change IP addrerss, DCHP, and other parameters. You can also setup the excellent free viewnetcam.com service and give your camera a real internet name. This is my third Panasonic Netcam, and I've been very pleased with them all.

The web setup is comprehensive and intuitive.  While the camera can stream RTSP, I've found enabling my router's UPnP setting and allowing the camera to auto-configure its port forwarding is a much better bet.

The web portal works like a charm on my iPhone, and supports display from upto 16 cameras.  So I'm officially on the lookout for as many as these little Pannies as I can!  If you're on the fence about picking up an easy, solid camera to keep an eye on your pets or kids or whatever, you cannot go wrong with the BL-C111A.  Git one!

Linksys WVC54GCA Woes...

So now that Mona has resumed her hectic full time school schedule, our 5 month old little puggies are going to spend more time home-alone.  They're housebroken and generally pretty lazy, but I've been thinking that I'd feel better if I could check in on them from work.  Enter the wonderful world of network enabled webcams.

 
IP Cameras have to-date been used for security and surveillance for small businesses, and recently have made appearances in consumer product lines from labels such as D-Link and Linksys.  Fundamentally a simple concept, IP cameras are essentially web-cams with an embedded server that can stream the video over HTTP or RTP RTSP.  Add in 802.11 b/g/n on the packaging and you'll shell out an extra $50-$75 for a wireless network cam.

Beyond the wired/wireless, the other main differentiating feature in network cameras is whether the device is fixed or capable of Pan, Tilt and Zoom (commonly referred to as PTZ).  The ability to move the lens around a room will roughly double the cost of the camera (new of course, but I've found that the more mature models are so new that the second hand market doesn't provide any better than a 15% discount).

After some typically obsessive research, I had my choices narrowed down to the Panasonic BL-C111A and two Linksys models WVC54GCA.  I decided that Pan, Tilt Zoom wasn't critical, but wireless would definitely be a requirement.  I settled on the WVC54GCA and ordered one up from Amazon.

After unboxing this neat little camera, and promptly tossing aside the manual and included CD (who needs directions), I plugged the device into my switch and powered it on.  The single blue LED in the front blinked a few times (which I later learned indicates the camera is booting up), and helpfully stayed on solid.

... Now What?

It turns out that configuring this thing is probably quite straightforward on a vanilla home network (managed by a dandy Linksys router no doubt) with few changes from the manufacturer's settings.  Of course, no such luck on my network.  Its not that I'm some uber hacker with obtuse or complex configurations at work, but I do have DD-WRT running on my router and have changed my internal network addressing from the default 192.168.1.255 subnet to 30.30.30.255. 

Next problem: I don't have a PC.  So even if I did swallow my pride and tried to use the "Discover cameras on my network" option on the installation CD, I would be out of luck.

After nearly a half hour of poking around (banging my head against my desk), I found this helpful tidbit on identifying new devices on your network; especially if you run macs at home.

Once you land on the Linksys web-based admin page, I have to admit it was pretty smooth sailing.  I'll point out a few points I wish I had stumbled upon sooner that maybe helpful to others, but first, my complaints:

1. I wanted to check in on my pups from work:  I didn't realize how much I had to compromise on performance if I wanted to rely on the wireless part of my wireless network camera.  I have a 15/3 cable  connection at home, and viewing my camera feed over the internet was dog-slow (ha ha!).
2. I thought it'd be cool if I could view my feed on my iPhone:  Documentation on RTSP support on the iPhone is rather sketchy.  I just found a fantastic app called Streamer that supports RTSP streams and does a terrific job with the WVC54GCA's feed.  That said, its still reliant on the quality of the network connection:  both on the iPhone, as well as your network that's attempting to up-stream the feed wirelesly from the camera.  In my case, with the camera just 30 feet from my wireless-N router, I couldn't do better than 10 fps.  Disappointing.
3. I wanted something plug/play/forget:  What can I say.  BonJour and zero-conf networking on the Macs have spoiled me.  Took me 2 hours to get the camera setup, streaming wirelessly, and accessible via the outside net.  And having it operational for just over 5 hours now, the camera has already dropped off the network twice.  Granted it came back pretty much on its own, its a bit frustrating when you're out to lunch and want to check in on something that ate up a chunk of your morning only to find the picture choppy, then stuck and then gone.

Lessons learned:

1. The default ip the camera grabs is 192.168.1.115 (admin/admin) .   This is useless on any customized network.  You'll need to figure out what the device's IP is using either a broadcast ping or something similar to these directions
2. The access key if you want to secure in mobile mode must be numeric only!  (You'll know what this means if your web-wizrd tells you that your text-overlay is invalid)
3. The fixed lens on the camera has a pretty narrow field of vision.  If you want to look at a room of any decent size ( I wanted to cover a room about 20x15), you'll need to either mount this thing on the ceiling in a corner, or accept a failry substantial blind-spot.
4. You need some seriously comprehensive wireless coverage wherever you intend to install the camera.
5. RTSP streaming is quite impressively supported on the Streamer iPhone app.  Get it!

Bottom Line:

1. I'm going to upgrade my network with a couple of wireless repeaters and give this camera another shot.  But I have to say, the performance better rock, otherwise, this thing is going back to the store.
2. I haven't given up on network cameras, but to get my approval, the wireless performance has to be much better, and I ill not compromise on the PTZ feature next time around.

YMMV, so I'm looking forward to feedback.