URL Shortener Design Flaw

With the rapid growth of Twitter and other microblogging services has come the rise of numerous URL shortening services.  Some, like TinyURL, existed long before Twitter, but they all share a common problem that has been exacerbated by the increasing use of microblogging.  They are a perfect mask for spammers. With these URL shortening services, anyone, including a spammer, can push any URL into the service and get back a shortened version under the domain of that particular shortening service.  When the new, shorter URL is posted on a microblogging service or anywhere else, viewers cannot easily determine where the link might take them.  Since almost everyone on microblogging services is using the shortening services, it is impractical to avoid them or to blacklist the associated domains. There are ways to safely extract the end URL from the short version, but those methods are not currently readily available to the majority of users.  Bit.ly, one of the shortening services, recently introduced warnings when they detected that a URL target might be malicious or intended for unsolicited use (spam).  This is a start, but it is still retroactive.  TinyURL offers, through a setting that users can enable (stored in a browser cookie), the option for users to see a preview of the target URL before being redirected.  Bit.ly offers a Firefox extension for the same purpose.  These options are better, but they still require that the end-user take some action for the extra protection. For administrators of sites like Twitter, there are very few options for screening shortened URLs.  Because of the variety of shortening services available, and the ability for users to pick from any of them (and post any arbitrary URL), the only way to dereference the URLs in a widely-supported manner is to actually attempt a simple HTTP request to each posted URL and, if the response code is 30x, then look at the "Location" response header.  Thus far, I have seen no evidence of a major microblogging service doing any filtering on shortened URLs, but I would not expect them to disclose their anti-spam measures if they did. Since malware and spam attacks are targeting microblogging services with increasing frequency, filtering and blacklisting of URLs may soon become a necessity.  I can imagine two things that would go a long way toward protecting users from the increased threat and return the balance.  First, browsers should include support for dereferencing links without visiting the targets, actively notifying users of the target URL when they are being redirected, and have that feature enabled by default only where the target is on a different domain.  Second, microblogging hosts should introduce filtering of shortened URLs by checking all links posted on their services for redirects and then filtering those redirection targets, and they should coordinate blocking efforts to increase the effectiveness of the filtering. Spammers are already taking advantage of shortened URLs, and the problem is only going to get worse unless we take action to destroy the advantage that shortening services currently give them. Update: The suggested browser feature from above has been proposed for Firefox at: https://bugzilla.mozilla.org/show_bug.cgi?id=453077

Comments

url shorteners are evil. twitter can fix this by either getting rid of the arbitrary character limits or offloading urls into a separate metadata field.

riffic,
Look into identi.ca, a microblog site that does resolve URLs into metadata fields.
See: http://identi.ca/notice/6562279

http://lnk.nu does some work to keep people from obfuscating the destination.