Sosolimited's 'Semantic Sabotage' Should Be the Future of Closed Captioning

In this upload-it-to-YouTube age, where three days worth of content are posted on the Internet broadcasting machine per minute, the toolbox of ways to remix and play with online media is bursting at its hinges.

With net artists and experimental coders working around the clock to create the next buzz-generating disrupt scripts, is the internet's endless fascination with remixing everything a sign we're getting bored with traditional media? Are the successes of tweaked pics (Instagram), subtly-composed film scenes (cinemagraphs), and sparkled selfies (Blingee), proof that we need to make media more dimensional, more fun, and more accessible?

Enter a new toolkit from the art and tech studio Sosolimited called Semantic Sabotage. It's a typographic visualizer for YouTube content with a fresh take on closed captioning. "The focus is on real-time typographic or graphic animations," Sosolimited's John Rothenberg told me in an e-mail exchange. "The audio from the clip is played back as these captions are streamed word by word."

Through both automatically created captioning, which is fed through Google Voice, and manually edited captions of the sort that are typically included with videos from the White House, Sosolimited's visualizations reimagine the same old, stale, pixilated, yellow letters we're used to seeing in hospital waiting rooms.

While one is able to download and mess with the visualizer's source code via GitHub, less technologically inclined patrons can still experiment with Semantic Sabotage by pasting any YouTube link into any of its players.

Although speech-to-text tech isn't super reliable, and, based on my trials, it doesn't seem the  application can scrape lyrics out of music videos, I still had some fun inputting a speech from last year's friendly fascist presidential candidate, Vermin Supreme:

There are several preloaded YouTube clips in Sosolimited's new machine, demonstrating a variety of different visualizers. Some of these include Yosemitebear (AKA the guy who saw the double rainbow), Obama addressing the Boston marathon bombing, and a JFK vs. Nixon debate. If you've deduced that Sosolimited has an obsession with politics, you'd be right: One of group's earliest projects was a live remix of 2008's presidential debates called ReConstitution.

Last year, Sosolimited teamed up with our sister site The Creators Project to produce Reconstitution 2012, which allowed viewers to watch Obama and Romney's second presidential debate come to life in a typographic transformation fueled by word analytics and on-the-fly statistical gathering.

Rothenberg left me with a broader idea of Sosolimited's mission, which runs congruent with recent political developments:

Aesthetically, we're interested in surveillance, and want to give users the tools to survey the language of others, particularly public figures… The language processing in our software includes the LIWC engine, which allows us to look at the emotional state of speakers and can even point to whether they are being deceptive… As speech to text improves even the automatic captions will be more accurate.

Looking forward, I'm sure it'll be some time before this type of machine shifts us out of closed caption's rusty old gear. As it develops, could such an application help researchers and developers find more watchable ways in which audiences—and the hearing-impaired—read along? Semantic Sabotage definitely feels like it's headed in that direction.

@danstuckey