Blog

Continuing with our recent theme of abstraction in data applications, Dave King gave a talk last month explaining his design principles for "Making Code Sing: Finding the Right Abstractions." Nailing the best abstractions is a quintessential software challenge. We strive for generality, flexibility, and reuse, but we are often forced to compromise in order to get the details right for one particular use case. We end up with projects that we know have amazing potential for use in other applications but are too hardcoded to make repurposing easy. It’s frustrating to see the possibilities locked away, just out of reach.

Six months ago I didn't understand the concept of abstraction. Now it comes up almost daily. It’s foundational to my thinking on everything from software to entrepreneurship. I can’t believe how simple it seems. When I finally grokked abstraction, it felt like my first taste of basic economics. Given a new framework, something that had always been there, intuited but blurry, came into focus.

I like to build things. I like analysis. I like programming. Interestingly, you often need to reverse that order before you’re in a position to build an application for analyzing something. You need programming knowledge to turn the analysis into a “thing.” The problem is, while I like programming, I’m still new to it. I mean, I’m Codecademy good, but that doesn’t translate into a user facing application leveraging Python, Javascript, and D3. So, when I recently sat down to build a minimally viable data application for looking at airline stocks, I wondered how long it might take to get to viable and, frankly, feared how minimal it might be.

I’m an algorithms guy. I love exploring data sets, building cool models, and finding interesting patterns that are hidden in that data. Once I have a model, then of course I want a great interactive, visual way to communicate it to anyone that will listen. When it comes to interactive visuals there is nothing better than JavaScript’s D3. It’s smooth and beautiful.

Do you feel like partisanship is running amok? It’s not your imagination. As an example, the modern State of the Union has become hyperpartisan, and topic modeling quantifies that effect.