The start of a new year is always a time for reflection, and this January has particular significance to me. This month marks fifteen years since my cofounders and I came up with the idea for 1010data. Fifteen years! In the tech world, that’s an eternity. Amazingly, this month also marks my 35th year in the technology business. If 15 years is an eternity, what is 35? Oh well, as they say, the only thing worse than getting old is not getting old.
How has the tech world changed in all those years? There certainly have been lots of changes, mostly positive. The biggest, of course, is the internet and the vast cultural changes it has caused. But a lot of things have stayed the same as well. Human nature is more resilient (stubborn?) than we often admit, and people adapt to new things in ways that keep the past alive.
One of the most powerful moments I experienced in my career is as fresh in my memory as last week’s ball drop. In 1979 I was a year out of college and working for one of the major TV networks as a programmer analyst. My project was to develop a business intelligence application that would allow management to monitor the status of commercial-slot sales. At the time, it was very difficult for them to find out how many slots were sold (and how many remained unsold) for a given show or daypart, for a given time period. I was part of a two-person team that was tasked with fixing the problem.
Using a computer in the days before spreadsheets generally required programming, but there was a tool called APL that provided some of the interactivity and ease of use that are now characteristic of spreadsheets. APL was used by programmers, scientists, actuaries and even non-technical business people as both a way of doing analysis on large amounts of data and building applications; it was both a user tool and a powerful programming language. I was doing my project in APL and one of the first things I needed to do was write some software that allowed me to access the company's CICS database from APL. The rough equivalent today would be to write an add-in to Excel that would allow users to pull data out of a database.
Now if someone were to write such a thing today, it would be obvious that it might have value beyond the immediate project at hand. For example, might it not be usable by others for other applications? So in that spirit, I shared it with some of my users in the finance department, who were also using APL and who appreciated the fact that they could now get to the data directly, bring it into APL, and analyze it to their hearts content.
Well, you probably guessed where this is going. My boss sat down with me with a serious expression on his face.
"I understand that you wrote something that allows you to pull data out of the CICS database," he began.
"Yes," I replied, "I needed it for my project."
"And you shared it with the users in finance?"
"Yes, they often need the same data and this will make it easy for them to get it ," I said, sensing where the conversation was going.
"Well," he said, and I will never forget his exact words, "If they can do that, then what's our job?"
So the BI problem in 1979 wasn't a technical one, it was a human one. And I don't think things have changed as much as we'd like to think. With every change, some people feel threatened and will attempt to put up roadblocks. They may not be as blatant as my former boss was, but that doesn't mean its not happening. Technology may change, but do people?
The answer is maybe. It looks like the market is finally recognizing that the end user needs to be empowered. Nascent ideas like Data Discovery seem to be getting more attention and some of the hottest recent IPOs were for companies whose products are aimed at the business user. (As an aside, in my view, Hadoop is actually a step in the wrong direction since it creates a lot of work for technologists but does not empower the user. You can't win 'em all.)
For me, it's been a long three decades of waiting for the world to change. Right now I'm optimistic that I will have a much happier story to tell 35 years from now. Check back with me then.