A Brief History of Databases

Peter James Thomas big data, infographics, technology,

A Brief History of Databases

Larger PDF version (opens in a new tab)

The pace of change in the field of database technology seems to be constantly accelerating. No doubt in five year’s time [1], Big Data and the Hadoop suite [2] will seem to be as old-fashioned as earlier technologies can appear to some people nowadays. Today there is a great variety of database technologies that are in use in different organisations for different purposes. There are also a lot of vendors, some of whom have more than one type of database product. I think that it is worthwhile considering both the genesis of databases and some of the major developments that have occurred between then and now.

The infographic appearing at the start of this article seeks to provide just such a perspective. It presents an abridged and simplified perspective on the history of databases from the 1960s to the late 2010s. It is hard to make out the text in the above diagram, so I would recommend that readers click on the link provided in order to view a much larger version with bigger and more legible text.

The infographic references a number of terms. Below I provide links to definitions of several of these, which are taken from The Data and Analytics Dictionary. The list progresses from the top of the diagram downwards, but starts with a definition of “database” itself:

To my mind, it is interesting to see just how long we have been grappling with the best way to set up databases. Also of note is that some of the Big Data technologies are actually relatively venerable, dating to the mid-to-late 2000s (some elements are even older, consisting of techniques for handling flat files on UNIX or Mainframe computers back in the day).

I hope that both the infographic and the definitions provided above contribute to the understanding of the history of databases and also that they help to elucidate the different types of database that are available to organisations today.
 

 
Acknowledgements

The following people’s input is acknowledged on the document itself, but my thanks are also repeated here:

Of course any errors and omissions remain the responsibility of the author.

 
Notes

 
[1]		  
If not significantly before then.
 
[2]		  
One of J K Rowling’s lesser-known works.

From: peterjamesthomas.com, home of The Data and Analytics Dictionary and The Anatomy of a Data Function

 

33 thoughts on “A Brief History of Databases”

  3. Would sure like to see the word “Codasyl” on your long-ago history. And really, relational database didn’t reach mainstream use until after Y2K, the machines were just too feeble to support it very well. The network and hierarchical, also inverted file systems that were not *quite* databases, still carried the load. Oh sure, and dbase, or “xbase” as it was later known, how about those? They were as much database – they were MORE database – than are the “NoSQL” tools today, much less stuff like Hadoop. Of course Teradata in 1983 was twenty or thirty or forty years ahead of their time!

    Reply

    • Sort of depends on what you mean by database. In my view incorporates network and hierarchical. I was personally using Oracle from 1988 in major organisations like BP, NEC etc.

      Reply

    • It is absolutely untrue that relational did not reach mainstream until after Y2K. Oracle, SQL Server and Db2 were running most new mission critical apps well before that. My background is heavy on Db2 for z/OS and I was writing banking apps for it in the late 80s and administering it ever since.

      Of course, I agree about the lack of CODASYL being mentioned (as well as IDMS). Additionally, the whole thing is a bout DBMS (or database management systems) not databases. But maybe that is just being pedantic. Another whole chapter of “database” history that is left out is the ODBMS craze of the 90s. Of course, the never really became mainstream.

      Reply

      • Craig, I’m interested in hearing more about the early years of relational! My first serious exposure to it was on a Wintel platform circa 1995, and Oracle as it ran then was – well, it was our platform, but we never finished, and my first role was writing a multi-thread driver for it as Oracle did not have one, not on Windows anyway (they might have released their first one six months later). And I believe Oracle still used a full image for each “thread”. It might have “worked” but I don’t see that as a mature product. SQL Server was still 4.2 then, and until SQL 7.0 came out in 1999 I don’t think you can say that SQL Server was ready for prime time either, although I had done one or two small system using 4.2. I’d also spec’d a medium to large system and talked to Oracle sales, and they thought it was too big for Wintel Oracle at the time, though again, we never got to prove it one way or the other. Frankly, it wasn’t until the Wintel platform went 64-bit and 8++ cores, that I would say it was a mature platform, and that’s more like 2007! Although I have since done maintenance and tuning work on systems that originated on SQL Server well before Y2K.

        Reply

        • I was using Oracle on VAX/VMS and then various flavours of UNIX. I agree that the WinTel versions were later / less robust initially.

          Peter

          Reply

  6. Where will Pick, Prime/Information, IBM/Universe, and IBM/Unidata fit in ?

    Reply

    • Hi Nagian,

      I obviously couldn’t be comprehensive – any suggestions from you where these would fit best?

      Peter

      Reply

      • Yikes! Do these also include MUMPS? Forth? This is what happens when you attempt a “full history”! At least chronologically these all began in the 1970s, I think some even into the late 1960s. Aren’t these precursors to Cassandra? Call them “compositional” perhaps. I’ve never spent much time with any of these, there may be other preferred terms.

        Reply

  8. I started programming with FORTRAN and COBOL in 1974 and spent the years 1986-1994 working for Information Builders, INC, the company that popularized 4GLs that nontechnical people could use to report on their own data.
    My opinion is that the introduction of relational databases along with the emergence of PCs and simple form-based data collection app-builders, e.g. dBase and its cousins, led to the dominance of relational(ish) data modeling, almost to the point where many, many people in IT cannot conceive of any other way to store data.
    This is a shame. It’s been a pox and a curse upon our world, inhibiting progress, suppressing other, better ways to store data.
    The fundamental flaw in relational database design is that it was created to benefit machines in their handling of data: properly set up a RDBMS can protect against data anomalies. But relational databases are impossible for nontechnical people to understand, and difficult for technical people to wrap their heads around and manage effectively. This was the in the main responsible for the dark ages of analytics, where it was necessary for IT departments to be the intermediaries between business stakeholders and their data.

    Reply

    • Thanks for taking the time to comment Chris :-).

      Can you tell me a bit more about what forms of data storage you would recommend?

      Peter

      Reply

Leave a Reply to A Brief History of Databases | 1A ContentCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.