Should You Learn About Data Structures and Algorithms?

During my years as a teacher, both at a technical college and a university, I've noticed something of a trend among new developers. Because of our modern culture of what I term "input => output," we tend to think that programming is just a matter of knowing what things to do in a certain set of situations. Think of it like being a machine operator on an assembly line. You learn how to work that specific machine to do its specific set of tasks.

Computers aren't like that though. You can program them to not only do anything you want but there are countless ways it can do a single task. Depending on your desired outcome, some ways are better than others. For starters let's talk about what exactly "data structures" and "algorithms" are.

What is a data structure?

A data structure, in programming, is a way of storing, organizing, and performing operations on data. Their operations include things like accessing or updating stored data, inserting or removing, and searching for data.

Some examples of data structures are lists and arrays. Each data structure stores data in a different way and each one is suited to different scenarios. It will be up to you to learn how they work, and when they can and ought to be used.

Choosing data structures

The selection of data structures used in a program depends on both the type of data being stored and the operations the program may need to perform on that data.

For example, if you need to store data in a way that you can easily search, but don’t care about the order, there is a data structure for that. If you need to quickly be able to add or remove items from a structure and preserve the order, there is a data structure best suited for that as well.

Choosing the correct data structure is vital to program performance. All of them will get the job done, but some of them will do a job much better than others.

Algorithms

My favorite definition of an algorithm is "what a developer tells you when they don’t want to explain to you what they did." I’ve used this before - because what most business people or project managers want to hear is whether something works or not - they don’t care how you did it, or wouldn’t understand it anyway.

An algorithm describes a sequence of steps to solve some computational problem or perform a calculation. They can be described in plain language, pseudocode, a programming language, or even hardware.

My time-costly mistake of not choosing the right data structure

Because we all want to know “how will I use this”, let me share with you a story from my early career about how using the wrong data structure cost me several hours of time and effort.

My first programming job was working on a desktop application that helped independent insurance agents give customer quotes from several companies at once. One day, I got an email with a pdf attachment that contained a list of 64k VINs. It was my job to figure out which of those VINs we already had in our database, and which ones we didn’t. They also gave me a list of all the VINs we had in the database - around 600 thousand of them.

They were big files.

As an aside, yes, it would have been much easier to have access to the database itself, but there was no ‘test’ environment with all the same data, we only had the production database.

Having just taken a Python class, I started whipping up a quick program that would read both lists from their respective files, and search the 600,000 item list for each of the 64,000 values. How bad could that be?

Well, I fired up the program and it started running.

After four hours, it still wasn’t finished. I didn’t expect it would take that long.

Then, I remembered something. I pulled up my Python program, changed two characters, and ran the program again.

This time, it finished in 4.6 seconds.

The two characters I changed indicated what data structure I was using for the VINs that needed to be searched. To be specific, I changed the type from a List to a Set. The defining feature of the Set is that searching one is instantaneous, while with a List, the time it takes to search it is directly correlated with how big the list is.

This is a small example of how important it is to use the correct data structure for your applications. No single data structure is perfect at everything, so by knowing how they all work, you will be able to make the right choices.

Learning some computer science is well worth your time

I can't say exactly when or where this will come in handy because, again, there are virtually limitless things you can program a computer to do. If a computer were like an assembly line machine then sure, I could tell you that when "Situation A" comes up, you should do "B".

Learning about all the different data structures and algorithms that are at your disposal is like adding more tools to your toolbox. The old saying "the right tool for the right job" is exactly what learning data structures and algorithms is about - when you know what is possible and how it works, you'll know how to use the "right tool" for the "right job".