A Post Mortem on Madness, or Why Process Matters

So, my first programming job was part of the Duke Basketball IT department, while I was enrolled as an undergraduate. To be fair, it wasn't really a programming job, mostly just churning out scouting reports and videos, but it was a lot of fun. I really liked basketball back then. I wasn't any good at it but I enjoyed playing all the same, and I had a lot of fun watching it. And as most people know, the culmination of the college basketball season is a 68-team single elimination tournament called March Madness. And it is just that, Madness; employee productivity plummets, players, coaches, and spectators spend thousands of dollars traveling all over the country with a single day's notice, and a lot of scouting reports get generated. Most people look forward to it, both the participants and the fans, but due to a series of mistakes and poor decisions, I have grown to dread March Madness...

It all started my first year out of college. I had landed a great job working for none other than Red Hat as a Middleware Consultant and was 6 months into my first engagement working with Spring MVC. I felt good; I had learned more in those 6 months than most of my time in college and was building some cool stuff in the process. Then one day I heard from one of my best friends from college who wanted to run something by me. He and his dad have been running a bracket pool for March Madness for years and have always done the scoring on paper. Oh yeah, and their pool usually consists of over 200 brackets. What he wanted was a way to score the brackets that didn't involve doing the math by hand (they had a custom rule set that didn't work with the existing services), and what I heard was "Build me a web app worthy of Mordor".

The problem was that this was in January, and it's obvious when March Madness is. At this point, I made one of the only intelligent decisions I made in the life of this project: "Let me see if someone has already made this and I'll get back to you". I found an old school php project that had all of the features they could dream of: user bracket selection in the web, scoring, newsletters, authentication, and even something called "Paths to Victory" that would show how likely a certain bracket was to win and what games they had to get right to get there. This all sounds great but what you have to remember is that I was in the process of writing enterprise level software, which has a whole different feel than an open source, built-in-a-basement type PHP project. Instead of figuring out requirements for an MVP, I was already thinking about distributed deployment, load balancing, responsive frontends and more. Luckily, I knew that there wasn't time to deliver all that in two months. That year, we put the site up on GoDaddy.com and a few of us "evaluated" it. The people that used it thought it was great, but I missed all that. I was disappointed with their object model and thought the front end was too 90's.

After the Madness ended, the real Madness set in. It was then that I decided I wanted to rebuild the application from the ground up, making it responsive, secure, efficient, and throw in features like user management, a blog, and some social media elements. I figured I had a year and was only going to get smarter. Well, the next six months went by and the Red Hat project got higher stress as deadlines loomed and the scope crept, and I started thinking "Ah well as long as I deliver all the things that the old site had, but do it right, everything will be good". In that time, I looked at a half a dozen frameworks and plugins that looked cool and that I could incorporate into the awesome project I was building but did very little actual development.

Around the end of November, the panic started to set in. Any "free" time was spent dreading how much work there was to do on the site, but at least at this point, I had fingers on the keyboard and was churning out functionality. For better or worse, I decided to stick with PHP,  leveraging the Laravel framework, with the hope that it would be simple to deploy on OpenShift Online (v2).

In February, I decided to take a week off from work to spend exclusively on the project and not to brag but I actually got darn close to delivering an MVP implementation. I even jumped through most of the hoops (heh) of getting it deployed at an enterprise level despite the lack of enterprise level funds. In the end, though, I fell short of an MVP and was only about 50% of the way to the functionality of the original site. About a week before tipoff was the second time I talked to my product owners, letting them know that I had something that I thought was "really close" (even though at that point I hadn't even started on the bracket scoring module), but might not be ready for production. We ended up deciding that it made more sense to do an alpha test on the site but do the actual submission and scoring on paper.

The alpha was a disaster. I had rushed along so much in the months leading up that I had made the same mistake every PM makes when the time gets tight: "Skip the tests, just be good developers and get it right the first time". What a dummy. We found out that my golden object model had nowhere near the performance of the old site and I had messed up the regional matchups so scoring was FUBAR. I took my licks and said it will be better next year. Then I proceeded to make another bonehead decision: "That felt awful, I'm never going to let that happen again... but I should rewrite this in Java".

I threw out the object model, the data model, most of the algorithms because they weren't "clean" and started from scratch, because I was way smarter than last year, and would never let things get out of hand. It was at this point that the project ceased being fun. The whole year leading up to the inevitable flop still had their fun moments, even when I was stressed out of my mind because it was all so new and interesting and I felt like I was getting something accomplished. That was when I should have tried to learn from my mistakes such as: "I'm going to be accountable to my POs this time" or even "Why don't I ask them what they actually want". Nope, my mind was back into the weeds. I started looking at data models and Spring projects that would make the application even more robust and flexible.

At this point, boredom started setting in, and I started only focusing on the fun problems to solve and only got about 25% of the feature coverage and didn't even bother with writing tests. While it sounds like I was laid back about getting the project done, it actually stressed me out more than the first time around.  I had a sense of how far behind I was and still remembered how bad it felt to fall short. With about a month until tipoff, my fiancee (oh did I not mention that I got engaged during the tournament the year before?) turned to me as we were walking around the neighborhood, and said, "Why are you even doing this project?"

I told her how I looked up to this friend and how much I wanted to deliver something after all these years of work. She responded with "But you did that two years ago remember? You were the only one who didn't like the first site, you did these 'refactors' that no one asked for only because you thought they needed them, and now they have done three more years of paper brackets, and you haven't got anything to show for it." She said it a little nicer than that, but it hit me like a ton of bricks mostly because she was right. Here I was holding myself to these high standards and imaginary requirements and these people were fine running their brackets on paper. On paper! I hadn't even talked to my product owners just to see if they wanted a site anymore...

It was a humbling moment in my career, but those moments are when you learn the most. The faults were mostly on my side, luckily for me, my livelihood wasn't riding on it and I'll never forget the lessons learned. Don't get me wrong there had still been good parts. I learned a lot of great tech, and had a decent bit of portfolio code. I absolutely nailed the Stream API and really dug into integrating the different Spring projects in a cohesive manner with 100% Java config. I was even able to use some of the work in POCs for my actual job. But the real lessons learned were on the management side.

If I had held myself to real deadlines, I would have delivered everything and more in the first rewrite. Instead, I only really got it working once the final deadline was looming over my head. I also had no communication with the people who actually wanted the site. When they first came to me, they barely even wanted a site, but I was too busy thinking about building a golden monument to Madness. Then when time ran out I was too worried about disappointing a friend to tell the truth and ended up delivering a pile of garbage. If there is one thing about this whole situation that I hope someone reading this post can learn. It's that there is a reason that there are strict processes to software development, and even when you are a one-man team working for a buddy for free, you should follow at least some process or you will let the whole team down.