In my life as a programmer, spanning almost 20 years since my BASIC adventures on a Tandy 1000EX, I have worked in either Microsoft languages or OpenSource. On one side, I've labored though PHP, spent far too many years debugging perl, and fell deeply in love with python. On the other side, I've learned win32, to have it replaced by MFC/OLE, to have OLE replaced by Active X, to have it all replaced by .Net and Forms, which is waiting to be replaced by XAML and WPF. All the while, perl stayed perl, python stayed python, and php is still craptastic.
In my younger years, I would say things like "Microsoft changes tech every two years because they have a marketing need to make everyone rebuy Visual Studio." I still agree there is some truth to that, but it's irrelevant. Ironically, I learned why with open source.
When I took over as "Director of Software Development" at a small web shop, I looked at our current amount of code reuse - there was none. We worked in PHP and perl, and while the turn around time on projects was okay, we weren't improving. This led me to my first realization:
There are bad languages - A language (and framework) need to be designed from the point of view of the developer, but too often the CPU is given the ultimate priority. Runtime performance is only one of many factors to "speed". How long it takes to write, and how much time is spent debugging are in many projects more important. Even if you are working on an application that will see a high load, it's usually one one or two components that will see that load. Treat them as the exception, rather than the rule.
As I looked for a better language I found python, and I was impressed. The idea of cutting out the bogus syntax (that's there for the parser, not the developer) plus the idea there should be a good reason *not* to include something, topped with "one and only one" way to do something is tempting God's wrath with envy. Being a web shop, I turned to selecting a web framework - and python has no shortage of those. In evaluating the many options, I came to my second realization:
Scale down first, then scale up - The ability of a framework to handle the simple tasks well tells you a whole lot more about how the framework will handle on the large projects. I'm not talking just about CPU cycles, but developer time. Put it this way, it's 2007 and why the hell do I still need to deal with table tags in my pages to display a grid of data? I should be spending more time on great user experience, not plumbing. My search ended up with Albatross, which we later forked into SnakeSkin.
Things starting happening at the shop - development time went down and reuse went up (like crazy). The team was excited and passionate about programming and we had many discussions about the best way to solve a problem. No management style change, no employee replacements, just getting the right tools in place. We wouldn't be immune from other contagious management diseases, but for a while there it was amazing. I learned the next lesson:
Patterns are great, Widgets are better - MVC, n-Tier design, Business objects, etc. all have their use, but having a widget is worth far more than applying a design pattern. In a theory world you would have a data access layer that is accessed only by a business logic layer. A bit of code would glue this to a view layer, which would tie to a rendering engine layer to support multiple views. What if you had a select box drivin by a simple "select * from table"? A widget that let you say sql="select * from table" would have you spend 2 minutes on the control, while implementing a good MVC patten might take half the morning. There is a trade off - for example if you have to move the datastore from a database to XML the MVC method would make that easier, but in reality how many times has that come up? Again, treat that as the exception, not the rule.
There are some thing we know we should do, but never get to. Unit testing and refactoring are the big two. A custom MVC application will need custom unit testing code - a widget on the other hand only needs unit testing code written once, then it's shared by every application that uses the widget. It's also rare that you can sit down and refactor an application - management and clients never see the point of working on code that works already. You will however be working on new projects with your old widgets, which you have a great chance at refactoring then updating all the old applications. This is the next lesson:
Frameworks get better, your code doesn't - Not saying you don't become a better programmer, you do. You don't go back and redo everything the "better way" however, you only apply it to the new projects because there never is the time (or in some cases desire) to return to the old projects.
Taking this rule a step further, there are things in the frameworks we use that we might be able to do better and "code it by hand." Don't, or at least make a very large effort to resist. That issue may not be an issue in the next update of the framework, making your code an unnecessary burden to support.
This last lesson was a bit of a pill to swallow. It came from that not only can frameworks get better, they should get better. We all know at some point, a rewrite is the next step, so:
Expect to have a new language/framework every two years - Otherwise you are accepting stagnation while the world moves on. Getting stuck on a non-evolving framework is how dinosaurs are created. All of those Microsoft changes were great improvements over their predecessors. While it does suck to learn new systems every few years, that is the life of a developer. That's why we get paid the big bucks (ha).
