Friday, April 17, 2009

Thoughts on Podcasts

I attended a neat BOF at PyCon, where production of podcasts were discussed. Really, I went to find out what podcasts were being made by Pythonistas, and to see if there were any lessons learned that I could take to NASA. I came away with a few realizations.

An hour? Really?

Even when we are at our most lovey-dovey, in general, we do not want to listen to one person speak for an hour. Even two people are hard to listen to. This is why radio shows have guests. After the first fifteen minutes, the mind starts to wander, especially if there's no visual stimuli. The only people who make their livings talking for an hour straight end up getting an HBO Special.

Are you that good?

My favorite podcasts that are done casually are done in no more than fifteen minutes. They're fast, focused... like popcorn. Filling in small amounts. Check out Writing Excuses or Grammar Girl for examples.

What's your focus?

Don't say 'programming.' You are not writing a text book that you're hoping will be adopted by some level one undergrad course. You are trying to pick up a geek hottie in a bar. "Hey baby, want to come to my server farm and see if my Python scales?"

C'mon. Be bold.

Archive or toss?

There's some podcasts that are always relevant. RadioLab is one of them. You can listen to any show in the last few seasons, and it will still be interesting and relevant. Others are shows that are old if you listen to them the next day. Don't mix the two.

What's your take?

You have to have an angle. What's your hook? Are you super funny? Have unique experience in the field? Have an extremely narrow focus that's of interest to a surprising number of people? Do you have a super sexy voice?

We will listen to Guido ramble off his recipe for biscuits, hoping to decode the mysteries within ("He uses a pinch of cornmeal? Maybe he's talking about C encoders. Quick! To the compiler!"). You? You'd better have something off the bat to keep me there. And no, cunning background music isn't it.

Cunning background music

Oh dear lord. Are there no depths that we will not dork to?

Is your name Jerry Holkins or Mike Krahulik?

Then you'd better put your stuff out on a regular schedule. The Penny Arcade guys are awesome enough that, if they only put out a show a few times a year, I'll stay subscribed anyway. Some guy jabbering on about his favorite framework that can't seem to stick to a regular schedule? The first time I start cleaning up podcasts, out you go.

I don't post this to keep anyone from doing a podcast. I want more people to do podcasts! I just don't want any more that make me want to take an ice pick to my poor iPod.

Wednesday, April 15, 2009

Dancing under the waterfall

I've been interviewing quite a bit recently, and in the interviews, I've had to explain what our development cycle is like. My response:

"NASA is a waterfall shop. We're an agile group."

Some of the more fascinated interviewees have broken out of the interview vibe to ask how that's done. Enough have done so that I've decided to write up how you get your group to be agile when everyone else wants to do waterfall.

What is Waterfall?

Waterfall is something that is best used when you want to make sure that components of a shuttle aren't going to fail. Sure, things can always go wrong. But you want to prove that at every point along the way that due diligence was given to all the details.

In essence, there's a bunch of steps, and traditionally, you have the engineers and developers at every point. Steps are completely done before moving on to the next step. There is no iteration. You get one shot to get it right. As a system goes, it should work, until you get people involved. People don't like saying 'Okay, we're done. On to the next bit,' especially when their buy-in group is large. There will always be one last thing to add.

You need a magician

If you're in an organization that's bought into Waterfall, it's not going away any time soon. Millions have probably been poured into the people that crafted it, the people that maintain it, and the systems that support it. So, to the larger organization, you need do some slight of hand.

It is a magic trick. The large organization is the audience. One person plays the magician, going along with the flow of the waterfall, making everything appear as if it's going along as it always has.

The developers are also an audience. To them, it should appear that they are working in an agile shop. Their development cycles are short, requirements are tightly controlled, and they stop coding on the day they thought they were going to.

You need a spokesperson

The waterfall method assumes that everyone will be available during all parts of the process. It assumes that developers are absolutely vital to requirements gathering and design, as well as the QA after implementation.

No geek I ever met has ever loved requirements gathering. If you want to punish a geek, don't take away telecommuting or the cappuccino machine. Tell them they have to sit in on requirement and design meetings. Just don't blame me if it ends in a envelope opener to the carotid.

You need someone that's technical to be the spokesperson to sit in on these meetings. They need to have the ability to be quizzed relentlessly about the capabilities of your system, and answer quickly and accurately. No "I'll take it back to my team..." Next time, they'll just ask for the team to come.

You need to forget

One disadvantage of dancing under the waterfall is that once you're done coding, you don't deploy quickly. There's lots more paperwork and testing to go through before you hit the build button. So the build procedure that you have at the end of the development cycle needs to be flawless... because you're not seeing it again for a few weeks. For our group, it can be up to six weeks before we actually do a build for any release.

The developers must come to terms with this. It's not elegant, it kills the momentum, but it's as good as one can usually get under waterfall. There's good news though!

Keep 'em busy

The biggest trick to keeping everyone under an agile system is being able to think about three releases at once. Why do you have to do this? Because while you're gathering requirements for one release and following the QA on another, they're writing a completely different release.

So, let's say we're working on NASA Science 2.1.

2.0 is in QA and being prepped for deployment.

2.2 is the release I'm gathering for.

Once 2.1 is done, they immediately get the next set of requirements, coded into tickets that are ready for work.

Sound like a lot of work?

Well, let's just say it's good that I'm dancing under a waterfall. One can get pretty sweaty.

Monday, April 13, 2009

Requirements and Dead Cats

Gathering requirements and executing them is not as simple as getting the requirement, then having someone code it. It's like Schrodinger's cat. That's the problem where you have a cat in a box, and you don't know if it's dead or alive. Theoretically, the cat is both dead and alive at the same time.

If you wait long enough, the question answers itself. The cat will always croak.

Every time a requirement is touched or thought about after its creation, it changes. Instead of the cat being dead or alive, we're not even sure if it's a cat. Maybe someone put a chinchilla in the box. The req is perverted in ways that, at the end of the development period, are beyond human comprehension. Ironically, the simpler the requirement, the more perverted and complex it becomes. A two-line requirement, I swear, will become something that can bring down the very foundations of our society if we ponder it too long.

Note: this is the true goal behind agile development: to keep us from thinking too long on any requirement, lest we all be destroyed.

Longer requirements are perverted less, but not because they are more exact. We simply do not like thinking about them. Put them in a ticket, and it's guaranteed that we will not open that ticket until week six of a six week development cycle.

This is a real problem in large buy-in groups like mine. We have lots of people who like to touch tickets all the time and think about them and morph them from dead cats to live ones and back again. Here are some of the ways tickets are perverted:


  • Comments on tickets

  • Comments on the comments on tickets

  • Comments on the comment on another ticket

  • Conversations had in meetings, dutifully noted by the meeting note taker

  • Conversations had at desk side, sometimes followed up by a confirmation email

  • Conversations had in the hallway, where one person is pensive and filled with ideas, and the other really, really has to pee

  • Alternate universe conversations, in which one person has one conversation, and the other person has a completely different conversation

  • Imaginary conversations, in which one person will swear they had a conversation with another person, and that person swears it never happened



So, what do we do?

Well, we're getting better about it, for certain. And I'm trying to introduce some things that will make the perversions cause fewer fist-fights at the end of a dev cycle, when I'm calling for a tag to be cut, and buy-in people are screaming for an extension.

Fewer requirements

At first, this seems counter-intuitive. Wouldn't more tickets give more of a buffer to the poking and prodding of any one ticket?

In fact, part of the perversion that goes on is due to people having so many tickets to touch. They touch one, then run off to touch fifty more. By the time they've returned to the first ticket, they've forgotten how they changed it the first time, then change it again.

Having fewer tickets means giving people the chance to actually remember what they wanted the first time, and keeps the developers from having to change code in fifty places every time someone decides to walk through the requirements again.

We do not talk in N-Space

If a conversation isn't recorded in a sensible location, it did not happen. I do not care if the President of Burundi is willing to vouch for you: those words were never said.

What's a sensible place? Oddly enough, it's not 'the wiki.' 'The wiki' might as well be synonymous with 'the trash pit' or 'a teenager's bedroom,' the way most people treat it. It's not 'the meeting notes'. No one refers to those things unless they want to prove you wrong about something. It's on the ticket where the requirement is. If you have a conversation, record it there. Not in another ticket. Not on a new wiki page. Not in an email that no one else will see. Not in some word document on the server.

On. The god-damn. Ticket.

Vicodin

As I said in the previous post, requirements getting perverted happens. Most of the time, the differences are minor. A satellite is not going to fall out of the sky because the navigation on a web page isn't indented quite enough.

When a requirement isn't quite hitting the peg for that release, and not everyone who has buy-in is loving it, don't hit the pause button. Momentum is a thing that's hard to rev up, once stopped. The second you hit the 'OMGWTF' button, the entire group's memory starts to decay, and, as they review the tickets that were closed until now, requirements get even more perverted.

Take a pill. Sometimes you just have to try again.

Cut a tag and try again

If your cat is indeed dead and smelly, cut a tag and start working on the next cycle. Non-developers don't know this, but cutting a tag feels good. It's like the feeling you get when walking into a fresh hotel room. The sheets are fresh, the soap is wrapped, you've got little bottles of shampoo, and everything is clean and shiny.

Give your devs a new hotel room. Cut a tag, and start over fresh, rather than hoping to squeeze one last change into a beleaguered release. Because if you look at the requirement too much, it does bring about the end of all things.

Friday, April 10, 2009

Why I almost did a sys.exit()

I almost quit programming.

It was back in the late nineties, early aughts, when I found myself working at a start-up. We provided hosting and VPN solutions, and I was employed partly as a customer service person and a programmer. I was thrilled. After doing cookie cutter code in school and projects that were never seen again, I was finally going to do some code that would be used in live environment!

Our desks were arranged in x's, with one person in each section of the x. There were no barriers, as cubes were both evil and very expensive. As a result, we chatted a lot over the top of our monitors.

I got placed with some other developers, all male. We were discouraged from wearing headphones, as that meant we were passing up learning opportunities, so I got to hear everything the guys talked about. This was in the midst of the *nix wars, and these guys were sworn enemies on that front.

Every day, I got to hear about what was better: SuSE? CentOS? Red Hat? Solaris? Pure, unadulturated Unix dug up from Bell Labs? They had lots of ammunition, too. Which kernel compiled faster. Which had more updates. Which performed slightly better. Why the other person's bench tests sucked. Why the other person obviously didn't know what the hell he was doing if he had to recompile his kernel all the time. Why certain *nix distros cause cancer.

It was so. F'ing. Boring.

Being hormonal and under a deadline (I was about seven months pregnant), I did finally threaten to find all the burned *nix disks, break them into shards, and stab them into their eyeballs. That moved the fights into email. At least the heated glares I could ignore.

When most of us were laid off some months later, I seriously considered whether I wanted the rest of my life to be sandwiched between two long-hairs that can't agree to disagree. I loved programming, but did I love the people enough?

After a long side-jaunt, I came into a job at NASA via pydanny. It wasn't a programming job. I was simply to be a requirements gatherer and needs analyzer. I was happy in that capacity, and probably would have stayed in that track had it not been for one thing:

Plone.

Say what you will about Plone, ye lovers of other frameworks, but of all things it may do right or wrong, it did one thing very right:

It got me back into the fold.

Pydanny was hot to use it for something, anything. We won approval to go ahead with using Plone for a new website. I was the requirements gatherer. Danny was the coder. After a while, he encouraged me to pick up Python so I could join him in coding as well. I did so, and loved it. Adored it. Got scolded because I was stepping outside of my career path. I didn't have to worry about all the piddly discussions about kernals or whether one algorithm was a smidgen faster than another. I could just code. And make things work.

Maybe this should be called "How to get more women into Python, part II." The sentiment of 'why are we arguing about this crap' has been fairly universal when it comes to the women I've spoken to. They've been willing to learn new things, learn best practices, but one thing they have not been willing to do is argue about which module is slightly faster. We just want to get our work done. Because, lord knows, there's more to life than code.

Arguing isn't a bad thing. But if it's all you do, if the arguments span mealtimes, perhaps it's time to step back and think about if the energy the fight is taking isn't just getting in the way of doing cool stuff, and getting others to do cool stuff as well.

Wednesday, April 8, 2009

How I learned to love hockey (and what that has to do with programming)

Growing up, I was not a sports fan in the least. I didn't enjoy watching them, I had no interest in learning the rules, and I had NO interest in playing any of them. In gym, I was so accident prone that I had to sit with the asthma kids, and I was damned happy to do so. I preferred playing SET to kicking a ball around.

I dated a few guys that were into sports. In the interest of being a couple, they'd try to teach me the rules while watching a game on TV. Liking systems, I'd try my best to learn. Football was the first attempt. I'd post some of the repartee, but honestly, it bored me so terribly, I can't even dredge up the few things that I did manage to pick up. What I came up with could probably fit on a 3x5 card, and have room for my Superbowl Dip recipe.

Fortunately, I married a guy who, while interested in sports, understood that I had no interest in watching or understanding them. I thought that would simply be something I'd never get into.

Then came Jim.

Jim, a friend of mine, has season tickets to the Washington Capitals, and invited me to join him for a game. Really, I was just interested in a night out rather than learning about hockey. For the first few games, I was just there to BS with Jim. He even joked about how I didn't know what the hell was going on.

Jim (watching me trying to look attentive): So, you know what they're doing?

Me: Oh, sure!

Jim: Okay. What team is our team?

(Long, painful pause)

Me: They're... in... white?

Jim: You know, you can ask if you don't get what's going on.

No, no, I was good. I was just there for the company and the Boardwalk Fries. I don't think I learned a damn thing that first season.

At least, that's what I thought.

The second season, I again joined Jim for weekend games, but I noticed something different. I was beginning to understand the game. I could tell when something bad happened. When the Caps were playing well. When they were having a rough night. When a player seemed 'off.'

Now, keep in mind, Jim hadn't explained any rules to me. I hadn't read anything. There wasn't anything I could think of that would have given my an understanding of the game at all. So, where had it come from?

I paid attention to the next few games, and started to get clues as to where this implicit knowledge had come from: the crowd. The crowd was filled with people who did, in fact, know hockey. They knew what a bad call was, they reacted to a team that was on fire, and a team that was tired and run down. In general, there's a buzz during games: people chatting, moving about, coughing, shuffling between seats. When things happen, that buzz turns to shouts... or silence. The changes in volume snap my attention to what was going on. I'd subconsciously replay the last few seconds in my head and try to map that to the sound.

I tested my implicit knowledge on my expert. While I was still very fuzzy, I was on the right track. With this in hand, I actually started looking up some of the rules that interested me, like penalties.

So what does this have to do with learning?

Rules are boring

They really, really are. I don't know many people who find rules interesting. Some people do become rule dictionaries, but I guarantee, the interest came first. People don't learn the rules first: they find the interest.

So when it comes to programming, why do we focus on the rules first? Loving computers and, more importantly, loving code should come first. However, every class I've ever been in has started with data structures and variables and syntax and, well, the rules.

Why not show them the buzz first? Take them to a users group in your area and let them see people talking about what they do. Most coders are more than happy to talk about their work, and are excited to talk about whatever neat thing has their attention at the moment. If you have someone you're really interested in seeing excel, see if you can get them to a conference. It's one thing to look at code and understand what an if statement is. It's another matter completely to understand the community that goes with it.

Rules are a good resource

Does this mean we should eschew rules completely? Of course not. They're a good resource. Once I was in love with hockey, I wanted to know more about the sport. While I'm no rulebook, I do know what the penalties are, the types of players, and some of the strategy behind the game. Hell, this year, I've even learned about standings, and will sometimes check up on other teams to see what their chances of overtaking us are.

As a coder learns more he/she will want to know more. They'll want to know more syntax, or how to make their code more efficient. They'll look at new frameworks and languages. They'll become an evangelist. If all we ever make are cube-rats, all we'll ever get is loveless code that does barely what it's supposed to do, and never anything more.

And by the by... Go Caps! Southeast Division Champs!

Monday, April 6, 2009

Developers need hugs too

I'm about to get a little crazy here, so bear with me. I just had to let people in on this way out realization I had:

Developers are people, too.

Now, developers, I know that you already knew this. This post is not for you. Go read the archives of XKCD or something. Those emotionally close to developers probably expected the above, so you can probably skip this post, too.

Who, then, am I writing to?

I write to those that have to work with developers. Specifically, I'm speaking to those that have to deal with evaluating the work of a developer.

Let's say your significant other asks for a cake. Great, you think, I know how to make a cake. I have all the ingredients here. This shouldn't be any problem. So, you collect your flour and sugar and vanilla and milk and eggs-- Oh, wait. Eggs. Have to go get eggs.

So you go to the store to get eggs, and start cooking. Except you can't remember how much flour to use, and you can't remember where you put that one recipe from Southern Living back in 87. A few hours of searching later, you've found it, only to discover that you need sour cream as well. You debate substituting some old creamer in the back of your fridge, but, remembering you actually love your significant other, you go back to the store.

Now with everything in hand, you start cooking. An hour later, you're covered in flour and sugar and egg bits, but the cakes are in the oven. You start to clean up, but by now, you're dragging, having been running about for most the day for a stupid cake. You are considering an existence without cake. A nocaken?

The cake finishes, and you icing it, then present it proudly to your significant other. And he/she frowns.

"But I wanted chocolate."

Want to know how the analogy maps?

We get requests all the time that start with a basic need. "I need a way to share files through a web interface." Great! we think. We have a CMS that does that out of the box! This will be sooo easy.

Easy until we discover that the servers are all already taken, and we have to beg for another one. Once we get the server, we find out we're out of licenses. Once we have a license, we discover there's issues installing the proper decoders onto our flavor of *nix, and we have to find a special buildout recipe. Finally, we get everything to build consistently, and we hand over something we've been working our butt off on for weeks, if not months.

"But I wanted it to have a preview function."

Cake is so much more satisfying to throw at someone than code.

"But I hear devs critique each other ALL THE TIME!"

Well, yes. But see, we're not in the same role with each other. We're peers, and some amount of critique/pissing content is expected. We even invite it. If Alton Brown came in and said, "Darling, that cake should have been chocolate," I would have been rushing off for some cocoa to make everything alright for ALTON FREAKING BROWN. My husband just gets cake shoved somewhere improbable.

So, what to do?

Take a deep breath. Perhaps the requirements were misread. Perhaps a major part of the requirement was missed (more on how that happens later). Perhaps hallway conversations and after-meeting asides perverted the requirement over time. It happens. Humans wrote the requirements, humans passed them along, humans implemented the requirements, and humans QA'd them. That's a lot of space for error.

There's things you can do in the future to help, but for now, getting spun up over the difference between chocolate and vanilla is going to do nothing for anyone. You might win a quick victory by slamming on the coders, but in the end, you'll end up hurting your team more. Is it worth killing the team's momentum for that?

Do you really want to have them throw the cake in your face and walk out the door?

Don't stop everything due to the cake being the wrong flavor. Go ahead and eat it, and make a note to ask for chocolate next time.

Friday, April 3, 2009

Web Two point Buh

We all know what Web 2.0 is by now: separation of form and content. The current batch of CMS's out there do a great job of doing this: databases and templates are kept in nicely separate environments, allowing data to be reused without having to pull data out of heinously formatted HTML files.

Bravo. Good job, all. But we're not where we need to be yet.

After working on NASAScience for two years, I've come to the conclusion that we need more walls than just between form and content.

We need a wall between the developers and designers.

Theoretically, we have that. In practice, we're not even close. Many of the people who design CMS's have never had to sit in meetings with designers in the midst of a crazed fit of inspiration, or with a interface expert who's trying to nudge things to be just right. They want to make changes to the sites they have to manage. They want ultimate freedom.

They want the right to have a better idea.

After a week off from my usual job, I have to say, I have a bit more sympathy for them. I can see the appeal for just doing a site in Flash, though it still makes my soul hurt. Flash, they can control. They don't have to wait for developers to get through their most recent agile cycle to deal with their needs. Six weeks is short for developers. For a designer, that is nigh on forever.

Inspired and refreshed, I sketched out an idea for giving designers much more control over the visual aspects of their websites. Some features:

Lots of views

Instead of making tons of content types, make a few that cover many cases.


  1. Flash object pages

  2. Image pages

  3. Image galleries

  4. Text heavy pages

  5. Video pages



Note: there's a few more, but I'm on the train and can't pull out my notes, which are on 11x17 sheets of paper.

We shouldn't restrict where objects should be put. They should be able to put new page types wherever they need. Updating the structure of the website shouldn't have to wait for a developer to be free.

The secret to just a few content types is to have lots of views. Give the designer lots of options, and make it easy for them to update these views if they need, or add some new ones. Don't make them drudge through the ZMI. For ghu's sake, who wants designers going through the hell that's the ZMI? Or poking at ambiguously named admin functions. Do you want your site deleted?! I've almost had dev vets of 20+ years delete my site. What do you think a designer would do?

CSS on demand

Sometimes you need to tweak the CSS. Sometimes, you just want a bit of CSS to do something you need. Why should you have to wait for a new deployment? However, you don't want them adding CSS inline, as that's impossible to find later. Instead, give them a slot to add chunks of CSS. Then, at the end of your dev period, you can do a report that pulls up all the new CSS chunks and incorporate them into your master CSS.

Customize everything on demand

Everything should be up for customization. The header image. The navigation. The footer. Elements that should be on every page, but a use case has come up where a widget needs to go away. Let them do it. Let them make a special page with pink ponies in the middle of the site, if that's what they've discovered they need to do.

Parts making up the whole

Sometimes, you have things that, in the initial design, are 'whole'. Headers are a good example. Normally, there's a nice, long header image. Give the designer the option to break that whole thing into parts. In our case, I'm adding two more elements that they can plug images or html or whatever they want into it, adding visual interest as needed.

Designer friendly

Sure, sure, this is usually all very doable in most CMS's without having to make huge fuss about the interface. But think about what a designer would have to do in order to affect most of these changes. They'd have to troll through dozens of dangerous, application killing options that are all speaking geek to find the option that will let them do what they need. This should all be exposed on the page they want to edit. Want to modify a template? There you go. A button will take you the the right screen with a warning that this changes -all- templates of that type. Want a new template? There you go. New template, and I'll even add it to the view options for you. Have some CSS for me? Why look, I have a box for that right here.

Why go to the bother?

Yes, this looks like a lot of work, and I'm sure it will be. I'm sure my devs are going to HATE me when they get back into work today. But in the end, I think we all do our job better when we don't have to worry about waiting on other people. I love my designers, I really do. I think they're awesome, creative people. I'd just much rather be playing Zombie! and talking about video games with them, rather than arguing about why we can't make the site like they need it right now.