Debugging Python with PDB

I suck at using debuggers, largely because I don’t launch them often enough.

Fortunately, this year’s PyCon had a great talk from Chris McDonough on how to get started with Python’s debugger, PDB. The friendly people at Stockholm’s Python meet-up suggested it.

I made notes while watching, to remove excuses from future me on launching pdb.

Where you want to have the debugger start, add the following code:

import pdb; pdb.set_trace()

This drops you into a PDB prompt. This is more helpful than putting in yet more
print statements.

Some helpful commands once you’re in:

  • l - list code
  • args - arguments to current function
  • p <var> - print a var
  • pp <var> - pretty print a var
  • n - next
  • s - step
  • w - where (stack trace for current position)
  • h - help

While in PDB, you can evaluate python code. The evaluated code won’t impact the running program.

For a bonus trick if you’re using Emacs, try out pdb track mode. Launch your python process to be debugged in an emacs shell (M-x shell). Stepping through the code in pdb will track with a source code buffer.

This worked out of the box with my emacs config. Your milage may vary.

I may actually start to enjoy debugging now!

Unit testing in Python with folder watching

Unit testing is good. Running unit tests manually is annoying.

Rspec has a great addition in autospec, which automatically re-runs specs that are changed or have the associated code changed. This model of working has a long history in the ruby community, with autotest.

I wanted a similar thing for python/nose, but hadn’t had much luck.

The solution is py.test, with some plugins added.

This was suggested to me at the Stockholm Python user group, and I am very grateful. My testing life is now much simpler.

To get started, try something like:

$ sudo pip install pytest pytest-xdist pytest-cov

The simple way to get started is to watch your tests folder:

py.test -f test/

Now as you change files, the relevant tests will automatically be re-run. Perfect to display on that second monitor.

As I was working on some legacy code, I was looking to improve the test coverage, so wanted to see how that was going with pytest-cov:

py.test -f --cov package.name test/

Adding the package.name for your main package means you won’t generate coverage for libraries you are using. Makes the output simpler.

Still not quite happy, it wasn’t showing me which lines I needed coverage on. No problem:

py.test -f --cov package.name --cov-report term-missing test/

For comparison, here is how to get similar results from nose, but without watch support:

nosetests test/
nosetests --with-coverage --cover-erase --cover-package=package.name test/

One hurdle I had with py.test was working with Twisted based test-cases. You need to ensure that you use a version of py.test later than 2.0. Debian squeeze does not package this by default, so use pip to install.

The annoyance is if you change the code so it doesn’t terminate. This still needs a context switch to kill off the tests and re-run. Keeps me focused on getting it right the first time.

pytest-xdist includes lots of really cool options, such as distribution of tests across multiple cores, versions, hosts and platforms.

I’ve since discovered this comprehensive summary of the options available. I’m happy I found this after selecting py.test, or I’d still be stuck in evaluation mode rather than writing code.

Update: check out sniffer if you want a more generic watcher framework.

Sharpen the Saw: Typing

I once explained my job as an overpaid typist.

There is some truth in that, as much of my day is spent typing. Writing code, taking notes for meetings, email, documentation, writing blogs posts.

It would make sense to become very good at typing.

The basics are often overlooked. You can get a lot of benefit from paying attention to them.

For typing, it comes down to three main areas:

  1. Accuracy
  2. Speed
  3. Touch typing

Touch typing is perhaps the most important aspect of this. You need to be able to type without looking at the keyboard. If you get really good, you shouldn’t need to even look at the screen to confirm what you’re typing.

Accuracy and speed can be built together. Focus on accuracy to start with, and then stretch for speed. There is no point going faster if you can’t maintain a high accuracy.

I found two websites to be helpful in my quest for improved typing skills:

  • Keyzen — a site focused on improving typing for programmers.
  • 10 fast fingers — general typing improvement with natural language exercises.

Of course, the other thing to do to improve typing is to do lots of typing. If you are writing lots of code, email or documents, then focus on your typing while you are doing it.

It helps to have a good environment for typing, such as a comfortable keyboard, good posture, but you know this already. Add it to the list of things to focus on.

Also, make sure that you are using a program that can keep up with you as you type. I find MS Word to lag when in full WYSIWG mode, whereas emacs and vi always manage to keep up!

Update: You may also want to check out Typespeed.

About Face, Edition 3

In any field, if you expand your view to know all the ecosystem around you it’s beneficial.

Davide Casali

If you are a developer, a product owner, or manage a development team, much of the input is going to be about product design. It is very useful to understand a bit about a good design process, so you can better ask questions and help refine the design based on technical constraints.

About Face, by Alan Cooper, is a great book to start with.

I highlighted much of it on my Kindle!

Here are some quotes:

Good design makes users more effective.

Customers, although they might be able to articulate the problems with an interaction, are not often capable of visualizing the solutions to those problems.

One of the most powerful tools designers bring to the table is empathy

One of the most dangerous practices in product development is isolating designers from the users because doing so eliminates empathic knowledge.

narrative is also one of our most powerful creative methods.

What would a helpful human do? What would a thoughtful, considerate interaction feel like?

All ideas are broken down into discrete sections, with lots of detail and examples on how to implement this with your customers.

Goal-Directed Design combines techniques of ethnography, stakeholder interviews, market research, detailed user models, scenario-based design, and a core set of interaction principles and patterns.

This process can be roughly divided into six phases: Research, Modeling, Requirements Definition, Framework Definition, Refinement, and Support

The book discusses the mindset required to be a good designer:

in anything at all, perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away

Next time you find yourself crowing about what cool interaction you’ve designed, just remember that the ultimate user interface for most purposes is no interface at all.

The first part of the book focuses on processes to elicit a good design. The latter part of the book covers specific examples found in user interface design and attempts to encode common patterns.

I found the first section much more valuable to how I think about designing a new product. The latter parts I felt were obvious, although that is perhaps their importance.

Well worth a read.

Note: Bret’s Magic Ink is a useful counterpoint to About Face.

Positioning

Positioning, by Al Ries and Jack Trout, has long been on my to read list, and I wish I had read it sooner.

The core message of how to position a product is becoming more relevant as the marketplace continues to crowd.

In our overcommunicated society, the name of the game today is positioning. And only the better players are going to survive.

The book draws on a wide range of examples of different products, where a product could be a type of beer, the Catholic Church or your own career.

The key message is that your products position is determined by how it fits into the consumer’s mind. All the things that matter to you, to your company, are irrelevant if you can’t create a distinct position to separate you from everything else.

Experience has shown that a positioning exercise is a search for the obvious. Those are the easiest concepts to communicate because they make the most sense to the recipient of a message.

The authors are quite scathing about “me too” products that attempt to beat out a competitor by going head-to-head. Many examples are provided within the book.

The suicidal bent of companies that go head-on against established competition is hard to understand.

To repeat, the first rule of positioning is: To win the battle for the mind, you can’t compete head-on against a company that has a strong, established position. You can go around, under or over, but never head to head.

The leader owns the high ground. The No. 1 position in the prospect’s mind. The top rung of the product ladder.

The book covers a range of detail levels, and dips into more detail on key areas such as product naming:

As a guide, the five most common initial letters are S, C, P, A, and T. The five least common are X, Z, Y, Q, and K. One out of eight English words starts with an S. One out of 3000 starts with an X.

The authors are often quite blunt as to how they see things. This is a refreshing change from most business books:

Creative people often resist positioning thinking because they believe it restricts their creativity. And you know what? It does. Positioning thinking does restrict creativity.

I gained a lot of insight into how products have succeeded, or not, by reading this book. The examples are clear and will easily map to situations you are experiencing.

I recommend Positioning for anyone involved in creating something new. This is a book that I will re-read.