Ceasar's Mind

Follow me: @Ceasar_Bautista

How to use default arguments with namedtuple

with 8 comments

I was running into some trouble earlier today trying to figure out how to use default argument with namedtuple.

According to the docs:

Default values can be implemented by using _replace() to customize a prototype instance:

>>> Account = namedtuple('Account', 'owner balance transaction_count')
>>> default_account = Account('<owner name>', 0.0, 0)
>>> johns_account = default_account._replace(owner='John')

This is of course not really what I was looking for.

Searching around some more led to the Python mailing list where a software engineer by the name of Issac Morland suggests exactly what I was looking for.

2. It would be nice to be able to have default values for named tuple fields.
Using Signature it’s easy to do this – I just specify a dictionary of defaults
at named tuple class creation time.

To which Raymond Hettinger (the creator of collections) responds:

This came-up before, but I’ll take another look at it this weekend. If it
significantly complicates the API or impacts performance, then it is a
non-starter; otherwise, I’ll put together an implementation and float it on
ASPN for comments.

And the trail ends. Not sure what the full story is.

In any case, the solution is fairly simple. Just subclass the result of namedtuple and override __new__ as follows:

from collections import namedtuple
class Move(namedtuple('Move', 'piece start to captured promotion')):
    def __new__(cls, piece, start, to, captured=None, promotion=None):
        # add default values
        return super(Move, cls).__new__(cls, piece, start, to, captured, promotion)

As you’ll notice, super is being used a little weird here, with cls being passed to both super and __new__. I’m not quite sure why that’s neccessary, but I can say I’ve tested it and it does work.

EDIT: It appears this question on SO explains the weirdness with using super together with __new__. tldr; __new__ is a static method, not a class method, so it requires an explicit passing of cls.

Written by Ceasar Bautista

2012/03/19 at 15:30

Posted in Uncategorized

Tagged with

8 Responses

Subscribe to comments with RSS.

  1. Have you sent these over to Isaac Morland and Raymond Hettinger?
    Might at least be worth mentioning on the thread you saw their conversation on, if possible.
    Seems like pretty cool stuff, although I have to admit it’s over my head

    Alex Rattray

    2012/03/19 at 18:42

  2. Thanks, this helped me.

    TimothyAWiseman

    2013/03/14 at 17:33

  3. Did you ever verify that using this technique retains the RAM-usage benefits of using a namedtuple? This looks promising, but I don’t understand enough about the way they work in order to evaluate it without testing.

    Wes Winham

    2014/03/14 at 13:12

    • I just did my own quick test. It looks like it does retain at least most of the benefits! Awesome. The strange thing, is that at least for python 2.6.5 on my AMD64 linux machine, a class with __slots__ was actually more memory efficient.

      This was using a namedtuple with 5 attributes, and each attribute set to the value 100. I measured the RAM usage via `ps`, and these are the VSZ and RSS values.

      ## Using the original namedtuple

      ### Before 1 million objects created

      272228 61076

      ### After

      413568 201748

      ## Using a subclass to add default arguments

      ### Before

      272264 61064

      ### After

      419736 207692

      ## Using dictionaries with attribute names

      ### Before

      271428 60212

      ### After

      593596 373732

      ## Using a normal object without __slots__ defined

      ### After

      650028 438000

      ## Using a normal object with __slots__ defined

      ### After

      394932 182864

      Wes Winham

      2014/03/14 at 13:40

    • I settled on just using a stand-alone method to provide the defaults. In my case _PointStat is the namedtuple and PointStat is a method that creates a _PointStat, using defaults. That resulted in the exact same memory usage as using the namedtuple directly.

      Wes Winham

      2014/03/14 at 13:47

  4. Nice solution to the defaults issue. The main downside to using namedtuples this way is it isn’t very DRY — the parameter names are repeated in three places.

    Chris

    2014/04/03 at 09:05

  5. Why not just use a class?

    fox7@gmail.com

    2017/05/03 at 07:42

  6. 偽物 激安 服 ブランド長袖luxurybrandsale2019
    シュプリーム 激安 サイト


Leave a reply to Chris Cancel reply