Comments on Valued Lessons: Python Memory Usage: What values are taking up so ...

I am getting wildly bigger values. Try this: #!/b...

2010-07-30T07:21:08.276-07:00

I am getting wildly bigger values. Try this:

#!/bin/sh
(
exec >test.py
echo "\
#!/usr/bin/python
import time"
i=300000
while test $i != 0; do
echo "i$i=$i"
: $((i--))
done
echo "time.sleep(3)"
)

chmod 755 test.py
echo "Before: `grep ^MemFree: /proc/meminfo`"
./test.py &
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
sleep 1
echo "During: `grep ^MemFree: /proc/meminfo`"
wait
echo "After: `grep ^MemFree: /proc/meminfo`"

# i=100000, output on my machine (x86-64):
#Before: MemFree: 308688 kB
#During: MemFree: 252144 kB
#During: MemFree: 227460 kB
#During: MemFree: 227468 kB
#During: MemFree: 227460 kB
#After: MemFree: 308200 kB
# Thus, (308200-227460)/100 = 807 bytes per each int variable

# i=300000
#Before: MemFree: 1007572 kB
#During: MemFree: 695952 kB
#During: MemFree: 421548 kB
#During: MemFree: 851084 kB
#During: MemFree: 835708 kB
#During: MemFree: 808800 kB
#During: MemFree: 795656 kB
#During: MemFree: 795656 kB
#During: MemFree: 795648 kB
#During: MemFree: 795656 kB
#After: MemFree: 1007448 kB
# Thus, (1007448-795648)/300 = 706 bytes per each int variable

Oops, I should have read the preceding comments be...

2009-09-02T07:27:18.776-07:00

Oops, I should have read the preceding comments before posting mine! Sorry.

It would be interesting to see the exact same test...

2009-09-02T05:25:55.701-07:00

It would be interesting to see the exact same test run with a new-style class with __slots__ to eliminate the __dict__ attribute. According to your chart, the instance takes up 345 bytes, but the __dict__ should be taking up 298 of that; that that is eliminated when you use __slots__.

Wow Gary! I tried that too and it came to about a ...

2009-09-02T04:11:51.468-07:00

Wow Gary! I tried that too and it came to about a quarter the size of virtual memory. This is way over my head but clearly indicates that __slots__ might be the way forward when memory matters.

Fun - I was just playing with the same sort of thi...

2008-11-21T11:42:00.000-08:00

Fun - I was just playing with the same sort of thing and came to a similar conclusion. One item that's useful is a slotted new style object:

class Foo(object):
  __slots__ = []

These come out to be (on average) slightly greater than 16 bytes / each.

If you add values:

class Foo(object):
  __slots__ = ['a','b']
  def __init__(self):
    self.a = None
    self.b = None

They come out to be ~65 bytes each.