diff options
Diffstat (limited to 'python-mem-top.spec')
| -rw-r--r-- | python-mem-top.spec | 327 |
1 files changed, 327 insertions, 0 deletions
diff --git a/python-mem-top.spec b/python-mem-top.spec new file mode 100644 index 0000000..917cc8a --- /dev/null +++ b/python-mem-top.spec @@ -0,0 +1,327 @@ +%global _empty_manifest_terminate_build 0 +Name: python-mem-top +Version: 0.2.1 +Release: 1 +Summary: Shows top suspects for memory leaks in your Python program. +License: MIT +URL: https://github.com/denis-ryzhkov/mem_top +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/c3/f0/8b6cc0ff682dee2760bfeb045fb4c0188a6040aee36f6c1e0e4568d4609e/mem_top-0.2.1.tar.gz +BuildArch: noarch + + +%description + +Usage:: + + pip install mem_top + from mem_top import mem_top + + # From time to time: + logging.debug(mem_top()) + # print(mem_top()) + + # Notice which counters keep increasing over time - they are the suspects. + +Counters: + +"mem_top" iterates all objects found in memory and calculates: + +* refs - number of direct references from this object to other objects, like keys and values of dict + + * E.g. a dict {("some", "complex", "key"): "value"} will have "refs: 2" - 1 ref for key, 1 ref for value + * Its key ("some", "complex", "key") will have "refs: 3" - 1 ref per item + +* bytes - size of this object in bytes +* types - number of objects of this type still kept in memory after garbage collection + +Real life example:: + + refs: + 144997 <type 'collections.defaultdict'> defaultdict(<type 'collections.deque'>, {<GearmanJobRequest task='...', unique='. + 144996 <type 'dict'> {'.:..............:.......': <GearmanJobRequest task='..................', unique='................. + 18948 <type 'dict'> {... + 1578 <type 'dict'> {... + 968 <type 'dict'> {... + 968 <type 'dict'> {... + 968 <type 'dict'> {... + 767 <type 'list'> [... + 726 <type 'dict'> {... + 608 <type 'dict'> {... + + types: + 292499 <type 'dict'> + 217912 <type 'collections.deque'> + 72702 <class 'gearman.job.GearmanJob'> + 72702 <class 'gearman.job.GearmanJobRequest'> + 12340 <type '... + 3103 <type '... + 1112 <type '... + 855 <type '... + 767 <type '... + 532 <type '... + +* Noticed a leak of 6GB RAM and counting. +* Added "mem_top" and let it run for a while. +* When got the result above it became absolutely clear who is leaking here: + the Python client of Gearman kept increasing its counters over time. +* Found its known bug - https://github.com/Yelp/python-gearman/issues/10 + leaking defaultdict of deques, and a dict of GearmanJobRequest-s, + just as the "mem_top" showed. +* Replaced "python-gearman" - long story: stale 2.0.2 at PyPI, broken 2.0.X at github, etc. +* "mem_top" confirmed the leak is now completely closed. + +Updates: + +* Pass e.g. "verbose_types=[dict, list]" to store their values, sorted by "repr" length, in "verbose_file_name". +* Added "bytes" top. + +Config defaults:: + + mem_top( + limit=10, # limit of top lines per section + width=100, # width of each line in chars + sep='\n', # char to separate lines with + refs_format='{num}\t{type} {obj}', # format of line in "refs" section + bytes_format='{num}\t {obj}', # format of line in "bytes" section + types_format='{num}\t {obj}', # format of line in "types" section + verbose_types=None, # list of types to sort values by `repr` length + verbose_file_name='/tmp/mem_top', # name of file to store verbose values in + ) + +See also: + +* https://docs.python.org/2/library/gc.html#gc.garbage +* https://pypi.python.org/pypi/objgraph + + + + + +%package -n python3-mem-top +Summary: Shows top suspects for memory leaks in your Python program. +Provides: python-mem-top +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-mem-top + +Usage:: + + pip install mem_top + from mem_top import mem_top + + # From time to time: + logging.debug(mem_top()) + # print(mem_top()) + + # Notice which counters keep increasing over time - they are the suspects. + +Counters: + +"mem_top" iterates all objects found in memory and calculates: + +* refs - number of direct references from this object to other objects, like keys and values of dict + + * E.g. a dict {("some", "complex", "key"): "value"} will have "refs: 2" - 1 ref for key, 1 ref for value + * Its key ("some", "complex", "key") will have "refs: 3" - 1 ref per item + +* bytes - size of this object in bytes +* types - number of objects of this type still kept in memory after garbage collection + +Real life example:: + + refs: + 144997 <type 'collections.defaultdict'> defaultdict(<type 'collections.deque'>, {<GearmanJobRequest task='...', unique='. + 144996 <type 'dict'> {'.:..............:.......': <GearmanJobRequest task='..................', unique='................. + 18948 <type 'dict'> {... + 1578 <type 'dict'> {... + 968 <type 'dict'> {... + 968 <type 'dict'> {... + 968 <type 'dict'> {... + 767 <type 'list'> [... + 726 <type 'dict'> {... + 608 <type 'dict'> {... + + types: + 292499 <type 'dict'> + 217912 <type 'collections.deque'> + 72702 <class 'gearman.job.GearmanJob'> + 72702 <class 'gearman.job.GearmanJobRequest'> + 12340 <type '... + 3103 <type '... + 1112 <type '... + 855 <type '... + 767 <type '... + 532 <type '... + +* Noticed a leak of 6GB RAM and counting. +* Added "mem_top" and let it run for a while. +* When got the result above it became absolutely clear who is leaking here: + the Python client of Gearman kept increasing its counters over time. +* Found its known bug - https://github.com/Yelp/python-gearman/issues/10 + leaking defaultdict of deques, and a dict of GearmanJobRequest-s, + just as the "mem_top" showed. +* Replaced "python-gearman" - long story: stale 2.0.2 at PyPI, broken 2.0.X at github, etc. +* "mem_top" confirmed the leak is now completely closed. + +Updates: + +* Pass e.g. "verbose_types=[dict, list]" to store their values, sorted by "repr" length, in "verbose_file_name". +* Added "bytes" top. + +Config defaults:: + + mem_top( + limit=10, # limit of top lines per section + width=100, # width of each line in chars + sep='\n', # char to separate lines with + refs_format='{num}\t{type} {obj}', # format of line in "refs" section + bytes_format='{num}\t {obj}', # format of line in "bytes" section + types_format='{num}\t {obj}', # format of line in "types" section + verbose_types=None, # list of types to sort values by `repr` length + verbose_file_name='/tmp/mem_top', # name of file to store verbose values in + ) + +See also: + +* https://docs.python.org/2/library/gc.html#gc.garbage +* https://pypi.python.org/pypi/objgraph + + + + + +%package help +Summary: Development documents and examples for mem-top +Provides: python3-mem-top-doc +%description help + +Usage:: + + pip install mem_top + from mem_top import mem_top + + # From time to time: + logging.debug(mem_top()) + # print(mem_top()) + + # Notice which counters keep increasing over time - they are the suspects. + +Counters: + +"mem_top" iterates all objects found in memory and calculates: + +* refs - number of direct references from this object to other objects, like keys and values of dict + + * E.g. a dict {("some", "complex", "key"): "value"} will have "refs: 2" - 1 ref for key, 1 ref for value + * Its key ("some", "complex", "key") will have "refs: 3" - 1 ref per item + +* bytes - size of this object in bytes +* types - number of objects of this type still kept in memory after garbage collection + +Real life example:: + + refs: + 144997 <type 'collections.defaultdict'> defaultdict(<type 'collections.deque'>, {<GearmanJobRequest task='...', unique='. + 144996 <type 'dict'> {'.:..............:.......': <GearmanJobRequest task='..................', unique='................. + 18948 <type 'dict'> {... + 1578 <type 'dict'> {... + 968 <type 'dict'> {... + 968 <type 'dict'> {... + 968 <type 'dict'> {... + 767 <type 'list'> [... + 726 <type 'dict'> {... + 608 <type 'dict'> {... + + types: + 292499 <type 'dict'> + 217912 <type 'collections.deque'> + 72702 <class 'gearman.job.GearmanJob'> + 72702 <class 'gearman.job.GearmanJobRequest'> + 12340 <type '... + 3103 <type '... + 1112 <type '... + 855 <type '... + 767 <type '... + 532 <type '... + +* Noticed a leak of 6GB RAM and counting. +* Added "mem_top" and let it run for a while. +* When got the result above it became absolutely clear who is leaking here: + the Python client of Gearman kept increasing its counters over time. +* Found its known bug - https://github.com/Yelp/python-gearman/issues/10 + leaking defaultdict of deques, and a dict of GearmanJobRequest-s, + just as the "mem_top" showed. +* Replaced "python-gearman" - long story: stale 2.0.2 at PyPI, broken 2.0.X at github, etc. +* "mem_top" confirmed the leak is now completely closed. + +Updates: + +* Pass e.g. "verbose_types=[dict, list]" to store their values, sorted by "repr" length, in "verbose_file_name". +* Added "bytes" top. + +Config defaults:: + + mem_top( + limit=10, # limit of top lines per section + width=100, # width of each line in chars + sep='\n', # char to separate lines with + refs_format='{num}\t{type} {obj}', # format of line in "refs" section + bytes_format='{num}\t {obj}', # format of line in "bytes" section + types_format='{num}\t {obj}', # format of line in "types" section + verbose_types=None, # list of types to sort values by `repr` length + verbose_file_name='/tmp/mem_top', # name of file to store verbose values in + ) + +See also: + +* https://docs.python.org/2/library/gc.html#gc.garbage +* https://pypi.python.org/pypi/objgraph + + + + + +%prep +%autosetup -n mem-top-0.2.1 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-mem-top -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Fri May 05 2023 Python_Bot <Python_Bot@openeuler.org> - 0.2.1-1 +- Package Spec generated |