%global _empty_manifest_terminate_build 0 Name: python-os-log-merger Version: 1.2.1 Release: 1 Summary: OpenStack Log merge tool License: Apache Software License URL: https://github.com/openstack/os-log-merger Source0: https://mirrors.nju.edu.cn/pypi/web/packages/3f/0c/af89e514e39a40390e9ac721b7946e59fad6bbbf6ede756e884daced1ff9/os-log-merger-1.2.1.tar.gz BuildArch: noarch Requires: python3-pbr Requires: python3-dateutil Requires: python3-six %description What is os-log-merger? ~~~~~~~~~~~~~~~~~~~~~~ os-log-merger stands for OpenStack LOG merger, it's a tool designed to take a bunch of openstack logs across different projects, and merge them in a single file, ordered by time entries. It should work as long as the logs are based on oslo logger output. Quick presentation: http://mangelajo.github.io/openstack-debugging-presentation/ Limitations ~~~~~~~~~~~ This tool is not able to properly (or meaningfully) merge logs if your servers are not time synced to a common time source. How to install ~~~~~~~~~~~~~~ pip install os-log-merger Basic Usage ~~~~~~~~~~~ $ os-log-merger ../bz/1257567/40-os1ctrl01/var/log/neutron/server.log:NS1 \ Please note that the :NS1, :NS2, :OVS1, :OVS2 are aliases and can be omitted, and in such case, the extra column used to associate a log file to a log line will use the original file path instead of the [ALIAS] The previous example would produce something like this:: 2015-08-25 09:37:15.463 [NS1] 15062 DEBUG neutron.context [req-b751a750-f5d8-4b6e-9af3-82d143ef9416 None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:83 2015-08-25 09:37:15.463 [NS1] 15062 DEBUG neutron.plugins.ml2.db [req-b751a750-f5d8-4b6e-9af3-82d143ef9416 None] get_ports_and_sgs() called for port_ids [u'4136d577-e02f-47c1-b543-f0bfd65ef85e', u'5d5ea109-4807-4df3-bef4-b5d89c3ffebc', u'6adcffbf-09d5-4a85-9339-9d6beb2bf82c', u'6b4d7b51-c87d-483e-9606-0e2a54ad8184', u'743ccaa6-7ed9-4195-aabd-3d55006338e1', u'dc662767-61a5-4807-b2ed-a7c76b541fd6', u'4decdd33-6f13-46df-b2f0-d9ff99878514', u'34b826df-9787-443c-9bef-084374827a85', u'7bbc404b-3df7-498a-b6fb-e81f9370a19f', u'c12e6e06-ff6a-44dc-b75f-78ec55dd3dd3', u'586cd86d-59d0-434b-ab27-76975ce5abc4', u'79b33879-3232-4b3a-a27c-c0a79da10379', u'ba6a28cc-9851-4cd7-acae-40034a19c761', u'05c4115a-da58-41db-b3f7-7326e1a22971'] get_ports_and_sgs /usr/lib/python2.7/site-packages/neutron/plugins/ml2/db.py:224 2015-08-25 09:37:15.463 [OVS1] 12613 DEBUG neutron.agent.linux.utils [req-588c942a-6526-464f-a447-782a5e2d436a None] Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'ha-2cdba01d-e4\nha-44dca3a9-44\nha-499d3db7-97\nha-55a19f5e-ef\nha-b2d04f15-f2\nha-b5b271a1-d8\nha-fa58d644-81\nint-br-enp7s0\nint-br-ex\nqr-34b826df-97\nqr-5d5ea109-48\nqr-6adcffbf-09\nqr-743ccaa6-7e\nqr-79b33879-32\nqr-c12e6e06-ff\nqr-dc662767-61\n' References to http url files instead of local files is also supported. Files will be cached locally to avoid re-downloading on next runs. Common Base ~~~~~~~~~~~ In many cases we'll have a common base directory where log reside and they'll probably share the .log extension. So for the sake of brevity os-log-merger allows setting the base directory and postfix for all files with the `-b` and `-p` option (`--log-base` and `--log-postfix` long options). Example for Cinder: $ os-log-merger -b /var/log/cinder/ -p .log api:api scheduler:sch volume:vol /var/log/messages ~~~~~~~~~~~~~~~~~ os-log-merger also supports /var/log/messages type of files with options `-ml` and `--msg-logs` options. Since the format for those files is missing year information -MAR 24 14:11:19- the year from the last file modification will be used. These files can also be specified with globs and they support alias definition as well. Beware that openstack files should be listed before `-ml` option files. Example for Cinder: $ os-log-merger -b /var/log/ cinder/api.log:API -ml messages:MSG *.log Timestamped logs ~~~~~~~~~~~~~~~~ os-log-merger also supports timestamped -[ 0.003036]- with options `-tl` and `--timestamp-logs` options. Since timestamp many times will not take epoc time as the source of the timestamp but the time the system started, the initial datetime will be calculated by substracting from the file modified datetime the last timestamp in the file. These files can also be specified with globs and they support alias definition as well. Beware that openstack files should be listed before `-tl` option files. Example for Cinder: $ os-log-merger -b /var/log/ cinder/api.log:API -tl dmesg:DMSG Auto Alias ~~~~~~~~~~ As we've seen above you can easily set you alias using `:ALIAS` after each log file, but since most of log files names and locations are well known, os-log-merger has an auto alias feature with different levels to adapt to your specific needs. If an alias has been defined in the command line it will disable the auto alias on that file. **Level 0** The most basic auto alias generation level is level 0, and is the default behavior explained above, where the file path is used as an alias. **Level 1** Since default configuration will create considerable long aliases, you can use level 1 when using base directory and log postfix options to remove them from the alias. Then the following command line: $ os-log-merger -a1 -b /var/log/cinder/ -p .log api scheduler volume Would use `api`, `scheduler` and `volume` aliases:: 2016-02-01 12:11:17.573 [api] ... 2016-02-01 12:11:17.701 [scheduler] ... 2016-02-01 11:11:18.667 [volume] ... **Level 2** In some cases we may want to use globbing patterns and auto alias level 1 is no longer useful, so you want to have the filename extensions removed as well as the common paths and reduce the well know log filenames. With level 2 os-log-merger will remove all common parts of the path as long as resulting paths can still uniquely identify the files within the prefixing path. It will also rename well known files like cinder/scheduler.log with c-sch like in this example: $ os-log-merger -a2 node?/var/log/{cinder,nova}/*.log That will give you:: 2016-02-01 10:23:34.680 [node1/C-API] ... 2016-02-01 10:24:34.690 [node1/C-SCH] ... 2016-02-01 10:25:34.700 [node1/C-VOL] ... 2016-02-01 10:26:34.710 [node1/N-API] ... 2016-02-01 10:27:34.680 [node2/N-CPU] ... **Level 3** Depending on the name of your non common directories in your log paths you may want to go one step further and reduce them to the minimum instead of preserving them unaltered. Replacing Level 2 auto alias generation in the previous command with the same files: $ os-log-merger -a3 node?/var/log/{cinder,nova}/*.log Would result in:: 2016-02-01 10:23:34.680 [1/C-API] ... 2016-02-01 10:24:34.690 [1/C-SCH] ... 2016-02-01 10:25:34.700 [1/C-VOL] ... 2016-02-01 10:26:34.710 [1/N-API] ... 2016-02-01 10:27:34.680 [2/N-CPU] ... %package -n python3-os-log-merger Summary: OpenStack Log merge tool Provides: python-os-log-merger BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-os-log-merger What is os-log-merger? ~~~~~~~~~~~~~~~~~~~~~~ os-log-merger stands for OpenStack LOG merger, it's a tool designed to take a bunch of openstack logs across different projects, and merge them in a single file, ordered by time entries. It should work as long as the logs are based on oslo logger output. Quick presentation: http://mangelajo.github.io/openstack-debugging-presentation/ Limitations ~~~~~~~~~~~ This tool is not able to properly (or meaningfully) merge logs if your servers are not time synced to a common time source. How to install ~~~~~~~~~~~~~~ pip install os-log-merger Basic Usage ~~~~~~~~~~~ $ os-log-merger ../bz/1257567/40-os1ctrl01/var/log/neutron/server.log:NS1 \ Please note that the :NS1, :NS2, :OVS1, :OVS2 are aliases and can be omitted, and in such case, the extra column used to associate a log file to a log line will use the original file path instead of the [ALIAS] The previous example would produce something like this:: 2015-08-25 09:37:15.463 [NS1] 15062 DEBUG neutron.context [req-b751a750-f5d8-4b6e-9af3-82d143ef9416 None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:83 2015-08-25 09:37:15.463 [NS1] 15062 DEBUG neutron.plugins.ml2.db [req-b751a750-f5d8-4b6e-9af3-82d143ef9416 None] get_ports_and_sgs() called for port_ids [u'4136d577-e02f-47c1-b543-f0bfd65ef85e', u'5d5ea109-4807-4df3-bef4-b5d89c3ffebc', u'6adcffbf-09d5-4a85-9339-9d6beb2bf82c', u'6b4d7b51-c87d-483e-9606-0e2a54ad8184', u'743ccaa6-7ed9-4195-aabd-3d55006338e1', u'dc662767-61a5-4807-b2ed-a7c76b541fd6', u'4decdd33-6f13-46df-b2f0-d9ff99878514', u'34b826df-9787-443c-9bef-084374827a85', u'7bbc404b-3df7-498a-b6fb-e81f9370a19f', u'c12e6e06-ff6a-44dc-b75f-78ec55dd3dd3', u'586cd86d-59d0-434b-ab27-76975ce5abc4', u'79b33879-3232-4b3a-a27c-c0a79da10379', u'ba6a28cc-9851-4cd7-acae-40034a19c761', u'05c4115a-da58-41db-b3f7-7326e1a22971'] get_ports_and_sgs /usr/lib/python2.7/site-packages/neutron/plugins/ml2/db.py:224 2015-08-25 09:37:15.463 [OVS1] 12613 DEBUG neutron.agent.linux.utils [req-588c942a-6526-464f-a447-782a5e2d436a None] Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'ha-2cdba01d-e4\nha-44dca3a9-44\nha-499d3db7-97\nha-55a19f5e-ef\nha-b2d04f15-f2\nha-b5b271a1-d8\nha-fa58d644-81\nint-br-enp7s0\nint-br-ex\nqr-34b826df-97\nqr-5d5ea109-48\nqr-6adcffbf-09\nqr-743ccaa6-7e\nqr-79b33879-32\nqr-c12e6e06-ff\nqr-dc662767-61\n' References to http url files instead of local files is also supported. Files will be cached locally to avoid re-downloading on next runs. Common Base ~~~~~~~~~~~ In many cases we'll have a common base directory where log reside and they'll probably share the .log extension. So for the sake of brevity os-log-merger allows setting the base directory and postfix for all files with the `-b` and `-p` option (`--log-base` and `--log-postfix` long options). Example for Cinder: $ os-log-merger -b /var/log/cinder/ -p .log api:api scheduler:sch volume:vol /var/log/messages ~~~~~~~~~~~~~~~~~ os-log-merger also supports /var/log/messages type of files with options `-ml` and `--msg-logs` options. Since the format for those files is missing year information -MAR 24 14:11:19- the year from the last file modification will be used. These files can also be specified with globs and they support alias definition as well. Beware that openstack files should be listed before `-ml` option files. Example for Cinder: $ os-log-merger -b /var/log/ cinder/api.log:API -ml messages:MSG *.log Timestamped logs ~~~~~~~~~~~~~~~~ os-log-merger also supports timestamped -[ 0.003036]- with options `-tl` and `--timestamp-logs` options. Since timestamp many times will not take epoc time as the source of the timestamp but the time the system started, the initial datetime will be calculated by substracting from the file modified datetime the last timestamp in the file. These files can also be specified with globs and they support alias definition as well. Beware that openstack files should be listed before `-tl` option files. Example for Cinder: $ os-log-merger -b /var/log/ cinder/api.log:API -tl dmesg:DMSG Auto Alias ~~~~~~~~~~ As we've seen above you can easily set you alias using `:ALIAS` after each log file, but since most of log files names and locations are well known, os-log-merger has an auto alias feature with different levels to adapt to your specific needs. If an alias has been defined in the command line it will disable the auto alias on that file. **Level 0** The most basic auto alias generation level is level 0, and is the default behavior explained above, where the file path is used as an alias. **Level 1** Since default configuration will create considerable long aliases, you can use level 1 when using base directory and log postfix options to remove them from the alias. Then the following command line: $ os-log-merger -a1 -b /var/log/cinder/ -p .log api scheduler volume Would use `api`, `scheduler` and `volume` aliases:: 2016-02-01 12:11:17.573 [api] ... 2016-02-01 12:11:17.701 [scheduler] ... 2016-02-01 11:11:18.667 [volume] ... **Level 2** In some cases we may want to use globbing patterns and auto alias level 1 is no longer useful, so you want to have the filename extensions removed as well as the common paths and reduce the well know log filenames. With level 2 os-log-merger will remove all common parts of the path as long as resulting paths can still uniquely identify the files within the prefixing path. It will also rename well known files like cinder/scheduler.log with c-sch like in this example: $ os-log-merger -a2 node?/var/log/{cinder,nova}/*.log That will give you:: 2016-02-01 10:23:34.680 [node1/C-API] ... 2016-02-01 10:24:34.690 [node1/C-SCH] ... 2016-02-01 10:25:34.700 [node1/C-VOL] ... 2016-02-01 10:26:34.710 [node1/N-API] ... 2016-02-01 10:27:34.680 [node2/N-CPU] ... **Level 3** Depending on the name of your non common directories in your log paths you may want to go one step further and reduce them to the minimum instead of preserving them unaltered. Replacing Level 2 auto alias generation in the previous command with the same files: $ os-log-merger -a3 node?/var/log/{cinder,nova}/*.log Would result in:: 2016-02-01 10:23:34.680 [1/C-API] ... 2016-02-01 10:24:34.690 [1/C-SCH] ... 2016-02-01 10:25:34.700 [1/C-VOL] ... 2016-02-01 10:26:34.710 [1/N-API] ... 2016-02-01 10:27:34.680 [2/N-CPU] ... %package help Summary: Development documents and examples for os-log-merger Provides: python3-os-log-merger-doc %description help What is os-log-merger? ~~~~~~~~~~~~~~~~~~~~~~ os-log-merger stands for OpenStack LOG merger, it's a tool designed to take a bunch of openstack logs across different projects, and merge them in a single file, ordered by time entries. It should work as long as the logs are based on oslo logger output. Quick presentation: http://mangelajo.github.io/openstack-debugging-presentation/ Limitations ~~~~~~~~~~~ This tool is not able to properly (or meaningfully) merge logs if your servers are not time synced to a common time source. How to install ~~~~~~~~~~~~~~ pip install os-log-merger Basic Usage ~~~~~~~~~~~ $ os-log-merger ../bz/1257567/40-os1ctrl01/var/log/neutron/server.log:NS1 \ Please note that the :NS1, :NS2, :OVS1, :OVS2 are aliases and can be omitted, and in such case, the extra column used to associate a log file to a log line will use the original file path instead of the [ALIAS] The previous example would produce something like this:: 2015-08-25 09:37:15.463 [NS1] 15062 DEBUG neutron.context [req-b751a750-f5d8-4b6e-9af3-82d143ef9416 None] Arguments dropped when creating context: {u'project_name': None, u'tenant': None} __init__ /usr/lib/python2.7/site-packages/neutron/context.py:83 2015-08-25 09:37:15.463 [NS1] 15062 DEBUG neutron.plugins.ml2.db [req-b751a750-f5d8-4b6e-9af3-82d143ef9416 None] get_ports_and_sgs() called for port_ids [u'4136d577-e02f-47c1-b543-f0bfd65ef85e', u'5d5ea109-4807-4df3-bef4-b5d89c3ffebc', u'6adcffbf-09d5-4a85-9339-9d6beb2bf82c', u'6b4d7b51-c87d-483e-9606-0e2a54ad8184', u'743ccaa6-7ed9-4195-aabd-3d55006338e1', u'dc662767-61a5-4807-b2ed-a7c76b541fd6', u'4decdd33-6f13-46df-b2f0-d9ff99878514', u'34b826df-9787-443c-9bef-084374827a85', u'7bbc404b-3df7-498a-b6fb-e81f9370a19f', u'c12e6e06-ff6a-44dc-b75f-78ec55dd3dd3', u'586cd86d-59d0-434b-ab27-76975ce5abc4', u'79b33879-3232-4b3a-a27c-c0a79da10379', u'ba6a28cc-9851-4cd7-acae-40034a19c761', u'05c4115a-da58-41db-b3f7-7326e1a22971'] get_ports_and_sgs /usr/lib/python2.7/site-packages/neutron/plugins/ml2/db.py:224 2015-08-25 09:37:15.463 [OVS1] 12613 DEBUG neutron.agent.linux.utils [req-588c942a-6526-464f-a447-782a5e2d436a None] Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'ha-2cdba01d-e4\nha-44dca3a9-44\nha-499d3db7-97\nha-55a19f5e-ef\nha-b2d04f15-f2\nha-b5b271a1-d8\nha-fa58d644-81\nint-br-enp7s0\nint-br-ex\nqr-34b826df-97\nqr-5d5ea109-48\nqr-6adcffbf-09\nqr-743ccaa6-7e\nqr-79b33879-32\nqr-c12e6e06-ff\nqr-dc662767-61\n' References to http url files instead of local files is also supported. Files will be cached locally to avoid re-downloading on next runs. Common Base ~~~~~~~~~~~ In many cases we'll have a common base directory where log reside and they'll probably share the .log extension. So for the sake of brevity os-log-merger allows setting the base directory and postfix for all files with the `-b` and `-p` option (`--log-base` and `--log-postfix` long options). Example for Cinder: $ os-log-merger -b /var/log/cinder/ -p .log api:api scheduler:sch volume:vol /var/log/messages ~~~~~~~~~~~~~~~~~ os-log-merger also supports /var/log/messages type of files with options `-ml` and `--msg-logs` options. Since the format for those files is missing year information -MAR 24 14:11:19- the year from the last file modification will be used. These files can also be specified with globs and they support alias definition as well. Beware that openstack files should be listed before `-ml` option files. Example for Cinder: $ os-log-merger -b /var/log/ cinder/api.log:API -ml messages:MSG *.log Timestamped logs ~~~~~~~~~~~~~~~~ os-log-merger also supports timestamped -[ 0.003036]- with options `-tl` and `--timestamp-logs` options. Since timestamp many times will not take epoc time as the source of the timestamp but the time the system started, the initial datetime will be calculated by substracting from the file modified datetime the last timestamp in the file. These files can also be specified with globs and they support alias definition as well. Beware that openstack files should be listed before `-tl` option files. Example for Cinder: $ os-log-merger -b /var/log/ cinder/api.log:API -tl dmesg:DMSG Auto Alias ~~~~~~~~~~ As we've seen above you can easily set you alias using `:ALIAS` after each log file, but since most of log files names and locations are well known, os-log-merger has an auto alias feature with different levels to adapt to your specific needs. If an alias has been defined in the command line it will disable the auto alias on that file. **Level 0** The most basic auto alias generation level is level 0, and is the default behavior explained above, where the file path is used as an alias. **Level 1** Since default configuration will create considerable long aliases, you can use level 1 when using base directory and log postfix options to remove them from the alias. Then the following command line: $ os-log-merger -a1 -b /var/log/cinder/ -p .log api scheduler volume Would use `api`, `scheduler` and `volume` aliases:: 2016-02-01 12:11:17.573 [api] ... 2016-02-01 12:11:17.701 [scheduler] ... 2016-02-01 11:11:18.667 [volume] ... **Level 2** In some cases we may want to use globbing patterns and auto alias level 1 is no longer useful, so you want to have the filename extensions removed as well as the common paths and reduce the well know log filenames. With level 2 os-log-merger will remove all common parts of the path as long as resulting paths can still uniquely identify the files within the prefixing path. It will also rename well known files like cinder/scheduler.log with c-sch like in this example: $ os-log-merger -a2 node?/var/log/{cinder,nova}/*.log That will give you:: 2016-02-01 10:23:34.680 [node1/C-API] ... 2016-02-01 10:24:34.690 [node1/C-SCH] ... 2016-02-01 10:25:34.700 [node1/C-VOL] ... 2016-02-01 10:26:34.710 [node1/N-API] ... 2016-02-01 10:27:34.680 [node2/N-CPU] ... **Level 3** Depending on the name of your non common directories in your log paths you may want to go one step further and reduce them to the minimum instead of preserving them unaltered. Replacing Level 2 auto alias generation in the previous command with the same files: $ os-log-merger -a3 node?/var/log/{cinder,nova}/*.log Would result in:: 2016-02-01 10:23:34.680 [1/C-API] ... 2016-02-01 10:24:34.690 [1/C-SCH] ... 2016-02-01 10:25:34.700 [1/C-VOL] ... 2016-02-01 10:26:34.710 [1/N-API] ... 2016-02-01 10:27:34.680 [2/N-CPU] ... %prep %autosetup -n os-log-merger-1.2.1 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-os-log-merger -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Wed May 10 2023 Python_Bot - 1.2.1-1 - Package Spec generated