emlop 0.3.0

A fast, accurate, ergonomic emerge.log parser
emlop-0.3.0 is not a library.

EMerge LOg Parser

Emlop parses emerge logs (as generated by portage, the Gentoo package manager) to yield useful info like merge history and merge time prediction.

It is heavily inspired by genlop but aims to be faster, more accurate, and more ergonomic. Other rewrites of Genlop exist, see COMPARISON.

Installation

Using portage

Not available in main portage tree yet, see gentoo bug 649904. In the meantime, get it from the moltonel overlay : install layman and run layman -a moltonel before running emerge emlop as ususal.

Using cargo

Install Rust using emerge rust (if you simply want to install Emlop) or rustup (if you want the latest Rust or plan to develop in Rust). Emlop releases should always work with portage's stable rust version.

Cargo installs binaries into ~/.cargo/bin/, which should be in your $PATH. If you wish to install them system-wide, edit the system $PATH or copy/symlink ~/.cargo/bin/* somewhere in $PATH.

From crates.io

cargo install -f emlop

From git

git clone https://github.com/vincentdephily/emlop
cd emlop
cargo test
cargo install -f

Usage

Emlop is split into log, predict and stats subcommands, which can be abbreviated by their first letter. This file doesn't show everything, see emlop --help and emlop <sucommand> --help for complete and up to date usage info.

Subcommands and arguments

Show log of sucessful merges and syncs:

emlop log [OPTIONS] [package]
    <package>                 Display only packages matching <package>.
    -s, --show <m,s,a>        Show (m)erges, (s)yncs, and/or (a)ll. [default: m]
    -e, --exact               Match package with a string instead of a regex.

Predict merge time for current or pretended merges:

emlop predict [OPTIONS]
    --limit <limit>   Use the last N merge times to predict next merge time. [default: 10]

Show statistics about sucessful merges and syncs:

emlop stats [OPTIONS] [package]
    <package>                  Show only packages matching <package>.
    -s, --show <m,t,s,a>       Show (m)erges, (t)otals, (s)yncs, and/or (a)ll. [default: m]
    -g, --groupby <y,m,w,d>    Group by (y)ear, (m)onth, (w)eek, or (d)ay.
    -e, --exact                Match package with a string instead of a regex.
        --limit <limit>        Use the last N merge times to predict next merge time. [default: 10]

Options common to all subcommands:

    --from <date>         Only parse log entries after <date>.
    --to <date>           Only parse log entries before <date>.
    --duration <hms,s>    Format durations in hours:minutes:seconds or in seconds. [default: hms]
-f, --logfile <file>      Location of emerge log file. [default: /var/log/emerge.log]
-v                        Show warnings (-v), info (-vv) and debug (-vvv) messages (errors are always displayed).
    --color <when>        Enable color (auto/always/never/y/n). [default: auto]
-h, --help                Show short (-h) or detailed (--help) help.

Examples

Show merge log with date, time, and package name:

$ emlop log | tail
2018-01-29 10:20:52 +00:00        13 net-wireless/iw-4.9
2018-01-29 10:21:21 +00:00        29 dev-libs/librdkafka-0.11.3
2018-01-29 10:22:27 +00:00      1:06 net-misc/curl-7.58.0
2018-01-29 11:09:20 +00:00      1:23 media-libs/openexr-2.2.0-r2
2018-01-29 11:12:18 +00:00      2:58 media-gfx/imagemagick-7.0.7.19
2018-01-29 11:12:42 +00:00        24 kde-frameworks/kimageformats-5.42.0
2018-01-29 11:25:32 +00:00     12:50 media-gfx/inkscape-0.92.2
2018-01-29 12:36:52 +00:00   1:11:20 dev-lang/rust-1.23.0-r1
2018-01-29 12:37:08 +00:00        16 virtual/rust-1.23.0
2018-01-29 12:41:54 +00:00      4:46 dev-util/cargo-0.24.0

Show same merge log but filter packages by regexp:

$ emlop l gcc | tail
2017-10-04 18:43:31 +01:00         8 sys-devel/gcc-config-1.8-r1
2017-10-16 13:54:34 +01:00        11 sys-devel/gcc-config-1.8-r1
2017-10-16 20:00:23 +01:00   1:51:55 sys-devel/gcc-5.4.0-r3
2017-10-19 11:57:21 +01:00        36 sys-devel/gcc-config-1.8-r1
2017-11-07 13:06:47 +00:00   1:56:37 sys-devel/gcc-6.4.0
2017-11-20 12:18:58 +00:00   2:24:20 sys-devel/gcc-6.4.0
2017-11-20 13:24:59 +00:00        46 sys-devel/gcc-config-1.8-r1
2017-12-04 18:12:03 +00:00      1:39 sys-devel/gcc-config-1.8-r1
2017-12-05 12:49:27 +00:00   2:59:33 sys-devel/gcc-6.4.0
2018-01-12 12:49:17 +00:00   1:48:28 sys-devel/gcc-6.4.0-r1

Show syncs of the last 7 days:

$ emlop l --from '1 week ago' -ss
2018-11-28 21:53:42 +00:00        13 Sync
2018-11-30 09:18:43 +00:00         7 Sync
2018-12-01 17:48:37 +00:00         3 Sync
2018-12-03 09:30:02 +00:00        11 Sync
2018-12-04 09:52:12 +00:00         8 Sync
2018-12-04 17:01:06 +00:00         8 Sync
2018-12-05 09:43:17 +00:00         4 Sync

Show currently emerging packages, how long they have been running, and predict how long is left:

$ emlop p
Pid 27455: ...n-exec/python3.5/emerge -O chromium         33
Pid 27848: ...on-exec/python3.5/emerge -O firefox         29
www-client/firefox-58.0.1                              53:37 - 24
www-client/chromium-65.0.3325.146                    6:01:02 - 28
Estimate for 2 ebuilds (0 unknown, 52 elapsed)       6:53:47

Predict merge time from an emerge --pretend output, taking currently elapsed time into account:

$ emerge -rOp | emlop p
Pid 8799: .../emerge -O chromium firefox konqueror   1:14:11
www-client/chromium-65.0.3325.146                    5:49:38 - 1:10:55
www-client/firefox-58.0.1                              53:37
kde-apps/konqueror-17.12.3                              3:46
Estimate for 3 ebuilds (0 unknown, 1:10:55 elapsed)  5:36:06

Show total merge time, merge count, and average merge time:

$ emlop s gtk
app-admin/gtkdiskfree                1:19    1      1:19
net-libs/webkit-gtk              63:17:43   44   1:57:33
x11-libs/gtk+                     3:10:20   40      4:21
x11-themes/gtk-engines-adwaita       1:23    4        20
dev-util/gtk-doc                     4:46    9        31
dev-python/pygtk                    16:05    7      2:17
dev-util/gtk-doc-am                  3:43   19        11
x11-libs/gtksourceview               4:54    6        49
dev-python/pygtksourceview           2:27    6        24
dev-perl/gtk2-ex-formfactory         2:29   10        14
dev-util/gtk-update-icon-cache       5:44   16        23
dev-cpp/gtkmm                       39:33   12      3:40
dev-python/pywebkitgtk                 13    1        13
dev-perl/gtk2-perl                  12:49    8      1:36

Show monthly merge stats for this year:

$ emlop s -gm -st --from '1 year ago'
2017-12 Merge    53:49:26    915      3:31
2018-01 Merge    42:15:12    531      4:46
2018-02 Merge    44:36:38    539      4:57
2018-03 Merge    25:36:50    527      2:54
2018-04 Merge    67:32:12   2114      1:55
2018-05 Merge    23:53:21    502      2:51
2018-06 Merge    35:16:01    442      4:47
2018-07 Merge    31:45:27    452      4:12
2018-08 Merge    28:36:45    260      6:36
2018-09 Merge    28:36:53    386      4:26
2018-10 Merge    31:15:24    515      3:38
2018-11 Merge    21:12:51    422      3:00
2018-12 Merge    12:18:01     42     17:34

Show yearly evolution of a package's average merge time:

$ emlop s -gy chromium
2013 www-client/chromium       33:37      1     33:37
2014 www-client/chromium    18:11:23     21   1:04:35
2015 www-client/chromium    41:52:19     20   2:46:10
2016 www-client/chromium    92:21:29     21   5:21:21
2017 www-client/chromium   125:26:42     22   5:20:28
2018 www-client/chromium   106:06:47     19   5:37:57

Show number of syncs per week:

$ emlop s -gw -ss | tail
2018-40 Sync        1:35      8        11
2018-41 Sync        1:22     10         8
2018-42 Sync        1:26     11         7
2018-43 Sync        1:02      9         6
2018-44 Sync        1:23     13         6
2018-45 Sync        1:06      9         7
2018-46 Sync        1:27     11         7
2018-47 Sync        1:14      8         9
2018-48 Sync          31      4         7
2018-49 Sync          31      4         7

Contributing

Thank you for taking the time to contribute. We're especially looking for

  • bug reports
  • feature requests
  • testing in non-gentoo/linux/amd64/portage environements
  • patches
  • praise and encouragements ;)

See CONTRIBUTING. Emlop is licensed as GPLv3.