Ghee: a command line tool for working with Linux extended attributes (xattrs)

The tastiest way to work with Linux extended attributes (xattrs) written in pure Rust and made of delicious, fibrous Open Source code.

Ghee provides tools for manipulating xattrs on individual files as well as for working with the filesystem as a document database where the filesystem paths act as primary keys and extended attributes provide non-indexed fields.

History

Ghee was developed in conjunction with the Audiotater audio annotation tool.

License

This software is licensed under GPL version 3 only.

Conventions

Extended attributes are parsed in a consistent manner by Ghee. Any xattr not preceded by the trusted, security, system, or user namespace will have the user namespace by default. For example, xattr trusted.uptime remains as is, while uptime would become user.uptime.

REPL

Running ghee with no arguments will enter a read-eval-print-loop (REPL), allowing for fluent command input.

  $ ghee
  Ghee 0.4.0

  ghee$ set ./test -s test=1

Subcommands

Ghee operates through a set of subcommands, each with a primary function. Run ghee --help to see the list of subcommands, and ghee $SUBCMD --help to get usage information for each subcommand.

Examples of each subcommand follow:

Move

Moves xattr values from one path to another.

• ghee mv path1.txt path2.txt: move all xattrs from path1.txt to path2.txt
• ghee mv -f id path1.txt path2.txt: move xattr id from path1.txt to path2.txt
• ghee mv -f id -f url path1.txt path2.txt: move xattrs id and url from path1.txt to path2.txt

Copy

Copies xattr values from one path to another.

• ghee cp path1.txt path2.txt: copy all xattrs from path1.txt to path2.txt
• ghee cp -f id path1.txt path2.txt: copy xattr id from path1.txt to path2.txt
• ghee cp -f id -f url path1.txt path2.txt: copy xattrs id and url from path1.txt to path2.txt

Remove

Removes xattr values.

• ghee rm path.txt: remove all xattrs on path.txt
• ghee rm -f id path.txt: remove xattr id from path.txt
• ghee rm -f id -f url path1.txt path2.txt path3.txt: remove xattrs id and url from path1.txt, path2.txt, and path3.txt

Set

Sets xattr values.

• ghee set -s id=123 path1.txt: set xattr id to value 123 on path1.txt
• ghee set -s id=123 -s url=http://example.com path1.txt path2.txt path3.txt: set xattr id to value 123 and xattr url to value http://example.com on path1.txt, path2.txt, and path3.txt

Get

Recursively get and print xattr values for one or more paths.

By default, the get subcommand outputs a tab-separated table with a column order of path, field, value. The value bytes are written to stdout as-is without decoding.

This excludes the user.ghee prefix unless -a --all is passed.

To opt out of the recursive default, use --flat.

• ghee get dir: print all xattrs for directory dir and all descendant files and directories, as raw (undecoded) TSV
• ghee get -f id path1.txt: print xattr id and its value on path1.txt as raw (undecoded) TSV
• ghee get -f id -f url path1.txt path2.txt path3.txt: print xattrs id and url and their respective values on path1.txt, path2.txt, and path3.txt as raw (undecoded) TSV

The get command can also output JSON - in which case values are decoded as UTF-8, filling in a default codepoint when decoding fails:

• ghee get -j --flat dir: print all xattrs for directory dir and all descendant files and directories, as UTF-8 decoded JSON
• ghee get -j -f id path1.txt: print xattr id and its value on path1.txt as UTF-8 decoded JSON
• ghee get -j -f id -f url path1.txt path2.txt path3.txt: print xattrs id and url and their respective values on path1.txt, path2.txt, and path3.txt as JSON

By adding --where (or -w), SQL WHERE-style clauses can be provided to select which files to include in the output. For example, ghee get -w age >= 65 ./patients will select all files under directory ./patients whose user.age attribute is 65 or greater.

Nested indices are always ignored in get output, though they will be used as appropriate to shortcut traversal when WHERE-style predicates are specified.

Init

Initializes a directory as a table with a specified primary key.

Examples:

• ghee init -k name ./people: marks the ./people directory as a table with primary key of name
• ghee init -k state -k id ./people-by-state-and-id: marks the ./people-by-state-and-id directory as a table with a compound primary key of [state, id].

Insert

Inserts JSON-formatted records into a table.

Records are read one per line from stdin.

• ghee ins ./people < ./people.json: inserts the records from ./people.json into the table at ./people, indexed by its primary key

Delete

Deletes records from a table.

They are unlinked from all table indices.

The records to be deleted are specified by providing either the components of the primary key or SQL-style WHERE clauses.

• ghee del ./people Von: because the table's primary key is name, deletes the record where name=Von from ./people and all indices.
• ghee del ./people -w name=Von: deletes ./people/Von as above, unlinking from all indices.

Index

Indexes a table.

When Ghee acts on a directory as if it were a database table, each file acts as a relational "record" with the primary key coming from the subpath under the table directory.

Each file's extended attributes act as the relational attributes [TODO with an overloadable virtual attribute id that represents the primary key. For each subcomponent $i of the primary key, there is also a virtual attribute id$i.]

Table directories created by Ghee also contain a special xattr user.ghee.key which gives the components of the primary key.

Examples:

• ghee idx -k name ./people ./people-by-name: recursively reindex the contents of ./people into a new directory ./people-by-name with primary key coming from xattr name and files hardlinked to the corresponding files in ./people.

  That means the ./people-by-name directory's files will have filenames taken from the names of the people as defined in xattr name.

  The new directory ./people-by-name will have xattr user.ghee.keyname=name so later commands can do efficient index lookups using user-friendly field names.

• ghee idx -k region -k name -s ./people-by-name ./people-by-region-and-name: recursively reindex the contents of ./people-by-name into a new directory ./people-by-region-and-name with primary key being the compound of xattr region and xattr name (in that order) and files hardlinked to the corresponding files in ./people, resolved via the hardlinks in ./people-by-name.

  The output directory ./people-by-region-and-name will have xattr user.ghee.keyname=region and each region-named subdirectory will have xattr user.ghee.keyname=name.

List

Like the ls command, lists directory contents, but annotated from Ghee's point of view.

Each path is marked as either a table or a record. For tables, the primary key is given.

• ghee ls: lists the current directory's contents
• ghee ls example: lists the contents of ./example