Expand description

memflow

Machine introspection made easy

Introduction

memflow is a library that enables introspection of various machines (hardware, virtual machines, memory dumps) in a generic fashion. There are 2 primary types of objects in memflow - Connectors and OS layers. Connector provides raw access to physical memory of a machine. Meanwhile, OS layer builds a higher level abstraction over running operating system, providing access to running processes, input events, etc. These objects are incredibly flexible as they can be chained together to gain access to a process running multiple levels of virtualization deep (see figure below).

+-----------+        +-----------+
| native OS |        | leechcore |
+-+---------+        +-+---------+
  |                    |
  |  +-----------+     |  +----------+
  +->|  QEMU VM  |     +->| Win32 OS |
     +-+---------+        +-+--------+
       |                    |
       |  +----------+      |  +-----------+
       +->| Win32 OS |      +->| lsass.exe |
          +-+--------+         +-----------+
            |
            |  +-----------+
            +->|  Hyper-V  |
               +-+---------+
                 |
                 |  +----------+
                 +->| Linux OS |
                    +-+--------+
                      |
                      |  +-----------+
                      +->| SSHD Proc |
                         +-----------+

(Example chains of access. For illustrative purposes only - Hyper-V Connector and Linux OS are not yet available)

As a library user, you do not have to worry about delicacies of chaining - everything is provided, batteries included. See one of our examples on how simple it is to build a chain (excluding parsing). All Connectors and OS layers are dynamically loadable with common interface binding them.

All of this flexibility is provided with very robust and efficient backend - memory interface is batchable and divisible, which gets taken advantage of by our throughput optimized virtual address translation pipeline that is able to walk the entire process virtual address space in under a second. Connectors and OS layers can be composed with the vast library of generic caching mechanisms, utility functions and data structures.

The memflow ecosystem is not bound to just Rust - Connector and OS layer functions are linked together using C ABI, thus users can write code that interfaces with them in other languages, such as C, C++, Zig, etc. In addition, these plugins can too be implemented in foreign languages - everything is open.

Overall, memflow is the most robust, efficient and flexible solution out there for machine introspection.

Structure

memflow is separated into modules that are concerned with different parts of the ecosystem. mem module is concerned with memory interfacing, os module is conerned with OS abstractions, architecture module defines specification of a computer architecture, as well as several built-in architectures, types concerns itself with data types used throughout memflow, while plugins module defines the dynamically loadable plugin types.

Getting started

To quickly get started with the memflow library, simply include its prelude:

use memflow::prelude::v1::*;

Afterwards, you will want to build a memflow object using the plugin inventory:

let inventory = Inventory::scan();

let conn = inventory.create_connector("dummy", None, None)?;

Core traits

While Connectors and OS layers are the primary user facing objects, functionality of these objects is provided through a set of traits.

MemoryView is the primary trait of issuing read and write operations. Required functions are a bit intimidating, because memflow wants IO to be batchable, which enables impressive performance, however, there are several helpers available for performing simple read and write operations.

Os and OsInner are the traits that deal with higher level OS abstractions. They enable access to Processes together with their MemoryViews. The reason for OsInner existance is lack of GATs, however, this complexity should be removed as soon as the feature is stabilized.

PhysicalMemory trait is implemented by connectors. It embeds special metadata which is used by our memory caches, however is not much different from MemoryView. Users performing physical reads may use the phys_view function to access a view to this physical address space and gain access to the helper methods.

VirtualTranslate trait is optionally provided by processes in order to translate virtual addresses into physical ones. This is a lower level trait.

Philosophy

The core idea of memflow is to generalize where possible, specialize when needed.

Best practice of writing memflow functions is to write them generically - use impl Trait notation to define the type of object needed. This will allow for your code to work on both dynamically loaded plugins, as well as custom, statically linked, and potentially more efficient memory/OS objects.

For instance, if you want to perform a memory read, define the function as follows:

use memflow::prelude::v1::*;

// Define the function with `impl Trait` notation
fn special_read(mem: &mut impl MemoryView) -> Result<u64> {
    mem.read(Address::from(0x42)).data()
}

// Use it with plugin object
let mut inventory = Inventory::scan();
let args = str::parse(":4m").unwrap();
let conn = inventory.create_connector("dummy", None, Some(&args))
    .unwrap();

assert!(special_read(&mut conn.into_phys_view()).is_ok());

// Use it with statically built connector
let mut mem = DummyMemory::new(size::mb(4));

assert!(special_read(&mut mem.phys_view()).is_ok());

// Use it with statically built process
let mut proc = DummyOs::quick_process(size::mb(4), &[]);

assert!(special_read(&mut proc).is_ok());

Modules

Module for handling different architectures in memflow.
Module containing basic connector and inventory related functions.
Specialized Error and Result types for memflow.
Special purpose iterators for memflow.
This module covers all implementations and traits related to reading/writing physical and virtual memory.
Describes an operating system in high level.
This module contains functions related to the Inventory system for Connectors and Os-Plugins.
Module with basic types used in memflow.

Macros