sn0int 0.20.1

Semi-automatic OSINT framework and package manager
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123

Database
========

There are a few things you need to understand how the database works to use it
efficiently.

The database that is backing sn0int is sqlite, but the api that is exposed to
the user and scripts is an nosql-ish object store. The query language that is
exposed to the user is still very similar to sql, except that it lacks a column
statement::

    select subdomains where value like %.example.com
    ^      ^          ^           ^    ^
    |      |          |           |    this value is going to be quoted automatically
    |      |          |           |
    |      |          |           this triggers automatic quoting
    |      |          |
    |      |          apply a filter, this translates to sql quite literally
    |      |
    |      the entity we want to select is a subdomain.
    |      this affects the table and the deserializer
    |
    select entities

This is how almost all user facing functions work that operate on the database.
The functions that are available for scripting are a bit more object based and
described below.

db_add
------

This operation is somewhat straight forward. It adds an entity to the
database:

.. code-block:: lua

    domain_id = db_add('domain', {
        value='example.com',
    })

If this entity conflicts with an entity that already exists, an upsert is
triggered and an db_update is performed instead.

.. note::
   This function may return ``nil`` if the entity already exists, but has been
   removed from scope with ``noscope``. Everytime you use ``db_add`` you need
   to make sure that the ID that has been returned is not ``nil``.

db_add_ttl
----------

Add a temporary entity to the database. This is commonly used to insert
temporary links that automatically expire over time. If the entity already
exists and is also marked as temporary the new ttl is going to replace the old
ttl. If the entity already exists but never expires we are not going to add a
ttl.

.. code-block:: lua

    -- this link is valid for 2min
    domain_id = db_add_ttl('network-device', {
        network_id=1,
        device_id=13,
    }, 120)

db_activity
-----------

Log an activity event. A basic event looks like this:

.. code-block:: lua

    db_activity({
        topic='harness/activity-ping:dummy',
        time=sn0int_time(),
        content={
            a='b',
            foo={
                bar=1337,
            },
            msg='ohai',
        },
    })

This function is explained in detail in the `activity <activity.html>`_
section.

db_update
---------

Update some mutable fields of an entity:

.. code-block:: lua

    db_update('ipaddr', arg, {
        asn=lookup['asn'],
        as_org=lookup['as_org'],
    })

The first parameter is usually the same arg that your script was called with.
Usually you can use db_add instead of db_update due to the upsert feature, but
db_update is still slightly faster.

.. note::
   Some fields are immutable and can not be updated.

db_select
---------

This function is used to check if something is in scope. If the entity has been
added to the database and has not been removed from scope, this function
returns that entities id. This is somewhat similar to ``db_add``, except that
``db_select`` never adds anything to the database.

.. code-block:: lua

    domain_id = db_select('domain', 'example.com')
    if domain_id ~= nil then
        -- do something
    end

This function only accepts a string instead of a lua table. This string is used
to filter on the ``value`` column.