1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
//! Run configuration tests from build.rs and set available features, similar to *autotools*
//! configure scripts.
//!
//!
//! # Description
//!
//! `ConfTest::run()` called from 'build.rs' parses 'Cargo.toml'. Then for each *[feature]*
//! defined, it checks if that feature was not set manually (with `--features`) and a test in
//! 'conf_tests/' exists. This test is then compiled and build. When that succeeds the
//! feature becomes enabled automatically.
//!
//!
//! # Rationale
//!
//! Operating Systems have sometimes subtle differences in features and standards
//! conformance. Sometimes non-standard features are added for improved performance. These
//! differences make it hard to write portable software that still offer optimal performance
//! for diffent Operatiing Systems. Moreover often a developer doesn't even know what
//! featureset other Operating Systems may provide or this may be changed by kernel or
//! userland version or configuration. Probing the presence of such features at build time
//! can solve these problems.
//!
//!
//! # How To
//!
//! When present 'cargo' (builds and) executes a *build.rs* script while building crate. This
//! is the place where *ConfTest* is hooked in at first:
//!
//! ```rust,ignore
//! use conf_test::ConfTest;
//!
//! fn main() {
//!     ConfTest::run();
//!     // any other build.rs steps follow below
//! }
//! ```
//!
//! Further one has to define a set of features and dependencies in the 'Cargo.toml'.  Note
//! that 'build.rs' will be run before the '[dependencies]' are build. Thus all dependencies
//! needed for the tests must go into '[build-dependencies]' as well. For Example:
//!
//! ```toml
//! [dependencies]
//! libc = "0.2.34"
//!
//! [build-dependencies]
//! libc = "0.2.34"
//! conf_test = "0.1"
//!
//! [features]
//! default = []
//! o_path = []
//! ```
//!
//! And as final step the crate directory 'conf_tests/' need to be created which contain rust
//! files named after the features to be probed. Containing a single `fn main()` which shall
//! probe one single thing.
//!
//! ```rust,ignore
//! // This goes into conf_tests/o_path.rs
//! extern crate libc;
//!
//! fn main() {
//!     unsafe {
//!         let conf_tests = std::ffi::CString::new("conf_tests").unwrap();
//!         // Compilation will fail when the libc does not define O_PATH
//!         libc::open(conf_tests.as_ptr(), libc::O_PATH);
//!     }
//! }
//! ```
//!
//! Later in the crate implementation source code one uses conditional compilation as usual
//! with `#[cfg(feature = "o_path")]`.
//!
//! ## Test depending on Features
//!
//! Tests may depend on features that are discovered by other tests or set manually. For
//! simplicity there is no dependency resolver about this but tests are run in sort order of
//! the feature name. Every subsequent test is compiled with the the feature flags already
//! discovered so far. To leverage this functionality one rarely needs to change the feature
//! names. For example when 'bar' depends on 'foo' it is required to enforce the sort order by
//! renaming these features to 'aa_foo' and 'bb_bar'. Only features that get discovered are
//! used for the test compilations features set by printing cargo instructions from the test
//! scripts are not used.
//!
//!
//! # Extra Features
//!
//! Tests can emit special instructions to cargo on stdout.
//! These become only effective when the test exits successful.
//! See https://doc.rust-lang.org/cargo/reference/build-scripts.html#outputs-of-the-build-script
//!
//! One can control ConfTest by setting the environment variable `CONF_TEST_INHIBIT` to one of
//! the following:
//! * **skip**
//!   Will not execute any conf_tests but proceed with 'build.rs'.
//! * **stop**
//!   Exits 'build.rs' sucessfully, not executing any tests.
//! * **fail**
//!   Exits 'build.rs' with an failure, not executing any tests.
//!
//! Any other value will make the script panic.
//!
//!
//! # Limitations
//!
//! * The tests running on the machine where the software is build, using the
//!   build-dependencies. This will be a problem when Software gets cross-compiled. For cross
//!   compilation set 'CONF_TEST_INHIBIT=skip' and set the desired features manually with the
//!   '--features' option.
//!
//! * Features can only be set, not unset. This is deliberate and not a limitation. Do only
//!   positive tests checking for the presence of a feature.
//!
//!
//! # Good Practices
//!
//! * Only use ConfTest when other things (like factoring out OS specific thing into their own
//!   crates) are not applicable.
//!
//! * Provide a baseline implementation which is portable with no features enabled. This may
//!   not perform as well or lack some special features but should compile nevertheless.
//!

use std::ffi::{OsStr, OsString};
use std::fs::File;
use std::io::prelude::*;

use std::env::var_os as env;
use std::path::{Path, PathBuf};
use std::str;

use cargo_metadata::{Message, MetadataCommand};
use std::process::{Command, Stdio};

use std::collections::{BTreeMap, BTreeSet};

// Empty Type for now, In future this may be extended without breaking existing code.
/// Implements the conf_test API
pub enum ConfTest {}

impl ConfTest {
    /// Run the configuration tests in 'conf_tests/'.
    #[allow(dead_code)]
    pub fn run() {
        if let Some(inhibit) = env("CONF_TEST_INHIBIT") {
            if inhibit == "skip" {
                println!("cargo:warning=Skipping ConfTest via CONF_TEST_INHIBIT");
                return;
            } else if inhibit == "stop" {
                std::process::exit(0);
            } else if inhibit == "fail" {
                println!("cargo:warning=Requested ConfTest failure via CONF_TEST_INHIBIT");
                std::process::exit(1)
            } else {
                // Bail on any unknown value to catch 'undefined' states/typos
                panic!("Unknown CONF_TEST_INHIBIT value: {:?}", inhibit)
            }
        }

        let metadata = MetadataCommand::new()
            .no_deps()
            .exec()
            .expect("Querying cargo metadata failed");

        let mut features = BTreeSet::new();
        let mut dependencies = BTreeSet::new();
        let mut edition: Option<String> = None;
        for package in metadata.packages {
            if edition == None {
                // just pick the first edition seen
                edition = Some(package.edition);
            }
            for (feature, _) in package.features {
                features.insert(feature);
            }
            for dep in package.dependencies {
                dependencies.insert(dep.name);
            }
        }
        let edition = edition.unwrap_or_else(|| String::from("2018"));

        let extern_libs = Self::get_extern_libs(&dependencies);

        let mut test_features = Vec::new();
        let mut outputs = Vec::new();
        for feature in features {
            if env(format!("CARGO_FEATURE_{}", feature.to_uppercase())).is_none() {
                outputs.push(format!("# checking for {}\n", &feature));
                let mut test_src = PathBuf::from("conf_tests");
                test_src.push(&feature);
                test_src.set_extension("rs");
                if test_src.exists() {
                    outputs.push(format!("# {} exists\n", test_src.display()));
                    outputs.push(format!("cargo:rerun-if-changed={}\n", test_src.display()));
                    if let Some(binary) =
                        Self::compile_test(&test_src, &edition, &extern_libs, &test_features)
                    {
                        outputs.push(format!("# compiling ConfTest for {} success\n", &feature));
                        if let Some(stdout) = Self::run_test(&binary) {
                            outputs
                                .push(format!("# executing ConfTest for {} success\n", &feature));
                            outputs.push(format!("cargo:rustc-cfg=feature=\"{}\"\n", &feature));
                            outputs.push(stdout);
                            test_features.push(feature.clone());
                        } else {
                            outputs.push(format!("# executing ConfTest for {} failed\n", &feature));
                        }
                    } else {
                        outputs.push(format!("# compiling ConfTest for {} failed\n", &feature));
                    }
                } else {
                    outputs.push(format!("# test for '{}' does not exist\n", &feature));
                }
            } else {
                outputs.push(format!("# test for '{}' manually overridden\n", &feature));
            }
            outputs.push(String::from("\n"));
            test_features.push(feature.clone());
        }

        let mut logfile = PathBuf::new();
        logfile.push(env("OUT_DIR").unwrap());
        logfile.push("conf_test");
        logfile.push("conf_test.log");

        let mut logfile = File::create(logfile).expect("Failed to create logfile");

        for output in outputs {
            logfile.write_all(output.as_bytes()).unwrap();
            print!("{}", output);
        }
    }

    fn run_test(test_binary: &Path) -> Option<String> {
        let command = Command::new(test_binary).output().ok()?;
        if command.status.success() {
            Some(String::from_utf8_lossy(&command.stdout).to_string())
        } else {
            None
        }
    }

    fn compile_test(
        src: &Path,
        edition: &str,
        extern_libs: &BTreeMap<OsString, (String, PathBuf)>,
        features: &[String],
    ) -> Option<PathBuf> {
        let mut out_file = PathBuf::new();
        out_file.push(env("OUT_DIR").expect("env var OUT_DIR is not set"));
        out_file.push("conf_test");
        out_file.push(src.file_stem().unwrap());

        let mut rust_cmd = Command::new(env("RUSTC").unwrap_or_else(|| OsString::from("rustc")));
        let rust_cmd = rust_cmd
            .arg("--crate-type")
            .arg("bin")
            .arg("--edition")
            .arg(edition)
            .arg("-o")
            .arg(&out_file)
            .arg("-v")
            .arg(src);

        for (name, filename) in extern_libs.values() {
            rust_cmd.arg("--extern").arg(format!(
                "{}={}", //FIXME: needs some better way to compose an OsString here
                name,
                filename.to_str().expect("invalid file name")
            ));
        }

        for feature in features {
            rust_cmd
                .arg("--cfg")
                .arg(format!("feature=\"{}\"", feature));
        }

        let rust_output = rust_cmd.output().ok()?;

        if rust_output.status.success() {
            Some(out_file)
        } else {
            None
        }
    }

    fn get_extern_libs(dependencies: &BTreeSet<String>) -> BTreeMap<OsString, (String, PathBuf)> {
        let mut extern_libs = BTreeMap::new();

        //PLANNED: get rid of extra target dir, is there any way to work around the build lock?
        let mut target_dir = PathBuf::new();
        target_dir.push(env("OUT_DIR").expect("env var OUT_DIR is not set"));
        target_dir.push("conf_test");

        // let cargo start a rustc process that does not build the project but returns the
        // metadata about compilation artifacts
        let mut cargo = Command::new(env("CARGO").unwrap_or_else(|| OsString::from("cargo")))
            .arg("rustc")
            .arg("--message-format")
            .arg("json")
            .arg("--target-dir")
            .arg(target_dir)
            .arg("--")
            .arg("--emit")
            .arg("metadata")
            .env("CONF_TEST_INHIBIT", "stop")
            .stdout(Stdio::piped())
            .spawn()
            .unwrap();

        let reader = std::io::BufReader::new(cargo.stdout.take().unwrap());

        for message in cargo_metadata::Message::parse_stream(reader) {
            if let Message::CompilerArtifact(artifact) = message.unwrap() {
                if dependencies.contains(&artifact.target.name) {
                    for filename in artifact.filenames {
                        let filename = PathBuf::from(filename);
                        let id = OsString::from(filename.file_stem().expect("invalid file name"));
                        let extension = filename.extension();
                        let name = String::from(&artifact.target.name);

                        match extension.and_then(OsStr::to_str) {
                            Some("rlib") => {
                                extern_libs.insert(id, (name, filename));
                            }
                            Some("rmeta") => {
                                if extern_libs.contains_key(&id) {
                                    let stored_extension = extern_libs[&id]
                                        .1
                                        .extension()
                                        .and_then(OsStr::to_str)
                                        .unwrap();
                                    if stored_extension == "rlib" {
                                        continue;
                                    }
                                    extern_libs.insert(id, (name, filename));
                                }
                            }
                            Some(_other) => {
                                if extern_libs.contains_key(&id) {
                                    let stored_extension = extern_libs[&id]
                                        .1
                                        .extension()
                                        .and_then(OsStr::to_str)
                                        .unwrap();
                                    if stored_extension == "rmeta" || stored_extension == "rlib" {
                                        continue;
                                    }
                                    extern_libs.insert(id, (name, filename));
                                }
                            }
                            None => {
                                panic!("extension is not utf8 {:?}", extension);
                            }
                        }
                    }
                }
            }
        }

        cargo.wait().expect("Couldn't get cargo's exit status");

        extern_libs
    }
}