^{1}

^{2}

^{3}

^{4}

^{5}

^{1}

^{2}

^{3}

^{4}

^{5}

Slice-to-volume reconstruction (SVR) method can deal well with motion artifacts and provide high-quality 3D image data for fetal brain MRI. However, the problem of sparse sampling is not well addressed in the SVR method. In this paper, we mainly focus on the sparse volume reconstruction of fetal brain MRI from multiple stacks corrupted with motion artifacts. Based on the SVR framework, our approach includes the slice-to-volume 2D/3D registration, the point spread function- (PSF-) based volume update, and the adaptive kernel regression-based volume update. The adaptive kernel regression can deal well with the sparse sampling data and enhance the detailed preservation by capturing the local structure through covariance matrix. Experimental results performed on clinical data show that kernel regression results in statistical improvement of image quality for sparse sampling data with the parameter setting of the structure sensitivity 0.4, the steering kernel size of

Magnetic resonance imaging (MRI) is an ideal diagnostic technique for researchers to investigate the development of the fetal brain [

The duration of an examination is typically 45 to 60 minutes for fetal brain MRI [

For the SVR framework [

The general SVR framework with the superresolution reconstruction method has been developed in [

The rest of the paper is organized as follows. The detailed methodology is discussed in Section

During data acquisition of fetal brain MRI, we collected several stacks of 2D slices in different orientations. Because of the fetal motion, the movement could be observed between these slices. Assume that the acquired

Based on the low-rank decomposition method, we can choose one stack with minimal motion as the target template and first perform the 3D rigid volumetric registration between the target template and the other stacks (stack to template registration). During the first registration, we can get the corresponding rigid global transformation matrix

The illustration of the whole transformation process from pixels

To model the actual appearance of sampling data points in physical space, the point spread functions (PSFs) are used to make the exact estimation for every voxel value in the reconstructed target volume. For the MRI ssFSE sequence in this paper, the exact shape of the PSF has been measured using a phantom and rotating imaging encoding gradient in [

Once the target volume is updated based on the Gaussian PSF, the simulated slices

In [

The variables

The purpose of the outlier removal is to make the framework more robust by rejecting the outlier slices. The outlier removal module is adopted directly from the cited previous work [

For sparse reconstruction, it is experimentally found that the reconstructed volume still remains unallocated or inaccurate voxels after PSF-based volume update and the reconstructed result is noise as shown in Figure

The reconstructed volume after PSF-based volume update.

In [

Assuming that the voxel

Based on the least-squares formula, we can optimize Equation (

For the computation simplicity, the Gaussian-based kernel function is chosen in the steering kernel regression [

Equation (

Once the reconstructed volume is updated based on the steering kernel regression, we update the simulated slices

The experiment computer is equipped with Intel Core i5 2.6 GHz CPU, and the operating system is Windows 7 64 bit. We have implemented the proposed algorithm using the Microsoft Visual Studio 2012 and Image Registration Toolkit (IRTK) software package which includes many useful methods to do registration, transformation, and other image processing. In this section, we discuss the key implementation details. The diagram of the total algorithm is expressed in Figure

Flowchart of the proposed algorithm.

The first step is to evaluate the stack motion according to the method of low-rank decomposition. We estimate the amount of the stack motion by the surrogate

The iterative steering kernel regression.

In the experimental evaluation, we used the datasets from the fetal MRI datasets [^{th} slice of the collected stack and its corresponding simulated spare slices are illustrated in Figure

The original and spare slices: (a) the typical 30th original slice; (b–j) the corresponding simulated sparse slice by removing once every 10% proportion pixels ranging from 10% to 90%.

For different data removal ratios, the sparse stacks are used to reconstruct the high-resolution 3D fetal brain MRI volume with the method of Kainz et al. [

Reconstruction results of different data removal ratio by Kainz et al. method (2015) and our proposed method. (a), (c), (e), (g), (i), (k), (m), (o), (q), (s) are the reconstructed results by Kainz et al. method for the sparsely sampled dataset with once every 10% data removal ratio ranging from 0% to 90% respectively. (b), (d), (f), (h), (j), (l), (n), (p), (r), (t) are the reconstructed results by the proposed methodfor the sparsely sampled dataset with once every 10% data removal ratio ranging from 0% to 90%, respectively. The red rectangle points to the obvious difference, which appears as artifacts in the reconstructed image if no steering kernel regression volume updated is used.

For the sake of quantitative evaluation, the image quality assessment index of root mean square error (RMSE) [

The structure similarity (SSIM) index explores the structural information for image quality assessment based on the main idea that the pixels have strong interdependency when they are spatially close. The SSIM metric is calculated based on the intensity, contrast, and structure and is computed as

For the clinical datasets, it is impractical to obtain the ground-truth volume in advance. For the sake of fair comparison among different methods, the quantitative evaluation is performed based on an average reconstructed volume. We first use the original stacks without data removal to reconstruct a complete volume by Kainz et al.’s method (2015) and our method (e.g., Figures

The RMSE and SSIM value comparison of fetal brain reconstruction with different removal proportions, respectively.

Different removal proportions | RMSE | MSSIM | ||
---|---|---|---|---|

Kainz et al.’s method (2015) | Our method | Kainz et al.’s method (2015) | Our method | |

0% | 19.096 | 19.096 | 1 | 1 |

10% | 32.578 | 29.120 | 0.9752 | 0.9759 |

20% | 38.043 | 33.947 | 0.9690 | 0.9691 |

30% | 43.171 | 36.790 | 0.9591 | 0.9608 |

40% | 49.194 | 41.027 | 0.9478 | 0.9458 |

50% | 55.894 | 45.480 | 0.9202 | 0.9366 |

60% | 67.053 | 53.305 | 0.9063 | 0.9271 |

70% | 81.522 | 62.964 | 0.8667 | 0.8993 |

80% | 111.917 | 78.886 | 0.7862 | 0.8449 |

90% | 180.483 | 112.249 | 0.6338 | 0.7407 |

Our approach is capable of reconstructing the accurate volume from the highly sparse sampling dataset, but it requires largely computational burden as well due to the iterative kernel regression estimation. To reduce the long processing time of the adaptive kernel regression, the proposed method is accelerated by the GPU-based parallel implementation based on the NVIDIA GeForce GTX 1080 and CUDA 8.0 libraries. In the experiment, we make the evaluation of the computational efficiency of the adaptive kernel regression method, including the computation of the gradient information, the covariance smoothing matrix, and the steering kernel regression. The computational efficiency of the other modules (i.e., motion estimation, stack-to-template registration, PSF-based volume update, robust outlier removal, and slice-to-volume registration) has been evaluated in detail in [

The running time comparison of the adaptive kernel regression method for fetal brain reconstruction based on CPU and GPU, respectively.

Processor | Single-threaded CPU | Multithreaded CPU | GPU | Single-threaded CPU vs. GPU | Multithreaded CPU vs. GPU |
---|---|---|---|---|---|

Gradient information (s) | 416.960 | 108.762 | 5.047 | 82.62 | 21.55 |

Covariance smoothing matrix (s) | 46.082 | 48.902 | 0.580 | 79.45 | 84.31 |

Steering kernel regression (s) | 1402.727 | 887.629 | 15.091 | 92.95 | 58.82 |

Total time (s) | 1865.769 | 1045.293 | 20.718 | 90.06 | 50.45 |

There are seven parameters which can be adjusted to affect the reconstructed image quality for the proposed method. These parameters include the kernel size

With the help of GPU-based fast implementation, we firstly adjust the parameters (i.e.,

The RMSE and MSSIM values and running time comparison of fetal brain reconstruction with different window sizes ranging from

Window size | ||||
---|---|---|---|---|

RMSE | 125.061 | 125.869 | 129.298 | 126.929 |

MSSIM | 0.6796 | 0.6663 | 0.6606 | 0.6763 |

TIME (s) |

Note: TIME denotes the time caused only by running the adaptive kernel regression method.

Reconstructed results of the MRI data with different window sizes

Table

The RMSE and MSSIM value comparison of fetal brain reconstruction with different structure sensitivities

Structure sensitivity | |||||
---|---|---|---|---|---|

RMSE | 125.06 | 112.10 | 99.927 | 91.073 | 97.556 |

MSSIM | 0.6823 | 0.7117 | 0.7584 | 0.7712 | 0.7523 |

Reconstructed results of the MRI data with different structure sensitivities

Under different regularization parameter settings, the RMSE and MSSIM measurements of the reconstructed results are calculated and shown in Table

The RMSE and MSSIM value comparison of fetal brain reconstruction with the regularization parameter

Regularization parameter | |||||
---|---|---|---|---|---|

RMSE | 91.073 | 91.447 | 91.276 | 91.566 | 91.071 |

MSSIM | 0.7720 | 0.7710 | 0.7708 | 0.7684 | 0.7732 |

Reconstructed results of the MRI data with different regularization parameters

The next group parameters (i.e.,

The RMSE and MSSIM values and running time comparison of fetal brain reconstruction with different steering kernel sizes

Steering kernel size | ||||
---|---|---|---|---|

RMSE | 91.07 | 79.554 | 79.005 | 81.502 |

MSSIM | 0.7791 | 0.7973 | 0.8054 | 0.7781 |

TIME (s) |

Note: TIME denotes the time caused only by running the adaptive kernel regression method.

Reconstructed results of the MRI data with different kernel window sizes: (a)

Table

The RMSE and MSSIM value comparison of fetal brain reconstruction with different steering smoothing parameters

Steering smoothing parameter | |||||
---|---|---|---|---|---|

RMSE | 79.005 | 78.886 | 79.230 | 79.159 | 82.87 |

MSSIM | 0.7927 | 0.8092 | 0.7933 | 0.7944 | 0.7946 |

Reconstructed results of the MRI data with different steering smoothing parameters: (a)

In this paper, we proposed an adaptive reconstruction method to deal with the sparse sampling dataset for fetal brain MRI. Our method combines the latest SVR framework, including the stack motion evaluation, PSF-based volume update, robust outlier removal, slice-to-volume registration, and the proposed adaptive kernel regression-based volume update. Compared with the existing SVR framework, our method has advantages of sparse volume reconstruction and is capable of reconstructing superresolution image even for 80%~90% data removal. With the capability of sparse reconstruction, the data sampling time can be greatly shortened and thus, the motion artifacts can be reduced indirectly. To accelerate the voxel estimation, we use the CUDA to implement the steering kernel regression approach. For the proposed method, the running times of GPU-based implementation are speeded up to 90x than that of the CPU. The GPU-based parallel implementation of the proposed method is more practical to meet the requirements of fetal brain MRI. Meanwhile, we make the detailed investigation on the choice of parameters for the adaptive kernel regression-based volume reconstruction with the help of GPU-based fast implementation. To summarize, the structure sensitivity

The test data was downloaded from the publicly available dataset on GitHub (

The authors declare that they have no conflicts of interest.

Qian Ni and Yi Zhang contributed equally to this work and should be considered co-first authors.

This study was financed partially by the National Key R&D Program of China (No. 2018YFA0704102), the National Natural Science Foundation of China (Nos. 81827805, 61401451), and the Shenzhen Key R&D Program (No. JCYJ20200109114812361).