用于医学图像分割的SinGAN-Seg综合训练数据生成_SinGAN-SegSyntheticTrainingDataG

版权申诉

60 浏览量 2022-02-05 02:09:30 上传评论收藏 4.99MB PDF 举报

用于医学图像分割的SinGAN-Seg综合训练数据生成_SinGAN-Seg Synthetic Training Data Generation for Medical Image Segmentation.pdf 在医疗图像分割领域，人工智能（AI）已经成为辅助医生自动处理医疗数据的重要工具，极大地减轻了医疗专家的工作负担。然而，AI模型的训练依赖大量数据，而获取这些数据面临诸多挑战，如隐私保护问题以及耗时费力的医疗数据注释过程。针对这些问题，研究人员提出了一种名为SinGAN-Seg的新方法，用于生成带有注释真值掩模的合成医学图像数据。 SinGAN-Seg是一种创新的合成数据生成流程，它能够产生具有相应标注的医疗图像数据，以解决实际数据获取的限制。该方法的核心是利用SinGAN（单张图像生成对抗网络）技术，通过分析单个医学图像的细节和模式，生成多样性且与原始图像特征相符的合成图像。SinGAN-Seg的优势在于，它能生成足够逼真的图像，同时避免了真实数据可能带来的隐私问题，为AI模型提供训练数据的同时保护了患者隐私。具体来说，SinGAN-Seg的工作流程包括以下几个步骤： 1. **数据预处理**：对原始医学图像进行预处理，包括增强、归一化等操作，确保输入到SinGAN模型的图像质量。 2. **SinGAN模型训练**：利用SinGAN模型对预处理后的图像进行训练，学习图像的层次结构和多样性，生成一系列不同尺度的合成图像。 3. **合成图像分割**：在生成的合成图像上应用图像分割算法，创建对应的标注真值掩模，以模拟真实的医学图像分割任务。 4. **数据融合**：将合成数据集与真实数据集结合，用于训练深度学习模型，如UNet++，以提升模型的泛化能力。实验结果显示，当使用真实图像分割数据集和SinGAN-Seg生成的合成数据集联合训练UNet++时，模型在识别异常（如息肉分割）方面的性能接近仅使用大量真实数据的情况。此外，当真实训练数据量非常有限时，SinGAN-Seg生成的合成数据能够显著提高分割算法的性能，弥补数据不足导致的训练效果下降。 SinGAN-Seg的贡献在于提供了一种有效的数据扩充策略，不仅解决了医疗数据获取的难题，还减轻了手动注释数据的负担。这一方法对于推动医疗图像分析的AI研究和应用具有重要意义，未来有望进一步应用于其他医学图像分割任务，如肿瘤检测、血管分析等，为医疗诊断自动化带来更大的潜力。

资源推荐

资源详情

资源评论

SinGAN-Seg: Synthetic Training Data Generation for Medical

Image Segmentation

Vajira Thambawita

1,2

, Pegah Salehi

, Sajad Amouei Sheshkal

, Steven A. Hicks

1,2

, Hugo L.

Hammer

, Sravanthi Parasa

, Thomas de Lange

, P˚al Halvorsen

, and Michael A. Riegler

SimulaMet, Oslo, Norway

Oslo Metropolitan University, Oslo, Norway

Department of Medical Research, Bærum Hospital, Gjettum, Norway

Department of Gastroenterology, Swedish Medical Group, Seattle, WA, USA

Abstract

Processing medical data to ﬁnd abnormalities is a

time-consuming and costly task, requiring tremen-

dous eﬀorts from medical experts. Therefore, artiﬁ-

cial intelligence (AI) has become a popular tool for

the automatic processing of medical data, acting as

a supportive tool for doctors. AI tools highly depend

on data for training the models. However, there are

several constraints to access to large amounts of med-

ical data to train machine learning algorithms in the

medical domain, e.g., due to privacy concerns and

the costly, time-consuming medical data annotation

process.

To address this, in this paper we present a

novel synthetic data generation pipeline called

SinGAN-Seg to produce synthetic medical data

with the corresponding annotated ground truth

masks. We show that these synthetic data genera-

tion pipelines can be used as an alternative to by-

pass privacy concerns and as an alternative way to

produce artiﬁcial segmentation datasets with corre-

sponding ground truth masks to avoid the tedious

medical data annotation process. As a proof of con-

cept, we used an open polyp segmentation dataset.

By training UNet++ using both real polyp segmenta-

tion dataset and the corresponding synthetic dataset

generated from the SinGAN-Seg pipeline, we show

that the synthetic data can achieve a very close per-

formance to the real data when the real segmenta-

tion datasets are large enough. In addition, we show

that synthetic data generated from the SinGAN-Seg

pipeline improving the performance of segmentation

algorithms when the training dataset is very small.

Since our SinGAN-Seg pipeline is applicable for any

medical dataset, this pipeline can be used with any

other segmentation datasets.

1 Introduction

AI has become a popular tool in medicine and has

been vastly discussed in recent decades to augment

performance of clinicians [1, 2, 3, 4]. According to the

statistics discussed by Jiang et al. [1], artiﬁcial neu-

ral networks (ANNs) [5] and support vector machines

(SVMs) [6] are the most popular machine learning

(ML) algorithms used with medical data. These ML

models learn from data; thus the medical data have

a direct inﬂuence on the success of ML solutions in

real applications. While the SVM algorithms are

popular within regression [7, 8] and classiﬁcation [9]

tasks, ANNs or deep neural networks (DNNs) are

used widely for all the types; regression, classiﬁca-

tion, detection and segmentation.

A segmentation model makes more advanced pre-

dictions than regression, classiﬁcation, and detection

as it performs pixel-wise classiﬁcation of the input

images. Therefore, medical image segmentation is

arXiv:2107.00471v1 [eess.IV] 29 Jun 2021

a popular application of AI in medicine, so it is

used more widely with diﬀerent kinds of medical im-

age data [10, 11, 12]. Polyp segmentation is one

of popular segmentation tasks that uses ML tech-

niques to detect and segment polyps in images/videos

collected from gastrointestinal tract (GI) screenings.

Early identiﬁcation of polyps in GI tract is criti-

cal to prevent colorectal cancers [13]. Therefore,

many ML models have been investigated to segment

polyps automatically in GI tract videos recorded

from endoscopy [14, 15, 16] or PilCams examina-

tions [17, 18, 19] to augment performance of doctors

by detecting polyps missed by experts, thereby both

decreasing the miss rates and reducing the observer

variations.

Most of polyp segmentation models are based

on convolutional neural networks (CNNs) and are

trained using publicly available polyp segmentation

datasets [20, 21, 22, 23, 24]. However, these datasets

have a limited number of images with corresponding

expert annotated masks. For examples, the CVC-

VideoClinicDB [21] dataset has 11, 954 images from

10 polyp videos and 10 non-polyp videos, the PIC-

COLO dataset [24] has 3, 433 manually annotated

images (2, 131 white-light images and 1, 302 narrow-

band images), and the Hyper-Kvasir [20] dataset has

only 1, 000 segmented images, but also contains of

100, 000 unlabeled images.

We identiﬁed two main reasons for having small

datasets in medical domain compared to other do-

mains. The ﬁrst reason is privacy concerns attached

with medical data, and the second is the costly and

time-consuming medical data annotation processes

that the medical domain experts must perform.

The privacy concerns can vary from country to

country and region to region according to data pro-

tection regulations introduced in the speciﬁc ar-

eas. For example, Norway should follow the rules

given by the Norwegian data protection authority

(NDPA) [25] and enforce the personal data act [26]

in addition to following the general data protec-

tion regulation (GDPR) [27] guidelines being the

same for all European countries. While there is

no central level privacy protection guideline in the

US like GDPR in Europe, US rules and regulations

are enforced through other US privacy laws, such as

Health Insurance Portability and Accountability Act

(HIPAA) [28] and California Consumer Privacy Act

(CCPA) [29]. In Asian counties, they follow their

own sets of rules, such as Japan’s Act on Protection of

Personal Information [30], the South Korean Personal

Information Protection Commission [31] and the Per-

sonal Data Protection Bill in India [32].

If research is performed with such privacy re-

strictions, the papers published are often theoretical

methods only. According to the analyzed medical im-

age segmentation studies in [33], 30% have used pri-

vate datasets. As a result, the studies are not repro-

ducible. Researchers must keep datasets private due

to medical data sharing restrictions. Furthermore,

universities and research institutes that use medical

domain data for teaching purposes use the same med-

ical datasets for years, which aﬀects the quality of

education. In addition to the privacy concerns, the

costly and time-consuming medical data labeling and

annotation process [34] is an obstacle to producing

big datasets for AI algorithms. Compared to other

already time-consuming medical data labeling pro-

cesses, a pixel-wise data annotation are far more de-

manding on the valuable medical experts’ time. The

experts in the medical domain can perform the an-

notations fully trustable in terms of correctness. If

the data annotations by experts are not possible, the

experts should do at least a review process to make

the annotations correct before using them in AI al-

gorithms. The importance of having accurate anno-

tations from experts for medical data is, for example,

discussed by Yu et al. [35] using a mandible segmenta-

tion dataset of CT images. In this regard, researching

a way to produce synthetic segmentation datasets is

important to overcome the timely and costly medical

data annotation process. Therefore, researching an

alternative way for medical data sharing, bypassing

both the privacy and time-consuming dataset gener-

ation challenges, is the main objective of this study.

In this regard, the contributions of this paper are

as follows.

• This study introduces the novel SynGAN-Seg

pipeline to generate synthetic medical image and

its corresponding segmentation mask using a

modiﬁed version of the state-of-the-art SinGAN

architecture with a ﬁne-tuning step using a style-

transfer method. We use polyp segmentation as

a case study, the SinGAN-Seg can be applied for

all types of segmentation tasks.

• We have published the biggest synthetic polyp

dataset and the corresponding masks at https:

//osf.io/xrgz8/. Moreover, we have pub-

lished our generators as a python package at

Python package index (PyPI) (https://pypi.

org/project/singan-seg-polyp/) to generate

an unlimited number of polyps and correspond-

ing mask images as needed. To the best of our

knowledge, this is the ﬁrst publicly available syn-

thetic polyp dataset and the corresponding gen-

erative functions as a PyPI package.

• We show that synthetic images and correspond-

ing mask images can improve the segmentation

performance when the size of a training dataset

is limited.

2 Method

In the pipeline of SinGAN-Seg, there are as de-

picted in Figure 1 two main steps: (1) training novel

SinGAN-Seg generative models and (2) style trans-

ferring. The ﬁrst step generates synthetic polyp im-

ages and corresponding binary segmentation masks

representing the polyp area. The novel four channels

SinGAN-Seg, based on the vanilla SinGAN architec-

ture [36], is introduced in this ﬁrst step. The novel

training process of four channels SinGAN-Seg models

is presented in this step. Using a single SinGAN-Seg

model, we can generate multiple synthetic images and

masks from a single real image and the correspond-

ing masks. Therefore this generation process can be

identiﬁed as 1 : N generations, and it is denoted using

[img]

, where N represents the number of samples

generated in the ﬁgure. The second step focuses on

transferring styles such as features of polyps’ texture

from real images into the corresponding generated

synthetic images. This second step is depicted in the

Step 2 in Figure 1.

SinGAN-Seg is a modiﬁed version of SinGAN [36]

which was designed to generate synthetic data from

a generative adversarial network (GAN) trained only

using a single image. The original SinGAN is trained

using diﬀerent scales of the same input image, the

so-called image pyramid. This image pyramid is a

set of images of diﬀerent resolutions of a single im-

age from low resolution to high resolution. SinGAN

consists of a GAN pyramid, which takes the corre-

sponding image pyramid. In this study, we build on

the implementation and the training process used in

SinGAN, except for the number of input and output

channels. The original SinGAN implementation [36]

uses a three-channel RGB image as the input and

produces a three-channel RGB image as the output.

However, our SinGAN-Seg uses four-channels images

as the input and the output. The four-channels im-

age consist of the input RGB image and the single

channel ground truth mask by stacking them together

as depicted in the SinGAN-Seg model in Figure 1.

The main purpose of this modiﬁcation is to generate

four-channels synthetic output, which consists of a

synthetic image and the corresponding ground truth

mask.

In the second step of the SinGAN-Seg pipeline, we

ﬁne-tune the output of the four channels SinGAN-Seg

model using the style-transfer method introduced by

Leon et al. [37]. This step aims to improve the quality

of the generated synthetic data by transferring real-

istic styles from real images to synthetic images. As

depicted in Step 2 in Figure 1, every generated image

is enhanced by transferring style form the cor-

responding real image im

. Then, the style trans-

ferred output image is presented using ST

where

M = [0, 1, 2...999] in this study, representing the 1000

images in thr training dataset. In this process, a suit-

able content : style ratio should be found, and it is a

hyper-parameter in this second stage. However, this

step is a separate training step from the training step

of the SinGAN-Seg generative models. Therefore,

this step is optional to follow, but we strongly rec-

ommend this style-transferring step to enhance the

quality of the output data from the ﬁrst step.

im_0 im_1 im_332 im_333 im_334 im_665 im_666 im_667 im_999

Fold 0 Fold 1 Fold 2

G_0

[img]

G_1

[img]

G_332

[img]

G_333

[img]

G_334

[img]

G_665

[img]

G_666

[img]

G_667

[img]

G_999

[img]

G_0 G_1 G_332 G_333 G_334 G_665 G_666 G_667 G_999

im_M

Style Transfer

(content:style)

ST_M

G_M

M = [0,1,2,...,999]

Step 1

Step 2

Fake Real

Training

Four channels SinGAN-Seg model

Figure 1: The complete pipeline of SinGAN-Seg to generate synthetic segmentation datasets. Step 1 :

represents the training of four channels SinGAN models. Step 2: represents ﬁne tuning step using the

neural style transfer [37]. Four channels SinGAN : Single training step of our four-channels SinGAN. Note

the stacked input and output compared to the original SinGAN implementation [36] which input only single

image with a noise vector and output only an image. In our SinGAN implementation, all the generators

(from G

to G

N −1

), except G

, get four channels image (a polyp image and a ground truth) as the input

in addition to the input noise vector. The ﬁrst generator, G

get only the noise vector as the input. The

discriminators also get four channels images which consist of a RGB polyp image and a binary mask as

input. The inputs to the discriminators can be either real or fake.

3 Experiments and results

This section demonstrates all the experiments and re-

sults collected using a polyp dataset as a case study.

For all the experiments discussed in the following

sections, we have used Pytorch deep learning frame-

work [38].

3.1 Data

We have used a polyp dataset published with Hy-

perKvasir dataset [20] which consists of polyp ﬁnd-

ings extracted from endoscopy examinations. This

polyp dataset has 1000 polyp ﬁndings and a corre-

sponding segmentation mask annotated by experts.

We use only the polyp dataset as a case study be-

cause of the time and resource-consuming training

process of the SinGAN-Seg pipeline. Furthermore,

we use three-fold cross-validation, which is another

time-consuming technique, for the experiments per-

formed to ﬁnd the validity of using synthetic data

instead of real data.

A few sample images and the corresponding masks

of the polyp dataset of HyperKvasir are depicted in

Figure 2. The polyp images of the dataset are RGB

images. The masks of the polyp images are single-

channel images with white (255) for true pixels, which

represent polyp regions, and black (0) for false pixels,

which represent clean colon or background regions.

In this dataset, there are diﬀerent sizes of polyps.

The distribution of polyp sizes as a percentage of the

full image size is presented in the histogram plot in

Figure 3. In this dataset, there are more relatively

small polyps compared to larger polyps according to

the plot presented in Figure 3. Additionally, this

dataset was used to prove that the performance of

segmentation models trained with small datasets can

be improved using our SinGAN-Seg pipeline.

This dataset was used for two purposes.

1. To train SinGAN-Seg models to generate syn-

thetic data.

2. To compare performance of real and synthetic

剩余17页未读，继续阅读

评论收藏

内容反馈

版权申诉

易小侠

粉丝: 6653
资源: 9万+

用于医学图像分割的SinGAN-Seg综合训练数据生成_SinGAN-Seg Synthetic Training Data G

图像分割在医学图像处理中的应用研究

一种新的生物医学图像分割水平集方法

图像的分割

KiTS 医学图像分割数据集

医学图像处理

图像分割算法或可提高万倍效率 将用于医疗影像

计算机系统-笔记-HUN2021级

cs1.6老版本供下载

港大CS（MSC）面试整理

SAP CS客户服务模块基本流程

Cobalt-Strike-4.5

shellcode加载器

cobaltstrike4.3.zip

SAMP算法实现.m

CobaltStrike V4.zip

课程设计报告数字式电缆对线器.docx

Cobalt-Strike-4.3

合成孔径雷达的经典成像算法cs(matlab)仿真代码（吐血整理，内容全，注释全）

common.js

基于C+++Mysql实现（CS界面）图书管理系统【100010034】

CS的一些重构算法.zip_CS_ROMP_omp samp_samp_压缩感知

运用html+css+js三件套实现动态圣诞树

复刻CS第一人称射击游戏Demo

软件技术方案(完整资料).doc.pdf

棉花库设计规范用电.pdf

浅谈数字化转型在施工现场4M1E中的应用.docx

通过MATLAB对比仿真了RD、RMA、CS三种成像算法+含代码操作演示视频

最新资源

图像分割算法或可提高万倍效率将用于医疗影像