CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels

Nazanin Mohammadi Sepahvand; Tal Hassner; Douglas L. Arnold; Tal Arbel

doi:10.1007/978-3-030-11723-8_6

CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels

Nazanin Mohammadi Sepahvand, Tal Hassner, Douglas L. Arnold, Tal Arbel

Computer Science discipline

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

New T2w and gadolineum-enhancing lesions in Magnetic Resonance Images (MRI) are indicators of new disease activity in Multiple Sclerosis (MS) patients. Predicting future disease activity could help predict the progression of the disease as well as efficacy of treatment. We introduce a convolutional neural network (CNN) framework for future MRI disease activity prediction in relapsing-remitting MS (RRMS) patients from multi-modal MR images at baseline and illustrate how the inclusion of T2w lesion labels at baseline can significantly improve prediction accuracy by drawing the attention of the network to the location of lesions. Next, we develop a segmentation network to automatically infer lesion labels when semi-manual expert lesion labels are unavailable. Both prediction and segmentation networks are trained and tested on a large, proprietary, multi-center, multi-modal, clinical trial dataset consisting of 1068 patients. Testing based on a dataset of 95 patients shows that our framework reaches very high performance levels (sensitivities of 80.11% and specificities of 79.16%) when semi-manual expert labels are included as input at baseline in addition to multi-modal MRI. Even with inferred lesion labels replacing semi-manual labels, the method significantly outperforms an identical end-to-end CNN which only includes baseline multi-modal MRI.

Original language	English
Title of host publication	Brainlesion
Subtitle of host publication	Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers
Editors	Spyridon Bakas, Mauricio Reyes, Farahani Keyvan, Alessandro Crimi, Theo van Walsum, Hugo Kuijf
Publisher	Springer Verlag
Pages	57-69
Number of pages	13
ISBN (Print)	9783030117221
DOIs	https://doi.org/10.1007/978-3-030-11723-8_6
State	Published - 2019
Event	4th International MICCAI Brainlesion Workshop, BrainLes 2018 held in conjunction with the Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2018 - Granada, Spain Duration: 16 Sep 2018 → 20 Sep 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11383 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	4th International MICCAI Brainlesion Workshop, BrainLes 2018 held in conjunction with the Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2018
Country/Territory	Spain
City	Granada
Period	16/09/18 → 20/09/18

Bibliographical note

Publisher Copyright:
© Springer Nature Switzerland AG 2019.

Keywords

Deep learning
Disease activity
Magnetic resonance imaging
Multiple sclerosis

Access to Document

10.1007/978-3-030-11723-8_6

Cite this

Sepahvand, N. M., Hassner, T., Arnold, D. L., & Arbel, T. (2019). CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels. In S. Bakas, M. Reyes, F. Keyvan, A. Crimi, T. van Walsum, & H. Kuijf (Eds.), Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers (pp. 57-69). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11383 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-11723-8_6

Sepahvand, Nazanin Mohammadi ; Hassner, Tal ; Arnold, Douglas L. et al. / CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers. editor / Spyridon Bakas ; Mauricio Reyes ; Farahani Keyvan ; Alessandro Crimi ; Theo van Walsum ; Hugo Kuijf. Springer Verlag, 2019. pp. 57-69 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{cadbd778866444d39ecaaaa329334c5c,

title = "CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels",

abstract = "New T2w and gadolineum-enhancing lesions in Magnetic Resonance Images (MRI) are indicators of new disease activity in Multiple Sclerosis (MS) patients. Predicting future disease activity could help predict the progression of the disease as well as efficacy of treatment. We introduce a convolutional neural network (CNN) framework for future MRI disease activity prediction in relapsing-remitting MS (RRMS) patients from multi-modal MR images at baseline and illustrate how the inclusion of T2w lesion labels at baseline can significantly improve prediction accuracy by drawing the attention of the network to the location of lesions. Next, we develop a segmentation network to automatically infer lesion labels when semi-manual expert lesion labels are unavailable. Both prediction and segmentation networks are trained and tested on a large, proprietary, multi-center, multi-modal, clinical trial dataset consisting of 1068 patients. Testing based on a dataset of 95 patients shows that our framework reaches very high performance levels (sensitivities of 80.11% and specificities of 79.16%) when semi-manual expert labels are included as input at baseline in addition to multi-modal MRI. Even with inferred lesion labels replacing semi-manual labels, the method significantly outperforms an identical end-to-end CNN which only includes baseline multi-modal MRI.",

keywords = "Deep learning, Disease activity, Magnetic resonance imaging, Multiple sclerosis",

author = "Sepahvand, {Nazanin Mohammadi} and Tal Hassner and Arnold, {Douglas L.} and Tal Arbel",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2019.; 4th International MICCAI Brainlesion Workshop, BrainLes 2018 held in conjunction with the Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2018 ; Conference date: 16-09-2018 Through 20-09-2018",

year = "2019",

doi = "10.1007/978-3-030-11723-8_6",

language = "אנגלית",

isbn = "9783030117221",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "57--69",

editor = "Spyridon Bakas and Mauricio Reyes and Farahani Keyvan and Alessandro Crimi and {van Walsum}, Theo and Hugo Kuijf",

booktitle = "Brainlesion",

address = "גרמניה",

}

Sepahvand, NM, Hassner, T, Arnold, DL & Arbel, T 2019, CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels. in S Bakas, M Reyes, F Keyvan, A Crimi, T van Walsum & H Kuijf (eds), Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11383 LNCS, Springer Verlag, pp. 57-69, 4th International MICCAI Brainlesion Workshop, BrainLes 2018 held in conjunction with the Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2018, Granada, Spain, 16/09/18. https://doi.org/10.1007/978-3-030-11723-8_6

CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels. / Sepahvand, Nazanin Mohammadi; Hassner, Tal; Arnold, Douglas L. et al.
Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers. ed. / Spyridon Bakas; Mauricio Reyes; Farahani Keyvan; Alessandro Crimi; Theo van Walsum; Hugo Kuijf. Springer Verlag, 2019. p. 57-69 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11383 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels

AU - Sepahvand, Nazanin Mohammadi

AU - Hassner, Tal

AU - Arnold, Douglas L.

AU - Arbel, Tal

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2019.

PY - 2019

Y1 - 2019

N2 - New T2w and gadolineum-enhancing lesions in Magnetic Resonance Images (MRI) are indicators of new disease activity in Multiple Sclerosis (MS) patients. Predicting future disease activity could help predict the progression of the disease as well as efficacy of treatment. We introduce a convolutional neural network (CNN) framework for future MRI disease activity prediction in relapsing-remitting MS (RRMS) patients from multi-modal MR images at baseline and illustrate how the inclusion of T2w lesion labels at baseline can significantly improve prediction accuracy by drawing the attention of the network to the location of lesions. Next, we develop a segmentation network to automatically infer lesion labels when semi-manual expert lesion labels are unavailable. Both prediction and segmentation networks are trained and tested on a large, proprietary, multi-center, multi-modal, clinical trial dataset consisting of 1068 patients. Testing based on a dataset of 95 patients shows that our framework reaches very high performance levels (sensitivities of 80.11% and specificities of 79.16%) when semi-manual expert labels are included as input at baseline in addition to multi-modal MRI. Even with inferred lesion labels replacing semi-manual labels, the method significantly outperforms an identical end-to-end CNN which only includes baseline multi-modal MRI.

AB - New T2w and gadolineum-enhancing lesions in Magnetic Resonance Images (MRI) are indicators of new disease activity in Multiple Sclerosis (MS) patients. Predicting future disease activity could help predict the progression of the disease as well as efficacy of treatment. We introduce a convolutional neural network (CNN) framework for future MRI disease activity prediction in relapsing-remitting MS (RRMS) patients from multi-modal MR images at baseline and illustrate how the inclusion of T2w lesion labels at baseline can significantly improve prediction accuracy by drawing the attention of the network to the location of lesions. Next, we develop a segmentation network to automatically infer lesion labels when semi-manual expert lesion labels are unavailable. Both prediction and segmentation networks are trained and tested on a large, proprietary, multi-center, multi-modal, clinical trial dataset consisting of 1068 patients. Testing based on a dataset of 95 patients shows that our framework reaches very high performance levels (sensitivities of 80.11% and specificities of 79.16%) when semi-manual expert labels are included as input at baseline in addition to multi-modal MRI. Even with inferred lesion labels replacing semi-manual labels, the method significantly outperforms an identical end-to-end CNN which only includes baseline multi-modal MRI.

KW - Deep learning

KW - Disease activity

KW - Magnetic resonance imaging

KW - Multiple sclerosis

UR - http://www.scopus.com/inward/record.url?scp=85063474814&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-11723-8_6

DO - 10.1007/978-3-030-11723-8_6

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85063474814

SN - 9783030117221

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 57

EP - 69

BT - Brainlesion

A2 - Bakas, Spyridon

A2 - Reyes, Mauricio

A2 - Keyvan, Farahani

A2 - Crimi, Alessandro

A2 - van Walsum, Theo

A2 - Kuijf, Hugo

PB - Springer Verlag

T2 - 4th International MICCAI Brainlesion Workshop, BrainLes 2018 held in conjunction with the Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2018

Y2 - 16 September 2018 through 20 September 2018

ER -

Sepahvand NM, Hassner T, Arnold DL, Arbel T. CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels. In Bakas S, Reyes M, Keyvan F, Crimi A, van Walsum T, Kuijf H, editors, Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers. Springer Verlag. 2019. p. 57-69. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-11723-8_6

CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this