Representation learning in limited and missing data scenarios

Clinical outcome or severity prediction from medical images has largely focused on learning representations from single-timepoint or snapshot scans, often from multiple modalities. It has been shown that disease progression can be better characterized by temporal imaging. However, temporal and multi-modal imaging data may not always be available in the model inference phase. To address this, our group has been developing  1) deep learning approaches that leverage temporal progression information to improve clinical outcome predictions from
single-timepoint images and 2) adversarial learning strategies to tackle missing modalities without synthesizing them in image or latent space. 

Representative publications: [1]

Multi-modal data fusion techniques for understanding COVID-19 progression and predicting clinical outcomes

COVID-19 image analysis has mostly focused on diagnostic tasks using single timepoint scans acquired upon disease presentation or admission. We have been working on developing deep learning and radiomics-based approaches to predict lung infiltrate progression from serial chest radiographs (CXRs) of COVID-19 patients. Further, we are working on fusing imaging and clinical data toward understanding the clinical trajectory of COVID-19. This involves development of new feature fusion and temporal modeling techniques.

Representative publications: [1], [2], [3]

Radiomics for Breast Cancer management

A patient’s response to pre-op chemotherapy may provide prognostic information that can supplement conventional prognostic data, such as initial staging, tumor grade, and receptor status. In fact, patients who achieve a pathological complete response (pCR), typically defined as the absence of residual invasive disease in the breast and axillary nodes at surgery, have improved long-term recurrence-free and overall survival compared to those who do not. Unfortunately, the current clinical workflow allows for the assessment of clinical response only at the end of pre-op therapy, thus precluding the switch to more aggressive pre-op therapy. The development of better predictive analytic techniques to assess response earlier in the treatment regimen would allow an early intervention to switch to another regimen if pCR is not likely, improving the chance of a better response to therapy. Towards this end, we are working on developing geometry and topology-inspired radiomic pipelines that can provide insight into the underlying tumor biology as reflected on radiologic imaging.

Representative publications: [1], [2]

Response monitoring in lung immunotherapy using machine learning on serial CT

Currently, there are no predictive biomarkers to point to whether non-small cell lung cancer (NSLC) patients will benefit from immune-checkpoint inhibitor therapy, a new form of cancer therapy that helps the body’s immune system fight cancer more effectively. We use radiomic techniques to find previously unseen changes in patterns, both intra- and peri-nodular, in CT scans taken when the lung cancer is first diagnosed compared to scans taken after the first 2-3 cycles of immunotherapy treatment.

Using multiple datasets for training and validation, our models were able to identify responders from non-responders as well as determine early response to immunotherapy. We also identified that the radiological features were associated with greater infiltration of immune cells into lung tissue, based on diagnostic biopsies performed on some of the patients in the study.