Publications

2021

1. IMWUT, 22

   BreathTrack: Detecting Regular Breathing Phases from Unannotated Acoustic Data Captured by a Smartphone

   Islam, Bashima, Rahman, Md Mahbubur, Ahmed, Tousif, Ahmed, Mohsin Yusuf, Hasan, Md Mehedi, Nathan, Viswam, Vatanparvar, Korosh, Nemati, Ebrahim, Kuang, Jilong, and Gao, Jun Alex

   Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2021

   Abs

   Breathing biomarkers, such as breathing rate, fractional inspiratory time, and inhalation-exhalation ratio, are vital for monitoring the user’s health and well-being. Accurate estimation of such biomarkers requires breathing phase detection, i.e., inhalation and exhalation. However, traditional breathing phase monitoring relies on uncomfortable equipment, e.g., chestbands. Smartphone acoustic sensors have shown promising results for passive breathing monitoring during sleep or guided breathing. However, detecting breathing phases using acoustic data can be challenging for various reasons. One of the major obstacles is the complexity of annotating breathing sounds due to inaudible parts in regular breathing and background noises. This paper assesses the potential of using smartphone acoustic sensors for passive unguided breathing phase monitoring in a natural environment. We address the annotation challenges by developing a novel variant of the teacher-student training method for transferring knowledge from an inertial sensor to an acoustic sensor, eliminating the need for manual breathing sound annotation by fusing signal processing with deep learning techniques. We train and evaluate our model on the breathing data collected from 131 subjects, including healthy individuals and respiratory patients. Experimental results show that our model can detect breathing phases with 77.33% accuracy using acoustic sensors. We further present an example use-case of breathing phase-detection by first estimating the biomarkers from the estimated breathing phases and then using these biomarkers for pulmonary patient detection. Using the detected breathing phases, we can estimate fractional inspiratory time with 92.08% accuracy, the inhalation-exhalation ratio with 86.76% accuracy, and the breathing rate with 91.74% accuracy. Moreover, we can distinguish respiratory patients from healthy individuals with up to 76% accuracy. This paper is the first to show the feasibility of detecting regular breathing phases towards passively monitoring respiratory health and well-being using acoustic data captured by a smartphone.

2. ASSETS,20

   Shared Privacy Concerns of the Visually Impaired and Sighted Bystanders with Camera Based Assistive Technologies

   Akter, Taslima, Ahmed, Tousif, Kapadia, Apu, and Swaminathan, Manohar

   ACM Trans. Access. Comput. 2021

   Abs

   Camera based assistive technologies can provide people with visual impairments (PVIs) visually derived information about people in their vicinity. Furthermore, the advent of smart glasses offers the possibility of not only analyzing visual information in front of the wearer, but also behind them through an extended field of view. Although such ‘visually available’ information can enhance one’s social interactions, the privacy and ethical implications for automated judgments about bystanders especially from the perspective of PVIs remains underexplored. To study the concerns of both bystanders and PVIs with such technologies, we conducted two online surveys with visually impaired participants as wearers (N=128) and sighted participants as bystanders (N=136). Although PVIs found some types of information to be improper or impolite (such as someone’s weight), our overarching finding is the shared ethical concern between PVIs and bystanders related to the fallibility of AI, where bystanders can be misrepresented (algorithmically) by the devices. These mischaracterizations can range from occasional, unexpected algorithmic errors (e.g., errors in facial recognition) to the questionable use of AI for determining subjective, social constructs (such as gender). Based on our findings, we discuss the design implications and directions for future work in the development of camera based assistive technologies while mitigating the ethical concerns of PVIs and bystanders.

3. EMBC, 21

   RRMonitor: A Resource-Aware End-to-End System for Continuous Monitoring of Respiration Rate Using Earbuds

   Ahmed, Tousif, M, Md Mahbubur Rahman, Ahmed, Mohsin, Nemati, Ebrahim, Ding, Minh, Folkman, Nathan, Kuang, Jilong, and Gao, Alex

   In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC) 2021

   Abs

   Respiration rate is considered as a critical vital sign, and daily monitoring of respiration rate could provide helpful information about any acute condition in the human body. While researchers have been exploring mobile devices for respiration rate monitoring, passive and continuous monitoring is still not feasible due to many usability challenges (e.g., active participation) in existing approaches. This paper presents an end-to-end system called RRMonitor that leverages the movement sensors from commodity earbuds to continuously monitor the respiration rate in near real-time. While developing the systems, we extensively explored some key parameters, algorithms, and approaches from existing literature that are better suited for continuous and passive respiration rate monitoring. RRMonitor can passively track the respiration rate with a mean absolute error as low as 1.64 cycles per minute without requiring active participation from the user.

4. Patent

   Adaptive respiratory condition assessment

   Rahman, Md Mahbubur, Ahmed, Tousif, Ahmed, Mohsin, Viswam, , Nemati, Ebrahim, Vatanparvar, Korosh, Kuang, Jilong, and Gao, Alex

   2021
5. Patent

   System and method for passive subject specific monitoring

   Vatanparvar, Korosh, Ahmed, Tousif, Nathan, Viswam, Nematihosseinabadi, Ebrahim, Rahman, Md Mahbubur, Kuang, Jilong, and Gao, Alex

   2021
6. IMWUT, 21

   Listen2Cough: Leveraging End-to-End Deep Learning Cough Detection Model to Enhance Lung Health Assessment Using Passively Sensed Audio

   Xu, Xuhai, Nemati, Ebrahim, Vatanparvar, Korosh, Nathan, Viswam, Ahmed, Tousif, Rahman, Md Mahbubur, McCaffrey, Daniel, Kuang, Jilong, and Gao, Jun Alex

   Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2021

   Abs

   The prevalence of ubiquitous computing enables new opportunities for lung health monitoring and assessment. In the past few years, there have been extensive studies on cough detection using passively sensed audio signals. However, the generalizability of a cough detection model when applied to external datasets, especially in real-world implementation, is questionable and not explored adequately. Beyond detecting coughs, researchers have looked into how cough sounds can be used in assessing lung health. However, due to the challenges in collecting both cough sounds and lung health condition ground truth, previous studies have been hindered by the limited datasets. In this paper, we propose Listen2Cough to address these gaps. We first build an end-to-end deep learning architecture using public cough sound datasets to detect coughs within raw audio recordings. We employ a pre-trained MobileNet and integrate a number of augmentation techniques to improve the generalizability of our model. Without additional fine-tuning, our model is able to achieve an F1 score of 0.948 when tested against a new clean dataset, and 0.884 on another in-the-wild noisy dataset, leading to an advantage of 5.8% and 8.4% on average over the best baseline model, respectively. Then, to mitigate the issue of limited lung health data, we propose to transform the cough detection task to lung health assessment tasks so that the rich cough data can be leveraged. Our hypothesis is that these tasks extract and utilize similar effective representation from cough sounds. We embed the cough detection model into a multi-instance learning framework with the attention mechanism and further tune the model for lung health assessment tasks. Our final model achieves an F1-score of 0.912 on healthy v.s. unhealthy, 0.870 on obstructive v.s. non-obstructive, and 0.813 on COPD v.s. asthma classification, outperforming the baseline by 10.7%, 6.3%, and 3.7%, respectively. Moreover, the weight value in the attention layer can be used to identify important coughs highly correlated with lung health, which can potentially provide interpretability for expert diagnosis in the future.

7. CSCW, 21

   Does This Photo Make Me Look Good? How Posters, Outsiders, and Friends Evaluate Social Media Photo Posts

   Das, Sanchari, Ahmed, Tousif, Kapadia, Apu, and Patil, Sameer

   Proc. ACM Hum.-Comput. Interact. 2021

   Abs

   In recent years, the use and importance of visual communication through photos have grown considerably. However, we have little understanding of the alignment between the intentions of the photo posters and the reactions of viewers. To address this gap, we replicated previous work that studied the alignment of poster and outsider judgments of text posts by extending it to photo posts. In our study of 573 users across four social media platforms, we found that outsiders generally judge photo posts more positively than anticipated by posters. Examining viewer engagement on social media revealed that photos depicting family and friends receive fewer reactions. We apply our insight to propose novel solutions that can help users create a more positive digital presence by aligning their photo posts with the expectations of their audiences.

2020

1. CHI, 20

   Assessing Severity of Pulmonary Obstruction from Respiration Phase-Based Wheeze-Sensing Using Mobile Sensors

   Chatterjee, Soujanya, Rahman, Md Mahbubur, Ahmed, Tousif, Saleheen, Nazir, Nemati, Ebrahim, Nathan, Viswam, Vatanparvar, Korosh, and Kuang, Jilong

   2020

   Abs HTML

   Obstructive pulmonary diseases cause limited airflow from the lung and severely affect patients’ quality of life. Wheeze is one of the most prominent symptoms for them. High requirements imposed by traditional diagnosis methods make regular monitoring of pulmonary obstruction challenging, which hinders the opportunity of early intervention and prevention of significant exacerbation. In this work, we explore the feasibility of developing a mobile sensor-based system as a convenient means of assessing the severity of pulmonary obstruction via respiration phase-based symptomatic wheeze sensing. We conduct a 131 subjects’ (91 patients and 40 healthy) study for the detection (F1: 87.96%) and characterization (F1: 79.47%) of wheeze. Subsequently, we develop novel wheeze metrics, which show a significant correlation (Pearson’s correlation: -0.22, p-value: 0.024) with standard spirometry measure of pulmonary obstruction severity. This work takes a principal step towards the unobtrusive assessment of pulmonary condition from mobile sensor interactions.

2. USENIX Security, 20

   "I am uncomfortable sharing what I can’t see": Privacy Concerns of the Visually Impaired with Camera Based Assistive Applications

   Akter, Taslima, Dosono, Bryan, Ahmed, Tousif, Kapadia, Apu, and Semaan, Bryan

   In 29th USENIX Security Symposium (USENIX Security 20) 2020

   Abs

   The emergence of camera-based assistive technologies has empowered people with visual impairments (VIP) to obtain independence in their daily lives. Popular services feature volunteers who answer questions about photos or videos (e.g., to identify a medical prescription). However, people with VIPs can (inadvertently) reveal sensitive information to these volunteers. To better understand the privacy concerns regarding the disclosure of background objects to different types of human assistants (friends, family, and others), we conducted an online survey with 155 visually impaired participants. In general, our participants had varying concerns depending on the type of assistants and the kind of information. We found that our participants were more concerned about the privacy of bystanders than their own when capturing people in images. We also found that participants were concerned about self-presentation and were more comfortable sharing embarrassing information with family than with their friends. Our findings suggest directions for future work in the development of human-assisted question-answering systems. Specifically, we discuss how humanizing these systems can give people a greater sense of personal security.

3. ASSETS, 20

   Privacy Considerations of the Visually Impaired with Camera Based Assistive Technologies: Misrepresentation, Impropriety, and Fairness

   Akter, Taslima, Ahmed, Tousif, Kapadia, Apu, and Swaminathan, Swami Manohar

   In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility 2020

   Abs

   Camera based assistive technologies such as smart glasses can provide people with visual impairments (PVIs) information about people in their vicinity. Although such ‘visually available’ information can enhance one’s social interactions, the privacy implications for bystanders from the perspective of PVIs remains underexplored. Motivated by prior findings of bystanders’ perspectives, we conducted two online surveys with visually impaired (N=128) and sighted (N=136) participants with two ‘field-of-view’ (FoV) experimental conditions related to whether information about bystanders was gathered from the front of the glasses or all directions. We found that PVIs considered it as ‘fair’ and equally useful to receive information from all directions. However, they reported being uncomfortable in receiving some visually apparent information (such as weight and gender) about bystanders as they felt it was ‘impolite’ or ‘improper’. Both PVIs and bystanders shared concerns about the fallibility of AI, where bystanders can be misrepresented by the devices. Our finding suggests that beyond issues of social stigma, both PVIs and bystanders have shared concerns that need to be considered to improve the social acceptability of camera based assistive technologies.

4. ICMI, 20

   Automated Time Synchronization of Cough Events from Multimodal Sensors in Mobile Devices

   Ahmed, Tousif, Ahmed, M., Rahman, M., Nemati, E., Islam, B., Vatanparvar, K., Nathan, V., McCaffrey, D., Kuang, J., and Gao, A.

   In Proceedings of the 2020 International Conference on Multimodal Interaction 2020

   Abs

   Tracking the type and frequency of cough events is critical for monitoring respiratory diseases. Coughs are one of the most common symptoms of respiratory and infectious diseases like COVID-19, and a cough monitoring system could have been vital in remote monitoring during a pandemic like COVID-19. While the existing solutions for cough monitoring use unimodal (e.g., audio) approaches for detecting coughs, a fusion of multimodal sensors (e.g., audio and accelerometer) from multiple devices (e.g., phone and watch) are likely to discover additional insights and can help to track the exacerbation of the respiratory conditions. However, such multimodal and multidevice fusion requires accurate time synchronization, which could be challenging for coughs as coughs are usually concise events (0.3-0.7 seconds). In this paper, we first demonstrate the time synchronization challenges of cough synchronization based on the cough data collected from two studies. Then we highlight the performance of a cross-correlation based time synchronization algorithm on the alignment of cough events. Our algorithm can synchronize 98.9% of cough events with an average synchronization error of 0.046s from two devices.

5. MobileHCI, 20

   Lung Function Estimation from a Monosyllabic Voice Segment Captured Using Smartphones

   Saleheen, Nazir, Ahmed, Tousif, Rahman, Md Mahbubur, Nemati, Ebrahim, Nathan, Viswam, Vatanparvar, Korosh, Blackstock, Erin, and Kuang, Jilong

   2020

   Abs

   Chronic respiratory diseases refer to a group of lung diseases that affect the airways and cause difficulty in breathing. Respiratory diseases are one of the leading causes of death and negatively impact the patients’ quality of life. Early detection and regular monitoring of lung functions might reduce the risk of death; however, lung function assessment requires the active supervision of a medical professional in a clinical setting. To make lung function tests more accessible and ubiquitous, researchers started leveraging mobile devices, which still require active supervision and demand extraneous effort from the user. In this work, we propose a convenient mobile-based approach that uses a monosyllabic voice segment called ‘A-vowel’ sound or ‘Aaaa...’ sound to estimate lung function. We conducted two studies (a lab study and an in-clinic study) with 201 participants to develop a detection model detecting ‘A-vowel’ sound from other acoustic events and a prediction model to estimate the lung function using the detected A-vowel sound. Our study shows that A-vowel sounds can be detected with 93% accuracy, and A-vowel sounds can estimate lung functions with 7.4-11.35% mean absolute error. We also conducted a validation study with 10 participants in a noisy environment and able to detect A-vowel segments with 71% F1-Score. Our results show auspicious directions to expand the horizon of mobile-based lung assessment.

2019

2018

1. SOUPS, 18

   "You dont want to be the next meme": College Students Workarounds to Manage Privacy in the Era of Pervasive Photography

   Rashidi, Yasmeen, Ahmed, Tousif, Patel, Felicia, Fath, Emily, Kapadia, Apu, Nippert-Eng, Christena, and Su, Norman Makoto

   In Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018) 2018

   Abs

   Pervasive photography and the sharing of photos on social media pose a significant challenge to undergraduates’ ability to manage their privacy. Drawing from an interview-based study, we find undergraduates feel a heightened state of being surveilled by their peers and rely on innovative workarounds—negotiating the terms and ways in which they will and will not be recorded by technology-wielding others—to address these challenges. We present our findings through an experience model of the life span of a photo, including an analysis of college students’ workarounds to deal with the technological challenges they encounter as they manage potential threats to privacy at each of our proposed four stages. We further propose a set of design directions that address our users’ current workarounds at each stage. We argue for a holistic perspective on privacy management that considers workarounds across all these stages. In particular, designs for privacy need to more equitably distribute the technical power of determining what happens with and to a photo among all the stakeholders of the photo, including subjects and bystanders, rather than the photographer alone.

2. IMWUT, 18

   Up to a Limit? Privacy Concerns of Bystanders and Their Willingness to Share Additional Information with Visually Impaired Users of Assistive Technologies

   Ahmed, Tousif, Kapadia, Apu, Potluri, Venkatesh, and Swaminathan, Manohar

   In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2018

   Abs

   The emergence of augmented reality and computer vision based tools offer new opportunities to visually impaired persons (VIPs). Solutions that help VIPs in social interactions by providing information (age, gender, attire, expressions etc.) about people in the vicinity are becoming available. Although such assistive technologies are already collecting and sharing such information with VIPs, the views, perceptions, and preferences of sighted bystanders about such information sharing remain unexplored. Although bystanders may be willing to share more information for assistive uses it remains to be explored to what degree bystanders are willing to share various kinds of information and what might encourage additional sharing of information based on the contextual needs of VIPs. In this paper we describe the first empirical study of information sharing preferences of sighted bystanders of assistive devices. We conducted a survey based study using a contextual method of inquiry with 62 participants followed by nine semi-structured interviews to shed more insight on our key quantitative findings. We find that bystanders are more willing to share some kinds of personal information with VIPs and are willing to share additional information if higher security assurances can be made by improving their control over how their information is shared.

2017

1. IEEE Journal

   Understanding Physical Safety, Security, and Privacy Concerns of People with Visual Impairments

   Ahmed, Tousif, Hoyle, Roberto, Shaffer, Patrick, Connelly, Kay, Crandall, David, and Kapadia, Apu

   In IEEE Internet Computing 2017

   Abs

   With the help of various assistive devices, people with visual impairments are able to live their lives with greater independence both online and offline. But significant work remains to understand and address their safety, security, and privacy concerns, especially in the physical, offline world. For example, people with visual impairments are particularly vulnerable to physical assault and theft, shoulder-surfing attacks, and being overheard during private conversations. The authors conducted two sets of interviews – one in a small-town setting and the other in a larger metropolitan area – with 33 participants with visual impairments to identify their physical safety, security, and privacy issues and find out how they manage these concerns and how assistive technologies could help alleviate them. The resulting research proposes design considerations for camera-based devices that would help people with visual impairments monitor for potential threats around them.

2. PoPET, 17

   To Permit or Not to Permit, That is the Usability Question: Crowdsourcing Mobile Apps Privacy Permission Settings

   Ismail, Q., Ahmed, Tousif, Caine, K., Kapadia, A., and Reiter, M.

   In Proceedings on Privacy Enhancing Technologies (PoPETs) 2017

   Abs

   Millions of apps available to smartphone owners request various permissions to resources on the devices including sensitive data such as location and contact information. Disabling permissions for sensitive resources could improve privacy but can also impact the usability of apps in ways users may not be able to predict. We study an efficient approach that ascertains the impact of disabling permissions on the usability of apps through large-scale, crowdsourced user testing with the ultimate goal of making recommendations to users about which permissions can be disabled for improved privacy without sacrificing usability. We replicate and significantly extend previous analysis that showed the promise of a crowdsourcing approach where crowd workers test and report back on various configurations of an app. Through a large, between-subjects user experiment, our work provides insight into the impact of removing permissions within and across different apps (our participants tested three apps: Facebook Messenger (N=218), Instagram (N=227), and Twitter (N=110)). We study the impact of removing various permissions within and across apps, and we discover that it is possible to increase user privacy by disabling app permissions while also maintaining app usability.

2016

1. SOUPS, 16

   Addressing Physical Safety, Security, and Privacy for People with Visual Impairments

   Ahmed, Tousif, Shaffer, Patrick, Connelly, Kay, Crandall, David, and Kapadia, Apu

   In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016) 2016

   Abs

   People with visual impairments face a variety of obstacles in their daily lives. Recent work has identified specific physical privacy concerns of this population and explored how emerging technology, such as wearable devices, could help. In this study we investigated their physical safety and security concerns and behaviors by conducting interviews (N=19) with participants who have visual impairments in the greater San Francisco metropolitan area. Our participants’ detailed accounts shed light on (1) the safety and security concerns of people with visual impairments in urban environments (such as feared and real instances of assault); (2) their behaviors for protecting physical safety (such as avoidance and mitigation strategies); and (3) refined design considerations for future assistive wearable devices that could enhance their awareness of surrounding threats.

2015

1. CHI, 15

   Privacy Concerns and Behaviors of People with Visual Impairments

   Ahmed, Tousif, Hoyle, Roberto, Connelly, Kay, Crandall, David, and Kapadia, Apu

   In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems 2015

   Abs

   Various technologies have been developed to help make the world more accessible to visually impaired people, and recent advances in low-cost wearable and mobile computing are likely to drive even moreadvances. However, the unique privacy and security needs of visually impaired people remain largely unaddressed. We conducted an exploratory user study with 14 visually impaired participants to understand the techniques they currently use for protecting privacy, their remaining privacy concerns,and how new technologies may be able to help. The interviews explored privacy not only in the physical world (e.g., bystanders overhearing private conversations) and the online world (e.g., determining if a URL is legitimate), but also in the interface between the two (e.g. bystanders ‘shoulder-surfing’ data from screens). The study revealed serious concerns that are not adequately solved by current technology, and suggested new directions for improving the privacy of this significant fraction of the population.

2. CHI,15

   Crowdsourced Exploration of Security Configurations

   Ismail, Qatrunnada, Ahmed, Tousif, Kapadia, Apu, and Reiter, Michael K.

   In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems 2015

   Abs PDF

   Smartphone apps today request permission to access a multitude of sensitive resources, which users must accept completely during installation (e.g., on Android) or selectively configure after installation (e.g., on iOS, but also planned for Android). Everyday users, however, do not have the ability to make informed decisions about which permissions are essential for their usage. For enhanced privacy, we seek to leverage crowdsourcing to find minimal sets of permissions that will preserve the usability of the app for diverse users. We advocate an efficient ’lattice-based’ crowd-management strategy to explore the space of permissions sets. We conducted a user study (N = 26) in which participants explored different permission sets for the popular Instagram app. This study validates our efficient crowd management strategy and shows that usability scores for diverse users can be predicted accurately, enabling suitable recommendations.