2026
-
No Pixel Left Behind: Filling Gaps in Anime Colorization
CHI 2026
Masahiro Kono, Akinobu Maejima, Yuki Koyama, Yotam Sechayk, and Takeo IgarashiIn Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 2026Animation production workflows often involve digital colorization of line art, where small unpainted regions ("gaps") frequently occur and remain an underexplored challenge. We conducted a formative study in Japanese animation (anime) pipelines and found that while the paint bucket tool is widely used for base coloring, tiny enclosed areas are frequently overlooked, resulting in time-consuming manual detection and filling. We introduce GapFill, a tool grounded in professional practices that reduces the effort of gap detection, zooming, and color selection. Our deep-learning method suggests appropriate fill colors by referencing surrounding regions, leveraging the flat-color nature of anime-style images. In a user study with 13 professional colorists, our system improved performance and usability in gap-filling tasks over conventional methods. The study also suggested that prediction accuracy alone is not the primary factor for usability, that appropriate colors can be contextually ambiguous, and that GapFill can complement existing tools depending on users’ trust in new AI-powered assistance.
@inproceedings{kono2026nopixel, author = {Kono, Masahiro and Maejima, Akinobu and Koyama, Yuki and Sechayk, Yotam and Igarashi, Takeo}, title = {No Pixel Left Behind: Filling Gaps in Anime Colorization}, year = {2026}, isbn = {9798400722783}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3772318.3790968}, booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems}, articleno = {300}, numpages = {19}, series = {CHI '26}, language = {english}, preview_alt = {First page of the No Pixel Left Behind paper, showing the title, authors, abstract, and a teaser figure of anime line art with unpainted gaps being detected and filled with suggested colors.} } -
Improving Low-Vision Chart Accessibility via On-Cursor Visual Context
CHI 2026
In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 2026Despite widespread use, charts remain largely inaccessible for Low-Vision Individuals (LVI). Reading charts requires viewing data points within a global context, which is difficult for LVI who may rely on magnification or experience a partial field of vision. We aim to improve exploration by providing visual access to critical context. To inform this, we conducted a formative study with five LVI. We identified four fundamental contextual elements common across chart types: axes, legend, grid lines, and the overview. We propose two pointer-based interaction methods to provide this context: Dynamic Context, a novel focus+context interaction, and Mini-map, which adapts overview+detail principles for LVI. In a study with N=22 LVI, we compared both methods and evaluated their integration to current tools. Our results show that Dynamic Context had significant positive impact on access, usability, and effort reduction; however, worsened visual load. Mini-map strengthened spatial understanding, but was less preferred for this task. We offer design insights to guide the development of future systems that support LVI with visual context while balancing visual load.
@inproceedings{sechayk2026visualcontext, author = {Sechayk, Yotam and Rave, Hennes and R\"{a}dler, Max and Colley, Mark and Zhou, Zhongyi and Shamir, Ariel and Igarashi, Takeo}, title = {Improving Low-Vision Chart Accessibility via On-Cursor Visual Context}, year = {2026}, isbn = {9798400722783}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3772318.3791165}, booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems}, articleno = {507}, numpages = {21}, series = {CHI '26}, language = {english}, preview_alt = {First page of the On-Cursor Visual Context paper, showing the title, authors, abstract, and a teaser figure of the Dynamic Context and Mini-map pointer interactions for reading charts with low vision.} } -
Exploring the Role of User Comments Throughout the Stages of Video-Based Task-Learning
CHI 2026 Poster
Nayoung Kim*, Yotam Sechayk*, Zhongyi Zhou, and Takeo IgarashiIn Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems, 2026Learning tasks through videos is a dynamic way to acquire skills by witnessing entire processes. However, compared to in-person demonstrations, videos may omit tacit knowledge, including subtle details and contextual nuances. Users’ unique circumstances, like missing ingredients in a recipe, may also require adaptation beyond the video content. To fill these gaps, many users turn to the comment section, seeking additional guidance and interactions with creators or peers to personalize their experience. Despite their importance, there is limited understanding of how users engage with and apply comments in task-learning scenarios. In our study, we explore the role of comments in video-based task-learning through interviews with 14 users, and co-watching sessions with eight. Our findings show that while comments are critical for learning, they are poorly integrated into all stages of the learning process. Based on our findings, we outline design opportunities to better utilize comments in video-based task-learning.
@inproceedings{kim2026enhancingrole, author = {Kim, Nayoung and Sechayk, Yotam and Zhou, Zhongyi and Igarashi, Takeo}, title = {Exploring the Role of User Comments Throughout the Stages of Video-Based Task-Learning}, year = {2026}, isbn = {9798400722813}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3772363.3799031}, booktitle = {Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems}, articleno = {283}, numpages = {7}, series = {CHI EA '26}, language = {english}, note = {poster}, preview_alt = {First page of the paper on the role of user comments in video-based task-learning, showing the title, authors, abstract, and a teaser figure of learners consulting video comments across the stages of learning a task.} } -
LandmarkLens: Predicting and Presenting Effective Landmarks for Mixed-Reality Urban Exploration
UIST 2026
Chu Li, Yotam Sechayk, Jared Hwang, Jon E. Froehlich, and Takeo IgarashiIn Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST ’26), 2026People with a poor sense of direction (SOD) struggle to build cognitive maps for effective spatial navigation, and existing navigation tools prioritize efficiency over spatial learning. To understand how navigation strategies differ by ability, we conducted a landmark attention study with 20 participants (ten good SOD, ten poor SOD) who navigated across four Tokyo neighborhoods in virtual reality (VR). We found systematic group differences in both gaze behavior and the types of landmarks they verbally identify as effective. Based on these findings, we built LandmarkLens, a mixed-reality (MR) navigation system that uses a vision-language model (VLM) to identify and highlight navigation-relevant landmarks. A follow-up study with eight poor-SOD participants showed improved performance in scene recognition, suggesting that guided landmark attention can support landmark-level spatial knowledge acquisition for people with poor SOD, a first step toward broader spatial learning.
@inproceedings{li2026landmarklens, author = {Li, Chu and Sechayk, Yotam and Hwang, Jared and Froehlich, Jon E. and Igarashi, Takeo}, title = {LandmarkLens: Predicting and Presenting Effective Landmarks for Mixed-Reality Urban Exploration}, year = {2026}, publisher = {Association for Computing Machinery}, address = {Detroit, MI, USA}, doi = {10.1145/3830398.3830505}, booktitle = {Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST '26)}, series = {UIST '26}, language = {english}, preview_alt = {First page of the LandmarkLens paper, showing the title, authors, abstract, and a teaser figure of a person walking with a mixed-reality headset while colored circles highlight predicted landmarks on a street and arrows point toward landmarks outside the field of view.} } -
Cyber-Physical Systems for Accessibility and Ability Augmentation: Bridging Diverse Communities
UIST 2026 Workshop
Shuchang Xu*, Riku Arakawa*, Mina Huh*, Nandi Zhang*, Tianyu Zhang*, Wazeer Zulfikar*, Ruei-Che Chang*, Yotam Sechayk*, Huamin Qu, Amy Pavel, Franklin Mingzhe Li, Yukang Yan, Brian A. Smith, and Pattie MaesIn Adjunct Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct ’26), 2026The powerful convergence of wearables, robotics, extended reality, and smart environments is expanding the design space for cyber-physical systems (CPS) that support and augment human abilities in daily life. By sensing real-world contexts, modeling user needs, and providing situated assistance, these systems can improve accessibility for people with disabilities while enhancing broader human abilities such as perception, memory, learning, and mobility. However, realizing this potential requires addressing key challenges in context sensing, user modeling, adaptive interaction, privacy, and evaluation to ensure that CPS are reliable and effective in real-world contexts. This workshop will bring together researchers and practitioners across HCI, AI, wearables, robotics, XR, smart environments, accessibility, and ability augmentation to examine shared strategies and challenges for designing accessibility- and ability-centered CPS. Through panel discussions, interactive demos, and mixed-group design activities, participants will identify recurring design principles, technical challenges, and future directions for CPS that support and augment human abilities in real-world settings.
@inproceedings{xu2026cps4all, author = {Xu, Shuchang and Arakawa, Riku and Huh, Mina and Zhang, Nandi and Zhang, Tianyu and Zulfikar, Wazeer and Chang, Ruei-Che and Sechayk, Yotam and Qu, Huamin and Pavel, Amy and Li, Franklin Mingzhe and Yan, Yukang and Smith, Brian A. and Maes, Pattie}, title = {Cyber-Physical Systems for Accessibility and Ability Augmentation: Bridging Diverse Communities}, year = {2026}, isbn = {9798400728556}, publisher = {Association for Computing Machinery}, address = {Detroit, MI, USA}, doi = {10.1145/3830397.3842691}, booktitle = {Adjunct Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct '26)}, numpages = {5}, series = {UIST Adjunct '26}, language = {english}, note = {workshop}, preview_alt = {First page of the workshop proposal on cyber-physical systems for accessibility and ability augmentation, showing the title, the fourteen organizers and their affiliations, and the start of the abstract. This page has no figures.} } -
Ups and Downs: Low-Vision Perspective on Smart Elevators and Industry Design Practices
ASSETS 2026 Experience report
Masaki Sawada, Yotam Sechayk, and Manabi MiyagiIn Proceedings of the 28th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’26), 2026Elevators are essential for accessing workplaces, educational institutions, transportation hubs, and other public spaces. Yet the experiences of people with low vision (LV) when using elevators remain largely unexplored. In this work, we draw on the lived experience of the first author, who identifies as having LV, and reflexive conversations with three additional LV individuals and one participant whose LV progressed to blindness. We complement these lived experiences with a site visit to a major Japanese elevator manufacturer, during which the first author directly experienced and evaluated existing and emerging accessibility features. By bringing together user and industry perspectives, we reveal challenges that arise in everyday elevator use, identify gaps in existing accessibility approaches, and highlight opportunities for future research and design.
@inproceedings{sawada2026upsanddowns, author = {Sawada, Masaki and Sechayk, Yotam and Miyagi, Manabi}, title = {Ups and Downs: Low-Vision Perspective on Smart Elevators and Industry Design Practices}, year = {2026}, isbn = {9798400725210}, publisher = {Association for Computing Machinery}, address = {Vila Nova de Gaia, Portugal}, doi = {10.1145/3797867.3832759}, booktitle = {Proceedings of the 28th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '26)}, numpages = {7}, series = {ASSETS '26}, language = {english}, note = {experience report}, preview_alt = {First page of the Ups and Downs paper, showing the title, authors, abstract, and a teaser figure contrasting how low-vision and sighted people experience finding and using an elevator.} } -
See What I Mean: A Technology Probe of AI Assistants for Chart Exploration with Low Vision
ASSETS 2026 Poster
Yotam Sechayk, Lucy Jiang, Yuhang Zhao, and Takeo IgarashiIn Proceedings of the 28th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’26), 2026Charts remain inaccessible to low vision individuals (LVI), who commonly magnify local details at the cost of the overview. AI assistants can help, but prior work has largely served non-visual access, while LVI prefer using residual vision. We built ChartAEye, a technology probe that lets LVI interact with an AI assistant visually, sharing their magnified view and point of interest as system context. Using the probe, we then asked whether the AI’s feedback also needs to be adapted to low vision, comparing a generic AI persona (GAI) with one designed around LVI needs and lived experience (LVAI). In a within-subjects study, 12 LVI used ChartAEye to complement their vision, reduce the effort of viewing chart elements, and practice more independence. Eight of 12 participants preferred the LVAI, whose responses more often carried spatial guidance (71.2% vs. 7.8%) and semantic guidance of visuals (84.8% vs. 29.1%). We close with design recommendations for AI assistants that support LVI in exploring charts with their residual vision.
@inproceedings{sechayk2026seewhatimean, author = {Sechayk, Yotam and Jiang, Lucy and Zhao, Yuhang and Igarashi, Takeo}, title = {See What I Mean: A Technology Probe of AI Assistants for Chart Exploration with Low Vision}, year = {2026}, isbn = {9798400725210}, publisher = {Association for Computing Machinery}, address = {Vila Nova de Gaia, Portugal}, doi = {10.1145/3797867.3841222}, booktitle = {Proceedings of the 28th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '26)}, series = {ASSETS '26}, language = {english}, note = {poster}, preview_alt = {First page of the See What I Mean paper, showing the title, authors, abstract, and a teaser figure of an AI assistant receiving a magnified view of a bar chart and the point the user indicated, with answers from a generic AI persona and a low-vision AI persona side by side.} } -
PreView: Rehearsing Real Routes in VR to Support Independent Navigation with Low-Vision
UIST 2026 Demo
Yotam Sechayk*, Xinyun Cao*, Dhruv Jain, and Takeo IgarashiIn Adjunct Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct ’26), 2026Low-vision individuals (LVI) must focus on their immediate surroundings while walking, a rational response to the risk of unseen curbs and obstacles. This protective attention leaves the landmarks that anchor cognitive maps unattended, and most navigation support concentrates on the walk itself, when attention is scarcest. We present PreView, a system that turns real 360-degree route imagery into a VR rehearsal of a route that a LVI plans to walk: realistic enough to learn from, safe enough to look up in. PreView highlights salient, permanent landmarks, layers an imagery-derived soundscape with surface-matched footsteps, and answers questions through an AI agent grounded in the user’s current view. Rehearsing a route in PreView lets LVI form a cognitive map in advance and navigate it more independently. In this demo, attendees freely explore real captured routes and experience how multimodal AI augmentation turns commodity imagery into route knowledge that LVI could acquire safely before ever walking the street.
@inproceedings{sechayk2026preview, author = {Sechayk, Yotam and Cao, Xinyun and Jain, Dhruv and Igarashi, Takeo}, title = {PreView: Rehearsing Real Routes in VR to Support Independent Navigation with Low-Vision}, year = {2026}, isbn = {9798400728556}, publisher = {Association for Computing Machinery}, address = {Detroit, MI, USA}, doi = {10.1145/3830397.3842475}, booktitle = {Adjunct Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct '26)}, numpages = {3}, series = {UIST Adjunct '26}, language = {english}, note = {demo}, preview_alt = {First page of the PreView paper, showing the title, authors, abstract, and the opening of the introduction. This page has no figures.} } -
Visual Communication for Low-Vision Information Access: From Personalized Highlights to AI Agents
UIST 2026 Doctoral symposium
Yotam SechaykIn Adjunct Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct ’26), 2026Charts, slides, and web pages remain difficult to access for the estimated 295 million people globally with low vision (LV). Much accessibility research translates visual content into speech, sound, or touch, but LV people often prefer to use their residual vision. My dissertation follows ability-based design to create tools motivated by residual vision use. My ongoing work extends this to ability-based AI systems that adapt AI interaction to LV users’ abilities. The central method is visual communication between tools and LV users. My completed work explores tool-to-user communication: VeasyGuide highlights instructor actions for LV learners in videos, while on-cursor interactions restore chart context hidden by magnification. Both systems improved access in studies with LV participants. My ongoing work examines user-to-tool communication through an AI assistant that receives the user’s magnified view and point of interest during chart exploration. My planned work asks how AI can answer visually, combine both directions, and balance assistance with user agency. I plan to extend this communication beyond charts to the web, documents, and eventually the physical world. Ultimately, my dissertation advances personalized AI that uses bidirectional visual communication to support the independence and agency of LV people.
@inproceedings{sechayk2026visualcommunication, author = {Sechayk, Yotam}, title = {Visual Communication for Low-Vision Information Access: From Personalized Highlights to AI Agents}, year = {2026}, isbn = {9798400728556}, publisher = {Association for Computing Machinery}, address = {Detroit, MI, USA}, doi = {10.1145/3830397.3841706}, booktitle = {Adjunct Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct '26)}, numpages = {5}, series = {UIST Adjunct '26}, language = {english}, note = {doctoral symposium}, preview_alt = {First page of the doctoral symposium paper Visual Communication for Low-Vision Information Access, showing the title, author, abstract, and a teaser figure of the research trajectory from VeasyGuide and On-Cursor Visual Context to Visual Context for AI and future bidirectional visual communication with AI agents.} }
2025
-
Graph Guide: 低視力者支援のためのセマンティック Focus+Context グラフ表示
WISS 2025
In WISS 2025: 第33回インタラクティブシステムとソフトウェアに関するワークショップ, 2025Best Paper Award
データ可視化グラフは情報伝達の手段として広く利用されているが，弱視者（Low Vision Individuals: LVI）にとっては依然として困難を伴うものであり，注目している情報とその周囲にある文脈情報の両方にアクセスすることが難しい．LVI を支援する最も一般的なツールは画面拡大ツールであるが，一様な拡大によってグラフの一部が視野から外れ，パン操作やレイアウトの記憶が求められるため，高い認知的負荷を引き起こす．これらの課題を明らかにするため，5 名の LVI を対象とした予備的調査を実施した．その結果，参加者は凡例といった文脈的要素を推測に頼りながら探すことが多く，それが余分な認知的負荷を生じさせていた．これらの知見に基づき，Graph Guide を開発した．Graph Guide は Focus+Context の発想に基づき，視野外にある文脈的要素（例: 軸や凡例）を意味的に抽出し，視野内へ投影する手法である．さらに Graph Guide を検証するため，6 名の LVI 参加者を対象に予備的評価を実施した．その結果，Graph Guide は既存の従来ツールによるワークフローと比較して，知覚的アクセスを改善し，労力を削減し，システムユーザビリティ尺度（System Usability Scale: SUS）においてより高いスコアを達成した．本研究の知見は，新たな機能を画面拡大ツールと統合する際に生じる視覚的な情報過多の管理といった課題を明らかにしており，今後さらなる研究の必要性を示唆している．
@inproceedings{sechayk2025graphguide, author = {Sechayk, Yotam and Li, Yuan and Rave, Hennes and Colley, Mark and Shamir, Ariel and Igarashi, Takeo}, title = {Graph Guide: 低視力者支援のためのセマンティック Focus+Context グラフ表示}, booktitle = {WISS 2025: 第33回インタラクティブシステムとソフトウェアに関するワークショップ}, year = {2025}, publisher = {情報処理学会 (IPSJ)}, address = {Japan}, language = {japanese}, preview_alt = {First page of the Graph Guide paper, written in Japanese, showing the title, authors, abstract, and scatter plot figures illustrating semantic Focus+Context chart display for low-vision readers.} } -
VeasyGuide: Personalized Visual Guidance for Low-vision Learners on Instructor Actions in Presentation Videos
ASSETS 2025
In Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25), 2025Honorable Mention Award
Instructors often rely on visual actions such as pointing, marking, and sketching to convey information in educational presentation videos. These subtle visual cues often lack verbal descriptions, forcing low-vision (LV) learners to search for visual indicators or rely solely on audio, which can lead to missed information and increased cognitive load. To address this challenge, we conducted a co-design study with three LV participants and developed VeasyGuide, a tool that uses motion detection to identify instructor actions and dynamically highlight and magnify them. VeasyGuide produces familiar visual highlights that convey spatial context and adapt to diverse learners and content through extensive personalization and real-time visual feedback. VeasyGuide reduces visual search effort by clarifying what to look for and where to look. In an evaluation with 8 LV participants, learners demonstrated a significant improvement in detecting instructor actions, with faster response times and significantly reduced cognitive load. A separate evaluation with 8 sighted participants showed that VeasyGuide also enhanced engagement and attentiveness, suggesting its potential as a universally beneficial tool.
@inproceedings{sechayk2025veasyguide, title = {VeasyGuide: Personalized Visual Guidance for Low-vision Learners on Instructor Actions in Presentation Videos}, author = {Sechayk, Yotam and Shamir, Ariel and Pavel, Amy and Igarashi, Takeo}, booktitle = {Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '25)}, publisher = {Association for Computing Machinery}, address = {Denver, CO, USA}, year = {2025}, doi = {10.1145/3663547.3746372}, language = {english}, preview_alt = {First page of the VeasyGuide paper, showing the title, authors, and a teaser figure of the VeasyGuide video player with highlighted instructor actions and its personalization settings panels.} } -
Task Mode: Dynamic Filtering for Task-Specific Web Navigation using LLMs
ASSETS 2025
Ananya Gubbi Mohanbabu, Yotam Sechayk, and Amy PavelIn Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25), 2025Modern web interfaces are unnecessarily complex to use as they overwhelm users with excessive text and visuals unrelated to their current goals. This problem particularly impacts screen reader users (SRUs), who navigate content sequentially and may spend minutes traversing irrelevant elements before reaching desired information compared to vision users (VUs) who visually skim in seconds. We present Task Mode, a system that dynamically filters web content based on user-specified goals using large language models to identify and prioritize relevant elements while minimizing distractions. Our approach preserves page structure while offering multiple viewing modes tailored to different access needs. Our user study with 12 participants (6 VUs, 6 SRUs) demonstrates that our approach reduced task completion time for SRUs while maintaining performance for VUs, decreasing the completion time gap between groups from 2x to 1.2x. 11 of 12 participants wanted to use Task Mode in the future, reporting that Task Mode supported completing tasks with less effort and fewer distractions. This work demonstrates how designing new interactions simultaneously for visual and non-visual access can reduce rather than reinforce accessibility disparities in future technology created by human-computer interaction researchers and practitioners.
@inproceedings{mohanbabu2025taskmode, title = {Task Mode: Dynamic Filtering for Task-Specific Web Navigation using LLMs}, author = {Mohanbabu, Ananya Gubbi and Sechayk, Yotam and Pavel, Amy}, booktitle = {Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '25)}, publisher = {Association for Computing Machinery}, address = {Denver, CO, USA}, year = {2025}, doi = {10.1145/3663547.3746401}, language = {english}, preview_alt = {First page of the Task Mode paper, showing the title, authors, abstract, and a teaser figure of web pages dynamically filtered down to task-relevant content.} } -
A Longitudinal Autoethnography of Email Access for a Professional with Chronic Illness and ADHD: Preliminary Insights
ASSETS 2025 Poster/demo
Veronica Pimenova, Yotam Sechayk, Fabricio Murai, Andrew Hundt, and Shiri Dori-HacohenIn Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25), 2025Email is a foundational infrastructure of professional environments, yet for chronically ill and neurodivergent individuals, it often becomes an invisible barrier to access. We share preliminary insights from a 14-year autoethnography of a professional with chronic illness and attention-deficit/hyperactivity disorder (ADHD). We detail this professional’s iterative adaptation of mainstream email features into Mail++, their personalized workplace communication workflow for managing executive function challenges and chronic illness flares. We propose three emerging themes: (1) from hacks to assistive technology, (2) evolving access needs, and (3) toll of inaccessible systems. Based on our findings, we present initial design insights for accessible workplace communication systems. As future work in this ongoing study, we discuss a more in-depth qualitative analysis of the autoethnographic data, and formal user testing of the Mail++ approach with a population of professionals with chronic illness and ADHD to better inform the design of assistive workplace technology.
@inproceedings{pimenova2025longitudinal, title = {A Longitudinal Autoethnography of Email Access for a Professional with Chronic Illness and ADHD: Preliminary Insights}, author = {Pimenova, Veronica and Sechayk, Yotam and Murai, Fabricio and Hundt, Andrew and Dori-Hacohen, Shiri}, booktitle = {Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '25)}, publisher = {Association for Computing Machinery}, address = {Denver, CO, USA}, year = {2025}, doi = {10.1145/3663547.3759764}, language = {english}, note = {poster/demo}, preview_alt = {First page of the longitudinal autoethnography paper on email access for a professional with chronic illness and ADHD, showing the title, authors, and abstract.} } -
ImprovMate: Multimodal AI Assistant for Improv Actor Training
DIS 2025 Work-in-progress
Riccardo Drago, Yotam Sechayk, Mustafa Doga Dogan, Andrea Sanna, and Takeo IgarashiIn Companion Publication of the 2025 ACM Designing Interactive Systems Conference (DIS ’25 Companion), 2025Improvisation training for actors presents unique challenges, particularly in maintaining narrative coherence and managing cognitive load during performances. Previous research on AI in improvisation performance often predates advances in large language models (LLMs) and relies on human intervention. We introduce ImprovMate, which leverages LLMs as GPTs to automate the generation of narrative stimuli and cues, allowing actors to focus on creativity without keeping track of plot or character continuity. Based on insights from professional improvisers, ImprovMate incorporates exercises that mimic live training, such as abrupt story resolution and reactive thinking exercises, while maintaining coherence via reference tables. By balancing randomness and structured guidance, ImprovMate provides a groundbreaking tool for improv training. Our pilot study revealed that actors might embrace AI techniques if the latter mirrors traditional practices, and appreciate the fresh twist introduced by our approach with the AI-generated cues.
@inproceedings{drago2025improvmate, author = {Drago, Riccardo and Sechayk, Yotam and Dogan, Mustafa Doga and Sanna, Andrea and Igarashi, Takeo}, title = {ImprovMate: Multimodal AI Assistant for Improv Actor Training}, year = {2025}, isbn = {9798400714863}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3715668.3736363}, doi = {10.1145/3715668.3736363}, booktitle = {Companion Publication of the 2025 ACM Designing Interactive Systems Conference (DIS '25 Companion)}, pages = {526--532}, numpages = {7}, language = {english}, note = {work-in-progress}, demo = {https://tomfluff.github.io/ImprovMate/}, preview_alt = {First page of the ImprovMate paper, showing the title, authors, abstract, and a teaser figure of the ImprovMate interface generating narrative stimuli and cues for improv actor training.} } -
Confidence Estimation of Few-shot Patch-based Learning for Anime-style Colorization
SIGGRAPH 2025 Poster/demo
Yuexiang Ji, Akinobu Maejima, Yotam Sechayk, Yuki Koyama, and Takeo IgarashiIn Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters (SIGGRAPH Posters ’25), 2025In hand-drawn anime production, automatic colorization is used to boost productivity, where line drawings are automatically colored based on reference frames. However, the results sometimes include wrong color estimations, requiring artists to carefully inspect each region and correct colors—a time-consuming and labor-intensive task. To support this process, we propose a confidence estimation method that indicates the confidence level of colorization for each region of the image. Our method compares local patches in the colorized result and the reference frame.
@inproceedings{ji2025confidence, author = {Ji, Yuexiang and Maejima, Akinobu and Sechayk, Yotam and Koyama, Yuki and Igarashi, Takeo}, title = {Confidence Estimation of Few-shot Patch-based Learning for Anime-style Colorization}, year = {2025}, isbn = {9798400715495}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3721250.3742964}, doi = {10.1145/3721250.3742964}, booktitle = {Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters (SIGGRAPH Posters '25)}, articleno = {40}, numpages = {2}, keywords = {Automatic colorization, Line drawing, Confidence estimation}, series = {SIGGRAPH Posters '25}, language = {english}, note = {poster/demo}, preview_alt = {First page of the confidence estimation paper for anime-style colorization, showing the title, authors, abstract, and a figure of colorized anime frames with per-region confidence maps.} }
2024
-
MyStoryKnight: A Character-drawing Driven Storytelling System Using LLM Hallucinations
INTERACTION 2024 Poster/demo
Yotam Sechayk, Gabriela A. Penarska, Ingrid A. Randsalu, Christian Arzate Cruz, and Takeo IgarashiIn インタラクション2024論文集 (IPSJ INTERACTION 2024 Proceedings), Feb 2024Storytelling is a valuable tradition that plays a crucial role in child development, fostering creativity and a sense of agency. However, many children often consume stories passively, missing out on the opportunity to participate in the creative process. To address this, we propose a storytelling system that creates adventure-type stories with multiple branches that users can explore. We generate these interactive stories using a character drawing as input, with visual features extraction using GPT-4. By leveraging LLM hallucinations, we generate interactive stories using user feedback as a prompt. Finally, we refine the quality of the generated story through a complexity analysis algorithm. We believe that the use of a drawing as input further improves the engagement in the story and characters.
@inproceedings{sechayk2024mystoryknight, author = {Sechayk, Yotam and Penarska, Gabriela A. and Randsalu, Ingrid A. and Cruz, Christian Arzate and Igarashi, Takeo}, title = {MyStoryKnight: A Character-drawing Driven Storytelling System Using LLM Hallucinations}, booktitle = {インタラクション2024論文集 (IPSJ INTERACTION 2024 Proceedings)}, year = {2024}, month = feb, pages = {1297--1300}, publisher = {情報処理学会 (IPSJ)}, address = {Japan}, note = {poster/demo}, language = {english}, demo = {https://tomfluff.github.io/MyStoryKnight/}, preview_alt = {First page of the MyStoryKnight paper, showing the title, authors, abstract, and the opening of the introduction. This page has no figures.} } -
SmartLearn: Visual-Temporal Accessibility for Slide-based e-learning Videos
CHI 2024 Late breaking work
Yotam Sechayk, Ariel Shamir, and Takeo IgarashiIn Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 2024In the realm of e-learning, video-based content is increasingly prevalent but brings with it unique accessibility challenges. Our research, beginning with a formative study involving 53 participants, has pinpointed the primary accessibility barriers in video-based e-learning: mismatches in user pace, complex visual arrangements leading to unclear focus, and difficulties in navigating content. To tackle these barriers, we introduced SmartLearn (SL), an innovative tool designed to enhance the accessibility of video content. SL utilizes advanced video analysis techniques to address issues of focus, navigation, and pacing, enabling users to interact with video segments more effectively through a web interface. A subsequent evaluation demonstrated that SL significantly enhances user engagement, ease of access, and learnability over existing approaches. We conclude by presenting design guidelines derived from our study, aiming to promote future efforts in research and development towards a more inclusive digital education landscape.
@inproceedings{sechayk2024smartlearn, author = {Sechayk, Yotam and Shamir, Ariel and Igarashi, Takeo}, title = {SmartLearn: Visual-Temporal Accessibility for Slide-based e-learning Videos}, year = {2024}, isbn = {9798400703317}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3613905.3650883}, doi = {10.1145/3613905.3650883}, booktitle = {Extended Abstracts of the CHI Conference on Human Factors in Computing Systems}, articleno = {294}, numpages = {11}, keywords = {Accessibility, E-learning, Online learning, Temporal Accessibility, Universal Design, Video Accessibility, Visual Accessibility}, location = {Honolulu, HI, USA}, language = {english}, note = {late breaking work}, series = {CHI EA '24}, preview_alt = {First page of the SmartLearn paper, showing the title, authors, abstract, and a teaser figure of the SmartLearn system pipeline and video interface for slide-based e-learning videos.} } -
Data Augmentation for 3DMM-based Arousal-Valence Prediction for HRI
ROMAN 2024
In 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), 2024Humans use multiple communication channels to interact with each other. For instance, body gestures or facial expressions are commonly used to convey an intent. The use of such non-verbal cues has motivated the development of prediction models. One such approach is predicting arousal and valence (AV) from facial expressions. However, making these models accurate for human-robot interaction (HRI) settings is challenging as it requires handling multiple subjects, challenging conditions, and a wide range of facial expressions. In this paper, we propose a data augmentation (DA) technique to improve the performance of AV predictors using 3D morphable models (3DMM). We then utilize this approach in an HRI setting with a mediator robot and a group of three humans. Our augmentation method creates synthetic sequences for underrepresented values in the AV space of the SEWA dataset, which is the most comprehensive dataset with continuous AV labels. Results show that using our DA method improves the accuracy and robustness of AV prediction in real-time applications. The accuracy of our models on the SEWA dataset is 0.793 for arousal and valence.
@inproceedings{sechayk2024data, author = {Cruz, Christian Arzate and Sechayk, Yotam and Igarashi, Takeo and Gomez, Randy}, booktitle = {2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)}, title = {Data Augmentation for 3DMM-based Arousal-Valence Prediction for HRI}, publisher = {IEEE}, year = {2024}, volume = {}, number = {}, pages = {2015-2022}, keywords = {Solid modeling;Accuracy;Three-dimensional displays;Human-robot interaction;Predictive models;Feature extraction;Data augmentation;Data models;Robustness;Robots}, doi = {10.1109/RO-MAN60168.2024.10731438}, language = {english}, preview_alt = {First page of the paper on data augmentation for 3DMM-based arousal-valence prediction, showing the title, authors, abstract, and a figure comparing original and augmented 3D face model data.} } -
ShowMe: 対話的な強調表示と拡大表示によるプレゼンテーションビデオの視覚的アクセシビリティの改善
WISS 2024
Yotam Sechayk, Ariel Shamir, and Takeo IgarashiIn WISS 2024: 第32回インタラクティブシステムとソフトウェアに関するワークショップ, 2024プレゼンテーションビデオを使った学習は広く一般的に行われている．講師は，ビデオ作成過程で，さまざまな視覚的補助動作を活用することが多い．具体的には，プレゼンテーション中のポインティング，マーキング，スケッチなどが，視覚的補助動作としてよく使われる．しかし，これらの動作は視覚的に認識が難しいことが多く，説明が不十分であることが多い．弱視の学習者は，このような動作に追従するために，常にプレゼンテーションのフレーム内を探索する必要があり，フラストレーションと疲労につながっている．我々は，この問題を理解し解決するために，3 人の弱視ユーザとユーザ参加型デザインを実施し，その結果にもとづき，講師の視覚的補助動作を強調表示し，拡大表示するツール ShowMe を開発した．ShowMe は，弱視ユーザがプレゼンテーションをフォローできるように支援し，疲労とフラストレーションを軽減する．
@inproceedings{sechayk2024showme, author = {Sechayk, Yotam and Shamir, Ariel and Igarashi, Takeo}, title = {ShowMe: 対話的な強調表示と拡大表示によるプレゼンテーションビデオの視覚的アクセシビリティの改善}, booktitle = {WISS 2024: 第32回インタラクティブシステムとソフトウェアに関するワークショップ}, year = {2024}, publisher = {情報処理学会 (IPSJ)}, address = {Japan}, pages = {137--145}, language = {japanese}, preview_alt = {First page of the ShowMe paper, written in Japanese, showing the title, authors, abstract, and a figure of instructor pointing and marking actions being highlighted in a presentation video.} }
2023
-
Smart Replay: eラーニング動画における視覚的・時間的アクセシビリティの向上
WISS 2023
Yotam Sechayk, Ariel Shamir, and Takeo IgarashiIn WISS 2023: 第31回インタラクティブシステムとソフトウェアに関するワークショップ, 2023Helpfeel Award for Corporate Excellence
eラーニングは，教材への幅広いアクセスを可能にすることを目的としている．しかし，動画コンテンツを多用することは，アクセシビリティに大きな課題をもたらす．多様な参加者を対象とした予備的調査に基づき，e ラーニングのビデオコンテンツに存在するアクセシビリティの障壁を明らかにする．これらには，ユーザーの理解速度と動画速度の不一致，どこに注意を向けてよいかわからない視覚的複雑さ，ナビゲーションのしにくさなどが含まれる．また，我々の調査結果は，アクセシビリティの問題が，障害のある利用者とない利用者の両方に影響を及ぼすことを示している．さらに，既存のアクセシビリティツールには限界があり，ささらなる対応が必要であることを示している．そこで我々は，e ラーニングアクセシビリティツールである「Smart Replay」を提案する．私たちのツールは，学習ビデオのビジュアルとコンテンツに基づいた分析を行い，アクセシブルな再生オプションを生成する．視覚的分野と時間的分野の両方を強化したビデオセクションの復習を可能にする．
@inproceedings{sechayk2023smartreplay, author = {Sechayk, Yotam and Shamir, Ariel and Igarashi, Takeo}, title = {Smart Replay: eラーニング動画における視覚的・時間的アクセシビリティの向上}, booktitle = {WISS 2023: 第31回インタラクティブシステムとソフトウェアに関するワークショップ}, year = {2023}, pages = {25--33}, publisher = {情報処理学会 (IPSJ)}, address = {Japan}, language = {japanese}, preview_alt = {First page of the Smart Replay paper, written in Japanese, showing the title, authors, abstract, and a figure of the Smart Replay interface for e-learning videos.} }