The aim of this research was to analyze the impact of latency on user behavior in a remote desktop-based telework system using captured traffic data. We statistically analyzed the time series data at multiple granularities in each TCP session to analyze the temporal dependency of latency and byte size relationship as well as the timescale of user behavior changes. Our research result suggests that latency affects user engagement, hindering data transmission in applications requiring user input. Our analysis further revealed that increased latency led to lower user activity levels, with the impact persisting for approximately at least 15 seconds.