Education
- March 2021 to February 2023,,Advisor Geehyuk Lee · GPA 3.81/4.3
- March 2017 to February 2021,,GPA 3.44/4.3
- Org
- DGIST
- Degree
- B.S.
- Major
- Interdisciplinary Engineering
- June 2017 to August 2017,
Experience
- May 2025 to present,,A company that turns buildings and construction sites into 3D digital twins people can look back through. I work on the frontend and as technical PM.
- What
- A company that turns buildings and construction sites into 3D digital twins people can look back through. I work on the frontend and as technical PM.
- Why
- Sites lose network, yet the desktop app fetched everything from the server, and QA filed bugs faster than the team could absorb them.
- How
- I built an offline mode that stores every entity's metadata and every image locally, and a hybrid layer that switches between the real API and the local database according to connectivity without changing a single call site. Edits made offline synchronise automatically on reconnection.
- New
- I proposed and ran the company's first workflow in which AI agents carry out development work, and served as technical PM for Beacon, its conversational AI product.
- February 2023 to April 2025,,A company making wearables that deliver touch through vibration. I worked on the apps and the SDK as technical research personnel.
- What
- A company making wearables that deliver touch through vibration. I worked on the apps and the SDK as technical research personnel.
- Why
- The hardware existed, but something had to carry what happens inside a PC or PSVR game onto the body.
- How
- I built the control apps on iOS (SwiftUI), Android (Jetpack Compose) and Windows (WPF), talked to the hardware over BLE and TCP including firmware update (DFU), and turned live PC-game events into haptic signals.
- New
- I designed the GitHub Actions CI/CD that automated release across all three platforms. In January 2024 I supported the company's booth demos at CES 2024 in Las Vegas.
- June 2022 to November 2022,,Measured how professional gamers aim, at NVIDIA Research.
- What
- Measured how professional gamers aim, at NVIDIA Research.
- Why
- The question was how aiming performance changes as display and input-device conditions change.
- How
- I measured and compared across conditions, and proposed a short-time, wavelet-inspired method for detecting mouse submovements.
- New
- arXiv 2026, first author.
- June 2020 to February 2021,,Undergraduate researcher at the DGIST Smart Input Device Lab.
- What
- Undergraduate researcher at the DGIST Smart Input Device Lab.
- Why
- When a mouse's polling rate falls out of step with the display refresh the cursor jitters, and how much that costs a person's aim was not known.
- How
- I built a rig that logs 8000 Hz input without loss and measured the cost experimentally, and analysed the Oculus Touch haptic feedback API.
- New
- ACM UIST 2021, first author.
- June 2019 to July 2020,,Undergraduate intern at the KAIST Interactive Media Lab.
- What
- Undergraduate intern at the KAIST Interactive Media Lab.
- Why
- Behavioural-measurement devices drift apart in time when run together, so they stop pointing at the same instant.
- How
- I implemented a UDP protocol that records eight sensor types — eye tracker, motion tracker, webcam, Arduino — against one timestamp, and built a Pupil Labs serialisation library and a CS:GO scene-analysis system.
- New
- Data gathered on that rig went into Secrets of Gosu (ACM CHI 2021).
- August 2018 to December 2018,,Software developer at a startup in the KAIST startup incubator.
- What
- Software developer at a startup in the KAIST startup incubator.
- Why
- Computers for blind users cost several thousand dollars, and a phone's voice guidance reads everything aloud to whoever is standing nearby.
- How
- I built a braille input and output device shaped like a phone case, and the iOS application that drives it over Bluetooth.
- New
- Demonstrated to investors at KAIST TECH DAY.
- June 2018 to August 2018,,A month at Can Tho University in Vietnam with the World Friends ICT volunteer programme.
- 2017,,Vice manager and head of outreach for a working group under the DGIST Supercomputing Center.
Publications
- 2026,,arXiv 2026 · first author
Auejin Ham, Ben Boudaoud
arXiv 2026
- What
- A wavelet-inspired decomposition that finds submovements in a mouse speed series and gives each a location, width and amplitude.
- Why
- Submovements overlap: the next starts before the previous ends. Segmentation cuts the time axis into intervals and assigns each instant to one movement, so it cannot measure overlap.
- How
- A CWT with a log-normal speed shape as mother wavelet seeds the fit; a peak-weighted loss refines it, repeating on the residual. Synthetic trials modelled on 13 users' FPS aiming data supply a known count.
- New
- Overlapping movements count separately rather than as one. Peak-weighted refinement recovered the correct submovement count in 67.124% of synthetic trials, above MSE refinement or the raw CWT guess.
- 2023,,International Journal of Human-Computer Interaction 2023
Sunmin Son, Jingun Jung, Auejin Ham, Geehyuk Lee
International Journal of Human-Computer Interaction 2023
- What
- A mouse wheel that senses touch and release, so a flick starts inertial scrolling and a touch stops it, with no mode switch.
- Why
- Long scrolls on a mouse wheel mean repeated, tiring clutching. Free-spinning modes avoid that but need a button to switch, and scrolling mixes short and long moves.
- How
- A touch sensor and encoder drive a virtual wheel that starts at release speed and slows under simulated friction. Twelve participants used it against Normal, FreeSpin and Toggle wheels on long and short tasks.
- New
- Participants chose flick-and-stop where it helped: up to 0.62 of wheel operations on long scrolls, near zero on short. There it took less time than Normal, fewer discrete actions than Toggle.
- 2022,,ACM CHI · Full Paper 2022 · acceptance 12.5%
Taejun Kim, Auejin Ham, Sunggeun Ahn, Geehyuk Lee
ACM CHI · Full Paper 2022 · acceptance 12.5%
- What
- A marking menu moved to gaze input: the user picks each level by looking at visual anchors in a lattice outside the pie slices.
- Why
- Eye movements are ballistic: once launched they cannot be steered, so a gaze gesture drawn in the air with nothing to look at easily slips onto the neighbouring item.
- How
- The paper implemented the menu on a VR headset with an integrated eye tracker and ran a 12-subject study against BorderPie, the border-crossing marking menu with no visual target to look at.
- New
- The point is that a lattice of anchors holds the gaze and keeps selection errors down: in experienced trials errors were about 5 times fewer than BorderPie's.
- 2022,,ACM CHI · Late-Breaking Work 2022 · acceptance 36.0% · co-first author
Junsu Lim*, Hyeonggeun Yun*, Auejin Ham*, Sunjun Kim
ACM CHI · Late-Breaking Work 2022 · acceptance 36.0%
- What
- A role-playing VR tutorial in which users hand-carry a privacy policy's data items, and a preliminary study pitting it against the policy text.
- Why
- VR headsets collect biometric data like gaze and body movement, unchangeable once leaked, yet the policy meant to disclose this is a long, jargon-heavy document nobody reads.
- How
- Each Oculus Privacy Policy data item becomes a ball the user grabs, reads and throws to VR content, the content provider or a third party. Fourteen people either read the policy or played the tutorial.
- New
- Privacy awareness was on par with reading the text and satisfaction significantly higher (p=0.007): an unread policy becomes an experience people rate higher, without losing what it conveys.
- 2022,,ACM SIGGRAPH · Emerging Technologies 2022 · acceptance 20.0%
HyeonBeom Yi, Yeeun Shin, Sehee Lee, Eunhye Youn, Auejin Ham, Geehyuk Lee, Woohun Lee
ACM SIGGRAPH · Emerging Technologies 2022 · acceptance 20.0%
- What
- A mobile peripheral that tracks the device's position and heading on a museum exhibit and overlays content there: near-surface AR.
- Why
- Mobile AR in museums pulls visitors' eyes to a screen away from the exhibit, and pinning content to the surface would mean a printed pattern or fixture on every exhibit.
- How
- An RFID tag identifies the exhibit and fixes the origin; Kalman filters fuse two displacement sensors, for coarse and glossy surfaces, with an IMU, and the app moves the content the opposite way.
- New
- The exhibit needs only a thin tag; tracking holds on acrylic, paper, wood and fabric while hovering up to 13 mm. Four forms followed, from smartphone magnifier to toy vehicle.
- 2021,,ACM UIST · Full Paper 2021 · acceptance 25.9% · first author
Auejin Ham, Junsu Lim, Sunjun Kim
ACM UIST · Full Paper 2021 · acceptance 25.9%
- What
- A study of the cursor jitter that arises when mouse polling and display refresh fall out of step: do people notice, and does pointing suffer?
- Why
- Mouse and display run on separate clocks, so on high-refresh monitors jitter is unavoidable. A model predicts its size, not whether people notice or how fast a mouse must be.
- How
- In a triangle test, 23 participants tried to tell lower polling rates from an 8000 Hz mouse; gamers then ran a Fitts' law test to measure throughput.
- New
- Twelve of 23 could tell 500 Hz from 8000 Hz, and 1000 Hz was borderline on some displays. Throughput was unaffected, yet perceptually today's 1000 Hz is not enough.
- 2021,,ACM CHI · Full Paper 2021 · acceptance 26.3%
Eunji Park, Sangyoon Lee, Auejin Ham, Minyeop Choi, Sunjun Kim, Byungjoo Lee
ACM CHI · Full Paper 2021 · acceptance 26.3%
- What
- A data-collection study that recorded eight professional and eight amateur Counter-Strike: Global Offensive players to test conjectures about what makes FPS professionals better.
- Why
- In-game statistics such as the kill/death ratio show who wins, not how, and the conjectures traded in online communities had never been scientifically tested.
- How
- The study built a logger that aligns body motion, gaze, device input and in-game state to one timestamp, staged one-on-one duels within each group and compared the groups on quantitative metrics.
- New
- Only six of 13 conjectures held up. Professionals pulled the mouse down more to counter recoil, so their crosshair sat farther from the target, contrary to the common advice.
- 2020,,ACM CHI Workshop (CUI@CHI) 2020
Hyeonggeun Yun, Auejin Ham, Jin Kim, Taeyeong Kim, Jeongeun Kim, Haechan Lee, Jongrae Park, Jinkyu Jang
ACM CHI Workshop (CUI@CHI) 2020
- What
- TwG adds touch gestures and an on-screen character to a text chatbot so a user can send an emotion or hand over the turn.
- Why
- Text-only chat carries no nonverbal cues, so a chatbot struggles to read the user's emotion or tell where a turn ends when one topic spans several short messages.
- How
- It maps six emotions to touch gestures the character answers with an expression; swiping the chat bubble up hands over the turn. In a Wizard of Oz study 28 participants compared it with text-only chat.
- New
- Touch and graphics stand in for nonverbal cues text chat lacks: emotional connection rose from 3.51 with text only to 4.39 with TwG, and turn-taking felt more natural.
Projects
- 2026,,An LLM chat server and panel UI over a SaaS that captures construction sites in 3D.
- What
- An LLM chat server and panel UI over a SaaS that captures construction sites in 3D.
- Why
- When someone asks how many safety findings sit on level three, one number stops work and settles invoices.
- How
- Code grades every search result before the model speaks — complete set or sample — and the grade constrains which sentence the model is permitted to write. Where the grade is low the system asks back and narrows the scope.
- New
- The model earns its confidence through conversation rather than imitating it: the path from a sample to a complete set runs through the dialogue itself.
- 2026,,A harness that makes multiple agents carry a natural-language development spec through to the end.
- What
- A harness that makes multiple agents carry a natural-language development spec through to the end.
- Why
- A spec written by a person leaves too much room for interpretation for a machine to judge how far it has got.
- How
- It pins the spec into a checkable state machine, splits it into work cards on a dependency graph, and runs agents with no human intervention until the graph is drained, producing verified commits.
- New
- Progress is measured by what is left in the graph rather than by what an agent reports.
- 2026,,An agent system that owns a production bug queue end to end.
unattended in production 5 months · issues ingested 924 · PRs generated automatically 571 · per issue ~30 min · merge rate 50.0% · root-cause accuracy 92.2% (as of 2026-08)
- What
- An agent system that owns a production bug queue end to end.
- Why
- Most of the time a bug costs goes not into the fix but into working out what broke under which conditions.
- How
- It polls Jira and picks its own issues, compresses the attached recording into keyframes for a VLM to recover the capture conditions, then reproduces in a real browser, fixes, refutes its own fix with a before/after check, and opens the pull request.
- New
- The point is that an LLM saying it is done never counts as the completion signal — verdicts are demoted to typed enums — and a biweekly self-audit files improvement pull requests against its own code.
- 2024,,An unattended kiosk that composites a person in a full-body costume between illustrated character layers and prints the result on the spot.
- What
- An unattended kiosk that composites a person in a full-body costume between illustrated character layers and prints the result on the spot.
- Why
- Convention booths have unreliable networks, and a costume's fur and trim make an outline that chroma keying cannot cut cleanly.
- How
- From a single webcam, SAM2 separates the person and Depth-Anything-V2 estimates depth, placing them between the illustration layers. Everything runs on the device.
- New
- No green screen and no server: the booth is self-contained. Run at seven conventions in South Korea between 2024 and 2026.
- 2024,,A desktop bot app for streamers on Chzzk, Naver's live-streaming platform.
downloads 56,000+ · broadcasts served 55,600+ (as of 2026-08)
- What
- A desktop bot app for streamers on Chzzk, Naver's live-streaming platform.
- Why
- Reading chat, answering donations and running screen effects while broadcasting is more than one person can hold at once.
- How
- It takes chat and donation events over a socket, runs eleven kinds of bot, and draws the results as a live overlay on the OBS scene — a Tauri v2, Angular and Rust (actix-web) desktop app with a NestJS cloud API.
- New
- Three operating-system builds are published through GitHub Actions, run single-handed without a commercial release service.
- 2024,,An iOS application (SwiftUI) that drives bHaptics vests and gloves.
downloads 8,700+ · active users 400+ (as of 2025-03)
- What
- An iOS application (SwiftUI) that drives bHaptics vests and gloves.
- Why
- To feel what happens in a game on your body, something has to sit between the PC or PSVR title and the hardware in real time.
- How
- It finds supported titles on the same Wi-Fi network by itself, takes their events, and relays them over BLE in under 20 ms, alongside firmware update (DFU) and account flows.
- New
- It ships with a Swift SDK folding device state and account state into the SwiftUI lifecycle, so the app and the SDK share one state machine.
- 2021,,A rig that mimics the physical behaviour of mechanical keys in order to find the right activation method for a touch keyboard.
- What
- A rig that mimics the physical behaviour of mechanical keys in order to find the right activation method for a touch keyboard.
- Why
- A touch keyboard gives no sensation at the moment of contact, so the hand cannot tell when the key registered.
- How
- Press depth, restoring force and other physical parameters are adjustable; capacitive sensing and an RF power detector measure contact, and multiplexing reads the key array. The control side is WPF and Arduino.
- New
- Activation is a variable to sweep rather than a fixed design constant, so the best point can be found by experiment.
More
- 2022,,A Python library that animates values arriving over a serial port in real time.
- 2021,,A real-time web platform for forming teams in KAIST classes.
- What
- A real-time web platform for forming teams in KAIST classes.
- Why
- Remote teaching through the pandemic removed the places where students get to know each other, leaving nothing to judge a prospective teammate by.
- How
- A quiz surfaces candidates with matching interests, an anonymised chat with a moderating bot lets them talk, and a credit record shows how much each person contributed to earlier group projects. Built on React and the Firebase realtime database.
- New
- Anonymity lowers the social cost of choosing while the credit record still exposes a history of free-riding, so a choice can be both comfortable and informed.
- 2021,,A system that lets two people in augmented-reality glasses see where the other is looking in the same room.
- What
- A system that lets two people in augmented-reality glasses see where the other is looking in the same room.
- Why
- Speech and pointing are thin instruments for singling out something far away, and the moment you look at a handheld device you look away from the site.
- How
- Using HoloLens 2 it measures each person's eye and head direction, finds the moment their regions of interest coincide, and outlines that target in both headsets at once. Built in Unity (C#), sharing a coordinate frame through Azure Spatial Anchors and replicating state over Photon Unity Networking.
- New
- Rather than showing gaze continuously it shows it only where two people's attention meets, so the cue itself signals agreement.
- 2020,,A toolkit that records several behavioural-measurement devices against a single timestamp.
- What
- A toolkit that records several behavioural-measurement devices against a single timestamp.
- Why
- Each device runs its own clock, so signals logged separately do not line up on the same instant afterwards.
- How
- It drives the devices together over UDP and serial, and cuts the number of system calls crossing the OS pipe to hold clock drift to 11 microseconds per second. Built with Arduino and Processing (Java).
- New
- Eight kinds of sensor at once — eye tracker, motion tracker, webcam, EMG. The data it collected went into Secrets of Gosu (ACM CHI 2021).
- 2020,,An interaction technique (TwG) that carries emotion and turn-taking to a chatbot through touch.
- What
- An interaction technique (TwG) that carries emotion and turn-taking to a chatbot through touch.
- Why
- Chatbots have been treated in terms of natural language processing for text conversation, leaving little means to convey a user's emotion or manage whose turn it is.
- How
- Users send emotion and hand over the turn by touching a graphic interface; a Wizard of Oz study evaluated the resulting experience.
- New
- Emotional expression improved over conventional text-only chatbot interaction, and turn-taking became more natural.
- 2020,,A library that brings Pupil Labs eye-tracker data into Processing (Java).
- 2019,,A VR interface for operating a crane.
- What
- A VR interface for operating a crane.
- Why
- Crane control carries three problems at once — dizziness, the safety of people nearby, and distance perception. HIAB Korea posed the subject.
- How
- The viewpoint and the control scheme were designed against all three together.
- New
- Winner of the logistics and physical-computing track at Junction X Seoul.
- 2019,,A system that throws a voice assistant's graphical interface onto a wall or desk with a projector.
- What
- A system that throws a voice assistant's graphical interface onto a wall or desk with a projector.
- Why
- A voice-only assistant has no way to hand over a list or a table, and a fixed screen only works if you walk to it.
- How
- Kinect v2 finds the person and the surface, Arduino PWM steers the projector, and the display lands where it can be seen. Built in WPF (C#) and evaluated with a Wizard of Oz study and NASA-TLX.
- New
- The display follows the person, so using the assistant is no longer tied to wherever the speaker sits.
- 2019,,A technique that takes mid-air pinch gestures as input before the hand reaches a vertical mouse.
- What
- A technique that takes mid-air pinch gestures as input before the hand reaches a vertical mouse.
- Why
- A vertical mouse is held with the wrist upright, so nothing can be entered until the hand is already on it.
- How
- It recognises the pinch trajectory in eight directions: left, right, upper left, lower left, upper right, lower right, up and down.
- New
- The seconds before the hand touches the mouse become usable input time.
- 2018,,An infographic of the lunchtime queue at the campus cafeteria.
- 2016,,An interface that operates an Android phone with the eyes alone.
- What
- An interface that operates an Android phone with the eyes alone.
- Why
- Someone with quadriplegia cannot touch the screen, and gaze by itself settles where to look but not when to press.
- How
- It follows the gaze trajectory through a webcam to place the point, takes a wink as the press signal, and injects the touch event over ADB.
- New
- Together they cover the whole design space of single-point touch — tap, long press, drag — using eyes and eyelids only.
Awards
- 2023,,iF International Forum Design · WonderScope: Practical Near-surface AR Device for Museum Exhibits
- 2022,,KAIST · Teaching Assistant @ Wearable Interface (CS492), KAIST
- By
- KAIST
- Year
- 2022
- 2022,,ACM SIGGRAPH · WonderScope: Practical Near-surface AR Device for Museum Exhibits
- 2021,,ACM SIGCHI · Secrets of Gosu: Understanding Physical Combat Skills of Professional Players in First-Person Shooters
- 2020,,DGIST · Chatbot with Touch and Graphics
- 2019,
- 2017,,Korea Student Aid Foundation
- 2016,,Ministry of Education · Winklick
- 2016,,Intel ISEF · Winklick
- 2015,,Seoul Metropolitan Science Exhibition Hall · Winklick
- By
- Seoul Metropolitan Science Exhibition Hall
- Rank
- Special Prize (Grade 1)
- Year
- 2015
- 2015,,National Information Society Agency · Winklick
- 2014,,Korea Institute of Information Scientists and Engineers
- 2014,,National Information Society Agency
- 2014,,Korea Institute of Information Scientists and Engineers · Winklick
- By
- Korea Institute of Information Scientists and Engineers
- Rank
- Special Award
- Year
- 2014
- 2013,,Ministry of Science, ICT and Future Planning
Teaching & Talks
Service
- 2025,
- 2024,
- 2023,
Skills
- Agent Systems, Reinforcement Learning,LLM multi-agent orchestration harness design (state-machine formalisation, task dependency graphs, failure taxonomies, checkpoint resume) · Model Context Protocol tool integration, VLM-based screen adjudication, browser automation · Agent Experience (AX) workflow design — AI-then-Human development processes wired through Jira, Slack, GitHub and Azure · Reinforcement learning (OpenAI Gym), computer vision (OpenCV), PyTorch
- Spatial Computing, Gesture-based Interface, Behavioral Sensors,AR/VR (Apple Vision Pro, Oculus Quest, Microsoft Hololens), Body Tracking (Microsoft Kinect, OptiTrack Motive, LEAP Motion) · Eye Gaze (Tobii Eye Tracker, SMI iViewRED, PupilLabs, FOVE VR), EMG (MYO Armband) · Touch & Press (bHaptics TactSuit, Sensel Morph, Wooting Keyboard)
- Rapid Prototyping,iOS & Mac & Apple Vision Pro (SwiftUI), Android (Jetpack Compose) · WPF & .NET MAUI & Microsoft Hololens (C#), Unity (C#), React & Electron or Tauri (TypeScript) · Processing 3 (Java), Figma · Firmware Development (ESP32, Arduino)
- User Experiment, Data Analysis & Visualization,Mixed-methods study design and operation (within- and between-subjects), performance measurement (completion time, accuracy, error rate), NASA-TLX, semi-structured interviews, Wizard of Oz evaluation · SPSS, JASP, Matlab, R, pandas, scipy, pingouin, seaborn, matplotlib, jupyter
- Real-time Data Collection, Messaging Platform Implementation,Sensing and Recording Low-level Sensor Data (SPI, I2C, Windows HID) · Designing Peer-to-Peer Wireless Networking Protocol (UDP, TCP, RESTful API, BLE UART, Wi-Fi) · Real-time Data Integration System (Photon Unity Networking RPC, ZeroMQ, MessagePack)
- Signal Processing, Sensor Fusion, Time Series Analysis,Recording, Denoising, and Analyzing High-frequency Time Series Data · Prediction Models, Clock Drift Analysis, Covariance-based Multi-Kalman-Filter Fusion