Jaerin Lee

I am a PhD student under Professor Kyoung Mu Lee at the Computer Vision Lab, Seoul National University (SNU). I have an unhealthy level of obsession with making things faster.






Research: Optimization

My recent interests are all about first-order (gradient-based) optimization methods for deep learning.

project image

A Journey to the Edge of Stability



arXiv preprint,
paper

Varying optimization algorithms and learning rates on a deep learning task reveals a distinction between the problem-intrinsic trajectory and optimizer-specific edge of stability.

project image

Greedy Alignment Principle for Optimizer Hyperparameter Selection



NeurIPS,
paper

A greedy alignment principle turns manual optimizer hyperparameter tuning into a solvable filter–statistics alignment maximization problem.

project image

The Road Taken: The Role of Optimizers at the Edge of Stability



arXiv preprint,
paper

The previous definition of the edge of stability mispredicts the stable sharpness with an optimizer-dependent systematic error; this work presents an interpretable alternative.

project image

Grokfast: Accelerated Grokking by Amplifying Slow Gradients



arXiv preprint,
project page / paper / code

Faster grokking with a two-timescale momentum optimizer acting as a low-pass amplification filter for gradients.


Research: Computer Vision

Research topics that are not directly related to optimization.

project image

Vector Scaffolding: Inter-Scale Orchestration for Differentiable Image Vectorization



ECCV,
project page / paper / poster

Structure-aligned optimization schedule adds 1.4 dB PSNR in x2.5 shorter training time for image vectorization.

project image

LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes



IEEE TVCG,   (Best Paper Runner-Up)
project page / paper / code / demo / demo #2

A 3D Gaussian splatting scene generation pipeline that recursively generates scenes from off-the-shelf diffusion models and a point cloud alignment algorithm.

project image

SemanticDraw: Towards Real-Time Interactive Content Creation from Image Diffusion Models



CVPR,
project page / paper / code / demo / demo #2 / poster

By combining previously incompatible region-based control & fast schedulers, Semantic Draw turns any image diffusion model into a live paintbrush tool that let you brush semantic intent.

project image

Rethinking RGB Color Representation for Image Restoration Models



arXiv preprint,
paper

An alternative to RGB colors, aRGB representation equips pairwise per-pixel loss functions with structural information for better supervision of various image restoration models.

project image

Clean Images are Hard to Reblur: Exploiting the Ill-Posed Inverse Task for Dynamic Scene Deblurring



ICLR,
paper / poster

CIHR improves the deblurring quality of images by exploiting the inverse task of deblurring, namely, reblurring.

project image

Structure-Resonant Discriminator for Image Super-Resolution



IEEE ICME,   (Oral Presentation)
paper / poster / slides

Highlighting that GAN discriminators are also data models, SRD is a discriminator exploiting three structural features of natural images: translation equivariance, rotation invariance, and hierarchy of scale.

project image

AIM 2020 Challenge on Video Temporal Super-Resolution



ECCV Workshop,
project page / paper / slides

An AIM challenge on example-based Video Temporal Super-Resolution for enhancing videos for better temporal resolution, held jointly with the 2nd AIM: Advances in Image Manipulation workshop, in conjunction with ECCV 2020 in Glasgow, UK.

project image

Progressively Growing Neural Networks for Image Recognition



Undergraduate Thesis,
code

Thesis advisor: Professor Kyoung Mu Lee. This work explores the optimal architecture for an image classifier by progressively growing a CNN with respect to intermediate activations, achieving better results than randomly wired networks.


ROK Army Projects

From January 2021 to July 2022, I worked for the Republic of Korea Army AI R&D Center located at the Korea Military Academy for my compulsory military service. I proposed and developed one of the first military field applications of computer vision and deep learning in Korea: the AI-driven unstaffed surveillance system at Korea Military Academy. My contribution was featured in two dedicated half-hour-long episodes of a documentary TV show by National Defence TV, Korea, a state media [EP 1] [EP 2].

project image

Completed Military Service at ROK Army AI R&D Center



An 18-month term of compulsory military service at the ROK Army AI R&D Center, completed with the final rank of Sergeant. The work spanned three projects commissioned by the Superintendent of the KMA (LTG ★ ★ ★), from proposals to deployments, with coverage in two documentary TV episodes from a state broadcaster, recognition through two Commander’s Commendation Ribbons (LTG ★ ★ ★, MG ★ ★), an award at the 5th Army Startup Competition (MG ★ ★), and one invited talk for the faculty members of the KMA.

project image

Division Commander's Commandation Ribbon ★ ★



A commendation from the Superintendent of the KMA, MG Jeon, Sungdae (★ ★), recognizing contributions to intelligent military surveillance through the project 'Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area.'

project image

Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area



An unstaffed, fully autonomous surveillance system for military operations in the ammunition storage area of the KMA. The first-ever field deployment of deep learning algorithms in the ROK Army, commissioned by LTG Chang-gu Kang (★ ★ ★), the Superintendent of Korea Military Academy, and recognized with a Commendation Ribbon (MG ★ ★).

project image

Al-Driven Real-Time Restoration of Command Control Center Surveillance Videos in Times of Inclement Weather



A real-time video restoration system for military surveillance cameras exposed to inclement weather. The first-ever military application of deep learning-based low-level vision algorithms in the ROK Army, commissioned by LTG Chang-gu Kang (★ ★ ★), the Superintendent of Korea Military Academy.

project image

Invited talk at the Korea Military Academy



Title of the talk: 'High Performance Computing for Artificial Intelligence.' A one-and-a-half-hour lecture for the faculty members of the KMA at the 1st Korea Military Academy Artificial Intelligence Seminar held by the ROK Army AI R&D Center.

project image

Featured on National Defence TV, Defence Media Agency, Korea (State Broadcaster)



video

Episode title: 'Dreaming of the Army with Advanced Science and Technology, Military Science and Technology Researcher in Korea Military Academy, Part 2, I Am a Republic of Korea Soldier, episode 176.' Another half-hour documentary TV show about enlisted researchers supporting cadet education with teaching assistance and technology demonstrations on AI in the KMA.

project image

Featured on National Defence TV, Defence Media Agency, Korea (State Broadcaster)



video

Episode title: 'Dreaming of the Army with Advanced Science and Technology, Military Science and Technology Researcher in Korea Military Academy, Part 1, I Am a Republic of Korea Soldier, episode 175.' A half-hour documentary TV show about enlisted researchers in the Army and the first AI field application in the ROK Army.

project image

Corp Commander's Commandation Ribbon ★ ★ ★



A commendation from the Superintendent of the KMA, LTG Kim, Jeong Soo (★ ★ ★), recognizing contributions to intelligent military surveillance through the project 'Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area.'

project image

Application of Artificial Intelligence on Intelligent Surveillance System of Korea Military Academy Command Control Center



An automated surveillance system integrating real-time multi-object tracking, super-resolution, and re-identification systems with a user-friendly interface. The first ever military application of AI in the ROK Army, commissioned by LTG Kim, Jeong Soo (★ ★ ★), the Superintendent of Korea Military Academy. The results were presented at the 21-2 semi-annual meeting of the Steering Committee of the KMA and recognized with a Commendation Ribbon (LTG ★ ★ ★).

project image

Innovation Award, 2021 5th Army Startup Competition, Republic of Korea Army



Proposal title: 'Military Consultation Chatbot for the Army Mental Health Services Using Large Language Models.' Awarded by the Commanding General of the ROK Army Personnel Command, MG Jeon, Byung-hee (★ ★).


Other Projects

Coursework, personal projects, and unpublished researches. Games, blockchains, but mostly rockets.

project image

Adaptive Dataset Sampling by Deep Reinforcement Learning


Coursework Stochastic Control and Reinforcement Learning, 2020 Spring

paper

An adaptive sampler improves on the random sampling used in training deep neural networks. The method models the optimal sampling problem as a Markov Decision Process (MDP) and trains an RNN policy network to create a sampling policy that is both model- and task-agnostic.

project image

Real-Time Ray-Marching with OpenGL


Coursework Graphics Programming, 2020 Spring

paper / video / video #2 / code

Ray marching algorithms render arbitrarily detailed geometric objects, including 3D fractals and infinite landscapes generated by procedural generation algorithms, in real time.

project image

Development and Demonstration of Hanging Pendulum Type Low Thrust Measurement System


Rocketry

A cold gas reaction control system and a corresponding measurement system for a sounding rocket. Funded by the SNU Undergraduate Research Program through the Faculty of Liberal Education, SNU (₩7M ≃ $6,000).

project image

Excellence Award, SNU Undergraduate Research Award


Award

news

A cold gas reaction control system for orientation control of a sounding rocket. Funded by the SNU Undergraduate Research Program through the Faculty of Liberal Education, SNU (₩7M ≃ $6,000).

project image

Accelerated Magnetic Flux Density Calculator for Solenoid Magnets


Development

code

A calculator for high-power magnet design at Applied Superconductivity Lab led by Professor Seungyong Hahn, using Gaussian quadratures of elliptic integrals.

project image

Successful Launch of the First ANCP-Based Rocket (Identity-III)


Rocketry

video / code

The Identity-III rocket successfully launched using a newly developed solid rocket booster; it flew with the maximum height of about 1 km.

project image

SNU Rocket Team HANARO


Rocketry

video

Passionate Rocket Scientists. A video for the SNU Rocket Team Hanaro.

project image

Gold Medal (the 1st Place) at the 27th Rocket Engineering Competition


Award

video

Hanaro, the Seoul National University Rocket Team, won a gold medal (the 1st place) in the rocket launch competition held by the National Universities’ Rocket Association (NURA), Korea.

project image

Avionics of the Identity-II Rocket


Rocketry

code

Electronic systems for the Identity-II and Identity-III rockets of Hanaro, the SNU Rocket Team.

project image

Data Acquisition (DAQ) System for Rocket Thrust Measurement Test


Rocketry

code

A data acquisition system for rocket thrust measurement tests for Hanaro, the SNU Rocket Team.

project image

The 1st Place at 2018 Hdac Hackathon, the Largest Blockchain Contest in Korea, 2018


Award

news / news #2

Proposal title: 'Development of a Platform-as-a-Service on edge computing-based blockchain middleware.' This was the largest software development contest (including of course the blockchain) held in Korea at that time. Awarded ₩50M (≃$45,000) for the first place.

project image

Firmware of the Identity-II Rocket


Rocketry

video / code

Control, communications, and in-flight data acquisition firmware for Identity-II, a sounding rocket (a small rocket for scientific research) of Hanaro, the SNU Rocket Team.

project image

A Game: Cool Guy Don't Look At Explosions


GameDev

code

A simple vertical platformer game where you walk grandiosely and parry enemy attacks without any hardship.

project image

Internship at Collain Healthcare, Georgetown, TX, USA


Experience

Market research on electronic health record (EHR) system providers for nursing homes in the U.S.


Design and source code from Jon Barron's website