Jaerin Lee

I am a PhD student under Professor Kyoung Mu Lee at the Computer Vision Lab, Seoul National University (SNU). My goal as a researcher is to contribute to our understanding of the intricacies and dynamics of this 4D world around us. The key, I believe, lies in the representation learning.

From January 2021 to July 2022, I worked for the Republic of Korea Army AI R&D Center located at the Korea Military Academy for my compulsory military service. I proposed and developed one of the first military field applications of computer vision and deep learning in Korea: the AI-driven unstaffed surveillance system at Korea Military Academy. My contribution was featured in two dedicated half-hour-long episodes of a documentary TV show by National Defence TV, Korea, a state media [EP 1] [EP 2].


Email  /  GitHub  /  X  /  Google Scholar  /  LinkedIn  /  CV





Research

My research interest mainly lies in the representation learning. This is where I believe computer vision, reinforcement learning, natural language processing and computer graphics meet together. All our thoughts stems from our beliefs.

project image

Grokfast: Accelerated Grokking by Amplifying Slow Gradients


Jaerin Lee, Bong Gyun Kang, Kihoon Kim and Kyoung Mu Lee
arXiv preprint, 2024
project page / paper / code

We accelerate the grokking phenomenon by amplifying low-frequencies of the parameter gradients with an augmented optimizer.

project image

StreamMultiDiffusion: Real-Time Interactive Generation with Region-Based Semantic Control


Jaerin Lee, Daniel Sungho Jung, Kanggeon Lee and Kyoung Mu Lee
arXiv preprint, 2024
project page / paper / code / demo / demo #2

We establish the compatibility between region-based control and acceleration methods for diffusion models to achieve up to x50 faster arbitrary-sized region-based text-to-image synthesis. Based on this, we propose a new drawing paradigm of semantic palette.

project image

Rethinking RGB Color Representation for Image Restoration Models


Jaerin Lee, JoonKyu Park, Sungyong Baik and Kyoung Mu Lee
arXiv preprint, 2024
paper

As an alternative to the RGB colors, we present a new representation for images, where pairwise per-pixel loss functions are equipped with structural information for better supervision of various image restoration models.

project image

LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes


Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee and Kyoung Mu Lee
arXiv preprint, 2023
project page / paper / code / demo / demo #2

We developed a 3D Gaussian splatting scene generation pipeline that recursively generates scenes from off-the-shelf diffusion models and our point cloud alignment algorithm.

project image

Clean Images are Hard to Reblur: Exploiting the Ill-Posed Inverse Task for Dynamic Scene Deblurring


Seungjun Nah, Sanghyun Son, Jaerin Lee, and Kyoung Mu Lee
ICLR, 2022
paper / poster

We proposed to improve the deblurring quality of images by exploiting the inverse task of deblurring, namely, reblurring. The strategy is to design two types of reblurring loss functions for better deblurring: a supervised reblurring loss at the training time and the self-supervised reblurring loss at the inference time.

project image

Structure-Resonant Discriminator for Image Super-Resolution


Jaerin Lee and Kyoung Mu Lee
IEEE ICME, 2021   (Oral Presentation)
paper / poster / slides

Highlighting that GAN discriminators are also data models, I developed a new discriminator exploiting three structural features of natural images: translation equivariance, rotation invariance, and hierarchy of scale.

project image

AIM 2020 Challenge on Video Temporal Super-Resolution


Sanghyun Son, Jaerin Lee, Seungjun Nah, Radu Timofe and Kyoung Mu Lee et al.
ECCV Workshop, 2020
project page / paper / slides

Jointly with the 2nd AIM: Advances in Image Manipulation workshop, in conjunction with ECCV 2020 in Glasgow, UK, we organized an AIM challenge on example-based Video Temporal Super-Resolution for enhancing videos for better temporal resolution.

project image

Progressively Growing Neural Networks for Image Recognition


Jaerin Lee
Undergraduate Thesis, 2019
code

Thesis advisor: Professor Kyoung Mu Lee. I explored for the optimal architecture for image classifier by growing a convolutional neural network progressively according to the intermediate activation intensity, and achieved better result than randomly wired networks.

project image

Development and Demonstration of Hanging Pendulum Type Low Thrust Measurement System


Deokhyeon Kim, Mingyu Park, and Jaerin Lee
Spring Conference of the Korean Society for Aeronautical & Space Sciences, 2019

We developed a cold gas reaction control system and a corresponding measurement system for a sounding rocket. The project was funded by the SNU Undergraduate Research Program through the Faculty of Liberal Education, SNU (₩7M ≃ $6,000).




ROK Army Projects

Projects I had run at Republic of Korea Army AI R&D Center at the Korea Military Academy.

project image

Completed Military Service at ROK Army AI R&D Center


2022-07-17

I finished my duty and discharged from the ROK Army with final rank: Sergeant. In my 18 months of compulsory military service, from the proposals to the deployments I have led three projects commissioned by the Superintendent of the KMA (LTG ★ ★ ★), featured on air in two episodes of documentary TV shows from a state broadcaster, honored with two Commander’s Commendation Ribbons (LTG ★ ★ ★, MG ★ ★), awarded at the 5th Army Startup Competition (MG ★ ★) and delivered one invited talk for the faculty members of the KMA. It was truly an honor for me to serve my country with the best of what I can do.

project image

Division Commander's Commandation Ribbon ★ ★


2022-06-30

I was awarded by the Superintendent of the KMA, MG Jeon, Sungdae (★ ★), for meritorious contributions to laying the groundwork for the intelligent military surveillance system by taking a leading role in the task 'Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area.'

project image

Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area


2022-05-22

I planned and solely developed the unstaffed, fully autonomous surveillance system for military operations in the ammunition storage area of the KMA. It was the first field deployment of machine learning algorithms in the history of the ROK Army. This project was commissioned by LTG Chang-gu Kang (★ ★ ★), the Superintendent of Korea Military Academy. I received a Commendation Ribbon (MG ★ ★) for my successful accomplishment.

project image

Al-Driven Real-Time Restoration of Command Control Center Surveillance Videos in Times of Inclement Weather


2022-05-21

I planned and led a project commissioned by LTG Chang-gu Kang (★ ★ ★), the Superintendent of Korea Military Academy. I solely developed a real-time video restoration system for military surveillance cameras exposed in inclement weather. It was the first military application of deep learning-based low-level vision algorithms in the ROK Army.

project image

Invited talk at the Korea Military Academy


2022-05-18

Title of the talk: 'High Performance Computing for Artificial Intelligence.' It was an unusual honor for me to deliver a full one-and-a-half-hour lecture for the faculty members of the KMA at the 1st Korea Military Academy Artificial Intelligence Seminar held by the ROK Army AI R&D Center.

project image

Featured on National Defence TV, Defence Media Agency, Korea (State Broadcaster)


2021-12-30
video

Episode title: 'Dreaming of the Army with Advanced Science and Technology, Military Science and Technology Researcher in Korea Military Academy, Part 2, I Am a Republic of Korea Soldier, episode 176.' Another half-hour documentary TV show is dedicated to my job as an enlisted research specialist and my contributions on supporting the cadet education with teaching assistance and technology demonstrations on artificial intelligence in the KMA.

project image

Featured on National Defence TV, Defence Media Agency, Korea (State Broadcaster)


2021-12-23
video

Episode title: 'Dreaming of the Army with Advanced Science and Technology, Military Science and Technology Researcher in Korea Military Academy, Part 1, I Am a Republic of Korea Soldier, episode 175.' A half hour-long documentary TV show dedicated to my role in the Army as an enlisted research specialist and my contributions on the first field application of artificial intelligence in the ROK Army, the project I have planned and executed.

project image

Corp Commander's Commandation Ribbon ★ ★ ★


2021-11-30

I was awarded by the Superintendent of the KMA, LTG Kim, Jeong Soo (★ ★ ★), for meritorious contributions to laying the groundwork for the intelligent military surveillance system by taking a leading role in the task 'Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area.'

project image

Application of Artificial Intelligence on Intelligent Surveillance System of Korea Military Academy Command Control Center


2021-11-29

I proposed and solely developed an automated survellance system integrating real-time multi-object tracking, super-resolution, and re-identification systems with a soldier-friendly interface. Being the first ever military application of artificial intelligence of modern kind in the ROK Army, this project was commissioned and supported by LTG Kim, Jeong Soo (★ ★ ★), the Superintendent of Korea Military Academy. I reported the results in front of the commander (★ ★ ★) at the 21-2 semi-annual meeting of the Steering Committee of the KMA and received a Commendation Ribbon (LTG ★ ★ ★) for the accomplishment.

project image

Innovation Award, 2021 5th Army Startup Competition, Republic of Korea Army


2021-06-11

Proposal title: 'Military Consultation Chatbot for the Army Mental Health Services Using Large Language Models.' Our project was awarded by the Commanding General of the ROK Army Personnel Command, MG Jeon, Byung-hee (★ ★).




Other Projects

Coursework, personal projects, and unpublished researches. Games, blockchains, but mostly rockets.

project image

Adaptive Dataset Sampling by Deep Reinforcement Learning


Coursework Stochastic Control and Reinforcement Learning, 2020 Spring
2020-06-29
paper

I proposed an adaptive sampler as an improvement to the random sampling used in training deep neural networks. The method models the optimal sampling problem as a Markov Decision Process (MDP) and train a RNN policy network to create a stochastic sampling policy that is both model-agnostic and task-agnostic.

project image

Real-Time Ray-Marching with OpenGL


Coursework Graphics Programming, 2020 Spring
2020-06-17
paper / video / video #2 / code

I demonstrated that ray marching alrogithms can render arbitrarily detailed geometric objects, including 3D fractals and infinite landscape generated by procedural generation algorithms, in real time.

project image

Excellence Award, SNU Undergraduate Research Award


Award
2019-01-02
news

We developed a cold gas reaction control system for orientation control of a sounding rocket and electronic and software systems for the thrust measuring devices using NI LabVIEW. The project was funded by the SNU Undergraduate Research Program through the Faculty of Liberal Education, SNU (₩7M ≃ $6,000). See the official press release from SNU.

project image

Accelerated Magnetic Flux Density Calculator for Solenoid Magnets


Development
2018-10-31
code

As a research apprentice at Applied Superconductivity Lab led by Professor Seungyong Hahn, I developed this calculator for high-power magnet design, using Gaussian quadratures of elliptic integrals. By parallelizing with Intel MKL and OpenMP library, the program can run 100M calculations in 35 seconds using a personal computer, a speed boost of X78 than a MATLAB equivalent.

project image

Successful Launch of the First ANCP-Based Rocket (Identity-III)


Rocketry
2018-10-13
video / code

We have successfully launched our new rocket, the Identity-III, using our newly developed solid rocket booster; it flew with the maximum height of about 1 km. Developing the new propellant, ANCP, required much burn tests, which was successful with my newly developed data acquisition system for the field test.

project image

The 2nd Place at the 9th College of Engineering User Created Contents Contest, SNU


Award
2018-09-13
video

Title: 'Passion Rocket Scientists.' The video I made for my rocket teammates in the SNU Rocket Team Hanaro, awarded ₩1M (≃$900) for the second place in the College of Enginnering UCC Contest.

project image

Gold Medal (the 1st Place) at the 27th Rocket Engineering Competition


Award
2018-08-05
video

We, Hanaro, the Seoul National University Rocket Team, won a gold medal (the 1st place) in the rocket launch competition held by the National Universities’ Rocket Association (NURA), Korea. As the avionics team lead, I was responsible for the electronics of our rocket.

project image

Avionics of the Identity-II Rocket


Rocketry
2018-07-31
code

As the avionics team lead, I built electonic systems for our rockets, Identity-II and Identity-III, at Hanaro, the SNU Rocket Team. My integrated system consists of a portable power supply, an automatic video capturing system, a flight computer, a communications system powered by Zigbee, and a control system with an altitude-aware parachute dispenser.

project image

Data Acquisition (DAQ) System for Rocket Thrust Measurement Test


Rocketry
2018-07-25
code

As the avionics team lead, I devloped data acquisition system for rocket thrust measurement test for Hanaro, the SNU Rocket Team. The system allows simultaneous acquisition of data from two pressure sensors and one load cell, with the delay below 1ms.

project image

The 1st Place at 2018 Hdac Hackathon, the Largest Blockchain Contest in Korea, 2018


Award
2018-07-05
news / news #2

Proposal title: 'Development of a Platform-as-a-Service on edge computing-based blockchain middleware.' This was the largest software development contest (including of course the blockchain) held in Korea at that time. Our idea was awarded ₩50M (≃$45,000) for the first place. See news article (Korean).

project image

Firmware of the Identity-II Rocket


Rocketry
2017-12-01
video / code

As the avionics team lead, I developed the entire system of control, communications, and in-flight data acquisition systems of a sounding rocket (a small rocket for scientific research), the Identity-II, for Hanaro, the SNU Rocket Team.

project image

A Game: Cool Guy Don't Look At Explosions


GameDev
2017-11-30
code

I coded a simple vertical platformer game where you walk grandiosely and parrying enemy attacks without any hardship. Motivated by the song, "Cool Guys Don’t Look At Explosions" by the Lonely Island.

project image

Internship at Collain Healthcare, Georgetown, TX, USA


Experience
2016-02-25

As an intern, I conducted market research on electronic health record (EHR) system providers for nursing homes in the U.S.


Design and source code from Jon Barron's website