Jaerin Lee

I am a PhD student under Professor Kyoung Mu Lee at the Computer Vision Lab, Seoul National University (SNU), where I am currently leading internal workshops on Gaussian splatting. My goal as a researcher is to contribute to our understanding of the intricacies and dynamics of this world around us. The key, I believe, lies in the representation learning.

I started my research career in low-level vision where I have gained valuable insights into complex structural priors within images. Now I am lifting my sight from 2D to 3D, and from 3D to 4D, where our reality really lies. I want to answer how we can precieve and act upon this four dimensional universe, and I wish to have the answers for them soon.

From January 2021 to July 2022, I worked for the Republic of Korea Army AI R&D Center located at the Korea Military Academy for my compulsory military service. I proposed and developed one of the first military field applications of computer vision and deep learning in Korea: the AI-driven unstaffed surveillance system at Korea Military Academy. My contribution was featured in two dedicated half-hour-long episodes of a documentary TV show by National Defence TV, Korea, a state media [EP 1] [EP 2].

Email / GitHub / X / Google Scholar / LinkedIn / CV

Research

My research interest mainly lies in the representation learning. This is where I believe computer vision, reinforcement learning, natural language processing and computer graphics meet together. All our thoughts stems from our beliefs.

	StreamMultiDiffusion: Real-Time Interactive Generation with Region-Based Semantic Control Jaerin Lee, Daniel Sungho Jung, Kanggeon Lee and Kyoung Mu Lee arXiv preprint, 2024 project page / paper / code / demo / demo #2 We establish the compatibility between region-based control and acceleration methods for diffusion models to achieve up to x50 faster arbitrary-sized region-based text-to-image synthesis. Based on this, we propose a new drawing paradigm of semantic palette.
	Rethinking RGB Color Representation for Image Restoration Models Jaerin Lee, JoonKyu Park, Sungyong Baik and Kyoung Mu Lee arXiv preprint, 2024 paper As an alternative to the RGB colors, we present a new representation for images, where pairwise per-pixel loss functions are equipped with structural information for better supervision of various image restoration models.
	LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee and Kyoung Mu Lee arXiv preprint, 2023 project page / paper / code / demo / demo #2 We developed a 3D Gaussian splatting scene generation pipeline that recursively generates scenes from off-the-shelf diffusion models and our point cloud alignment algorithm.
	Clean Images are Hard to Reblur: Exploiting the Ill-Posed Inverse Task for Dynamic Scene Deblurring Seungjun Nah, Sanghyun Son, Jaerin Lee, and Kyoung Mu Lee ICLR, 2022 paper / poster We proposed to improve the deblurring quality of images by exploiting the inverse task of deblurring, namely, reblurring. The strategy is to design two types of reblurring loss functions for better deblurring: a supervised reblurring loss at the training time and the self-supervised reblurring loss at the inference time.
	Structure-Resonant Discriminator for Image Super-Resolution Jaerin Lee and Kyoung Mu Lee IEEE ICME, 2021 (Oral Presentation) paper / poster / slides Highlighting that GAN discriminators are also data models, I developed a new discriminator exploiting three structural features of natural images: translation equivariance, rotation invariance, and hierarchy of scale.
	AIM 2020 Challenge on Video Temporal Super-Resolution Sanghyun Son, Jaerin Lee, Seungjun Nah, Radu Timofe and Kyoung Mu Lee et al. ECCV Workshop, 2020 project page / paper / slides Jointly with the 2nd AIM: Advances in Image Manipulation workshop, in conjunction with ECCV 2020 in Glasgow, UK, we organized an AIM challenge on example-based Video Temporal Super-Resolution for enhancing videos for better temporal resolution.
	Progressively Growing Neural Networks for Image Recognition Jaerin Lee Undergraduate Thesis, 2019 code Thesis advisor: Professor Kyoung Mu Lee. I explored for the optimal architecture for image classifier by growing a convolutional neural network progressively according to the intermediate activation intensity, and achieved better result than randomly wired networks.
	Development and Demonstration of Hanging Pendulum Type Low Thrust Measurement System Deokhyeon Kim, Mingyu Park, and Jaerin Lee Spring Conference of the Korean Society for Aeronautical & Space Sciences, 2019 We developed a cold gas reaction control system and a corresponding measurement system for a sounding rocket. The project was *funded by the SNU Undergraduate Research Program through the Faculty of Liberal Education, SNU* (₩7M ≃ $6,000).

ROK Army Projects

Projects I had run at Republic of Korea Army AI R&D Center at the Korea Military Academy.

	Completed Military Service at ROK Army AI R&D Center 2022-07-17 I finished my duty and discharged from the ROK Army with final rank: *Sergeant*. In my 18 months of compulsory military service, from the proposals to the deployments I have led three projects commissioned by the Superintendent of the KMA (LTG ★ ★ ★), featured on air in two episodes of documentary TV shows from a state broadcaster, honored with two Commander’s Commendation Ribbons (LTG ★ ★ ★, MG ★ ★), awarded at the 5th Army Startup Competition (MG ★ ★) and delivered one invited talk for the faculty members of the KMA. It was truly an honor for me to serve my country with the best of what I can do.
	Division Commander's Commandation Ribbon ★ ★ 2022-06-30 I was awarded by the Superintendent of the KMA, MG Jeon, Sungdae (★ ★), for meritorious contributions to laying the groundwork for the intelligent military surveillance system by taking a leading role in the task '*Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area*.'
	Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area 2022-05-22 I planned and solely developed the *unstaffed, fully autonomous surveillance system for military operations* in the ammunition storage area of the KMA. It was the *first field deployment of machine learning algorithms* in the history of the ROK Army. This project was commissioned by LTG Chang-gu Kang (★ ★ ★), the Superintendent of Korea Military Academy. I received a Commendation Ribbon (MG ★ ★) for my successful accomplishment.
	Al-Driven Real-Time Restoration of Command Control Center Surveillance Videos in Times of Inclement Weather 2022-05-21 I planned and led a project commissioned by LTG Chang-gu Kang (★ ★ ★), the Superintendent of Korea Military Academy. I solely developed a real-time video restoration system for military surveillance cameras exposed in inclement weather. It was the *first military application of deep learning-based low-level vision algorithms* in the ROK Army.
	Invited talk at the Korea Military Academy 2022-05-18 Title of the talk: '*High Performance Computing for Artificial Intelligence.' It was an unusual honor for me to deliver a full one-and-a-half-hour lecture for the faculty members of the KMA at the 1st Korea Military Academy Artificial Intelligence Seminar* held by the ROK Army AI R&D Center.
	Featured on National Defence TV, Defence Media Agency, Korea (State Broadcaster) 2021-12-30 video Episode title: '*Dreaming of the Army with Advanced Science and Technology, Military Science and Technology Researcher in Korea Military Academy, Part 2, I Am a Republic of Korea Soldier, episode 176.' Another half-hour documentary TV show is dedicated to my job as an enlisted research specialist and my contributions on supporting the cadet education with teaching assistance and technology demonstrations on artificial intelligence in the KMA*.
	Featured on National Defence TV, Defence Media Agency, Korea (State Broadcaster) 2021-12-23 video Episode title: '*Dreaming of the Army with Advanced Science and Technology, Military Science and Technology Researcher in Korea Military Academy, Part 1, I Am a Republic of Korea Soldier, episode 175.' A half hour-long documentary TV show dedicated to my role in the Army as an enlisted research specialist and my contributions on the first field application of artificial intelligence in the ROK Army, the project I have planned and executed*.
	Corp Commander's Commandation Ribbon ★ ★ ★ 2021-11-30 I was awarded by the Superintendent of the KMA, LTG Kim, Jeong Soo (★ ★ ★), for meritorious contributions to laying the groundwork for the intelligent military surveillance system by taking a leading role in the task '*Development of AI Autonomous Surveillance System in Korea Military Academy Ammunition Storage Area*.'
	Application of Artificial Intelligence on Intelligent Surveillance System of Korea Military Academy Command Control Center 2021-11-29 I proposed and solely developed an automated survellance system integrating real-time multi-object tracking, super-resolution, and re-identification systems with a soldier-friendly interface. Being the *first ever military application of artificial intelligence of modern kind in the ROK Army, this project was commissioned and supported by LTG Kim, Jeong Soo (★ ★ ★), the Superintendent of Korea Military Academy. I reported the results in front of the commander (★ ★ ★) at the 21-2 semi-annual meeting of the Steering Committee of the KMA* and received a Commendation Ribbon (LTG ★ ★ ★) for the accomplishment.
	Innovation Award, 2021 5th Army Startup Competition, Republic of Korea Army 2021-06-11 Proposal title: '*Military Consultation Chatbot for the Army Mental Health Services Using Large Language Models*.' Our project was awarded by the Commanding General of the ROK Army Personnel Command, MG Jeon, Byung-hee (★ ★).

Other Projects

Coursework, personal projects, and unpublished researches. Games, blockchains, but mostly rockets.

	Adaptive Dataset Sampling by Deep Reinforcement Learning Coursework Stochastic Control and Reinforcement Learning, 2020 Spring 2020-06-29 paper I proposed an adaptive sampler as an improvement to the random sampling used in training deep neural networks. The method models the optimal sampling problem as a Markov Decision Process (MDP) and train a RNN policy network to create a stochastic sampling policy that is both model-agnostic and task-agnostic.
	Real-Time Ray-Marching with OpenGL Coursework Graphics Programming, 2020 Spring 2020-06-17 paper / video / video #2 / code I demonstrated that ray marching alrogithms can render arbitrarily detailed geometric objects, including 3D fractals and infinite landscape generated by procedural generation algorithms, in real time.
	Excellence Award, SNU Undergraduate Research Award Award 2019-01-02 news We developed a cold gas reaction control system for orientation control of a sounding rocket and electronic and software systems for the thrust measuring devices using NI LabVIEW. The project was *funded by the SNU Undergraduate Research Program through the Faculty of Liberal Education, SNU* (₩7M ≃ $6,000). See the official press release from SNU.
	Accelerated Magnetic Flux Density Calculator for Solenoid Magnets Development 2018-10-31 code As a research apprentice at Applied Superconductivity Lab led by Professor Seungyong Hahn, I developed this calculator for high-power magnet design, using Gaussian quadratures of elliptic integrals. By parallelizing with Intel MKL and OpenMP library, the program can run 100M calculations in 35 seconds using a personal computer, a speed boost of X78 than a MATLAB equivalent.
	Successful Launch of the First ANCP-Based Rocket (Identity-III) Rocketry 2018-10-13 video / code We have successfully launched our new rocket, the *Identity-III, using our newly developed solid rocket booster; it flew with the maximum height of about 1 km. Developing the new propellant, ANCP, required much burn tests, which was successful with my newly developed data acquisition system* for the field test.
	The 2nd Place at the 9th College of Engineering User Created Contents Contest, SNU Award 2018-09-13 video Title: 'Passion Rocket Scientists.' The video I made for my rocket teammates in the SNU Rocket Team Hanaro, awarded ₩1M (≃$900) for the second place in the College of Enginnering UCC Contest.
	Gold Medal (the 1st Place) at the 27th Rocket Engineering Competition Award 2018-08-05 video We, Hanaro, the Seoul National University Rocket Team, won a gold medal (*the 1st place*) in the rocket launch competition held by the National Universities’ Rocket Association (NURA), Korea. As the avionics team lead, I was responsible for the electronics of our rocket.
	Avionics of the Identity-II Rocket Rocketry 2018-07-31 code As the avionics team lead, I built electonic systems for our rockets, *Identity-II* and *Identity-III*, at Hanaro, the SNU Rocket Team. My integrated system consists of a portable power supply, an automatic video capturing system, a flight computer, a communications system powered by Zigbee, and a control system with an altitude-aware parachute dispenser.
	Data Acquisition (DAQ) System for Rocket Thrust Measurement Test Rocketry 2018-07-25 code As the avionics team lead, I devloped data acquisition system for rocket thrust measurement test for Hanaro, the SNU Rocket Team. The system allows simultaneous acquisition of data from two pressure sensors and one load cell, with the delay below 1ms.
	The 1st Place at 2018 Hdac Hackathon, the Largest Blockchain Contest in Korea, 2018 Award 2018-07-05 news / news #2 Proposal title: '*Development of a Platform-as-a-Service on edge computing-based blockchain middleware.' This was the largest software development contest (including of course the blockchain) held in Korea at that time. Our idea was awarded ₩50M (≃$45,000) for the first place*. See news article (Korean).
	Firmware of the Identity-II Rocket Rocketry 2017-12-01 video / code As the avionics team lead, I developed the entire system of control, communications, and in-flight data acquisition systems of a sounding rocket (a small rocket for scientific research), the *Identity-II*, for Hanaro, the SNU Rocket Team.
	A Game: Cool Guy Don't Look At Explosions GameDev 2017-11-30 code I coded a simple vertical platformer game where you walk grandiosely and parrying enemy attacks without any hardship. Motivated by the song, "Cool Guys Don’t Look At Explosions" by the Lonely Island.
	Internship at Collain Healthcare, Georgetown, TX, USA Experience 2016-02-25 As an intern, I conducted market research on electronic health record (EHR) system providers for nursing homes in the U.S.