Hi, I'm Nathan! I recently graduated from The University of Michigan
(Computer Engineering BSE, Physics Minor) and have been building out original, creative
applications and proof-of-concepts
utilizing Artificial Intelligence & Machine Learning.
(If you want to get in touch with
me, my email address is nnakkapa at umich.edu)
Physically play Tic-Tac-Toe on a piece of paper against a smart robot
I'm excited about the creative freedom and high impact of AI-driven software.
1.) In the past few months, I have architected and
built a novel vision-based AI Agent and Framework using multimodal LLMs to
autonomously interact (scroll/click) on websites and complete nontrivial tasks (a quick 1 min demo video is shown at the top of the page).
The Framework portion consists of high-level operations (read, type, click_button, scroll, extract_info, etc.) that allows the AI to
execute those actions and control the computer. The Framework uses
Set-of-Marks Prompting, removing the need to use the mouse at all since
all GUI interactions (including clicking buttons and scrolling) can be driven by the keyboard.
The AI Agent portion automates dynamic tasks where the input can vary
(unlike in traditional automation where actions are hard-coded).
This capability is thanks to the vision-mechanism of the AI Agent and allows for "more human-like automation"
since the Agent can see, interpret, and interact with website GUIs (just like humans).
Beyond the proof-of-concept demo, there is a lot more engineering work (experiments, usecases, code development, etc.)
and creative work (unqiue content creation & video demos, collaborations, community feedback, etc.)
that can be done to take this project to the next level.
Note, this project is very similar to
Claude's Computer Use. However, some of the engineering design and implementation choices differ.
For more info, checkout the AI Agent Github Repo here.
2.) I created the first Open Source Youtube Comment & Associated Subreplies Scraper (No API key needed).
AFAIK, no other open source Youtube Scraper
scrapes and groups/labels Comments with their associated Subreplies
for any Youtube video.
The original inspiration for this project was to identify/report sophisticated yet
subtle scam comment threads in Youtube videos.
The future work consists of training or finetuning a ML model to
identify those scam comment threads with high accuracy and then automate the reporting of those scams.
SmileClips (create smile compilations from Youtube videos), Cracked Chess (chess but rules change dynamically, forcing players to make moves based on partial info),
a better application of a well-known HCI device on laptops, and the Shortake App (a better alternative to the social media app BeReal).
In the past, I spent a summer as an Embedded Systems Intern
at Intrepid Control Systems (a company making hardware &
software for analyzing/testing onboard computer networks in vehicles).
I supported firmware validation efforts by creating an
ultrafast C++ GUI application for Ethernet Data Analysis and engineered 12 custom-made cables for internal testing purposes.
During the school year, I co-taught one of Michigan's introductory engineering classes,
E110,
to 40 first-year students. For my final semester at Michigan, I was honored to be apart of
TechLab at MCity, where I worked with a team to produce
accurate 3D scenes from a set of 2D images using
Gaussian Splatting
in partnership with
Capoom.
Outside of work, I try to get involved with my surrounding community.
I have participated in multiple case competions and pitch events. My favorite
was a Consulting Case Competition for P&G (Proctor and Gamble),
where my teammate and I
won 2nd place, providing actionable recommendations to
increase P&G’s Hispanic shopper growth with a $12,000,000 budget.
I was also the co-founder and Vice President of
Michigan Forensics Speech and Debate Club,
where I helped make the club website,
recruit 150+ members,
and hosted a grant-funded public speaking tournament.
Growing up, my two favorite sports were basketball and football.
Looking forward, I hope to inspire and help younger generations experience the blessing of sports.