One by one

Under the influence of excitement, in my first semester at ETHZ, I enrolled in far too many courses. With almost 40 ECTS credits on my plate, I began to pass them one by one. While Spanish B1.1 and the trolley problem-related course were not much of an issue, the vision courses posed a significant challenge. Computer Vision and Vision Algorithms for Mobile Robots focused on the geometric properties of images, deep learning, image segmentation, etc. Planning and Decision Making for Autonomous Robots provided exercises with various planning algorithms, controllers, and optimization techniques, while another course delved into database knowledge. The group projects from two courses brought me the most pride - a baking app user study for Human-Computer Interaction and Spot arm control with the Hololens AR headset for Mixed Reality.

Two sides of studying

In contrary to the first semester, the second one was more chilled. I had only a Seminar on Research Topics in Software Engineering and Digital Humans, during which I participated in a project on 3D human pose LISST parameters estimation. Apart from that, a semester-long project on smooth coverage path planning for underwater robots in cooperation with Tethys Robotics startup gave me an insight into implementing path-related solutions onto real robots. With that, I finished all the course-related credits for my Master.

The grand finale

The last semester at ETH was dedicated to my "Master thesis, done in collaboration with Microsoft Mixed Reality and AI Lab (now Spatial AI Lab), and Computer Vision and Geometry Group at ETH. The aim was to make a Large Language Model reason about spacial properties of the scenes. To this end, I developed a RAG-based system, which by leveraging various modalities (such as images, texts, navigation meshes and SQL databases) was able to answer spatial questions. Additionally, I created a dataset of 1'000 spatial questions with answers, along with a benchmarking system, to assess the accuracy of the solution. The findings, along with the published paper and codebase, can be found on the project's webpage.