Transformation of the Week
Ben Gibbons

Transcript

Lusk: Hey Ben, congrats on winning the Transformation of The Week, tell me a little bit more about your background.

Ben: So yeah, I’ve been building software for a little bit over a decade now. I’ve got a bunch of experience with C++, virtual reality, I’ve worked as a principal engineer, I’ve worked as a VR creative lead, and right now I’m working as a technical consultant.

Lusk: So what got you interested in Gen AI specifically?

Ben: Oh, that’s a good question. I think it was, it felt like something that had never existed before, something out of science fiction that was just amazing. Suddenly we had this tool that could write code almost as well as humans. And if you’ve been a long time programmer that’s kind of terrifying. But if you’re like me, your instinct when you’re faced with some new technology that’s scary and unknown, is to get really excited about it and try and learn everything you can about it.

Lusk: Now you graduated from the AI Engineering Bootcamp Cohort number 5 last week, congrats on that. Have you been able to put anything that you learned into practice at work or on personal projects?

Ben: Oh yeah definitely! So immediately after I graduated, I got hit up by an old former boss to go do AI contract work for his new company. And because of everything I learned through AI Makerspace, I absolutely crushed the interview that they gave me. But I’ve been using everything I learned through AI Makerspace to just accelerate my personal projects like no one’s business. I’ve been working a ton on my AI to 3D world prototype, and I’ve been making massive progress in a tiny amount of time.

Lusk: Now I know you’re doing something really cool with gen AI and VR, tt’s way over my head, why don’t you tell me and everyone listening a little bit more about it?

Ben: I’ve been working on a full end-to-end prompt to virtual reality system. So, a system where you type in just like a short text description of what you want, you put on your VR headset and boom, you’re in a 3D world that matches the prompt that you typed in. Why is that difficult and why is no one done that before? The data set to build something like that doesn’t really exist yet. Gen AI is built all around data, and then the model is a compressed down version of that data.

Data for 3D worlds just doesn’t really exist. It’s present in video games, but those video games aren’t published publicly. At least the datasets aren’t, which makes sense. And even if they were, the datasets would be in a whole bunch of different formats that you couldn’t just collect and feed into a model. We can’t just off-the-bat, train a single model to accomplish our task. So what do we do instead? Well, I think it’s really important to reframe the problem of generation of 3D worlds from it’s not just in a Gen AI problem, it’s not just a modeling problem, it’s actually a synthetic data generation problem.

So in the same way that we might, a tool we use a lot in the AI Makerspace is, RAGAS. So in the same way RAGAS generates synthetic data, this is actually, think of this as the same kind of synthetic data generation but for three dimensions. So it’s a combination of language models and vision models. And so we have the language model on one hand which is spitting out suggestions for where things; what kind of object should be in this environment; what kind of shapes can the environment be; what kind of train features should it have? And then on the other hand you have this vision model that is looking at renders that are coming out real-time of your 3D environment and going, “hey this one is good, this one’s not so good”; let’s head in this direction; let’s ignore this other direction. So it’s almost this like genetic evolving system that’s being tuned and pushed in different directions by these, these existing AI models.

Lusk: So what words of wisdom do you have for someone out there watching this right now who is considering a move into Gen AI?

Ben: Don’t wait to have everything all figured out just start building. It’s not the people who are who appear the most confident and or appear the most accomplished. It’s the people who show up early and are willing to do a whole bunch of trial and error. People who are willing to fail. If you’re any kind of engineer right now, you absolutely have the tools and the ability to start learning these things right now.

Lusk: Ben, congrats again on winning the Transformation of The Week. How can people connect with you, work with you, and follow your journey?

Ben: Yeah, so I think the best place to follow me is on LinkedIn. I’ve been really trying to post everything that I’ve been up to, all my personal projects, every all my certifications, everything that I’ve been learning there. GitHub is a good second place I try and promote semi-regularly. And in terms of working with me, please reach out through LinkedIn, email is also good.