Generative AI + Industrial Automation

Education
1

Hi Asmbly Makerspace!

The Intro

I’m Andrew (LlamBit); I recently joined at the suggestion of a colleague and am very excited about the space already.

The Question

I wanted to get a pulse on a couple of areas that I might be able to offer some instruction on, depending on interest.

The Pitch

Generative AI and LLMs:

Do you want to prevent FAANG (and Tesla, and OpenAI, and on and on…) from doing to Generative AI what they did to the Internet? Do you want to wrangle your own, locally hosted LLMs for the greater good? Do you want to get Python savvy? Then I have a proposition for you…

The Multiverse School, based in San Francisco, is an online educational cooperative looking to put generative AI in the hands of the people. Hosted by Liz Howard and a bunch of other tech smarty pants, The Multiverse School empowers the maker, the entrepreneur, and the social activist to leverage this rapidly developing technology for people and not for profit (thanks KOOP). I recently stumbled across this GitHub (shoot me a message and I’d be happy to provide the link, it’s called HuggingFace - Lerobot) while attending the Prompt Engineering course with TMS. The project enables you to 3d print, assemble, and leverage LLM APIs to train your very own robot arm. Unfortunately, I do not have access to a 3D printer, nor know anyone that does.

Enter: Asmbly Makerspace. I signed up originally to get printer access, and had the thought that there may be more nerds out there that want to learn this stuff. I’m unsure if any folks in Austin are attending this course (or the School at large), but I thought it would be really great if there were. Attend classes, meet in person, build stuff, and generally nerd out.

Industrial Automation

By trade, I am an industrial automation and controls engineer. I’ve worked on a number of applications (from heavy industrial manufacturing, AI-based robotics, flexible circuit manufacture and more) and would love to start giving back to the community. I wanted to gauge if folks in this area wanted to learn the industrial side of microcontrollers (less Arduino-y, more Allen Bradley-y) and develop some industry-hardened skills. Which provides an excellent segue into…

The Robots

I got started in the industrial automation space with KUKA Robotics as a corporate trainer, and have been wanting to teach people interested in the space how accessible these tools really are. Maybe with the right kind of networking, Asmbly could even get its very own robot… Wouldn’t that be cool??

The latter two would require some hardware that I don’t currently possess (beyond a couple PLCs I could provide to the cause), but first I wanted to see if anyone would actually be interested in any of these topics.

So drop me a line!

4 Likes
2

Welcome to ASMBLY! While this is all a bit outside my time/goals scope, it’ sounds amazing. I hope it sparks some interest here.

1 Like
3

Large Language Models for the greater good, controlling my bot AND taking it to the next level with AI? Count me in. And how

When I realized I should be able to fairly easily train a LLM to recognize a certain type object in a large field of other stuff with as few as 200 example images (correct me if I’m wrong) then put it on a camera and make it portable, my mind was blown.

I’m old school and after seeing several decades evolution in electronics technology first hand I find the possibilities truly amazing. There are two applications I’ve been working on and the other is a paintbot. It’s essentially xyz fluid delivery and distribution on an easel. So being mostly a hardware guy I can build right up to the point of applying the fuzzy bits in-between before I start to struggle. But I know good results should be possible with a little Python and compute power.
Nice

2 Likes
4

Hey @LlamBit ! I’m super excited about your idea of implementing an LLM API to optimize a robotic arm—it’s such a fantastic concept! All the ideas you mentioned sound like incredible opportunities for Asmbly, and I’d love to see them come to life here.

I’m currently in touch with Asmbly’s education coordinators for starting a class to control a simple robotic arm using Arduino, stepper motors, and servo motors. The goal is to introduce members to the basics of microcontrollers and robotics, especially for those who are new to this space. Additionally, I am personally investigating how to build miniature smart cars that allow robots to roam freely.

I’d love to collaborate and discuss how we can align our efforts. Feel free to DM me—I’d be thrilled to connect!

2 Likes
5

Check out the SO-100 robot model on GitHub; it’s an interesting idea when it comes to robot training in that you have a “leader” (i.e. human driven) and "follower (i.e. LLM driven). It’s intended to tie into HuggingFace for image processing, and manages to do a LOT in just a few training sessions. the goal is to get as many of these printed as I can over the next few days; would be fun to meet up and do a build out in person. I’ll create a separate post for it.

When are you typically at Asmbly? Would be great to meet and chop it up~

1 Like
6

Give “LM Studio” a look up. It’s based on similar training that Anthropic and OpenAI are using, but it lives locally. Rule of thumb is match the number of GB ram your laptop has to the number of Billion parameters of the model. I.e. if you’re running 16GB ram, I’d recommend ~7B models (Qwen is a good one to start with).

Would love to talk more w/ you on your robo-Picasso! When are you usually in?

7

Nerd out! In person, over Discourse, existentially!

1 Like