China disappointed me a little. World Humanoid Robot Games.1 Fun, just like in real life Rock them, sock them robots what kind of people Really care about that robots do their job. That's why videos of robotic laundry folding are so popular: a few years ago, we didn't know how to do it. And that's definitely what people want! But how This article so well stated that basic laundry folding is in a better place considering the methods we have now. It might seem like if our AI methods can fold laundry then maybe they can do anything, but that's not true, and we'll have to invent new methods that are truly universal and useful.
With this in mind, I challenge the roboticists: here my Humanoid Olympic events. Each event will require us to introduce the latest technologies and open up new opportunities in robotic manipulation. I'll update my substack As people reach these goals, they mail real, real-life medals to the winners.
Current state of affairs
To talk about why each of these problems is driving the development of modern robotic manipulation, let's first talk about what's working now. What I see at work is learning by demonstration. Usually people use puppet interfaces. The most common are two copies of the robot, so that a person can grab and move one while the other follows. virtual reality headset with controllers for hand tracking. They then record the 10-30 second activity hundreds of times. Based on this data, the neural network is trained to imitate these examples. This method has unlocked tasks whose steps are somewhat chaotic (such as pulling a towel around a corner to make it lie flat) or have high state space (such as how a towel can be folded in many different ways).
When thinking about this method of teaching robots to do certain things, it should be clear what some of the limitations are. Each of them has exceptions, but together they form a general trend.
- No force feedback at the wrists.2 A robot can only perform as well as a human operating it remotely, but we don't yet have good standardized ways to communicate high-resolution force information to a human teleoperator.
- Limited finger control.3 It is difficult for the teleoperator (and the base model) to see and control all of the robot's fingers with much more precision than simply opening and closing them.
- No sense of touch.4 Human hands are literally stuffed with sensors. To get closer to this feeling from robot hands in such a way that it can be used by a human cameraman is currently not possible.
- Average accuracy.5 Based on the videos I've seen, I think the accuracy of the tasks is around 1-3cm.
Now about the events!
Event 1: Doors
Event 2: Laundry
Event 3: Tools
Event 4: Fingertip Manipulation
Event 5: Wet Manipulation
Event 1: Doors
Things like doors are tricky because of asymmetrical forces: you need to grab and turn the knob or knob quite hard, but if you pull hard beyond the arc of the door, you'll tend to slip out of the grip. Additionally, getting through the door requires full-body manipulation, something I've never seen anyone do before.
Bronze medal: Entrance to a door with a round handle.
I think this is very close to the modern state (or maybe it has already happened and I haven't seen it). I expect this medal to be claimed by December.
Silver Medal: Entry into a self-closing push door with a lever handle.
Adding a self-closing makes this significantly more difficult due to the force involved, although a lever handle is arguably simpler (I just don't see many self-closing doors with a round handle).6
Gold Medal: Lever handle self-closing door entry.
Boss, fight the doors.7 You either need to use your other limb to block the door from closing again, or get through the door fast enough to use the dynamics.
Event 2: Laundry
We're just starting to do laundry.
Bronze Medal: Fold the T-shirt inside out.
This is probably doable using the methods we have now, but it is a longer time horizon task and may require some tricky two-handed manipulation to pull the shirt right side out.8
Silver medal: Turn the sock inside out.
I think both hand insertion and inner sock pinching present interesting new challenges.
Gold medal: Hang up a man's shirt.
A medium-sized shirt begins to be unbuttoned by turning one sleeve inside out. It should be positioned correctly on the hanger, with the sleeve attached and at least one button fastened. I think there are 3-10 years left until this point, because the buttons are very difficult, and also because you need to become strong, deft a hand small enough to fit into the sleeve will find it difficult.
Event 3: Tools
Humans are technological creatures, and as useful as our hands are, we primarily use them to hold and manipulate tools. This challenge involves developing the strength and dexterity to use basic tools.
Bronze Medal: Window cleaner and paper towels.
The window cleaner bottle is very gentle in terms of how you hold it, but you do need to move your finger yourself, and your finger needs to be quite strong to spray the liquid.9
Silver medal: Peanut butter sandwiches.
The challenge here is to grab the knife and then adjust your grip so that it is strong and stable enough to scoop and spread the peanut butter. People use “strong tool skills” for all kinds of activities, but they find it very difficult. robot grippers.10
Gold Medal: Use the key.
A key fob containing at least 2 keys and a fob is thrown into the robot's waiting palm/gripper.. Without putting the keys down, Align the correct key, insert it and turn it into the lock. This requires very complex hand manipulations as well as highly precise force interactions.
Event 4: Fingertip Manipulation
We humans perform all sorts of manual manipulations, using the structure of our hands to manipulate the things we hold.
Bronze medal: socks of the correct size.
Requires dexterity and precision, but not very much strength.
Silver medal: Use a dog poop bag.
When I use the dog bag, I have to slide between my fingertips to separate the bag's opening, which is a complex force interaction, as well as a movement that I'm not even sure most robot arms are capable of. It's also difficult to tear off one packet rather than having to pull a large, long spool out of the holder if you decide to use it.11
Gold medal: peel an orange.
Made without third party tools. It is very difficult: high strength and high precision with your fingertips.
Event 5: Wet Manipulation
If you sit down and write down what you would like a robot to do for you, many tasks will end up being unconvincing. Robots generally don't like being wet, but we'll have to change that if we want them to be clean for us. And wet things can be difficult to pick up and use.
Bronze Medal: Wipe the countertop with a sponge.
Slightly damp, but with a huge risk of submerging your entire hand in water if you're not careful. You'll probably need at least splash-proof hands (or a whole bunch of spare parts).
Silver Medal: Clean the peanut butter from the paddle.
This one naturally follows the sandwich. Water is everywhere. This seems to be an important skill to acquire after several hours of collecting training data on the dog poop task.
Gold medal: Use a sponge to scrub grease off the pan in the sink.
Water, soap, grease and an unpleasant task that no one wants to do.
Terms and Conditions
To be eligible to win, a general-purpose robotic arm operating autonomously must demonstrate successful completion of a task on real-time video without any cuts. You are allowed a maximum of 10 times the time I spent on each task (a 4 second task could take your robot up to 40 seconds). I reserve the right to be arbitrary in making a decision if everything does not correspond to the spirit of the task. The first robot to achieve this will receive a prize!
To receive a medallion, send an email [email protected] with the address to which I should send it. If you give me a photo of your robot with a medal, I will be very happy. I will also accept future applicants who are at least 25% faster than the current winner. Good luck and may the odds always be on your side.
Thanks to Jeff Bingham for advice, fact checking and coolness robot video. And thanks to my patient wife for spending an hour filming me doing stupid things in a stupid costume.
Notes
1 As far as I can tell, kickboxing was just Unitree Mini Humanoid Robotand everyone had the same code working, so… I guess he won?
2 TRI has some pretty cool stuff with force control on a large training stand.
3 Tesla Optimus has 22 degrees of freedom via cable drive (because you can't fit into those motors in hand). In 2008 I was working on this robot which also had 22 degrees of freedom, and was incredibly difficult to control (as was properly tensioning all the cables). The other hand was large, with two fingers. capture which I ended up using for most teleop tasks.
4 Meta worked with some machine vision systems that seem cool.
5 It's most likely teleoperation accuracy limitation than model limitation. Here is the video Generalist Robotics, which performs tasks with subcentimeter precision. I love that hockey sticks have become a traditional “robot fight” tool, even for such ridiculous things.
6 Yes, I wore this to my workplace to get this video. Please.
7 I programmed (not trained) a general purpose mobile manipulator to go through a self-closing door, but it took over 4 minutes (disqualified for taking too long) and required a special door stop. Also, the video is not public (also disqualified). Plus, it's very tacky to have a competition and award yourself the gold before it even starts.
8 The T-shirt starts completely inside out in the tutu. The trim is tolerably folded, right side out.
9 You should spray 3 good sprays on the window and wipe it off with paper towels so there are no unsightly streaks left. Paper towels start with a roll of paper towels, not pre-torn and scrunched up.
10 The peanut butter jar starts and ends closed. The sandwich should be cut in half. (Both triangular and rectangular cuts are acceptable, although your three-year-old may disagree.)
11 Mock poops are allowed. The bag launches on a roll but can be held in a standard dog holder held by a robot.
This post originally appeared on General robotsBenji Holson's Substack on starting a general purpose robotics company.
Articles from your site
Related articles on the Internet
