ChatGPT can control Robots

Happy Friday everyone ! Welcome to The Alignment, where we help you understand what’s happening in the world of AI without any of the jargon or bs.

Here’s what we have lined up for you today -

1. Microsoft is using ChatGPT for Robotics

2. Samsung can use your AI generated voice to take calls for you

3. Nvidia hints at new cloud software offering

Microsoft is using ChatGPT for Robotics

The autonomous systems and robotics group at Microsoft is experimenting with using ChatGPT to control robots intuitively with natural language without any fine tuning of the model.

The group published a list of design principles aimed to guide language models towards solving robotic tasks. These principles include -

1. Define a set of high-level robot APIs or function library mapping to existing low-level implementations from the robot’s control stack or a perception library. It’s very important to use descriptive names for the high-level APIs so ChatGPT can reason about their behaviours.

2. Next, a text prompt for ChatGPT which describes the task while also explicitly stating which functions from the high-level library are available. The prompt can also contain information about task constraints, or how ChatGPT should form its answers (specific coding language, using auxiliary parsing elements)

3. The user stays on the loop to evaluate ChatGPT’s code output, either through direct inspection or using a simulator. If needed, the user uses natural language to provide feedback to ChatGPT on the answer’s quality and safety.

4. When the user is happy with the solution, the final code can be deployed onto the robot.

Here’s what ChatGPT could do :

1. Intuitively control a drone by asking clarification questions and writing complex code structures to determine the drone’s flight path.

2. Manipulate robotic arms using conversational feedback from users and eventually building the Microsoft logo using wooden blocks.

3. Plan a robot’s path to a particular object after it was given access to functions such as object detection and object distance APIs along with textual feedback about the object from the user.

Samsung can use your AI generated voice to take calls for you

Samsung has launched a custom voice creator which lets users record sentences to create an AI generated copy of their voice and tonal quality. The goal is to let the AI Generated voice answer phone calls for you when you’re unable to answer on your own. Samsung is using this feature as a test bed to expand the AI generated voice to other capabilities on Samsung devices.

Nvidia hints at new cloud software offering

Nvidia is soon going to take a jab at the cloud software market competing directly with the likes of established cloud platforms like Azure, AWS and GCP. At a time when AI interest is surging it makes sense for NVIDIA to try and break into the high margin software market given their dominance in the hardware market.

While not surprising, the market dynamics will be one to watch as the world’s biggest cloud providers are mostly all powered by GPUs built by NVIDIA.

Product Watch Luma AI -

Capture lifelike photorealistic 3D images and videos on the web from off the shelf traditional 2D images taken from your devices. The video demo does an amazing job showcasing this!

Around the industry

1. Agriculture tech startup raises 23 million to use AI software to optimise crop yield and educate farmers on how to grow the best produce to ensure food abundance.

2. A comprehensive survey of all pre-trained foundation ML Models.

3. Runway ML hosts the first ever AI film festival.

4. JP Morgan restricts staff from using ChatGPT.

5. McKinsey & Company joins Corporate Affiliate Program at Stanford HAI.

Weekend learning with Andrej Karpathy

Sit down with Andrej Karpathy as he goes over the fundamentals of neural networks and deep learning in this 2.5 hour deep dive. Seriously, we can’t believe this stuff is available for free on the internet.