Gemini Now Hears You, Sam Altman & Jony Ive AI Device, Virtual Influencers by TikTok

Welcome to our weekly recap! How was your week? Looking at all our newsletter heroes' accomplishments, I sometimes feel like I'm not productive enough. I hope you're doing better! Anyway, today, we have a lot of AI news to talk about.

We'll discuss some exciting rumors about TikTok and the partnership between Sam Altman and Jony Ive, announcements from Google and OpenAI, and who wants to displace Nvidia in the AI chip market. Stay tuned; it's going to be interesting!

Recent YC’s Demo Day is one of the most significant news stories in the AI world. That's why we've highlighted it in a separate post. Follow this link if you missed it:

This Creators’ AI Edition:

Featured Materials 🎟️
News of the week 🌍
Useful tools ⚒️
Weekly Guides 📕
AI Meme of the Week 🤡
AI Tweet of the Week 🐦
(Bonus) Materials 🎁

Do you have a minute to participate in Subscription Giveaway and make our newsletter even better? Take a short survey and tell us about your experience + participate in the free Giveaway of a Premium Subscription for 6 months.

Start Survey!

# Featured Material 🎟️

Google’s Gemini 1.5 Pro Now Hears You

Google's update to Gemini 1.5 Pro gives the popular model an ears! The platform can now listen to uploaded audio files and generate information from things like earnings calls or audio from video without having to refer to a written transcript.

Gemini 1.5 Pro is already available to the public for the first time through its platform for building AI applications, Vertex AI.

This new version of Gemini Pro already surpasses the biggest and most powerful model, Gemini Ultra, in performance. Google claims Gemini 1.5 Pro can understand complicated instructions and eliminate the need to fine-tune models.

Gemini's massive context window (starting at 250,000 like the powerful Claude 3 Opus and scaling to over a million for select users) eliminates the need for fine-tuning. Simply load your data and start asking questions for immediate, tailored insights.

The update also means Gemini can now generate transcripts for video clips regardless of how long they might run and find a specific moment within the audio or video file.

Right now, most users encounter Gemini language models through the Gemini chatbot. But you have to remember that Gemini Ultra powers the Gemini Advanced chatbot, and while it is powerful and can understand long commands, it’s not as fast as Gemini 1.5 Pro.

In addition, Google revealed some important updates to the Gemini API:

System instructions: Developers can now control model responses using instructions available in Google AI Studio and the Gemini API. You can define roles, formats, goals, and rules to control model behavior for a specific use case.
JSON mode: You can tell the model to output only JSON objects. This mode allows you to extract structured data from text or images. Developers can get started with cURL; Python SDK support is coming soon.
Improvements to function calling: Users can now choose modes to restrict model output, which improves reliability. You can select text, function call, or just the function itself.

More about the new Gemini API features for developers can be found here.

Gemini Pro 1.5 is already available in more than 180 countries worldwide.

Keep your mailbox updated with key knowledge & news from the AI industry

Subscribe now

# News Of The Week 🌍

TikTok Plots Using Virtual Influencers for Ads

According to The Information, TikTok plans to add generative AI to its platform, allowing it to create its own virtual influencers. This feature is being developed specifically for TikTok Shop sellers. With its help, avatars will promote various products by reading scripts based on text prompts.

The new feature is in the early stages of development, and insiders say it may be subject to change before the official release.

You can, of course, wait until TikTok unveils its tool, but you already can create a virtual influencer right now. Follow this newsletter to learn more:

Jony Ive and Sam Altman Seeking Funding for Personal AI Device

OpenAI CEO Sam Altman and former Apple design chief Jony Ive have teamed up to design an AI-powered personal device and are seeking funding. Not much is known about the partnership, but insiders say that the new device won't look like a smartphone. However, given that Altman is one of the main investors in Humane AI, we may see something similar.

Ive and Altman are looking to raise $1 billion for the project!

Another big Apple executive to join Jony Ive, Sam Altman to build new AI device - BusinessToday

This isn't the first time we've heard about Jony Ive and Sam Altman's stealthy startup. The first such news appeared last fall, but then the project was only at an early discussion stage.

Google and Meta Expand In-House AI Chip Efforts

Google and Meta made two high-profile (and similar) announcements on the same week: the companies have made progress on developing CPUs designed for AI. Google's new Arm-based CPU, called Axion, will support Google's AI workloads before it rolls out to Cloud business customers "later this year." At the same time, Meta talked about its next-generation AI infrastructure. It's called MTIA and promises to accelerate the training of generative AI models.

These platforms promise to break Nvidia's monopoly on the AI chip market.

You can learn how Nvidia came to this status from this article:

Sharing is caring! Refer someone who started a learning Journey in AI!

Refer a friend

OpenAI Makes GPT-4 Turbo with Vision Available Through its API

The company announced on its X accounts that its GPT-4 Turbo with Vision model is now “generally available” through its API. GPT-4’s vision capabilities were released alongside audio uploads in September 2023, and GPT-4 Turbo was announced at OpenAI’s developer conference in November.

The latter promised speed improvements, larger input context windows (up to 128,000 tokens—equivalent to about a 300-page book or document), and increased affordability. In addition, requests for using the model’s vision recognition and analysis capabilities can now be made through the text format JSON and function calls, which generate a JSON code snippet that developers can use to automate actions within their connected apps.

Google Brings AI Editing Tools to all Google Photos Users

Google announced that a handful of enhanced editing features—including its AI-powered Magic Editor—will now be available for free to all Google Photos users. This expansion includes Google’s Magic Eraser, which removes unwanted items from photos; Photo Unblur, which uses machine learning to sharpen blurry images; and Portrait Light, which lets you change the light source on pictures after the fact.

Creators’ AI Subscription is the best gift for Creators diving into AI.

Give a gift subscription

Useful Tools ⚒️

eezyCollab – AI-powered influencer marketing tool

Infinity AI – Make your own GenAI meme videos in 2 minutes

SkimAI - The Ultimate Copilot for Your Inbox

Odaptos – Customer research powered by Artificial Intelligence

Muraena – Find sales leads with AI

Muraena AI is a lead generation tool designed explicitly for B2B sales teams. It uses AI to simplify finding the right prospects, replacing complex filters with a system where you define your ideal customer. The platform provides detailed information like a company's technology stack and funding status, allowing you to tailor your sales outreach for better results.