Nvidia has unveiled a new application named Chat with RTX that enables users to run large language models (LLMs) locally on their RTX 30 series and RTX 40 series graphics cards. This app, designed for Windows, is capable of handling queries related to various content types, such as documents, photos, and videos, when they are connected to the application. A standout feature includes its ability to answer questions about a YouTube video, using transcriptions as its source.
Chat with RTX presents a clear advantage for users who want to leverage LLMs on their personal computers without relying on an internet connection. Nvidia touts this offline functionality as a way to achieve faster and more secure results. It appears that the application is currently in a demonstration or beta phase, yet Nvidia has generously offered it as a free download. With a file size of 35GB, it’s conveniently available for download from Nvidia’s official website.
As the first application to locally harness the power of LLMs on personal GPUs, Chat with RTX is breaking new grounds and widening accessibility to advanced language processing technologies. With the potential for Nvidia to integrate additional capabilities, the evolutionary trajectory of the app seems poised for substantial growth and innovation. Considering that it has been just a short time since the emergence of platforms like ChatGPT, the rapid advancements in the field of Generative AI (GenAI) suggest an exhilarating future ahead for the technology.
For individuals looking to explore the functionalities of Chat with RTX and undertake personalized queries on their Nvidia RTX 30 or 40 series GPUs, this application marks an exciting step forward in the realm of on-device AI processing.
