{"id":1355,"date":"2024-10-21T04:47:31","date_gmt":"2024-10-21T04:47:31","guid":{"rendered":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2024\/10\/21\/announcing-ibm-granite-ai-models-now-available-on-docker-hub\/"},"modified":"2024-10-21T04:47:31","modified_gmt":"2024-10-21T04:47:31","slug":"announcing-ibm-granite-ai-models-now-available-on-docker-hub","status":"publish","type":"post","link":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2024\/10\/21\/announcing-ibm-granite-ai-models-now-available-on-docker-hub\/","title":{"rendered":"Announcing IBM Granite AI Models Now Available on Docker Hub"},"content":{"rendered":"

We are thrilled to announce that Granite models<\/a>, IBM\u2019s family of open source and proprietary models built for business, as well as Red Hat InstructLab model alignment tools, are now available on Docker Hub<\/a>.\u00a0<\/p>\n

Now, developer teams can easily access, deploy, and scale applications using IBM\u2019s AI models specifically designed for developers.<\/p>\n

This news will be officially announced during the AI track of the keynote at IBM TechXchange<\/a> on October 22. Attendees will get an exclusive look at how IBM\u2019s Granite models on Docker Hub accelerate AI-driven application development across multiple programming languages.<\/p>\n

Why Granite on Docker Hub?<\/strong><\/h2>\n

With a principled approach to data transparency, model alignment, and security, IBM\u2019s open source Granite models represent a significant leap forward in natural language processing. The models are available under an Apache 2.0 license<\/a>, empowering developer teams to bring generative AI into mission-critical applications and workflows.\u00a0<\/p>\n

Granite models deliver superior performance in coding and targeted language tasks at lower latencies, all while requiring a fraction of the compute resources and reducing the cost of inference. This efficiency allows developers to experiment, build, and scale generative AI applications both on-premises and in the cloud, all within departmental budgetary limits.<\/p>\n

Here\u2019s what this means for you:<\/p>\n

Simplified deployment<\/strong>: Pull the Granite image from Docker Hub and get up and running in minutes.<\/p>\n

Scalability<\/strong>: Docker offers a lightweight and efficient method for scaling artificial intelligence and machine learning (AI\/ML) applications. It allows you to run multiple containers on a single machine or distribute them across different machines in a cluster, enabling horizontal scalability.<\/p>\n

Flexibility<\/strong>: Customize and extend the model to suit your specific needs without worrying about underlying infrastructure.<\/p>\n

Portability: <\/strong>By creating Docker images once and deploying them anywhere, you eliminate compatibility problems and reduce the need for configurations.\u00a0<\/p>\n

Community support<\/strong>: Leverage the vast Docker and IBM communities for support, extensions, and collaborations.<\/p>\n

In addition to the IBM Granite models, Red Hat also made the InstructLab model alignment tools available on Docker Hub. Developers using InstructLab can adapt pre-trained LLMs using far less real-world data and computing resources than alternative methodologies. InstructLab is model-agnostic and can be used to fine-tune any LLM of your choice by providing additional skills and knowledge.<\/p>\n

With IBM Granite AI models and InstructLab available on Docker Hub, Docker and IBM enable easy integration into existing environments and workflows.<\/p>\n

Getting started with Granite<\/strong><\/h2>\n

You can find the following images available on Docker Hub:<\/p>\n

InstructLab<\/strong><\/a>: Ideal for desktop or Mac users looking to explore InstructLab, this image provides a simple introduction to the platform without requiring specialized hardware. It\u2019s perfect for prototyping and testing before scaling up.<\/p>\n

InstructLab with CUDA support<\/strong><\/a>: Designed for running full training workflows on GPU-equipped Linux<\/a> servers, this image accelerates the synthetic data generation and training process by leveraging NVIDIA GPUs.<\/p>\n

Granite-7b-lab<\/strong><\/a>: This image is optimized for model serving and inference on desktop or Mac environments, using the Granite-7B model. It allows for efficient and scalable inference tasks without needing a GPU, perfect for smaller-scale deployments or local testing.<\/p>\n

Granite-7b-lab with CUDA support<\/strong><\/a>: For those with GPU-equipped Linux servers, this image supports faster model inference and serving through CUDA acceleration. This is ideal for high-performance AI applications where response times and throughput are critical.<\/p>\n

How to pull and run IBM Granite images from Docker Hub\u00a0<\/strong><\/h3>\n

IBM Granite provides a toolset for building and managing cloud-native applications. Follow these steps to pull and run an IBM Granite image using Docker and the CLI.\u00a0You can follow similar steps for the Red Hat InstructLab images.<\/p>\n

Authenticate to Docker Hub<\/h4>\n

Enter your Docker username and password when prompted.<\/p>\n

Pull the IBM Granite Image<\/h4>\n

Pull the IBM Granite image from Docker Hub. There are two versions of the image:\u00a0<\/p>\n

redhat\/granite-7b-lab-gguf: For Mac\/desktop users with no GPU support<\/p>\n

redhat\/granite-7b-lab-gguf-cuda: For Linux NVIDIA\u00ae CUDA\u00ae support<\/p>\n

Run the Image in a Container<\/h4>\n

Start a container with the IBM Granite image. The container can be started in two modes: CLI (default) and server.<\/p>\n

To start the container in CLI mode, run the following:
docker run –ipc=host -it redhat\/granite-7b-lab-gguf\u00a0<\/p>\n

This command opens an interactive bash session within the container, allowing you to use the tools.<\/p>\n