Create an AI-Powered Image Analysis QA System Using Granite

Advancements in AI-Powered Image Analysis

As AI-driven technologies evolve, the realm of image analysis is making remarkable strides. With advancements in machine learning, AI can process images with a level of sophistication that provides deeper insights into visual data. This shift is crucial for various sectors, enabling automated processing, content moderation, and predictive modeling in applications such as pricing and image generation. By integrating these innovative data-driven approaches, businesses can streamline workflows and enhance decision-making, paving the way for intelligent visual interpretation.

Transforming Industries Through Computer Vision

Businesses and researchers alike are harnessing the power of AI to revolutionize how we analyze visual information. From image classification and Optical Character Recognition (OCR) to segmentation and video analysis, AI-powered tools are reshaping the landscape of image-based technologies. In industries such as social media, AI enhances content moderation by analyzing images at a pixel level, ensuring compliance while boosting engagement. Organizations can leverage Vision APIs for automated document processing, converting scanned files, spreadsheets, and reports into structured data. This efficiency not only streamlines workflows but also equips companies to extract meaningful insights from extensive visual datasets.

AI-Powered Q&A for Presentations

Large language models (LLMs) have transformed machine learning by enabling intelligent insights from extensive datasets of unstructured text. However, challenges persist when it comes to image analysis, particularly in interpreting charts, diagrams, and other visual elements in presentations. IBM Granite™ Vision 3.2 is bridging this gap by integrating AI tools with advanced object detection algorithms, allowing users to automate multimodal analysis. This article focuses on leveraging these capabilities to enhance PowerPoint presentations, facilitating interactive Q&A sessions based on both text and images.

Building an AI-Driven Q&A System

This tutorial guides you through constructing a system that can answer users’ questions in real-time based on PowerPoint slide content, utilizing both text and images as context. Here’s a glimpse of the key steps involved:

1. PowerPoint Processing: Extract text and images from .pptx files for AI-based analysis.
2. Text-Based Q&A: Utilize IBM Granite to generate answers using the extracted text from slides.
3. Image-Based Q&A: Leverage AI to analyze images, charts, and diagrams from the slides.
4. Optimizing Question Formulation: Craft effective questions to receive accurate AI responses.

Cutting-Edge Technologies and Tools

The tutorial employs several state-of-the-art technologies, including:

• IBM Granite Vision: A robust vision-language model adept at processing both text and images.
• Python-PPTX: A library facilitating the extraction of text and images from PowerPoint (.pptx) files.
• Transformers: A framework designed for efficiently processing AI model inputs.

The potential applications of this technology are vast, assisting developers, researchers, content creators, and business professionals in enhancing presentations with actionable AI-driven insights.

Getting Started: Step-by-Step Instructions

Step 1: Set Up Your Environment

To participate in this tutorial, you will need an IBM Cloud account. Log in and create a watsonx.ai project. Be sure to note your project ID, which will be required later.

Step 2: Install Required Dependencies

Start by installing essential Python libraries necessary for extracting and processing PowerPoint content:

bash
!pip install –upgrade transformers
!pip install –upgrade torch
!pip install python-pptx

Step 3: Import Required Libraries

Next, import necessary libraries to facilitate file handling and interact with the IBM Granite Vision model. Libraries include os, torch, pptx, PIL, and others to handle cloud storage access.

python
import os
import io
import torch
from pptx import Presentation
from PIL import Image
from io import BytesIO
from transformers import AutoProcessor, AutoModelForVision2Seq

Step 4: Connect to IBM Cloud Object Storage

Now you’ll establish a connection to IBM Cloud Object Storage to access your PowerPoint files. You’ll need your IBM Cloud API key and the appropriate endpoint URLs.

python
cos_client = ibm_boto3.client(
service_name=’s3′,
ibm_api_key_id=’Enter your API Key’,
ibm_auth_endpoint='[Enter your auth end-point url]’,
config=Config(signature_version=’oauth’),
endpoint_url='[Enter your end-point url]’
)

Step 5: Define Storage Parameters

Specify the IBM Cloud Object Storage bucket and file details for locating the PowerPoint presentation for processing.

python
bucket = ‘Enter your bucket key’
object_key = ‘Your_Presentation_Name.pptx’

Step 6: Retrieve the PowerPoint File

Download the PowerPoint file from IBM Cloud Object Storage to process it locally.

python
streaming_body = cos_client.get_object(Bucket=bucket, Key=object_key)[‘Body’]
pptx_bytes = streaming_body.read()

Step 7: Save the PowerPoint File Locally

Now save the retrieved PowerPoint content to a local file for further processing.

python
pptx_path = "downloaded_presentation.pptx"
with open(pptx_path, ‘wb’) as f:
f.write(pptx_bytes)

Step 8: Confirm File Save Location

Print a confirmation to ensure your PowerPoint file is saved successfully.

python
print(f"PPTX file saved as: {pptx_path}")

Step 9: Extract Text and Images from the PowerPoint File

Define a function that processes the PowerPoint file to extract both text and images, allowing the AI to answer questions based on this content.

python
def extract_text_and_images_from_pptx(pptx_path):
presentation = Presentation(pptx_path)
slide_texts = []
slide_images = []
for slide_number, slide in enumerate(presentation.slides):
slide_text = []
for shape in slide.shapes:
if hasattr(shape, "text"):
slide_text.append(shape.text)
slide_texts.append("\n".join(slide_text))

    for shape in slide.shapes:
        if hasattr(shape, "image"):
            image_stream = BytesIO(shape.image.blob)
            image = Image.open(image_stream)
            slide_images.append((slide_number, image))
return slide_texts, slide_images

Step 10: Process and Display Extracted Content

Call the function to process and display the extracted text and images from the PowerPoint file.

python
slide_texts, slide_images = extract_text_and_images_from_pptx(pptx_path)
for i, text in enumerate(slide_texts):
print(f"Slide {i + 1} Text:\n{text}\n{‘-‘*40}")

Following these steps, you can build an intelligent system capable of real-time interaction with your presentations, answering questions based on textual and visual content. This effective integration of AI not only enhances the user experience but also empowers organizations to derive insightful, actionable data from their visual presentations.