Build an Image Labeler: React + Flask + OpenAI (Step-by-step)

What we’re building

A clean little app with two main actions:

Upload image → show a preview + filename
Predict → send the image to Flask → Flask calls OpenAI → return a single label

Architecture (super simple)

React (Vite)  ──►  Flask API  ──►  OpenAI
  UI + upload      /predict        returns label

Your project layout

We’ll keep frontend and backend in one repo folder, but separated. This structure avoids a classic beginner mistake (putting code inside the virtualenv).

Project tree (aim for this)

image-predictor/
  src/                 # React app code (frontend)
  index.html
  package.json
  backend/
    app.py              # Flask app code (backend)
    .env                # API key (local only)
    .venv/              # Python virtualenv (do NOT put your code here)

Gotcha we hit

We accidentally created app.py inside backend/.venv/. Flask couldn’t import the app. Your code must be next to .venv, not inside it.

Part 1 — Create the React app (Vite)

Go to a parent folder where you keep projects (for example, Documents/projects), then run Vite’s create command.

Terminal

npm create vite@latest image-predictor -- --template react

Gotcha we hit

Don’t pre-create the image-predictor folder. Vite creates it for you. If you create it first, you can end up with image-predictor/image-predictor.

Then:

Terminal

cd image-predictor
npm install
npm run dev

Checkpoint ✅

You should see the default “Vite + React” page in your browser.

Part 2 — The only file we care about (for now)

In the beginning, ignore the noise. You’ll spend most of your time in:

src/App.jsx — your UI
src/main.jsx — boots React (rarely touched)

First edit: prove you control the screen

Replace src/App.jsx with:

src/App.jsx

function App() {
  return <h1>Hello from React</h1>;
}

export default App;

Why it’s written like this

We use the “two-step style” because it’s beginner-friendly: define the function, then export it. React just needs an exported component.

Checkpoint ✅

Your browser updates automatically and shows “Hello from React”.

Part 3 — Build the “dumb UI” (no backend)

Start extremely simple: two buttons and some spacing.

src/App.jsx (snippet)

function App() {
  return (
    <div style={{ padding: "24px" }}>
      <h1>My first React app</h1>

      <div style={{ display: "flex", gap: "12px" }}>
        <button>Upload image</button>
        <button disabled>Predict</button>
      </div>
    </div>
  );
}

export default App;

Checkpoint ✅

You see two buttons. Predict is disabled (greyed out).

Part 4 — Use the standard file input first

We tried a couple “fancy label tricks” and it got messy fast. The standard, reliable approach is: use the native file input. We’ll style it later.

Add state + filename

src/App.jsx (snippet)

import { useState } from "react";

function App() {
  const [file, setFile] = useState(null);

  return (
    <div style={{ padding: "24px" }}>
      <h1>My first React app</h1>

      <div style={{ display: "flex", gap: "12px" }}>
        <input
          type="file"
          accept="image/*"
          onChange={(e) => setFile(e.target.files[0] ?? null)}
        />

        <button disabled={!file}>Predict</button>
      </div>

      {file && <p>Selected file: {file.name}</p>}
    </div>
  );
}

export default App;

Checkpoint ✅

Pick an image → filename appears. Predict enables only when a file exists.

Part 5 — Show an image preview

We create a temporary preview URL from the selected file. Then we clean it up when the file changes.

src/App.jsx (snippet)

import { useEffect, useState } from "react";

function App() {
  const [file, setFile] = useState(null);
  const [previewUrl, setPreviewUrl] = useState("");

  useEffect(() => {
    if (!file) {
      setPreviewUrl("");
      return;
    }
    const url = URL.createObjectURL(file);
    setPreviewUrl(url);
    return () => URL.revokeObjectURL(url);
  }, [file]);

  return (
    <div>
      {/* ... */}
      {previewUrl && (
        <img
          src={previewUrl}
          alt="preview"
          style={{ maxWidth: "100%", marginTop: "12px" }}
        />
      )}
    </div>
  );
}

export default App;

Gotcha we hit

The tag is <img />, not <image />. Also, React inline style keys are camelCase: maxWidth, not maxWidt.

Checkpoint ✅

After selecting a file, the preview appears below your controls.

Part 6 — Add the “prediction flow” (still mocked)

Before we wire the backend, we make the UI feel real: a Predict click shows loading, then a label.

src/App.jsx (snippet)

const [label, setLabel] = useState("");
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState("");

async function handlePredict() {
  setError("");
  setLabel("");
  setIsLoading(true);

  try {
    await new Promise((r) => setTimeout(r, 800));
    setLabel("cat");
  } catch (e) {
    setError(String(e));
  } finally {
    setIsLoading(false);
  }
}

Checkpoint ✅

Click Predict → it briefly shows “Predicting…” and then displays a label.

Part 7 — Create a local Flask backend

Create a backend/ folder inside your project. Then make a Python virtualenv there.

Terminal

cd backend
python -m venv .venv
# activate it (command depends on your OS)
pip install flask

Create backend/app.py:

backend/app.py

from flask import Flask, jsonify

app = Flask(__name__)

@app.get("/health")
def health():
    return "ok"

@app.post("/predict")
def predict():
    return jsonify({"label": "cat"})

Run it:

Terminal (inside backend/)

python -m flask --app app:app run --port 5000 --debug

Checkpoint ✅

Open http://127.0.0.1:5000/health → you should see ok.

Part 8 — Connect React → Flask (CORS)

Replace the mock delay with a real fetch() to Flask. At first you’ll likely see a failure (CORS), then we fix it.

Why this happens (CORS)

Your React dev server runs on one origin (like localhost:5173), and Flask runs on another (127.0.0.1:5000). Browsers block cross-origin requests unless the server allows them.

Fix CORS in Flask

Terminal (inside backend/)

pip install flask-cors

backend/app.py (snippet)

from flask_cors import CORS

app = Flask(__name__)
CORS(app)

Checkpoint ✅

Click Predict from React → you can successfully hit Flask and receive JSON.

Part 9 — Send the image file to Flask (FormData)

Files don’t go in JSON. In the browser, the standard approach is FormData.

src/App.jsx (snippet)

const formData = new FormData();
formData.append("image", file);

const res = await fetch("http://127.0.0.1:5000/predict", {
  method: "POST",
  body: formData,
});

In Flask, read the uploaded file from request.files:

backend/app.py (snippet)

from flask import request

@app.post("/predict")
def predict():
    file = request.files.get("image")
    if not file:
        return jsonify({"error": "No image uploaded"}), 400

    return jsonify({"label": "cat"})

Gotcha we hit

formData.append("image", file) needs two arguments. If you accidentally do append("image"), you’ll get “Not enough arguments”.

Checkpoint ✅

React uploads an image → Flask receives it → still returns “cat”.

Part 10 — Call OpenAI Vision from Flask

Now the fun part: Flask sends the image to OpenAI and returns a label. We’ll use a local .env file so your API key doesn’t live in code.

Create `backend/.env`

backend/.env

OPENAI_API_KEY=sk-your-key-here

Install dependencies

Terminal (inside backend/)

pip install openai python-dotenv

Use base64 data URLs (no third-party image hosting)

Why base64/data URLs are not “third-party hosting”

Base64 is just a way to represent bytes as text. A data URL is the image data “inline”. You’re not uploading to an image host. You’re sending the image directly to OpenAI in the API request.

backend/app.py (snippet)

import base64
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI()

# inside predict():
image_bytes = file.read()
mime = file.mimetype or "image/jpeg"
b64 = base64.b64encode(image_bytes).decode("utf-8")
data_url = f"data:{mime};base64,{b64}"

response = client.responses.create(
    model="gpt-4.1-mini",
    input=[{
        "role": "user",
        "content": [
            {"type": "input_text", "text": "Return a single simple label for the main object in this image. Reply with ONLY the label, 1-3 words."},
            {"type": "input_image", "image_url": data_url},
        ],
    }],
)

label = response.output_text.strip()
return jsonify({"label": label})

Checkpoint ✅

Upload an image → Predict → you get a real label from OpenAI.

Part 11 — Clean the label (tiny backend polish)

Models can return extra whitespace or newlines. We normalize it so your UI is consistent.

backend/app.py (snippet)

raw = (response.output_text or "").strip()

# normalize whitespace + take first line only
label = " ".join(raw.splitlines()[0].split())

if not label:
    label = "unknown"
if len(label) > 40:
    label = label[:40]

return jsonify({"label": label})

Checkpoint ✅

No weird spacing, no multi-line output — just a clean label.

Part 12 — UI polish (nice upload button + clear)

Once everything works, we can make the upload control look clean: hide the native input, trigger it with a button, and add Clear.

src/App.jsx (snippet)

import { useEffect, useRef, useState } from "react";

const fileInputRef = useRef(null);

<input
  ref={fileInputRef}
  type="file"
  accept="image/*"
  style={{ display: "none" }}
  onChange={(e) => setFile(e.target.files[0] ?? null)}
/>

<button type="button" onClick={() => fileInputRef.current?.click()}>
  Upload image
</button>

function handleClear() {
  setFile(null);
  setLabel("");
  setError("");
  if (fileInputRef.current) fileInputRef.current.value = "";
}

Checkpoint ✅

Your UI feels like a real product: upload button, preview, clear, prediction box.

Wrap-up (before deployment)

At this point, you have a working full-stack app locally: React sends an image → Flask receives it → Flask calls OpenAI → React displays the label.

Next chapter (not in this tutorial)

Deploying to PythonAnywhere: environment variables, WSGI config, and making the React frontend talk to your deployed backend.

Appendix — Full final code

This is a reference. In the tutorial, we built this step-by-step.

Frontend: `src/App.jsx`

src/App.jsx

import { useEffect, useRef, useState } from "react";

function App() {
  const fileInputRef = useRef(null);

  const [file, setFile] = useState(null);
  const [previewUrl, setPreviewUrl] = useState("");
  const [label, setLabel] = useState("");
  const [isLoading, setIsLoading] = useState(false);
  const [error, setError] = useState("");

  useEffect(() => {
    if (!file) {
      setPreviewUrl("");
      return;
    }
    const url = URL.createObjectURL(file);
    setPreviewUrl(url);
    return () => URL.revokeObjectURL(url);
  }, [file]);

  function handleClear() {
    setFile(null);
    setLabel("");
    setError("");
    if (fileInputRef.current) fileInputRef.current.value = "";
  }

  async function handlePredict() {
    setError("");
    setLabel("");
    setIsLoading(true);

    try {
      const formData = new FormData();
      formData.append("image", file);

      const res = await fetch("http://127.0.0.1:5000/predict", {
        method: "POST",
        body: formData,
      });

      if (!res.ok) {
        const text = await res.text();
        throw new Error(`HTTP ${res.status}: ${text}`);
      }

      const data = await res.json();
      setLabel(data.label);
    } catch (e) {
      setError(String(e));
    } finally {
      setIsLoading(false);
    }
  }

  return (
    
      Image labeler

      {/* hidden file input */}
       setFile(e.target.files?.[0] ?? null)}
      />

      {/* controls row */}
      
        

        

        
      

      {/* selected filename */}
      {file && (
        
          Selected
          {file.name}
        
      )}

      {/* preview */}
      {previewUrl && (
        
          
        
      )}

      {/* error + result */}
      {error && {error}}

      {label && (
        
          Prediction: {label}
        
      )}
    
  );
}

export default App;

Backend: `backend/app.py`

backend/app.py

import base64

from dotenv import load_dotenv
from flask import Flask, jsonify, request
from flask_cors import CORS
from openai import OpenAI

load_dotenv()

app = Flask(__name__)
CORS(app)

client = OpenAI()

@app.get("/health")
def health():
    return "ok"

@app.post("/predict")
def predict():
    file = request.files.get("image")
    if not file:
        return jsonify({"error": "No image uploaded"}), 400

    image_bytes = file.read()
    mime = file.mimetype or "image/jpeg"
    b64 = base64.b64encode(image_bytes).decode("utf-8")
    data_url = f"data:{mime};base64,{b64}"

    response = client.responses.create(
        model="gpt-4.1-mini",
        input=[{
            "role": "user",
            "content": [
                {
                    "type": "input_text",
                    "text": "Return a single simple label for the main object in this image. Reply with ONLY the label, 1-3 words.",
                },
                {"type": "input_image", "image_url": data_url},
            ],
        }],
    )

    raw = (response.output_text or "").strip()
    label = " ".join(raw.splitlines()[0].split())

    if not label:
        label = "unknown"
    if len(label) > 40:
        label = label[:40]

    return jsonify({"label": label})

Local secrets: `backend/.env`

backend/.env

OPENAI_API_KEY=sk-your-key-here

What we’re building

Your project layout

Part 1 — Create the React app (Vite)

Part 2 — The only file we care about (for now)

First edit: prove you control the screen

Part 3 — Build the “dumb UI” (no backend)

Part 4 — Use the standard file input first

Add state + filename

Part 5 — Show an image preview

Part 6 — Add the “prediction flow” (still mocked)

Part 7 — Create a local Flask backend

Part 8 — Connect React → Flask (CORS)

Fix CORS in Flask

Part 9 — Send the image file to Flask (FormData)

Part 10 — Call OpenAI Vision from Flask

Create backend/.env

Install dependencies

Use base64 data URLs (no third-party image hosting)

Part 11 — Clean the label (tiny backend polish)

Part 12 — UI polish (nice upload button + clear)

Wrap-up (before deployment)

Appendix — Full final code

Frontend: src/App.jsx

Image labeler

Backend: backend/app.py

Local secrets: backend/.env

Create `backend/.env`

Frontend: `src/App.jsx`

Backend: `backend/app.py`

Local secrets: `backend/.env`