Automated Subjective Answer Sheet Evaluation System

1.1 Problem Statement

Traditional answer-sheet evaluation works reasonably well for objective questions, but subjective answer sheets are much harder to assess automatically. In descriptive exams, students do not always write the same words as the ideal answer — they may express the same concept in a different sentence structure, draw diagrams differently, or write mathematical expressions in equivalent but non-identical forms.

Because of this, simple keyword matching is not enough. A practical evaluation system must understand:

1.2 Project Goal

To create a usable academic evaluation platform that can accept an ideal answer sheet, rubric, and student answer sheets; extract and evaluate textual content semantically; separately process diagrams and formulas; and generate per-student scores, feedback, and downloadable PDF reports.

1.3 Why This Project Matters

Manual checking of subjective answer sheets is time-consuming, repetitive, and inconsistent across large batches. This system reduces manual effort while preserving a rubric-guided and explainable workflow — especially valuable for faculty members, researchers in educational technology, and experiments in multimodal grading.

1. Build a pipeline that evaluates descriptive answers beyond exact keyword matching.
2. Support both printed and handwritten answer sheets through multiple OCR paths.
3. Add diagram-aware scoring for visual answers.
4. Add formula-aware scoring for mathematical expressions.
5. Support two evaluation modes: local SBERT and LLM-based Gemini evaluation.
6. Generate human-readable PDF reports for each student.
7. Provide both a research-facing Streamlit interface and a product-style Next.js web interface.

3.1 In Scope

• Subjective answer sheet evaluation with OCR-based text extraction
• Printed sheet OCR (EasyOCR), handwritten OCR (Google Vision AI + Azure)
• Semantic text similarity scoring, diagram extraction + visual comparison
• Formula extraction + symbolic evaluation via SymPy
• Rubric-based weighted scoring with JSON and PDF report generation
• Streamlit research UI + FastAPI backend + Next.js web frontend

3.2 Current Assumptions

• One page is treated as one question.
• The ideal answer sheet acts as the reference solution.
• Rubric JSON provides per-question weights and maximum marks.
• Diagram regions are expected mainly in lower or distinct visual sections of a page.
• Formula extraction depends on OCR layout heuristics.

The main users of the system are:

• Faculty / examiners — grading descriptive answer sheets at scale
• Project supervisors and reviewers — evaluating BTP or research submissions
• Researchers — working on answer-sheet automation and multimodal grading
• Students / demo audiences — observing the evaluation system in action

The project is a multimodal pipeline built around three core ideas: extract evidence from answer sheets, evaluate each evidence type using the most suitable model, and combine results according to rubric weights.

Every run starts with three required inputs:

The project uses a JSON rubric. A simplified example:

{
  "1": {
    "max_marks": 10,
    "text_weight": 0.6,
    "diagram_weight": 0.4,
    "formula_weight": 0.2,
    "penalize_missing_diagram": true,
    "penalize_missing_formula": true
  }
}

Rubric Field Reference

Workflow Stages

1. Input validation — verify all three required files are present
2. OCR on ideal answer sheet — extract ground-truth text and layouts
3. OCR on each student sheet — extract per-page text blocks
4. Formula extraction — isolate math regions and convert to LaTeX (SBERT mode)
5. Diagram extraction — connected-component based visual isolation
6. Student-wise evaluation — compare all modalities against ideal
7. JSON output generation — structured per-student result files
8. PDF report generation — faculty-friendly individual reports
9. Summary display — ranking table and download links

Implemented in src/ocr_pipeline.py. Converts PDFs to images, runs the selected OCR backend, and saves page-wise JSON files with bounding boxes and confidence scores.

OCR Output Format

{
  "page": 1,
  "text": "full extracted text",
  "blocks": [
    {
      "bbox": [[x1,y1],[x2,y2],[x3,y3],[x4,y4]],
      "text": "block text",
      "confidence": 0.92
    }
  ]
}

OCR Backend Comparison

EasyOCR Processing Steps

1. Convert PDF pages to images using pdf2image.
2. Convert PIL image to NumPy array.
3. Run EasyOCR reader on the page.
4. Save text blocks and full page text as JSON.

Google Vision AI Processing Steps

1. Convert PDF pages to images.
2. Compress pages to fit Vision API request limits.
3. Call document_text_detection.
4. Extract paragraph/word blocks with bounding boxes.
5. Save page-wise OCR JSON.

Azure Document Intelligence Steps

1. Convert PDF pages to images.
2. Compress to fit Azure request limits.
3. Call prebuilt-read model.
4. Extract line/word content with polygon coordinates.
5. Save OCR results as JSON.

Implemented in src/diagram_extractor.py. Many answer sheets contain figures or circuit-style diagrams that cannot be judged through text OCR alone. The project separates diagrams from text and evaluates them independently.

Connected-Component Method

Output Path

results/diagrams/<BaseName>_page<PageNo>_diagram.png

Implemented in src/formula_pipeline.py. Mathematical expressions should not be treated as ordinary text because OCR noise makes direct string comparison unreliable. This module isolates formula-like lines and converts them to LaTeX for symbolic analysis.

Processing Steps

1. Read page-wise OCR blocks.
2. Convert each OCR block bounding box into rectangular regions.
3. Group nearby blocks into line candidates.
4. Detect whether a line looks formula-like (_looks_formula_like()).
5. Crop the formula region with padding.
6. Run pix2tex to convert the crop into LaTeX.
7. Save page-wise formula JSON.

Formula Detection Heuristics

The _looks_formula_like() function checks for patterns such as:

• Math operators: =, +, *, ^
• Fraction-like patterns
• Trig/calculus keywords: sin, cos, log, lim, dx
• High symbolic density
• Equation-like alphanumeric patterns

Formula Output Format

{
  "page": 1,
  "formula_count": 2,
  "formulas": [
    {
      "index": 1,
      "bbox": [x1, y1, x2, y2],
      "ocr_text": "E = mc^2",
      "latex": "E=mc^2",
      "crop_path": "results/formula_crops/...",
      "error": ""
    }
  ]
}

Implemented in src/formula_evaluator.py. Two formulas may be mathematically equivalent but look different as strings, e.g. a/b vs \frac{a}{b} vs algebraically rearranged forms. A formula-aware evaluator must check equivalence, not just string similarity.

Scoring Strategy

• 1.0 — exact or symbolically equivalent formulas
• Partial score — close but imperfect matches (string similarity fallback)
• 0.0 — incorrect or missing required formulas

Feedback Labels

Implemented in src/evaluation_core.py. The primary local research evaluation path combines semantic text, diagram, and formula similarity.

Text Similarity — SBERT

• Model: sentence-transformers/all-MiniLM-L6-v2
• Converts ideal and student answers into embeddings, computes cosine similarity
• Allows judging semantic closeness even when exact wording differs

Scoring Curve

Diagram Evaluation — CLIP

• Model: openai/clip-vit-base-patch32
• Ideal and student diagram images are embedded and compared by cosine similarity

Weight Normalization

For each question: text, diagram, and formula weights are read from the rubric, normalized, and each modality contributes proportionally to the final score. If the rubric's penalize_missing_diagram is set and the student has no diagram, marks for that modality reduce to zero.

Implemented in src/llm_evaluator.py. Provides rubric-aware LLM grading as an alternative to local SBERT evaluation.

Prompt Strategy

For each question, the evaluator builds a prompt containing: question id, rubric JSON, ideal answer text, student answer text, text and diagram maximum marks, and whether ideal/student diagrams are available. If diagrams exist, the evaluator sends both images alongside the prompt.

Expected JSON Response

{
  "text_score": 7.2,
  "diagram_score": 3.5,
  "score": 8.1,
  "max_marks": 10,
  "feedback": "Student answer covers key concepts but misses..."
}

Implemented in src/pipeline_service.py. Coordinates the full run: write rubric JSON, configure OCR / diagram / formula extraction, select evaluator, generate reports, and publish progress callbacks.

Progress Messages Published

• Running OCR on ideal answer sheet
• Extracting formulas
• Extracting diagrams
• Evaluating student [name]
• Generating report

Implemented in src/report_generator.py using fpdf2. Converts evaluation JSON results into faculty-friendly downloadable PDF reports.

Report Contents (Per Student)

Each question entry also includes a natural-language feedback string explaining why marks were awarded or deducted.

Implemented in backend_api/main.py. Acts as the bridge between the web interface and the evaluation pipeline — handling file uploads, job creation, synchronous evaluation, run history, report downloads, and runtime configuration.

API Endpoints

Health Endpoints

Informational Endpoints

Job & Run Endpoints

Job Metadata Stored Per Run

run id, status, engine, OCR backend, student count, created time, start time, completion time, current step, total steps, progress percent, events timeline, error message, report directory, evaluation directory, summary rows.

Implemented in src/app.py. Provides a fast experimentation and demonstration interface for uploading files, running local evaluations, observing workflow progress, showing ranking tables, and downloading reports.

Tabs & Sections

.venv\Scripts\python.exe -m streamlit run src\app.py

Product-style web interface in the web/ directory.

Evaluate Page Features

• File uploads (ideal PDF, rubric JSON, student PDFs)
• Rubric sample download
• Evaluation engine selection (SBERT / Gemini)
• OCR backend selection (EasyOCR / Vision AI / Azure)
• Live workflow tracking via progress callbacks
• Recent run history
• Summary tables and per-student report downloads

cd web
npm install
npm run dev

Docker Backend — Key Points

• Base image: python:3.10-slim
• Installs system dependencies: Poppler, OpenCV runtime libraries
• Installs backend-only Python requirements (requirements.backend.txt)
• Exposes port 8001
• Starts FastAPI via start-backend.sh → uvicorn backend_api.main:app

Helper Scripts

• run_all_local.cmd — start all three services at once
• run_api_8001.cmd — start FastAPI backend
• run_streamlit_8502.cmd — start Streamlit UI
• run_web_3100.cmd — start Next.js frontend

A practical walkthrough of one complete evaluation run:

1. Upload Ideal Answer Sheet.pdf
2. Upload rubric.json
3. Upload one or more student PDFs
4. Choose SBERT or Gemini as the evaluation engine
5. Choose OCR backend: EasyOCR, Google Vision AI, or Azure Document Intelligence
6. Click Run evaluation
7. OCR JSON is created for ideal + all student sheets
8. Diagram and formula outputs are extracted per page
9. Student answers are compared page by page against ideal
10. Weighted scores are calculated per question per student
11. Evaluation JSON and PDF reports are generated
12. Final summary ranking table is displayed with download links

1. Better question segmentation beyond one-page-one-question assumption
2. Stronger handwritten formula recognition (specialized models)
3. Diagram object detection instead of only connected components
4. Better dashboard analytics and charts
5. Cloud storage for long-term report persistence
6. Human-in-the-loop override and review mode
7. Multi-rubric support for different courses
8. Better benchmarking on large answer-sheet datasets

The Automated Subjective Answer Sheet Evaluation System is a research-driven, multimodal answer-sheet evaluation platform designed to handle the real complexity of descriptive academic assessment. Instead of relying only on exact keywords, it combines OCR, semantic similarity, diagram analysis, formula parsing, symbolic math evaluation, rubric-based weighting, and report generation into one connected workflow.

This makes the project much more realistic and extensible than a simple text-matching prototype. It already demonstrates a strong foundation for an academic BTP project and also forms the base for a future product-grade evaluation platform.