📊 Excel Team Progress Tracker

Arabic-first dashboard for weekly department Excel reports: financial requests, task progress, validation alerts, and cross-week completion deltas — without a traditional business database.

🎯 Project overview

Organizations submit one Excel file per department per week, stored under folders named yyyy-mm-dd. This system:

Parses .xlsx files with Python and xlwings (Windows + Microsoft Excel)
Normalizes Arabic text and Eastern Arabic numerals (٠–٩ → 0–9)
Flags data issues (missing/duplicate task codes, unapproved financial rows)
Caches a canonical JSON payload in Laravel
Displays an RTL Filament admin UI in Arabic

Key capabilities

Area	What you get
💰 Financials	Weekly/department/grand totals; unapproved rows visible but excluded from sums
✅ Tasks	Full SRS column mapping; completion % deltas across weeks
⚠️ Alerts	System-wide warnings for bad source data
🏢 Views	Global financial & projects; department hub (financials, projects, tasks); integrity matrix

Tech stack

Layer	Technology
Extractor	Python 3.11+, xlwings, pandas, PyYAML, openpyxl
Backend	Laravel 11, Filament 3.3
Storage	Filesystem + JSON (`latest.json`); SQLite only for auth/cache
UI	Filament, Cairo font, RTL, brand `#FF2D20`

🏗️ Project architecture

flowchart LR
  subgraph sources [Filesystem]
    WeekDirs["yyyy-mm-dd folders"]
    Xlsx["Department.xlsx"]
    WeekDirs --> Xlsx
  end
  subgraph extract [Python]
    Discover[discovery.py]
    Parse[parsers + normalize]
    Validate[validate.py]
    JSON[latest.json]
    Discover --> Parse --> Validate --> JSON
  end
  subgraph laravel [Laravel]
    Cmd["progress:parse"]
    Cache[ProgressCacheService]
    Metrics[ProgressMetricsService]
    UI[Filament RTL admin]
    Cmd --> Parse
    JSON --> Cache --> Metrics --> UI
  end
  Config[config/progress.yaml] --> Discover
  Config --> Cmd
  Xlsx --> Discover

Repository layout

excel_summary/
├── config/progress.yaml       # Shared YAML: paths, ranges, Arabic headers
├── extractor/                 # Python venv + run.py CLI
│   └── scripts/generate_samples.py
├── backend/                   # Laravel + Filament (/admin)
├── samples/                   # Dummy weekly Excel (dev)
├── scripts/                   # PowerShell: setup, dev-start, parse, tests, …
├── README.md                  # This file
├── DEVELOPMENT.md             # Daily vs one-time commands
├── SYSTEM_DOCUMENTATION.md    # Technical reference
└── UX_Logic_SRS.md            # UX / information architecture

📦 Dependencies

System requirements

Requirement	Notes
Windows	Recommended; xlwings uses Excel COM
Microsoft Excel	Required for parsing (`run.py`); not for generating samples
PHP 8.2+	Laravel backend
Composer	PHP dependencies
Python 3.11+	Extractor venv

PHP (`backend/composer.json`)

Package	Purpose
`laravel/framework` ^11	Web app, cache, scheduler
`filament/filament` 3.3	Arabic RTL admin UI
`laravel/tinker`	REPL (dev)

Python (`extractor/requirements.txt`)

Package	Purpose
`xlwings`	Read Excel via installed Excel
`pandas`	Data manipulation
`PyYAML`	Load `config/progress.yaml`
`openpyxl`	Generate dummy `.xlsx` without Excel
`pytest`	Extractor tests

🛠️ Installation

Shortcut (Windows): from repo root run .\scripts\setup-first-time.ps1 — see scripts/README.md.

1️⃣ Python extractor

cd extractor
python -m venv .venv
.\.venv\Scripts\pip install -r requirements.txt

2️⃣ Laravel backend

cd backend
composer install
copy .env.example .env
php artisan key:generate
php artisan migrate
php artisan db:seed

3️⃣ Dummy data (first run)

cd extractor
.\.venv\Scripts\python scripts\generate_samples.py --clean

cd ..\backend
php artisan progress:parse --force

Daily commands: see DEVELOPMENT.md for what to run every session vs once.

▶️ How to run

Start the dashboard

.\scripts\dev-start.ps1

Or manually:

cd backend
composer run serve-dev

Open http://127.0.0.1:8000/admin

Field	Value
Email	`admin@progress.local`
Password	`password`

On some Windows + PHP 8.4 setups, php artisan serve fails — use composer run serve-dev (PHP built-in server).

Refresh data after Excel changes

php artisan progress:parse --force

Or click «تحديث البيانات» in the admin header.

Generate dummy data (GUI or CLI)

From the home page (الرئيسية): «توليد بيانات تجريبية» — recreates samples/ and re-parses (only in local env or when EXCEL_ROOT points at samples/).

From terminal:

php artisan progress:generate-samples --clean
php artisan progress:parse --force

⚙️ Configuration

Configuration has two layers: shared YAML for Excel layout, and optional .env overrides.

`config/progress.yaml`

Key	Description
`paths.excel_root`	Root folder of weekly directories (default: `samples`)
`paths.output_json`	Parsed JSON output path
`excel.week_folder_pattern`	Regex for week folders (`yyyy-mm-dd`)
`excel.financial.amount_range`	e.g. `E8:E25`
`excel.financial.description_range`	e.g. `H8:H25`
`excel.tasks.header_row`	Task header row (default: `28`)
`excel.tasks.data_start_row`	First data row (default: `29`)
`excel.tasks.columns`	Arabic header → internal field map
`validation.*`	Flags for missing/duplicate task codes, unapproved totals
`python.venv_python`	Path to venv Python for Laravel bridge

`backend/.env` overrides

Variable	Purpose
`EXCEL_ROOT`	Override weekly reports directory (absolute or relative to repo)
`PROGRESS_CONFIG_PATH`	Override YAML path
`PYTHON_EXECUTABLE`	Override Python binary
`PYTHON_SCRIPT`	Override `extractor/run.py` path

⚠️ Do not set env vars to empty strings (EXCEL_ROOT=). Leave them commented out so defaults apply.

Example production path:

EXCEL_ROOT=Z:/Reports/WeeklyProgress

Production deployment (security)

Before going live, set these in backend/.env (never commit .env):

Variable	Value
`APP_ENV`	`production`
`APP_DEBUG`	`false`
`APP_KEY`	Run `php artisan key:generate` once per environment
`ADMIN_PASSWORD`	Strong unique password (required before `php artisan db:seed`)
`ADMIN_EMAIL`	Optional; defaults to `admin@progress.local`

cd backend
php artisan key:generate
php artisan migrate
# Set ADMIN_PASSWORD in .env first, then:
php artisan db:seed

Restrict /admin (HTTPS, VPN, or firewall). Do not use the local dev password password in production.

See DEVELOPMENT.md for secret scanning, git safety rules, and confidential data handling.

🖥️ Dashboard views (Arabic RTL, V3)

View	Description
🏠 الرئيسية (`/admin/home`, page title صحة النظام)	System health (scoped by global date range), matrix, alerts, sort controls
💰 الاحتياج المالي (`/admin/global-financial`)	All departments by week tab + weekly grand total + range rollup
📁 المشاريع (`/admin/global-projects`)	Projects across all departments with dept badges
🏢 مركز المديرية (`/admin/departments/{dept}`)	Hub: per-dept financials, projects, tasks
Global date bar	On every admin page — filters weeks for all views above

Weekly Excel folders may use yyyy-mm-dd or yyyy_mm_dd (normalized to hyphens in the app).

See UX_Logic_SRS.md and SYSTEM_DOCUMENTATION.md §7 for the full IA.

Browser tab icon

Custom favicon and sidebar logo: backend/public/images/favicon.svg (brand #FF2D20), registered in Filament AdminPanelProvider.

📋 Artisan commands

Command	Description
`php artisan progress:parse`	Parse if Excel files changed
`php artisan progress:parse --force`	Force re-parse
`php artisan progress:generate-samples`	Generate dummy Excel under `samples/`
`php artisan progress:generate-samples --clean`	Delete `samples/` first, then generate
`php artisan progress:ensure-admin`	Create or reset the Filament admin user

Hourly auto-parse: backend/routes/console.php.

📚 Related documents

DEVELOPMENT.md — everyday dev commands
SYSTEM_DOCUMENTATION.md — architecture, services, GUI reference
UX_Logic_SRS.md — information architecture and UX behavior
scripts/README.md — PowerShell helper scripts

Optional local-only docs (Progress_Tracking_Dashboard_SRS.md, progress_tracker.md, styling_guide.md) are gitignored and may exist on your machine but are not part of the tracked repo.

📄 License

See LICENSE.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

📊 Excel Team Progress Tracker

Table of contents

🎯 Project overview

Key capabilities

Tech stack

🏗️ Project architecture

Repository layout

📦 Dependencies

System requirements

PHP (`backend/composer.json`)

Python (`extractor/requirements.txt`)

🛠️ Installation

1️⃣ Python extractor

2️⃣ Laravel backend

3️⃣ Dummy data (first run)

▶️ How to run

Start the dashboard

Refresh data after Excel changes

Generate dummy data (GUI or CLI)

⚙️ Configuration

`config/progress.yaml`

`backend/.env` overrides

Production deployment (security)

🖥️ Dashboard views (Arabic RTL, V3)

Browser tab icon

📋 Artisan commands

📚 Related documents

📄 License

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 19 Commits
.github/workflows		.github/workflows
backend		backend
config		config
extractor		extractor
scripts		scripts
.gitignore		.gitignore
.gitleaks.toml		.gitleaks.toml
.pre-commit-config.yaml		.pre-commit-config.yaml
.secrets.baseline		.secrets.baseline
DEVELOPMENT.md		DEVELOPMENT.md
LICENSE		LICENSE
README.md		README.md
SYSTEM_DOCUMENTATION.md		SYSTEM_DOCUMENTATION.md
UX_Logic_SRS.md		UX_Logic_SRS.md

Folders and files

Latest commit

History

Repository files navigation

📊 Excel Team Progress Tracker

Table of contents

🎯 Project overview

Key capabilities

Tech stack

🏗️ Project architecture

Repository layout

📦 Dependencies

System requirements

PHP (backend/composer.json)

Python (extractor/requirements.txt)

🛠️ Installation

1️⃣ Python extractor

2️⃣ Laravel backend

3️⃣ Dummy data (first run)

▶️ How to run

Start the dashboard

Refresh data after Excel changes

Generate dummy data (GUI or CLI)

⚙️ Configuration

config/progress.yaml

backend/.env overrides

Production deployment (security)

🖥️ Dashboard views (Arabic RTL, V3)

Browser tab icon

📋 Artisan commands

📚 Related documents

📄 License

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

PHP (`backend/composer.json`)

Python (`extractor/requirements.txt`)

`config/progress.yaml`

`backend/.env` overrides

Packages