Skip to content

Commit 2f4b06e

Browse files
iTerminateiTerminate
authored
Merge pull request #9 from AdvancedPhotonSource/mqtt-integration
Release 2026.3
2 parents 2e21fb7 + 24c136f commit 2f4b06e

209 files changed

Lines changed: 20094 additions & 800 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/test-mqtt-broker.yml‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
name: Test BELY MQTT Message Broker
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- 'tools/developer_tools/bely-mqtt-message-broker/**'
7+
- '.github/workflows/test-mqtt-broker.yml'
8+
workflow_dispatch:
9+
10+
jobs:
11+
test:
12+
name: Run MQTT Broker Tests
13+
runs-on: ubuntu-latest
14+
15+
defaults:
16+
run:
17+
working-directory: tools/developer_tools/bely-mqtt-message-broker
18+
19+
strategy:
20+
matrix:
21+
python-version: ['3.11', '3.12']
22+
# python-version: ['3.11']
23+
24+
steps:
25+
- name: Checkout code
26+
uses: actions/checkout@v4
27+
28+
- name: Set up Python ${{ matrix.python-version }}
29+
uses: actions/setup-python@v5
30+
with:
31+
python-version: ${{ matrix.python-version }}
32+
33+
- name: Install dependencies
34+
run: |
35+
python -m pip install --upgrade pip
36+
make install-dev
37+
38+
- name: Run code quality linter
39+
run: |
40+
make lint
41+
42+
- name: Run code quality type-check
43+
run: |
44+
make type-check
45+
46+
- name: Run tests with coverage
47+
run: |
48+
make test-cov
49+
50+
# - name: Upload coverage reports
51+
# if: matrix.python-version == '3.11'
52+
# uses: actions/upload-artifact@v4
53+
# with:
54+
# name: coverage-report
55+
# path: tools/developer_tools/bely-mqtt-message-broker/htmlcov/

‎.github/workflows/test-plugin-utilities.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: Test Plugin Utilities
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- 'tools/developer_tools/logr_plugins/**'
7+
- '.github/workflows/test-plugin-utilities.yml'
8+
workflow_dispatch:
9+
10+
jobs:
11+
test:
12+
name: Run Plugin Utility Tests
13+
runs-on: ubuntu-latest
14+
15+
defaults:
16+
run:
17+
working-directory: tools/developer_tools/logr_plugins
18+
19+
strategy:
20+
matrix:
21+
python-version: ['3.11', '3.12']
22+
23+
steps:
24+
- name: Checkout code
25+
uses: actions/checkout@v4
26+
27+
- name: Set up Python ${{ matrix.python-version }}
28+
uses: actions/setup-python@v5
29+
with:
30+
python-version: ${{ matrix.python-version }}
31+
32+
- name: Install pytest
33+
run: python -m pip install --upgrade pip pytest
34+
35+
- name: Run tests
36+
run: python -m pytest test/ -v

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
.vscode
33
.DS_Store
44
src/python/.loglogin
5+
.loglogin
56

67
# Compiled files
78
*.pyc
@@ -56,3 +57,5 @@ docs/python/_build/
5657
# Temporary testing csv files
5758
tools/developer_tools/python-client/cdbCli/service/cli/Spreadsheets/
5859
.env
60+
tools/developer_tools/bely-mqtt-message-broker/conda-bld
61+
tools/developer_tools/bely-mqtt-message-broker/dev-config

‎CLAUDE.md‎

Lines changed: 237 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
BELY (Best Electronic Logbook Yet) is a Java EE web application for electronic logbook management. It is deployed on Payara Server (GlassFish fork) and uses MySQL/MariaDB as its database. The project includes:
8+
9+
- **LogrPortal**: Main Java EE web application (JSF/PrimeFaces frontend + REST API)
10+
- **Python web service**: Python-based web service components
11+
- **MQTT integration**: Python framework for handling MQTT events and notifications
12+
- **CLI utilities**: Command-line tools for searching and querying the system
13+
14+
## Build and Development Commands
15+
16+
### Environment Setup
17+
18+
Always source the environment setup script before running any commands:
19+
```bash
20+
source setup.sh
21+
```
22+
23+
This sets up critical environment variables:
24+
- `LOGR_ROOT_DIR`: Root directory of the repository
25+
- `LOGR_SUPPORT_DIR`: Support software directory (Payara, Java, MySQL, etc.)
26+
- `LOGR_INSTALL_DIR`: Installation directory
27+
- `LOGR_DATA_DIR`: Data directory
28+
- `PYTHONPATH`: Includes `src/python` and Python client paths
29+
30+
### Initial Setup
31+
32+
```bash
33+
# Install support software (Java, Payara, etc.)
34+
make support
35+
36+
# Install MySQL (if needed)
37+
make support-mysql
38+
39+
# Install NetBeans IDE
40+
make support-netbeans
41+
42+
# Create development configuration
43+
make dev-config
44+
```
45+
46+
### Database Management
47+
48+
```bash
49+
# Create clean database with schema
50+
make clean-db
51+
52+
# Create test database with test data
53+
make test-db
54+
55+
# Backup database
56+
make backup
57+
58+
# Development database variants
59+
make clean-db-dev
60+
make backup-dev
61+
```
62+
63+
Database SQL files are located in `db/sql/`:
64+
- `clean/`: Clean database schema initialization scripts
65+
- `test/`: Test data initialization scripts
66+
- `static/`: Static lookup data (e.g., notification providers)
67+
- `updates/`: Database migration scripts (e.g., `updateTo2025.3.sql`)
68+
- `create_logr_tables.sql`: Main table creation script
69+
- `create_stored_procedures.sql`: Stored procedures
70+
- `create_views.sql`: Database views
71+
72+
### Building and Deployment
73+
74+
```bash
75+
# Build the web portal
76+
cd src/java/LogrPortal
77+
ant clean
78+
ant dist
79+
80+
# Deploy to Payara (from project root)
81+
make deploy-web-portal
82+
83+
# Deploy web service
84+
make deploy-web-service
85+
86+
# Undeploy
87+
make undeploy-web-portal
88+
make undeploy-web-service
89+
90+
# Development deployment variants
91+
make deploy-web-portal-dev
92+
make deploy-web-service-dev
93+
```
94+
95+
### Running Tests
96+
97+
```bash
98+
# Run full test suite (backs up DB, deploys test DB, runs tests, restores DB)
99+
make test
100+
101+
# Manual API tests
102+
cd tools/developer_tools/python-client/
103+
pytest test/api_test.py
104+
105+
# Test requirements
106+
pip install -r tools/developer_tools/python-client/test/requirements.txt
107+
```
108+
109+
### Python Web Service Development
110+
111+
```bash
112+
# Start the Python web service
113+
./sbin/cdbWebService.sh
114+
```
115+
116+
Python code is in `src/python/cdb/`.
117+
118+
## Architecture
119+
120+
### Java Package Structure
121+
122+
All Java code is under `src/java/LogrPortal/src/java/gov/anl/aps/logr/`:
123+
124+
**portal** - Main web portal application
125+
- `controllers/` - JSF managed beans for UI logic
126+
- `model/db/entities/` - JPA entity classes (e.g., `Log`, `UserInfo`, `NotificationConfiguration`)
127+
- `model/db/beans/` - EJB facades for database operations (stateless session beans)
128+
- `view/` - View objects and JSF utilities
129+
- `import_export/` - Import/export functionality
130+
- `plugins/` - Plugin support system
131+
- `utilities/` - General utility classes
132+
133+
**rest** - REST API
134+
- `routes/` - JAX-RS resource classes (e.g., `LogbookRoute`, `SearchRoute`)
135+
- `authentication/` - Authentication filters and utilities
136+
- `entities/` - DTO classes for API responses
137+
- `provider/` - JAX-RS providers
138+
139+
**common** - Shared utilities
140+
- `mqtt/` - MQTT integration models and utilities
141+
- `objects/` - Common data objects
142+
- `utilities/` - Shared utility classes
143+
- `search/` - Search functionality
144+
145+
**api** - API client interfaces
146+
147+
### Web Application Structure
148+
149+
Location: `src/java/LogrPortal/web/`
150+
151+
- `views/` - XHTML pages organized by entity type (e.g., `log/`, `userInfo/`, `notificationConfiguration/`)
152+
- `templates/` - XHTML template files
153+
- `resources/` - Static resources (CSS, JavaScript, images)
154+
- `WEB-INF/` - Web application configuration
155+
156+
### Technology Stack
157+
158+
- **Framework**: Java EE 8 (JSF 2.3, EJB 3.2, JPA 2.2, JAX-RS 2.1)
159+
- **UI**: PrimeFaces 11 + PrimeFaces Extensions, OmniFaces 3
160+
- **App Server**: Payara 5.2022.5 (GlassFish fork)
161+
- **Database**: MySQL/MariaDB (driver: mariadb-java-client-3.1.0.jar)
162+
- **ORM**: EclipseLink (JPA implementation)
163+
- **Build**: Apache Ant + NetBeans project
164+
- **API Docs**: Swagger/OpenAPI 2.1.5
165+
- **PDF Generation**: iText 5.5.13.1
166+
- **Markdown**: Flexmark 0.64.8
167+
- **Logging**: Log4j 2.17.0
168+
169+
### MQTT Integration
170+
171+
The MQTT notification framework is a pluggable Python system for handling MQTT events:
172+
- Location: `tools/developer_tools/bely-mqtt-message-broker/`
173+
- Features: Type-safe Pydantic models, topic matching with wildcards, notification handlers (email, Slack, Discord)
174+
- See `tools/developer_tools/bely-mqtt-message-broker/README.md` for details
175+
176+
### Plugins System
177+
178+
BELY supports custom plugins for extending functionality:
179+
- Templates: `tools/developer_tools/logr_plugins/pluginTemplates/`
180+
- Deployment: `make deploy-cdb-plugin` or `make deploy-cdb-plugin-dev`
181+
182+
## Development Workflow
183+
184+
### NetBeans Setup
185+
186+
1. Open NetBeans: `netbeans &`
187+
2. File > Open Project > Select `src/java/LogrPortal`
188+
3. Resolve missing server: Right-click project > "Resolve Missing Server Problem"
189+
4. Add Payara Server pointing to `$LOGR_SUPPORT_DIR/netbeans/payara`
190+
5. Copy MariaDB driver to Payara:
191+
```bash
192+
cp src/java/LogrPortal/lib/mariadb-java-client-3.1.0.jar \
193+
$LOGR_SUPPORT_DIR/netbeans/payara/glassfish/domains/domain1/lib/
194+
```
195+
6. Run project from NetBeans
196+
197+
### Adding New Entities
198+
199+
When adding new database entities (e.g., notification system):
200+
201+
1. Update `db/sql/create_logr_tables.sql` with new table definitions
202+
2. Add static data SQL files to `db/sql/static/` if needed
203+
3. Create JPA `@Entity` class in `portal/model/db/entities/`
204+
4. Create `@Stateless` facade in `portal/model/db/beans/` (extends `AbstractFacade`)
205+
5. Create JSF controller in `portal/controllers/` (extends appropriate base controller)
206+
6. Create XHTML views in `web/views/<entity_name>/`
207+
7. Update database with `make clean-db` or create migration in `db/sql/updates/`
208+
209+
### Database Migrations
210+
211+
Version updates go in `db/sql/updates/updateToYYYY.X.sql` (e.g., `updateTo2025.3.sql`)
212+
213+
## Key Patterns
214+
215+
- **Entity/Facade/Controller**: Standard Java EE pattern - JPA entities, EJB facades for CRUD, JSF controllers for UI
216+
- **Named Queries**: Use JPA `@NamedQuery` annotations on entities for common queries
217+
- **Lazy Loading**: JSF data tables use lazy loading models
218+
- **Session Management**: Session-scoped JSF beans maintain user state
219+
- **REST Authentication**: Token-based auth via `rest/authentication/`
220+
- **MQTT Events**: Notification configurations trigger MQTT messages handled by Python framework
221+
222+
## Important Files
223+
224+
- `Makefile`: Top-level make targets for building, deploying, testing
225+
- `setup.sh`: Environment variable setup (must be sourced)
226+
- `sbin/`: Deployment and utility scripts
227+
- `src/java/LogrPortal/build.xml`: Ant build file
228+
- `src/java/LogrPortal/nbproject/project.properties`: NetBeans project configuration
229+
- `db/sql/create_logr_tables.sql`: Main database schema
230+
231+
## Notes
232+
233+
- The project is also known as "ComponentDB" or "CDB" in some contexts
234+
- Default database name: `logr` (development: `logr_dev`)
235+
- Application URL after deployment: `https://<hostname>:8181/bely` or `https://<hostname>:8181/cdb`
236+
- The main branch is `master` (not `main`)
237+
- Always run database operations and deployments from the repository root after sourcing `setup.sh`

‎Makefile‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,9 @@ test-db:
4343
test:
4444
$(TOP)/sbin/cdb_test.sh
4545

46+
test-plugins:
47+
cd $(TOP)/tools/developer_tools/logr_plugins && python -m pytest test/ -v
48+
4649
db:
4750
$(TOP)/sbin/cdb_create_db.sh
4851

0 commit comments

Comments
 (0)