Perform final parse after termination trigger set

[wk9874]

📝 Summary

If there is a long running callback, and the final lines of a results file are written while the callback is being executed and the termination trigger is called, then when the callback finishes it will stop the thread loop and never parse these final lines.

This change makes it so that after the termination trigger is set, the thread does one final loop to check for any final lines in the file, before exiting.

Hopefully will fix #155

✅ Checks

• Unit tests running locally.
• Pre-commit hooks run and passing.
• If a feature/bug fix, an issue has been opened and linked to this pull request.
• Documentation has been updated both within the code and in docs where appropriate.

[wk9874]

@kzscisoft can use this script to test functionality:

from multiparser import FileMonitor
import csv
import threading
import time
import multiprocessing
import multiparser.parsing.tail as mp_tail_parser
import pathlib
fieldnames = ['number']

TRIGGER = multiprocessing.Event()


def write_rows():
    pathlib.Path("results.csv").unlink(missing_ok=True)
    with open("results.csv", "w") as file:
        # Create a DictWriter object
        writer = csv.DictWriter(file, fieldnames=fieldnames)

        # Write the header row
        writer.writeheader()
        for i in range(50):
            # Write data rows
            writer.writerow({"number": i})
            file.flush()
            time.sleep(0.1)
            
        TRIGGER.set()
            
def callback(data: dict, *_):
    print(data)
    time.sleep(0.5)
    
thread = threading.Thread(target=write_rows)
        
thread.start()

with FileMonitor(termination_trigger=TRIGGER) as monitor:
    monitor.tail(
        path_glob_exprs="results.csv",
        parser_func=mp_tail_parser.record_csv,
        callback=callback,
    )
    monitor.run()