GitHub - linuxrocks123/dish: This is the Display Shell, which implements a possibly novel technique for using AI-based OCR for desktop control. Dish can be used by itself as an automation tool or in conjunction with the included Sonic program as an accessibility aid. Dish depends on PaddleOCR and Sonic depends on voice2json.

linuxrocks123 / dish Public

Notifications You must be signed in to change notification settings
Fork 0
Star 1

This is the Display Shell, which implements a possibly novel technique for using AI-based OCR for desktop control. Dish can be used by itself as an automation tool or in conjunction with the included Sonic program as an accessibility aid. Dish depends on PaddleOCR and Sonic depends on voice2json.

GPL-2.0 license

1 star 0 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
Build		Build
Dimension_Swapper.hpp		Dimension_Swapper.hpp
FieldNames.h		FieldNames.h
LICENSE		LICENSE
MultiArray.hpp		MultiArray.hpp
Number.cpp		Number.cpp
Number.hpp		Number.hpp
README		README
StringFunctions.cpp		StringFunctions.cpp
StringFunctions.h		StringFunctions.h
accessibility_usage.txt		accessibility_usage.txt
chartbl.h		chartbl.h
dish-atk-backend.c		dish-atk-backend.c
dish.h		dish.h
dish.tab.c		dish.tab.c
dish.y		dish.y
paddle_example.txt		paddle_example.txt
relops.cxx		relops.cxx
rtolvalue.hpp		rtolvalue.hpp
sonic.cpp		sonic.cpp
spawn.cpp		spawn.cpp
spawn.hpp		spawn.hpp
test.dsh		test.dsh
test2.dsh		test2.dsh
test3.dsh		test3.dsh
test4.dsh		test4.dsh
test5.dsh		test5.dsh
test6.dsh		test6.dsh
test7.dsh		test7.dsh
test8.dsh		test8.dsh
xmacroplay.c		xmacroplay.c
xmacroplay.h		xmacroplay.h

Repository files navigation

Run ./Build to build
See dish.y for language syntax
Needs "pip3 install paddleocr" and "pip3 install paddlepaddle"
Tested on Linux.  Should work on other X11-based systems.
If you want to port to Windows or OS X, reimplement the stuff in xmacroplay.c using those systems' GUI automation hooks.  Ditto for Wayland.

Based partly on code from xmacrorec project.  I thank theem for their work.

dish is copyright (c) 2024 Patrick Simmons and licensed under the GPLv2.