feat/Allow PDF partitioning without unstructured_inference

[flash1293]

Is your feature request related to a problem? Please describe.
Up until unstructured 0.10.27 it was possible to use the fast and ocr_only strategy without having unstructured_inference installed (which pulls in a lot of transitive dependencies). However, starting from 0.10.28 there is a hard dependency on unstructured_inference for PDF partitioning in two ways:

Top level import of unstructured.partition.ocr which in turn has a top level import from unstructured_inference:

unstructured/unstructured/partition/pdf.py

Line 76 in 2931cb3

from unstructured.partition.ocr import (

This makes it impossible to use pdf partitioning without having unstructured_inference installed as importing from unstructured.partition.pdf will fail.

For OCR partitioning, there is another explicit check in place to require unstructured_inference:

unstructured/unstructured/partition/pdf.py

Line 324 in 2931cb3

@requires_dependencies("unstructured_inference")

Describe the solution you'd like

Ideally, both fast and ocr_only partitioning are possible without having to install all of unstructured_inference including transitive dependencies, basically the state of 0.10.27. This can be done by guarding all imports with explicit checks in various places.

Describe alternatives you've considered

• Installing unstructured_inference. In my environment, the application using unstructured is packaged in a docker image - adding the unstructured_inference dependency increases the size of the docker image by more than 3GB which makes distribution difficult.
• Restoring fast partitioning by avoiding top-level imports from unstructured.partition.ocr in unstructured.partition.pdf for the code path of the fast strategy. While this restores basic functionality, it reduces the number of parseable PDFs considerably.

Additional context

Happy to provide a PR if you agree with this being a useful feature.

[artdent]

I don't know what version fixed this, but I think it's fixed as of 0.12.5: I can partition PDFs again without having unstructured_inference installed.

[MthwRobinson]

Yeah we refactored the imports and and dependencies a bit and I don't think this is still and issue. We can reopen the issue if it pops up again.

[artdent]

This regressed again in 0.16.4 due to #3753, which added an unconditional import of unstructured_inference inside unstructured.partition.pdf.

[artdent]

Oops: also looks like #3234 added a dependency path from pdf -> pdf_image -> unstructured_inference, all the way back in 0.14.9.

[skulltech]

Any update on this? @artdent @MthwRobinson Is it fixed in recent versions or is still an issue?

[artdent]

Yes, https://github.com/Unstructured-IO/unstructured/blob/main/unstructured/partition/pdf.py still has a top-level import of unstructured.partition.pdf_image.analysis, so this is still an issue.

[artdent]

This is still an issue as of unstructured 0.21.5. e1f75a3 improved the situation by removing some imports from unstructured.partition.pdf_image, but others remain. https://github.com/Unstructured-IO/unstructured/blob/main/unstructured/partition/pdf.py imports from unstructured.partition.pdf_image.pdfminer_processing, which in turn imports from unstructured_inference.config, unstructured_inference.constants, and unstructured_inference.inference.elements.

That said, it's probably doable to make those last few imports lazy (or TYPE_CHECKING only).

[artdent]

I've sent #4284 as a proposed fix.