Unstructured 非结构化文档解析

this will drop you into a bash shell where the Docker image is running

docker exec -it unstructured bash

You can also build your own Docker image. Note that the base image is `wolfi-base`, which is
updated regularly. If you are building the image locally, it is possible `docker-build` could
fail due to upstream changes in `wolfi-base`.

If you only plan on parsing one type of data you can speed up building the image by commenting out some
of the packages/requirements necessary for other data types. See Dockerfile to know which lines are necessary
for your use case.

this will drop you into a bash shell where the Docker image is running

make docker-start-bash

Once in the running container, you can try things directly in Python interpreter's interactive mode.

Installing the library

Use the following instructions to get up and running with unstructured and test your installation.

- Install the Python SDK to support all document types with pip install "unstructured[all-docs]" - For plain text files, HTML, XML, JSON and Emails that do not require any extra dependencies, you can run pip install unstructured - To process other doc types, you can install the extras required for those documents, such as pip install "unstructured[docx,pptx]" - Install the following system dependencies if they are not already available on your system. Depending on what document types you're parsing, you may not need all of these. - libmagic-dev (filetype detection) - poppler-utils (images and PDFs) - tesseract-ocr (images and PDFs, install tesseract-lang for additional language support) - libreoffice (MS Office docs) - pandoc is bundled automatically via the pypandoc-binary Python package (no system install needed)

- For suggestions on how to install on the Windows and to learn about dependencies for other features, see the installation documentation here.

At this point, you should be able to run the following code:

from unstructured.partition.auto import partition

elements = partition(filename="example-docs/eml/fake-email.eml")
print("\n\n".join([str(el) for el in elements]))

Installation Instructions for Local Development

The following instructions are intended to help you get up and running with unstructured locally if you are planning to contribute to the project.

This project uses uv for dependency management. Install it first:

```bash

:eight_pointed_black_star: Quick Start

There are several ways to use the unstructured library: Run the library in a container or Install the library 1. Install from PyPI 2. Install for local development * For installation with conda on Windows system, please refer to the documentation

PDF Document Parsing Example

The following examples show how to get started with the unstructured library. The easiest way to parse a document in unstructured is to use the partition function. If you use partition function, unstructured will detect the file type and route it to the appropriate file-specific partitioning function. If you are using the partition function, you may need to install additional dependencies per doc type. For example, to install docx dependencies you need to run pip install "unstructured[docx]". See our installation guide for more details.

from unstructured.partition.auto import partition

elements = partition("example-docs/layout-parser-paper.pdf")

Run print("\n\n".join([str(el) for el in elements])) to get a string representation of the output, which looks like:

LayoutParser : A Unified Toolkit for Deep Learning Based Document Image Analysis

Zejiang Shen 1 ( (cid:0) ), Ruochen Zhang 2 , Melissa Dell 3 , Benjamin Charles Germain Lee 4 , Jacob Carlson 3 , and
Weining Li 5

Abstract. Recent advances in document image analysis (DIA) have been primarily driven by the application of neural
networks. Ideally, research outcomes could be easily deployed in production and extended for further investigation.
However, various factors like loosely organized codebases and sophisticated model configurations complicate the easy
reuse of important innovations by a wide audience. Though there have been ongoing efforts to improve reusability and
simplify deep learning (DL) model development in disciplines like natural language processing and computer vision, none
of them are optimized for challenges in the domain of DIA. This represents a major gap in the existing toolkit, as DIA
is central to academic research across a wide range of disciplines in the social sciences and humanities. This paper
introduces LayoutParser, an open-source library for streamlining the usage of DL in DIA research and applications.
The core LayoutParser library comes with a set of simple and intuitive interfaces for applying and customizing DL models
for layout detection, character recognition, and many other document processing tasks. To promote extensibility,
LayoutParser also incorporates a community platform for sharing both pre-trained models and full document digitization
pipelines. We demonstrate that LayoutParser is helpful for both lightweight and large-scale digitization pipelines in
real-word use cases. The library is publicly available at https://layout-parser.github.io

Keywords: Document Image Analysis · Deep Learning · Layout Analysis · Character Recognition · Open Source library ·
Toolkit.

Introduction

Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of document image analysis (DIA) tasks
including document image classification [11,

See the partitioning section in our documentation for a full list of options and instructions on how to use file-specific partitioning functions.