.net 下如何将文档文件(Word, Pdf等) 中的文本提取出来
经常有人问我怎么将类似word，pdf这样的文档转换为文本然后索引，.net 这方面的解决方案不是很多，为了方便大家，我花了一天时间自己做了一个。
Java 版本的 lucence 提供了一个 tika 的工具用于将 word, excel, pdf 等文档转换为文本，然后进行索引。但这个工具没有 .net 版本，要在 .net 下用，需要用 IKVM.net，很麻烦。而且这个工具实际上底层是调用 POI 和 PDFParse 来转换的。从网上搜索到的信息看,POI 对 office 2007 以上版本的文档处理有问题，不知道最新版本是否解决了，我没有试过。 PDFParse 这个东西，我用过 .net 版本，对中文不支持，不知道 Java 版本是否支持。
其实 .net 下完全不需要用这些开源解决方案来解决，因为微软提供了一个官方的解决方案，这个解决方案叫 IFilter，这个过滤器是为 SQL SERVER 的全文索引设计的，但第三方软件可以调用API来完成文档的提取工作。
为了方便大家，我把 IFilter 转换的功能封装到了一个开源的组件中去，大家可以到下面地址去下载源码： HBTextParse .
调用很简单：
这个是提取文件中的文本到字符串的代码
if (openFileDialog.ShowDialog() == DialogResult.OK)
{
//要转换的文件
textBoxFilePath.Text = openFileDialog.FileName;
//实例化 TextParse ，传入要转换的文件路径
TextParse textParse = new TextParse(textBoxFilePath.Text);
//提取文件中的文本，并输出
richTextBoxView.Text = textParse.ConvertToString();
}
这个是将文件转换为文本文件的代码：
if (saveFileDialog.ShowDialog() == DialogResult.OK)
{
try
{
//实例化 TextParse，传入要转换的文件的路径
TextParse textParse = new TextParse(textBoxFilePath.Text);
//将文件转换到 saveFileDialog.FileName 指定的文本文件中
textParse.ConvertToFile(saveFileDialog.FileName);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Error" , MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
要注意的问题是提取 Pdf 文档，如果机器是 64为操作系统，必须要安装 Adobe PDF iFilter 9 for 64-bit platforms . 否则会报异常。这个问题我搞了将近一天才搞定。
支持的文档类型：
目前这个组件可以支持所有 Microsoft office 提供的文档类型，包括 *.rtf, *.doc, *.docx, *.xls, *.xlsx, *.ppt, *.pptx 等等
除了微软Office的文档外，还可以转换
html 文档:可以把html文档中的文本提取出来（不包含标签） Pdf 文档:我测试过，对中文支持没有问题 Txt 文档这个代码的核心部分是一个叫 FilterCode 的类。这个类是从 http://ifilter.codeplex.com/ 这个地方下载的，我对这个类做了改进，加入了转换到文件的方法。我把这个类的代码贴出来，如果对如何调用IFilter的windows API感兴趣，可以参考这段代码
IFilter 的相关API函数如下：通常这些API函数就可以通过IFilter接口提取文本。
[DllImport( "query.dll" , SetLastError = true , CharSet = CharSet.Unicode)]
static extern int LoadIFilter( string pwcsPath,
[MarshalAs(UnmanagedType.IUnknown)] object pUnkOuter,
ref IFilter ppIUnk);
[ComImport, Guid( "89BCB740-6119-101A-BCB7-00DD010655AF" )]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IFilter
{
/// <summary>
/// The IFilter::Init method initializes a filtering session.
/// </summary>
[PreserveSig]
IFilterReturnCodes Init(
//[in] Flag settings from the IFILTER_INIT enumeration for
// controlling text standardization, property output, embedding
// scope, and IFilter access patterns.
IFILTER_INIT grfFlags,
// [in] The size of the attributes array. When nonzero, cAttributes
// takes
// precedence over attributes specified in grfFlags. If no
// attribute flags
// are specified and cAttributes is zero, the default is given by
// the
// PSGUID_STORAGE storage property set, which contains the date and
// time
// of the last write to the file, size, and so on; and by the
// PID_STG_CONTENTS
// 'contents' property, which maps to the main contents of the
// file.
// For more information about properties and property sets, see
// Property Sets.
int cAttributes,
//[in] Array of pointers to FULLPROPSPEC structures for the
// requested properties.
// When cAttributes is nonzero, only the properties in aAttributes
// are returned.
IntPtr aAttributes,
// [out] Information about additional properties available to the
// caller; from the IFILTER_FLAGS enumeration.
out IFILTER_FLAGS pdwFlags);
/// <summary>
/// The IFilter::GetChunk method positions the filter at the beginning
/// of the next chunk,
/// or at the first chunk if this is the first call to the GetChunk
/// method, and returns a description of the current chunk.
/// </summary>
[PreserveSig]
IFilterReturnCodes GetChunk( out STAT_CHUNK pStat);
/// <summary>
/// The IFilter::GetText method retrieves text (text-type properties)
/// from the current chunk,
/// which must have a CHUNKSTATE enumeration value of CHUNK_TEXT.
/// </summary>
[PreserveSig]
IFilterReturnCodes GetText(
// [in/out] On entry, the size of awcBuffer array in wide/Unicode
// characters. On exit, the number of Unicode characters written to
// awcBuffer.
// Note that this value is not the number of bytes in the buffer.
ref int pcwcBuffer,
// Text retrieved from the current chunk. Do not terminate the
// buffer with a character.
[Out(), MarshalAs(UnmanagedType.LPWStr)]
StringBuilder awcBuffer);
/// <summary>
/// The IFilter::GetValue method retrieves a value (public
/// value-type property) from a chunk,
/// which must have a CHUNKSTATE enumeration value of CHUNK_VALUE.
/// </summary>
[PreserveSig]
IFilterReturnCodes GetValue(
// Allocate the PROPVARIANT structure with CoTaskMemAlloc. Some
// PROPVARIANT
// structures contain pointers, which can be freed by calling the
// PropVariantClear function.
// It is up to the caller of the GetValue method to call the
// PropVariantClear method.
// ref IntPtr ppPropValue
// [MarshalAs(UnmanagedType.Struct)]
ref IntPtr PropVal);
/// <summary>
/// The IFilter::BindRegion method retrieves an interface representing
/// the specified portion of the object.
/// Currently reserved for future use.
/// </summary>
[PreserveSig]
IFilterReturnCodes BindRegion( ref FILTERREGION origPos,
ref Guid riid, ref object ppunk);
}
从文档中提取文本的代码如下：
/// <summary>
/// Utilizes IFilter interface in Windows to parse the contents of files.
/// </summary>
/// <param name="path">Path - Location of file to parse</param>
/// <param name="buffer">Buffer - Return text artifacts</param>
/// <returns>Raw set of strings from the document in plain text format.</returns>
public void GetTextFromDocument( string path, ref StringBuilder buffer)
{
IFilter filter = null ;
int hresult;
IFilterReturnCodes rtn;
// Initialize the return buffer to 64K.
buffer = new StringBuilder(64 * 1024);
// Try to load the filter for the path given.
hresult = LoadIFilter(path, new IntPtr(0), ref filter);
if (hresult == 0)
{
IFILTER_FLAGS uflags;
// Init the filter provider.
rtn = filter.Init(
IFILTER_INIT.IFILTER_INIT_CANON_PARAGRAPHS |
IFILTER_INIT.IFILTER_INIT_CANON_HYPHENS |
IFILTER_INIT.IFILTER_INIT_CANON_SPACES |
IFILTER_INIT.IFILTER_INIT_APPLY_INDEX_ATTRIBUTES |
IFILTER_INIT.IFILTER_INIT_INDEXING_ONLY,
0, new IntPtr(0), out uflags);
if (rtn == IFilterReturnCodes.S_OK)
{
STAT_CHUNK statChunk;
// Outer loop will read chunks from the document at a time. For those
// chunks that have text, the contents will be pulled and put into the
// return buffer.
bool bMoreChunks = true ;
while (bMoreChunks)
{
rtn = filter.GetChunk( out statChunk);
if (rtn == IFilterReturnCodes.S_OK)
{
// Ignore all non-text chunks.
if (statChunk.flags != CHUNKSTATE.CHUNK_TEXT)
continue ;
// Check for white space items and add the appropriate breaks.
switch (statChunk.breakType)
{
case CHUNK_BREAKTYPE.CHUNK_NO_BREAK:
break ;
case CHUNK_BREAKTYPE.CHUNK_EOW:
buffer.Append( ' ' );
break ;
case CHUNK_BREAKTYPE.CHUNK_EOC:
case CHUNK_BREAKTYPE.CHUNK_EOP:
case CHUNK_BREAKTYPE.CHUNK_EOS:
buffer.AppendLine();
break ;
}
// At this point we have a text chunk. The following code will pull out
// all of it and add it to the buffer.
bool bMoreText = true ;
while (bMoreText)
{
// Create a temporary string buffer we can use for the parsing algorithm.
int cBuffer = DefaultBufferSize;
StringBuilder sbBuffer = new StringBuilder(DefaultBufferSize);
// Read the next piece of data up to the size of our local buffer.
rtn = filter.GetText( ref cBuffer, sbBuffer);
if (rtn == IFilterReturnCodes.S_OK || rtn == IFilterReturnCodes.FILTER_S_LAST_TEXT)
{
// If any data was returned, scrub it and then add it to the buffer.
CleanUpCharacters(cBuffer, sbBuffer);
buffer.Append(sbBuffer.ToString());
// If we got back some text but there is no more, terminate the loop.
if (rtn == IFilterReturnCodes.FILTER_S_LAST_TEXT)
{
bMoreText = false ;
break ;
}
}
// Once all data is exhausted, we are done so terminate.
else if (rtn == IFilterReturnCodes.FILTER_E_NO_MORE_TEXT)
{
bMoreText = false ;
break ;
}
// Check for any fatal errors. It is a bug if you land here.
else if (rtn == IFilterReturnCodes.FILTER_E_NO_TEXT)
{
System.Diagnostics.Debug.Assert( false , "Should not get here" );
throw new InvalidOperationException();
}
}
}
// Once all chunks have been read, we are done with the file.
else if (rtn == IFilterReturnCodes.FILTER_E_END_OF_CHUNKS)
{
bMoreChunks = false ;
break ;
}
else if (rtn == IFilterReturnCodes.FILTER_E_EMBEDDING_UNAVAILABLE ||
rtn == IFilterReturnCodes.FILTER_E_LINK_UNAVAILABLE)
{
continue ;
}
else
{
throw new COMException( "IFilter COM error: " + rtn.ToString());
}
}
}
}
else
{
// If you get here there is no filter for the file type you asked for. Throw an
// exception for the caller.
throw new InvalidOperationException( "Failed to find IFilter for file " + path);
}
}
分类: Hubble.Net , 搜索引擎
标签: Hubble.Net , 索引
作者： Leo_wl
出处： http://www.cnblogs.com/Leo_wl/
本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。
版权信息查看更多关于.net 下如何将文档文件(Word, Pdf等) 中的文本提取出来的详细内容...
声明：本文来自网络，不代表【好得很程序员自学网】立场，转载请注明出处：http://haodehen.cn/did47244